A step-by-step tutorial on fine-tuning XLM-RoBERTa-large for multilingual text classification using Habana Gaudi HPUs on AWS DL1 instances. Covers environment setup, dataset preparation with the Amazon Science Massive dataset (7 languages, ~80K training samples), single-HPU training with GaudiTrainer, and scaling to distributed training across all 8 HPUs. Includes a cost-performance benchmark showing Habana Gaudi is 8.4x faster and 7.7x cheaper than a comparable NVIDIA V100-based p3.8xlarge instance for the same training job.

7m read timeFrom philschmid.de
Post cover image
Table of contents
1. Setup Habana Gaudi instance2. Load and process the dataset3. Create a GaudiTrainer and an run single HPU fine-tuning4. Run distributed data parallel training with GaudiTrainer5. Cost performance benefits of Habana Gaudi on AWSConclusion