Step-by-step guide to fine-tuning Llama 3 70B using PyTorch FSDP (Fully Sharded Data Parallel) combined with Q-LoRA, enabling training on consumer-grade GPUs. Covers environment setup, dataset preparation using the HuggingFaceH4/no_robots dataset, distributed training configuration with torchrun, memory usage trade-offs (from 4x24GB GPUs with CPU offloading to 4x H100s), cost estimates (~$255 on A10G vs ~$25-50 on H100s), and inference testing. Uses Hugging Face TRL's SFTTrainer, PEFT, Transformers, and Flash Attention v2 via PyTorch SDPA.
Table of contents
1. Setup development environment2. Create and prepare the dataset3. Fine-tune the LLM with PyTorch FSDP, Q-Lora and SDPA4. Test Model and run Inference3 Impressions