Step-by-step guide to fine-tuning Llama 3 70B on Amazon SageMaker using PyTorch FSDP (Fully Sharded Data Parallel) and Q-LoRA with Hugging Face TRL, PEFT, and Flash Attention v2. Covers environment setup, dataset preparation using the no_robots dataset, launching a SageMaker training job on ml.p4d.24xlarge (8x A100 GPUs), and deploying the fine-tuned model to a SageMaker endpoint using the Hugging Face LLM Inference DLC with TGI. Training Llama 3 70B for 2 epochs on 10k samples takes ~84 minutes and costs ~$50.
