A comprehensive guide to fine-tuning open LLMs in 2025 using Hugging Face tools, focusing on optimization and distributed training. Covers QLoRA (4-bit quantization with low-rank adaptation), Spectrum (SNR-based selective layer fine-tuning), Flash Attention, and Liger Kernels. Uses the TRL SFTTrainer with a reusable Python script and YAML configs rather than notebooks. Includes benchmark results comparing training times across different GPU setups (1x to 8x L4 GPUs), evaluation on GSM8K math dataset, and appendix sections on DeepSpeed distributed training, vLLM inference, and Spectrum configuration. Spectrum with 30% SNR layers achieved 58% GSM8K accuracy vs 54% for QLoRA.
Table of contents
1. Define a good use case for fine-tuning2. Setup development environment3. Create and prepare the dataset4. Fine-tune the model using trl and the SFTTrainer with QLoRA5. Test Model and run InferenceConclusionDistributed TrainingInferenceSpectrum1 Impression