> ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. --- title: "Fine-Tuning & Post-Training" url: https://daily.dev/agentic-ai-hub/fine-tuning-post-training/ description: "Reach for the cheapest tool that works, in this order:" lastUpdated: "2026-07-22" --- ## When to fine-tune vs prompt vs RAG Reach for the cheapest tool that works, in this order: 1. **Prompting / in-context learning.** Change behavior with instructions and a few examples. Zero training and instant iteration. Try this first. 2. **RAG ([RAG & Retrieval](/agentic-ai-hub/rag-retrieval/)).** Inject *knowledge* the model lacks. Use when facts are large, dynamic, or must be cited. 3. **Fine-tuning.** Change the model's *default behavior*: output format/schema adherence, tone/persona, a narrow skill, following domain conventions, or compressing a long prompt into learned weights (latency/cost win). Also for teaching structure or style that's hard to specify in words. The classic mistake is fine-tuning to add facts. It is an expensive, leaky way to do what RAG does better, and it bakes in staleness. Fine-tune for **form and behavior** and retrieve for **knowledge**. These combine: a fine-tuned model *plus* RAG is common. ## The post-training stack Modern "post-training" (everything after base pretraining) has a few layers: - **SFT (Supervised Fine-Tuning).** Train on curated (prompt, ideal-response) pairs. The workhorse. It teaches the model to imitate demonstrations. Data quality dominates the outcome. - **Preference optimization.** Align to human/AI preferences over pairs of responses: - **RLHF (Reinforcement Learning from Human Feedback).** Train a reward model on preference data, then optimize the policy with RL (e.g., PPO). It is powerful but complex and unstable to run. - **DPO (Direct Preference Optimization).** Skips the separate reward model and RL loop and optimizes directly on preference pairs with a simple classification-style loss. It is far easier to run than PPO and is the common default for open-model alignment. Variants like **ORPO**, **KTO**, and **SimPO** trade off data format and stability. - **RLAIF / Constitutional-style.** Use an AI model to generate the preference labels, which reduces human labeling cost. - **RLVR (RL from Verifiable Rewards).** For tasks with checkable answers (math, code, tool use), reward correctness directly. Central to 2025 to 2026 reasoning-model training. ## Parameter-efficient fine-tuning (PEFT): LoRA / QLoRA Full fine-tuning updates every weight, which requires enormous memory and compute and a full model copy per task. **PEFT** methods freeze the base model and train a small number of new parameters instead. - **LoRA (Low-Rank Adaptation).** Freeze the base weights and inject small trainable low-rank matrices into (typically) the attention/projection layers. You train and ship a few megabytes of "adapter" instead of a full model, can host many adapters over one base, and lose little quality on most tasks. The dominant PEFT method. - **QLoRA.** LoRA on top of a **quantized** (commonly 4-bit, NF4) frozen base, so the base's memory footprint collapses. This is what lets people fine-tune large models on a single consumer/prosumer GPU. You pay a little training-speed and precision cost for a large memory saving. Variants like **DoRA** push quality closer to full fine-tuning. - **PEFT** more broadly (adapters, prefix/prompt tuning) is packaged in Hugging Face's `peft` library. ## Tooling | Tool | Best at | Watch out | | :---- | :---- | :---- | | [Unsloth](https://unsloth.ai/) | Fast, cheap single-GPU tuning: custom kernels give large speedups and big VRAM cuts for LoRA/QLoRA, plus ready notebooks | Single-GPU focus (multi-GPU historically more limited) | | [Axolotl](https://github.com/axolotl-ai-cloud/axolotl) | Reproducible, scalable training jobs without writing loops: config-driven (YAML), SFT/DPO/etc. across many models and multi-GPU | Broad config surface, so a learning curve | | [Hugging Face TRL](https://huggingface.co/docs/trl) | Custom pipelines and new algorithms: reference SFT, DPO, GRPO/PPO, and reward modeling; integrates with `peft`/`accelerate` | More code and moving parts than the wrappers | | [LLaMA-Factory](https://github.com/hiyouga/LLaMA-Factory) | Broad model/method coverage with a UI | Many knobs; verify recipes for your model | | [torchtune](https://github.com/meta-pytorch/torchtune) | Native-PyTorch fine-tuning recipes | Development wound down in 2025 (maintenance-only), so treat it as a reference, not a live default | | [MLX-LM](https://github.com/ml-explore/mlx-lm) | Fine-tuning on Apple Silicon (MLX) | Mac-only, smaller-scale | | [veRL](https://github.com/verl-project/verl) | Scaled RLVR/GRPO/PPO post-training (HybridFlow); the go-to for reasoning-model RL at scale | Heavier infra (Ray-based); overkill for simple SFT/LoRA | ## GPU / precision constraints The practical question is always "will it fit in my GPU's memory?" A rough guide: - **Just running the model (inference).** Weights need about `2 bytes × parameters` at FP16, so a 7B model is ≈ 14 GB, plus some headroom for the KV-cache. Quantizing to 4-bit roughly quarters that, so the same 7B fits in ~4 GB. - **Full fine-tuning.** Much heavier: you need room for the weights *plus* the gradients, optimizer state, and activations, which together run several times the model's size. This is why full fine-tuning usually needs a multi-GPU A100/H100-class setup. - **QLoRA (the cheap path).** Loads the base model in 4-bit and trains only small adapters on top, so a mid-size model fine-tunes on a single 24 GB card. This is what puts fine-tuning within reach of one consumer GPU. Precision formats you'll run into: **BF16** (the training default), **FP8** (newer, for faster training/inference on Hopper/Blackwell), and **4-bit NF4/INT4** (QLoRA and low-memory inference). The lower you go, the more accuracy you trade for memory, so validate on your eval set.