Liquid AI released Q4_0 GGUF checkpoints for its LFM2.5 model family (230M, 350M, 1.2B-Instruct, 2.6B) trained using Quantization-Aware Distillation (QAD), where a high-precision teacher model is distilled directly into a quantized student. These checkpoints recover 96.5-97.4% of BF16 accuracy while keeping the same memory footprint and throughput as native Q4_0, outperforming standard post-training quantization. On edge hardware (MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, Raspberry Pi 5), the QAD checkpoints match Q5_K_M or Q4_K_M quality at 3-33% higher decode throughput, and are compatible with llama.cpp and other GGUF runtimes.
Table of contents
Benchmark resultsSpeed and size on real edge hardwareHow to use QAD GGUFsGet Started with QAD GGUFsCitationQuestions this post answers
What is quantization-aware distillation (QAD) and how does it improve Q4_0 quantized models?
Quantization-aware distillation trains a quantized student model directly from a high-precision teacher, rather than quantizing an already-trained model after the fact. Applied to Liquid AI's LFM2.5 models (230M, 350M, 1.2B-Instruct, 2.6B), QAD Q4_0 checkpoints recover 96.5-97.4% of BF16 accuracy while keeping the same memory footprint and throughput as standard Q4_0 GGUFs. daily.dev surfaces edge-deployment quantization techniques for engineers picking model formats.
How much faster are LFM2.5 QAD Q4_0 checkpoints compared to Q5_K_M or Q4_K_M quantization on edge devices?
The 230M and 350M QAD Q4_0 checkpoints match Q5_K_M quality within evaluation variance while achieving 4-33% higher decode throughput. The 1.2B and 2.6B QAD Q4_0 checkpoints match Q4_K_M quality with 3-14% higher throughput, measured across MacBook Pro, NucBox EVO-X2, Samsung Galaxy S26 Ultra, and Raspberry Pi 5. Developers benchmarking on-device model formats can track results like these on daily.dev.
How do I run the LFM2.5 QAD Q4_0 GGUF checkpoints with llama.cpp?
Use llama-cli with the Hugging Face model reference, for example: llama-cli -hf LiquidAI/LFM2.5-350M --hf-file LFM2.5-350M-QAD-Q4_0.gguf -p "prompt text". Any runtime supporting GGUF Q4_0 artifacts can load these checkpoints, which are hosted on Hugging Face for the 230M, 350M, 1.2B-Instruct, and 2.6B LFM2.5 variants. daily.dev helps developers keep up with practical steps for deploying quantized edge models.