--- title: Developing Nemotron 3.5 Lightning NVFP4 with QAD Using... description: NVIDIA describes how quantization-aware distillation (QAD) was used to build the Nemotron 3.5 Lightning NVFP4 checkpoint, compressing the model from 66 GB to... canonical: https://daily.dev/posts/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer-1sxxbgtvp twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer | daily.dev og:description: NVIDIA describes how quantization-aware distillation (QAD) was used to build the Nemotron 3.5 Lightning NVFP4 checkpoint, compressing the model from 66 GB to... og:url: https://daily.dev/posts/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer-1sxxbgtvp og:image: https://api.daily.dev/og/posts/1SXxbgtvP.png og:image:alt: Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer **[NVIDIA Developer](https://daily.dev/sources/nvidiadev)** · 20 min read · 0 upvotes · 0 comments ## Summary NVIDIA describes how quantization-aware distillation (QAD) was used to build the Nemotron 3.5 Lightning NVFP4 checkpoint, compressing the model from 66 GB to 22 GB while preserving accuracy and unlocking up to 4x faster throughput. The process starts with post-training quantization (PTQ) to create an aggressively quantized student model, then distills the full-precision teacher into that student using a KL divergence loss on logits. Detailed benchmark tables compare BF16, PTQ, and QAD checkpoints, showing QAD recovers most accuracy lost to aggressive quantization, especially on agentic and reasoning benchmarks. The post walks through the full training pipeline using NVIDIA Model Optimizer and Megatron-Bridge, including specific configs, sequence lengths, learning rates, and hardware setups, with code snippets for reproducing the recipe. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is quantization-aware distillation (QAD) and how does it differ from post-training quantization (PTQ)? Quantization-aware distillation trains a quantized student model to match a frozen full-precision teacher using a KL divergence loss on logits, while every forward pass runs through simulated quantization so the model adapts to quantization noise. Unlike PTQ, which just fixes weights to a low-precision grid and accepts the resulting error, QAD continues training against that error, recovering most of the accuracy PTQ alone would lose. _daily.dev surfaces deep dives like this for teams deciding how to compress models without losing accuracy._ ### How much accuracy recovery does QAD achieve compared to PTQ alone when quantizing a model to NVFP4? On an aggressively quantized intermediate checkpoint, PTQ alone reached 96.33% median accuracy recovery versus a BF16 baseline, while adding QAD lifted that to 99.72%, improving 10 of 11 benchmarks. A second checkpoint showed a similar pattern, moving from 95.84% with PTQ to 98.53% with QAD, both at an identical 21.19 GB footprint versus 65.85 GB for BF16. _engineers comparing quantization approaches track results like these on daily.dev before committing compute budget._ ### How large is the NVIDIA Nemotron 3.5 Lightning NVFP4 checkpoint compared to the full precision model? The Nemotron 3.5 Lightning NVFP4 checkpoint is compressed to 22 GB compared to 66 GB for the full precision BF16 checkpoint, achieved by quantizing many weights to 4 bits. This size reduction, combined with quantization-aware distillation, unlocks up to 4x faster throughput while preserving accuracy close to the original model. _developers weighing model size against throughput can follow coverage of releases like this on daily.dev._ ## Similar posts on daily.dev - [Creating the NVIDIA Nemotron 3 Ultra NVFP4 Checkpoint with NVIDIA Model Optimizer](https://daily.dev/posts/creating-the-nvidia-nemotron-3-ultra-nvfp4-checkpoint-with-nvidia-model-optimizer-o6e6ls0pg) · NVIDIA Developer · 1 upvotes · 0 comments - [Nemotron 3 Nano 4B: A Compact Hybrid Model for Efficient Local AI](https://daily.dev/posts/nemotron-3-nano-4b-a-compact-hybrid-model-for-efficient-local-ai-oifexpu69) · Hugging Face · 0 upvotes · 0 comments - [NVIDIA Nemotron 3.5 Lightning · Ollama Blog](https://daily.dev/posts/nvidia-nemotron-3-5-lightning-ollama-blog-uy1kba1ur) · Ollama · 3 upvotes · 0 comments - [Run Highly Efficient and Accurate Multi-Agent AI with NVIDIA Nemotron 3 Super Using vLLM](https://daily.dev/posts/run-highly-efficient-and-accurate-multi-agent-ai-with-nvidia-nemotron-3-super-using-vllm-cpvrpiesj) · vLLM · 1 upvotes · 0 comments --- Tags: [#data-science](https://daily.dev/tags/data-science) [View this post on daily.dev](https://daily.dev/posts/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer-1sxxbgtvp) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer","url":"https://daily.dev/posts/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer-1sxxbgtvp","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer-1sxxbgtvp"},"datePublished":"2026-08-17T18:15:42.108Z","dateModified":"2026-08-17T18:20:24.102Z","description":"NVIDIA describes how quantization-aware distillation (QAD) was used to build the Nemotron 3.5 Lightning NVFP4 checkpoint, compressing the model from 66 GB to...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/74f8125d53516ca840988b98d43d7d2d?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/74f8125d53516ca840988b98d43d7d2d?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"NVIDIA Developer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"NVIDIA Developer","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/86e45aab42ba48ce83103d01b1119910","url":"https://daily.dev/sources/nvidiadev"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer-1sxxbgtvp","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science","timeRequired":"PT20M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"NVIDIA Developer","item":"https://daily.dev/sources/nvidiadev"},{"@type":"ListItem","position":3,"name":"Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/developing-nemotron-3-5-lightning-nvfp4-with-qad-using-nvidia-model-optimizer-1sxxbgtvp#faq","mainEntity":[{"@type":"Question","name":"What is quantization-aware distillation (QAD) and how does it differ from post-training quantization (PTQ)?","acceptedAnswer":{"@type":"Answer","text":"Quantization-aware distillation trains a quantized student model to match a frozen full-precision teacher using a KL divergence loss on logits, while every forward pass runs through simulated quantization so the model adapts to quantization noise. Unlike PTQ, which just fixes weights to a low-precision grid and accepts the resulting error, QAD continues training against that error, recovering most of the accuracy PTQ alone would lose. daily.dev surfaces deep dives like this for teams deciding how to compress models without losing accuracy."}},{"@type":"Question","name":"How much accuracy recovery does QAD achieve compared to PTQ alone when quantizing a model to NVFP4?","acceptedAnswer":{"@type":"Answer","text":"On an aggressively quantized intermediate checkpoint, PTQ alone reached 96.33% median accuracy recovery versus a BF16 baseline, while adding QAD lifted that to 99.72%, improving 10 of 11 benchmarks. A second checkpoint showed a similar pattern, moving from 95.84% with PTQ to 98.53% with QAD, both at an identical 21.19 GB footprint versus 65.85 GB for BF16. engineers comparing quantization approaches track results like these on daily.dev before committing compute budget."}},{"@type":"Question","name":"How large is the NVIDIA Nemotron 3.5 Lightning NVFP4 checkpoint compared to the full precision model?","acceptedAnswer":{"@type":"Answer","text":"The Nemotron 3.5 Lightning NVFP4 checkpoint is compressed to 22 GB compared to 66 GB for the full precision BF16 checkpoint, achieved by quantizing many weights to 4 bits. This size reduction, combined with quantization-aware distillation, unlocks up to 4x faster throughput while preserving accuracy close to the original model. developers weighing model size against throughput can follow coverage of releases like this on daily.dev."}}]} ```