A deep dive into how NVIDIA quantized the 550B Nemotron 3 Ultra model to NVFP4 using NVIDIA Model Optimizer. The post explains why different layers use different precision formats (NVFP4, FP8, BF16), compares quantization scaling strategies including max scaling, MSE scaling, and the novel four-over-six approach, and shows how bits-per-element sweeps identified 5.03 BPE as the accuracy-performance sweet spot. The result is a 3.2x model size reduction (1,121 GB to 352.3 GB) with near-BF16 accuracy and up to 5.9x higher inference throughput. Practical code examples and YAML recipes for reproducing the pipeline with any Hugging Face model are included, along with a comparison of HuggingFace Transformers vs. Megatron-LM quantization paths (120 min vs. 45 min total).
