vLLM now offers day-0 support for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter sparse MoE model built on the Qwen 3.5 architecture with 512 experts and hybrid linear/full attention across 92 layers. FP8 and BF16 checkpoints are available from Qwen, with Inferact contributing NVFP4 and MXFP4 quantized weights that closely match full-precision benchmark scores on GSM8K and AIME25. Running the model requires at least two NVIDIA B300 or AMD MI355X nodes, or a single node with FP4 quantization. The release includes co-developed kernels with NVIDIA and AMD for attention, GEMMs, and MoE routing, plus deployment guidance and recommended generation parameters.
Table of contents
TL;DRQuick startFP4 quantization: quality at lower costOptimizationsDeployment tipsAcknowledgementsQuestions this post answers
What hardware do I need to run Qwen3.8-2.4T-A95B in vLLM?
Running Qwen3.8-2.4T-A95B requires at least two NVIDIA B300 or AMD MI355X nodes for the FP8/BF16 checkpoints, or a single node when using the FP4-quantized version. The model has 2.4 trillion parameters with 512 experts across a 92-layer hybrid backbone, where full attention runs every 4th layer and the rest use linear attention. daily.dev surfaces hardware and deployment details like these for teams sizing large MoE model inference.
Does FP4 quantization hurt accuracy for Qwen3.8-2.4T-A95B on vLLM?
No, NVFP4-quantized Qwen3.8-2.4T-A95B actually scored slightly higher than FP8 on benchmarks tested: 90.37%/91.05% versus 89.61%/90.52% on GSM8K, and 92.22%/96.67% versus 87.78%/93.33% on AIME25 pass rates. Quantization used Round-to-Nearest with activation calibration for 4-bit activations, though reproducing results requires a larger reasoning token budget. daily.dev helps engineers weighing quantization trade-offs track real benchmark data like this.
Does Qwen3.8-2.4T-A95B require architecture changes to run on vLLM?
No, Qwen3.8-2.4T-A95B reuses the Qwen 3.5 architecture, so it runs on vLLM from day one with no architecture changes needed. Support was enabled through optimized kernels co-developed with NVIDIA and AMD for linear attention, GQA attention, dense GEMMs, and MoE routing, built on the existing Qwen 3.5 integration. daily.dev keeps model-serving engineers current on which new releases need no integration work.