--- title: "Day 0 Support for Qwen3.8-2.4T-A95B on vLLM" url: https://daily.dev/posts/day-0-support-for-qwen3-8-2-4t-a95b-on-vllm-phczewpdv source_url: https://vllm.ai/blog/2026-08-12-qwen3.8 type: article source: "vLLM" published: 2026-08-12T15:44:12.151Z updated: 2026-08-13T03:03:18.886Z tags: ["data-science", "ai-inference", "vllm", "mixture-of-experts", "qwen"] reading_time: 3 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Day 0 Support for Qwen3.8-2.4T-A95B on vLLM **[vLLM](https://daily.dev/sources/vllm)** · 3 min read · 0 upvotes · 0 comments ## Summary vLLM now offers day-0 support for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter sparse MoE model built on the Qwen 3.5 architecture with 512 experts and hybrid linear/full attention across 92 layers. FP8 and BF16 checkpoints are available from Qwen, with Inferact contributing NVFP4 and MXFP4 quantized weights that closely match full-precision benchmark scores on GSM8K and AIME25. Running the model requires at least two NVIDIA B300 or AMD MI355X nodes, or a single node with FP4 quantization. The release includes co-developed kernels with NVIDIA and AMD for attention, GEMMs, and MoE routing, plus deployment guidance and recommended generation parameters. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What hardware do I need to run Qwen3.8-2.4T-A95B in vLLM? Running Qwen3.8-2.4T-A95B requires at least two NVIDIA B300 or AMD MI355X nodes for the FP8/BF16 checkpoints, or a single node when using the FP4-quantized version. The model has 2.4 trillion parameters with 512 experts across a 92-layer hybrid backbone, where full attention runs every 4th layer and the rest use linear attention. _daily.dev surfaces hardware and deployment details like these for teams sizing large MoE model inference._ ### Does FP4 quantization hurt accuracy for Qwen3.8-2.4T-A95B on vLLM? No, NVFP4-quantized Qwen3.8-2.4T-A95B actually scored slightly higher than FP8 on benchmarks tested: 90.37%/91.05% versus 89.61%/90.52% on GSM8K, and 92.22%/96.67% versus 87.78%/93.33% on AIME25 pass rates. Quantization used Round-to-Nearest with activation calibration for 4-bit activations, though reproducing results requires a larger reasoning token budget. _daily.dev helps engineers weighing quantization trade-offs track real benchmark data like this._ ### Does Qwen3.8-2.4T-A95B require architecture changes to run on vLLM? No, Qwen3.8-2.4T-A95B reuses the Qwen 3.5 architecture, so it runs on vLLM from day one with no architecture changes needed. Support was enabled through optimized kernels co-developed with NVIDIA and AMD for linear attention, GQA attention, dense GEMMs, and MoE routing, built on the existing Qwen 3.5 integration. _daily.dev keeps model-serving engineers current on which new releases need no integration work._ --- Tags: [#data-science](https://daily.dev/tags/data-science), [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts), [#qwen](https://daily.dev/tags/qwen) [View this post on daily.dev](https://daily.dev/posts/day-0-support-for-qwen3-8-2-4t-a95b-on-vllm-phczewpdv)