--- title: Day 0 Support for Qwen3.8-2.4T-A95B on vLLM | daily.dev description: vLLM now offers day-0 support for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter sparse MoE model built on the Qwen 3.5 architecture with 512 experts and hybrid... canonical: https://daily.dev/posts/day-0-support-for-qwen3-8-2-4t-a95b-on-vllm-phczewpdv twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Day 0 Support for Qwen3.8-2.4T-A95B on vLLM | daily.dev og:description: vLLM now offers day-0 support for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter sparse MoE model built on the Qwen 3.5 architecture with 512 experts and hybrid... og:url: https://daily.dev/posts/day-0-support-for-qwen3-8-2-4t-a95b-on-vllm-phczewpdv og:image: https://api.daily.dev/og/posts/PHczewPDV.png og:image:alt: Day 0 Support for Qwen3.8-2.4T-A95B on vLLM og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Day 0 Support for Qwen3.8-2.4T-A95B on vLLM **[vLLM](https://daily.dev/sources/vllm)** · 3 min read · 0 upvotes · 0 comments ## Summary vLLM now offers day-0 support for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter sparse MoE model built on the Qwen 3.5 architecture with 512 experts and hybrid linear/full attention across 92 layers. FP8 and BF16 checkpoints are available from Qwen, with Inferact contributing NVFP4 and MXFP4 quantized weights that closely match full-precision benchmark scores on GSM8K and AIME25. Running the model requires at least two NVIDIA B300 or AMD MI355X nodes, or a single node with FP4 quantization. The release includes co-developed kernels with NVIDIA and AMD for attention, GEMMs, and MoE routing, plus deployment guidance and recommended generation parameters. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What hardware do I need to run Qwen3.8-2.4T-A95B in vLLM? Running Qwen3.8-2.4T-A95B requires at least two NVIDIA B300 or AMD MI355X nodes for the FP8/BF16 checkpoints, or a single node when using the FP4-quantized version. The model has 2.4 trillion parameters with 512 experts across a 92-layer hybrid backbone, where full attention runs every 4th layer and the rest use linear attention. _daily.dev surfaces hardware and deployment details like these for teams sizing large MoE model inference._ ### Does FP4 quantization hurt accuracy for Qwen3.8-2.4T-A95B on vLLM? No, NVFP4-quantized Qwen3.8-2.4T-A95B actually scored slightly higher than FP8 on benchmarks tested: 90.37%/91.05% versus 89.61%/90.52% on GSM8K, and 92.22%/96.67% versus 87.78%/93.33% on AIME25 pass rates. Quantization used Round-to-Nearest with activation calibration for 4-bit activations, though reproducing results requires a larger reasoning token budget. _daily.dev helps engineers weighing quantization trade-offs track real benchmark data like this._ ### Does Qwen3.8-2.4T-A95B require architecture changes to run on vLLM? No, Qwen3.8-2.4T-A95B reuses the Qwen 3.5 architecture, so it runs on vLLM from day one with no architecture changes needed. Support was enabled through optimized kernels co-developed with NVIDIA and AMD for linear attention, GQA attention, dense GEMMs, and MoE routing, built on the existing Qwen 3.5 integration. _daily.dev keeps model-serving engineers current on which new releases need no integration work._ ## Similar posts on daily.dev - [Systems Engineering Playbook: Optimizing Qwen 3.5-397B MoE on Ironwood \(TPU7x\)](https://daily.dev/posts/systems-engineering-playbook-optimizing-qwen-3-5-397b-moe-on-ironwood-tpu7x--quc6gr0fd) · Google Developers · 1 upvotes · 0 comments - [Intel Updates LLM-Scaler-vLLM With Support For More Qwen3/3.5 Models](https://daily.dev/posts/intel-updates-llm-scaler-vllm-with-support-for-more-qwen3-3-5-models-eu6bsj1e6) · Phoronix · 0 upvotes · 0 comments --- Tags: [#data-science](https://daily.dev/tags/data-science), [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts), [#qwen](https://daily.dev/tags/qwen) [View this post on daily.dev](https://daily.dev/posts/day-0-support-for-qwen3-8-2-4t-a95b-on-vllm-phczewpdv) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Day 0 Support for Qwen3.8-2.4T-A95B on vLLM","url":"https://daily.dev/posts/day-0-support-for-qwen3-8-2-4t-a95b-on-vllm-phczewpdv","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/day-0-support-for-qwen3-8-2-4t-a95b-on-vllm-phczewpdv"},"datePublished":"2026-08-12T15:44:12.151Z","dateModified":"2026-08-13T03:03:18.886Z","description":"vLLM now offers day-0 support for Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter sparse MoE model built on the Qwen 3.5 architecture with 512 experts and hybrid...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/08f40aa592be7f8f08876d760a6f17a7?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/08f40aa592be7f8f08876d760a6f17a7?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"vLLM","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"vLLM","logo":"https://media.daily.dev/image/upload/s--hTxEuls9--/f_auto/v1744613054/logos/vllm","url":"https://daily.dev/sources/vllm"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/day-0-support-for-qwen3-8-2-4t-a95b-on-vllm-phczewpdv","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science,ai-inference,vllm,mixture-of-experts,qwen","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"vLLM","item":"https://daily.dev/sources/vllm"},{"@type":"ListItem","position":3,"name":"Day 0 Support for Qwen3.8-2.4T-A95B on vLLM"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/day-0-support-for-qwen3-8-2-4t-a95b-on-vllm-phczewpdv#faq","mainEntity":[{"@type":"Question","name":"What hardware do I need to run Qwen3.8-2.4T-A95B in vLLM?","acceptedAnswer":{"@type":"Answer","text":"Running Qwen3.8-2.4T-A95B requires at least two NVIDIA B300 or AMD MI355X nodes for the FP8/BF16 checkpoints, or a single node when using the FP4-quantized version. The model has 2.4 trillion parameters with 512 experts across a 92-layer hybrid backbone, where full attention runs every 4th layer and the rest use linear attention. daily.dev surfaces hardware and deployment details like these for teams sizing large MoE model inference."}},{"@type":"Question","name":"Does FP4 quantization hurt accuracy for Qwen3.8-2.4T-A95B on vLLM?","acceptedAnswer":{"@type":"Answer","text":"No, NVFP4-quantized Qwen3.8-2.4T-A95B actually scored slightly higher than FP8 on benchmarks tested: 90.37%/91.05% versus 89.61%/90.52% on GSM8K, and 92.22%/96.67% versus 87.78%/93.33% on AIME25 pass rates. Quantization used Round-to-Nearest with activation calibration for 4-bit activations, though reproducing results requires a larger reasoning token budget. daily.dev helps engineers weighing quantization trade-offs track real benchmark data like this."}},{"@type":"Question","name":"Does Qwen3.8-2.4T-A95B require architecture changes to run on vLLM?","acceptedAnswer":{"@type":"Answer","text":"No, Qwen3.8-2.4T-A95B reuses the Qwen 3.5 architecture, so it runs on vLLM from day one with no architecture changes needed. Support was enabled through optimized kernels co-developed with NVIDIA and AMD for linear attention, GQA attention, dense GEMMs, and MoE routing, built on the existing Qwen 3.5 integration. daily.dev keeps model-serving engineers current on which new releases need no integration work."}}]} ```