--- title: Qwen3.8-2.4T-A95B lands with day-0 vLLM support | daily.dev description: Qwen released Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter sparse mixture-of-experts model built on the Qwen 3.5 architecture, with 512 experts, hybrid... canonical: https://daily.dev/posts/qwen3-8-2-4t-a95b-lands-with-day-0-vllm-support-vojioedbc twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Qwen3.8-2.4T-A95B lands with day-0 vLLM support | daily.dev og:description: Qwen released Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter sparse mixture-of-experts model built on the Qwen 3.5 architecture, with 512 experts, hybrid... og:url: https://daily.dev/posts/qwen3-8-2-4t-a95b-lands-with-day-0-vllm-support-vojioedbc og:image: https://api.daily.dev/og/posts/vojIOeDbc.png og:image:alt: Qwen3.8-2.4T-A95B lands with day-0 vLLM support og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Qwen3.8-2.4T-A95B lands with day-0 vLLM support **[Collections](https://daily.dev/sources/collections)** · 4 min read · 0 upvotes · 0 comments ## Summary Qwen released Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter sparse mixture-of-experts model built on the Qwen 3.5 architecture, with 512 experts, hybrid linear/full attention, and 92 layers. vLLM shipped day-0 support including deployment guidance and recommended generation parameters. Qwen released FP8 and BF16 checkpoints, while Inferact followed with NVFP4 and MXFP4 quantized versions that reportedly land close to full-precision scores on GSM8K and AIME25. Running it requires at least two NVIDIA B300 or AMD MI355X nodes, or a single node with FP4 quantization. The release includes kernels co-developed with NVIDIA and AMD covering attention, GEMMs, and MoE routing. ## Content Alibaba dropped the open weights for Qwen3.8-2.4T-A95B (also called Qwen3.8-Max), and it's a genuinely huge release — 2.4 trillion total parameters, 95B activated per token through a mixture-of-experts setup with 512 experts (10 routed + 1 shared). Built on the Qwen3.5 architecture, it uses hybrid full/linear attention across 92 layers, supports a native context window of 262,144 tokens (extensible to over 1 million), and can output up to 128K tokens. A few things stand out. It's text-only — no vision this time, which disappointed some people waiting for a multimodal Max release, though as one HF community member put it, you can still do a lot with just the text decoder. It also ships in thinking-mode-only: there's no way to turn off reasoning, but you get configurable depth (low/medium/xhigh) and a `preserve_thinking` flag to carry reasoning context across turns. Alibaba is clearly aiming this at coding, professional/research work, and long-horizon agentic tasks, and benchmark tables put it up against Claude Opus 4.8, Claude Fable 5, and GPT-5.6 Sol. ## Getting it running The scale here (4.9TB in full precision) means running this thing is its own project, and the ecosystem moved fast: - **vLLM** shipped day-0 support, verified on both NVIDIA and AMD hardware. Inferact contributed ready-made 4-bit checkpoints for each vendor — NVFP4 (1.32 TiB, runs on one NVIDIA 8xB300 node) and MXFP4 (1.45 TiB, one AMD 8xMI355X node). No conversion or calibration needed on the user's end; just point vLLM at the checkpoint. The kernels for attention, GEMMs, and MoE routing were co-developed with NVIDIA and AMD. FP8/BF16 checkpoints also work, and running unquantized needs at least two B300 or MI355X nodes.//These NVFP4/MXFP4 weights reportedly track full-precision scores closely on GSM8K and AIME25.//DigitalOcean and Together were early testers.// - **Unsloth** shrank the model dramatically — from 4.9TB down to 397GB (a 91% reduction) using dynamic 1-bit quantization, making it feasible to run locally without a data-center rack. - **NVIDIA** got over 4,000 tokens/sec per GPU and 350+ tokens/sec per user on GB300 NVL72 in FP8, out of the box, no extra tuning — with NVFP4 expected to push that higher. It's servable via SGLang, vLLM, NVIDIA Dynamo, or a model-free NIM container, and fine-tunable through NeMo AutoModel. - **Modal** partnered with Qwen ahead of launch for day-zero support through Auto Endpoints, running on SGLang plus a custom DFlash speculator specifically tuned for this model's architecture — trained on tool-call-heavy sequences to improve token acceptance. They're offering text-only access for a limited one-month window. - **DigitalOcean** now serves it on Serverless Inference and the Inference Router, on NVIDIA HGX B300 GPUs with NVFP4 weights (via Inferact again). Pricing is $2/$6 per million input/output tokens — compare that to Claude Fable 5's $10/$50. The endpoint is OpenAI-compatible with function calling, structured outputs, and server-side tools like web search, MCP, and RAG. Measured numbers: ~1.1s time-to-first-token at 1,000 input tokens, ~16,000 tokens/sec prefill, 8-9 tokens/sec per stream, scaling to roughly 1,900 tokens/sec aggregate at 256 concurrent requests. ## How it stacks up Benchmarks are mixed in an interesting way — it reportedly leads on PaperBench and IFBench but trails Claude Fable 5 on SWE-bench Pro and HLE. Some are calling it #1 on the agentic AA index. Whether it actually beats Opus 4.6 on general use, as some enthusiastic takes on social media claimed, probably depends heavily on the task.//The same week also saw Liquid release LFM2.5-VL-3B and Cohere release North Micro Vision, two small vision-language models — so there was a lot happening in open releases all at once.// What I find genuinely notable isn't just the size — it's how fast the whole serving stack came together around it. Day-0 vLLM support, verified on two different GPU vendors, with quantized checkpoints ready to go and no calibration required on the user side. That kind of coordinated launch used to be rare for open weights this large. Whether a 2.4T-parameter MoE model is the right shape for where open models are heading, I'm less sure — but the tooling ecosystem clearly isn't waiting around to find out. ## Questions this post answers ### What hardware do I need to run Qwen3.8-2.4T-A95B? At least two NVIDIA B300 or AMD MI355X nodes are required for full-precision inference, or a single node if using FP4 quantization. The model is a 2.4-trillion-parameter sparse mixture-of-experts model with 512 experts, hybrid linear/full attention, and 92 layers, built on the Qwen 3.5 architecture. _Teams sizing GPU clusters for frontier MoE models can track hardware requirement changes like these on daily.dev._ ### Does vLLM support Qwen3.8-2.4T-A95B on release day? Yes, vLLM shipped day-0 support for Qwen3.8-2.4T-A95B, including deployment guidance and recommended generation parameters. The release also includes kernels co-developed with NVIDIA and AMD covering attention, GEMMs, and MoE routing, meaning support goes beyond just loading the model. _Engineers evaluating new model releases for production inference can follow vLLM support updates on daily.dev._ ### How much accuracy do quantized versions of Qwen3.8-2.4T-A95B lose compared to full precision? Little to none, according to benchmarks on GSM8K and AIME25 for the NVFP4 and MXFP4 quantized versions released by Inferact, which land close to full-precision scores. Qwen itself shipped FP8 and BF16 checkpoints at launch, giving multiple precision options depending on available hardware. _Anyone choosing a quantization strategy for large MoE models can compare accuracy trade-offs like these on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 3 discussions and 191 comments across hackernews, x (as of 2026-08-15). **TL;DR:** Reactions center on the surprise that a frontier-scale model can be quantized down to run on consumer laptops/desktops, with excitement about local/offline privacy and cost savings tempered by confusion over hardware requirements and skepticism about real-world performance versus closed models. Qwen/Qwen3.8-2.4T-A95B: Discussion focuses heavily on the practicalities of serving and quantizing this huge MoE model (no QAT, only bf16/fp8 at launch), with real excitement reserved for the smaller upcoming 27B variant, while some are disappointed that the open weights drop vision support and cap context length compared to the hosted Max version. Qwen3.8-2.4T: Discussion focuses mostly on confusion over naming and identity (is this the same as a smaller variant?) and skepticism about who could actually run a model this massive, rather than praise or criticism of its capabilities. **Sentiment:** 36% positive · 40% mixed · 24% skeptical **The case for** - Running it locally removes dependency on API uptime, pricing changes, and rate limits. - Benchmarks look competitive with top frontier models, trading blows with Opus-tier performance at a much lower price point. - Offline/local execution is valued for privacy and keeping data off external servers. - Large MoE models are noted to degrade less than dense models under aggressive quantization, making extreme compression more viable. - Some report it handling coding tasks (e.g., a simple pygame demo) that other local models couldn't manage. **The pushback** - Confusion and disagreement over minimum hardware (RAM vs VRAM, GPU tier) needed to actually run it well. - Lack of QAT/low-bit quantization at launch makes the model much harder and more expensive to serve than rival Kimi k3. - Doubt that anyone outside major labs or huge corporations has the hardware to run a model this size. - Reports that it runs painfully slowly on consumer GPUs compared to other models. - The open-weight release strips vision input and caps context at ~250k versus the proprietary Max version's 1M context and vision support, seen as an unnecessary capability nerf. **By community** - hackernews (mixed): Interest in the model's scale and benchmark positioning is tempered by frustration over serving difficulty, missing vision/context features, and much stronger anticipation for the smaller 27B release. - x (mixed): A mix of hype about running near-frontier capability locally, practical confusion over hardware requirements, some negative firsthand performance reports, and scattered political/trust objections about the model's origin. **Hottest debate:** Whether the compressed/quantized local model can genuinely rival closed frontier models in practice, versus being slow or degraded compared to cloud alternatives. **Open questions** - What is the actual minimum hardware (RAM/VRAM combo) needed for acceptable performance? - Whether the model can be effectively quantized below 4-bit without major capability loss, given the lack of QAT. - Is this truly a distinct model from the smaller '3.8-27B' variant or the same lineage rebranded? - How does real-world token throughput on consumer hardware compare to vendor-claimed benchmarks? - How real-world coding/agent performance will compare to benchmark tables, given Qwen's historical benchmark-to-usage gap. **Highlights** > @AlexFinn Local weights change who can run it, not what it can finish. The three tasks my agent failed on had working code and needed a credential on someone else's dashboard, and no model size fixes that > — [SSShken on x · 1 points](https://x.com/SSShken/status/2088306316003483654) > > In particular, Qwen3.8-Max is the official version based on Qwen3.8-2.4T-A95B with more features, such as vision input & non-thinking support, 1M context length by default, official built-in tools, etc. That is unfortunate, that the open weight model doesn't have vision support or the 1M context length... > — [l72 on hackernews · 2 comments](https://news.ycombinator.com/item?id=49274223) > @AlexFinn I don’t know… every time I try an open source model to do development it does stupid things. Creates tickets no one asked builds code no one asked , when I mention it, it wipes everything. I’ll stick to gpt 5.6 for now… > — [AB10578 on x](https://x.com/AB10578/status/2088389811271008411) > @AlexFinn Just tried it on the 3090 it was sooooo slow compared to muse glimmer totally not worth the extra bump in capability. Muse just flies and Qwen 3.8 was crawling. > — [alexhuth on x](https://x.com/alexhuth/status/2088375374283415761) > @OliverFinn_ @AGreatInsight @AlexFinn @grok Not really hardware that’s within reach of the Everyman is it which is sad. I still wouldn’t call these accessible local models. > — [cupcake\_trader on x](https://x.com/cupcake_trader/status/2088340926192120159) **Source threads** - [x](https://x.com/AlexFinn/status/2088305234418504099) · 0 points · 108 comments - [hackernews](https://news.ycombinator.com/item?id=49273478) · 216 points · 65 comments - [hackernews](https://news.ycombinator.com/item?id=49274950) · 79 points · 18 comments --- Tags: [#data-science](https://daily.dev/tags/data-science), [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts), [#qwen](https://daily.dev/tags/qwen) [View this post on daily.dev](https://daily.dev/posts/qwen3-8-2-4t-a95b-lands-with-day-0-vllm-support-vojioedbc) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Qwen3.8-2.4T-A95B lands with day-0 vLLM support","url":"https://daily.dev/posts/qwen3-8-2-4t-a95b-lands-with-day-0-vllm-support-vojioedbc","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/qwen3-8-2-4t-a95b-lands-with-day-0-vllm-support-vojioedbc"},"datePublished":"2026-08-12T15:44:52.546Z","dateModified":"2026-08-15T01:59:14.630Z","description":"Qwen released Qwen3.8-2.4T-A95B, a 2.4-trillion-parameter sparse mixture-of-experts model built on the Qwen 3.5 architecture, with 512 experts, hybrid...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/08f40aa592be7f8f08876d760a6f17a7?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/08f40aa592be7f8f08876d760a6f17a7?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/qwen3-8-2-4t-a95b-lands-with-day-0-vllm-support-vojioedbc","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-science,ai-inference,vllm,mixture-of-experts,qwen","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"Qwen3.8-2.4T-A95B lands with day-0 vLLM support"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/qwen3-8-2-4t-a95b-lands-with-day-0-vllm-support-vojioedbc#faq","mainEntity":[{"@type":"Question","name":"What hardware do I need to run Qwen3.8-2.4T-A95B?","acceptedAnswer":{"@type":"Answer","text":"At least two NVIDIA B300 or AMD MI355X nodes are required for full-precision inference, or a single node if using FP4 quantization. The model is a 2.4-trillion-parameter sparse mixture-of-experts model with 512 experts, hybrid linear/full attention, and 92 layers, built on the Qwen 3.5 architecture. Teams sizing GPU clusters for frontier MoE models can track hardware requirement changes like these on daily.dev."}},{"@type":"Question","name":"Does vLLM support Qwen3.8-2.4T-A95B on release day?","acceptedAnswer":{"@type":"Answer","text":"Yes, vLLM shipped day-0 support for Qwen3.8-2.4T-A95B, including deployment guidance and recommended generation parameters. The release also includes kernels co-developed with NVIDIA and AMD covering attention, GEMMs, and MoE routing, meaning support goes beyond just loading the model. Engineers evaluating new model releases for production inference can follow vLLM support updates on daily.dev."}},{"@type":"Question","name":"How much accuracy do quantized versions of Qwen3.8-2.4T-A95B lose compared to full precision?","acceptedAnswer":{"@type":"Answer","text":"Little to none, according to benchmarks on GSM8K and AIME25 for the NVFP4 and MXFP4 quantized versions released by Inferact, which land close to full-precision scores. Qwen itself shipped FP8 and BF16 checkpoints at launch, giving multiple precision options depending on available hardware. Anyone choosing a quantization strategy for large MoE models can compare accuracy trade-offs like these on daily.dev."}}]} ```