Alibaba released model weights for Qwen3.8-Flash-Next, a preview of the upcoming Qwen4 architecture, as a multimodal mixture-of-experts model with 176B total parameters (6B active per token) and a native 262,144-token context window extensible to 1M with YaRN. It combines Gated DeltaNet and Qwen Sparse Attention layers to cut KV cache growth and attention compute at long context lengths, delivering up to 7.6x prefill and 4.9x decoding speedups over full attention. NVIDIA provides Day 0 support via SGLang, vLLM, and TensorRT LLM, validated on GB300 NVL72, which delivers over 16K tokens/sec per GPU and 200 tokens/sec per user. The model also runs on DGX Station, DGX Spark, and RTX PRO 6000 workstations, with fine-tuning support through NeMo AutoModel and NeMo RL.
Table of contents
Architectural innovations for long-context inferenceRunning Qwen3.8-Flash-Next on NVIDIA GB300 NVL72Post-train Qwen3.8-Flash-Next and serve it with your preferred inference engineGet started with Qwen3.8-Flash-NextQuestions this post answers
What is the parameter count and architecture of Qwen3.8-Flash-Next?
Qwen3.8-Flash-Next is a multimodal mixture-of-experts model with 176B total parameters, including 51B N-gram embedding parameters, activating 6B parameters per token. It uses a hybrid architecture where three out of every four layers use Gated DeltaNet to compress historical context into a fixed-size recurrent state, while the remaining layer uses Qwen Sparse Attention for precise full-context retrieval. Track new MoE model releases like this one on daily.dev as agentic coding options evolve.
How much throughput does Qwen3.8-Flash-Next get on NVIDIA GB300 NVL72?
Running on NVIDIA GB300 NVL72, Qwen3.8-Flash-Next delivers over 16,000 tokens per second per GPU and over 200 tokens per second per user. The GB300 NVL72 integrates 72 Blackwell Ultra GPUs with a 130 TB/s NVLink domain enabling efficient all-to-all expert traffic, supporting high-throughput, low-latency agentic coding applications. Developers benchmarking inference hardware for agentic coding can follow results like this on daily.dev.
How does Qwen Sparse Attention improve long-context performance compared to full attention?
Qwen Sparse Attention aggregates sequences into micro-blocks, estimates block-level importance, and selects only the most relevant regions, avoiding the growing cost of token-level indexers. Its attention kernel achieves up to 7.6x speedup during prefill and 4.9x during decoding versus full attention, and at a 1M-token context with a 90% prefix-cache hit rate, Qwen3.8-Flash-Next reached 8.6x the prefill throughput of Qwen3.7-Plus. Engineers optimizing long-context inference can follow attention architecture advances on daily.dev.