vLLM announces day-0 production support for Kimi K3, Moonshot AI's 2.8-trillion-parameter Mixture-of-Experts model with a 1M-token context window and native vision. Key highlights include: DSpark speculative decoding achieving 370 tok/s (3.14× speedup) on 16 NVIDIA GB300 GPUs; a redesigned hybrid prefix caching system that manages both recurrent KDA state and paged KV blocks; prefill/decode disaggregation with NIXL for large-scale deployments; sequence parallelism with custom reduce-scatter/all-gather kernels 1.7×–4.5× faster than NCCL; and multiple fused CUDA/Triton kernels for KDA decode, attention residuals, and LatentMoE tail fusion. The post covers architecture adaptations, deployment tips, benchmark results (0.976 GSM8K, 0.939 GPQA-Diamond), and a roadmap including Decode Context Parallelism and RL training support. NVIDIA Hopper/Blackwell and AMD MI355X are supported at launch.
