--- title: Kimi K3 Is Here: Efficient Day-0 Support on vLLM | daily.dev description: vLLM announces day-0 production support for Kimi K3, Moonshot AI's 2.8-trillion-parameter Mixture-of-Experts model with a 1M-token context window and native... canonical: https://daily.dev/posts/kimi-k3-is-here-efficient-day-0-support-on-vllm-6alao2vmf twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Kimi K3 Is Here: Efficient Day-0 Support on vLLM | daily.dev og:description: vLLM announces day-0 production support for Kimi K3, Moonshot AI's 2.8-trillion-parameter Mixture-of-Experts model with a 1M-token context window and native... og:url: https://daily.dev/posts/kimi-k3-is-here-efficient-day-0-support-on-vllm-6alao2vmf og:image: https://api.daily.dev/og/posts/6alaO2vMf.png og:image:alt: Kimi K3 Is Here: Efficient Day-0 Support on vLLM og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Kimi K3 Is Here: Efficient Day-0 Support on vLLM **[vLLM](https://daily.dev/sources/vllm)** · 22 min read · 3 upvotes · 0 comments ## Summary vLLM announces day-0 production support for Kimi K3, Moonshot AI's 2.8-trillion-parameter Mixture-of-Experts model with a 1M-token context window and native vision. Key highlights include: DSpark speculative decoding achieving 370 tok/s (3.14× speedup) on 16 NVIDIA GB300 GPUs; a redesigned hybrid prefix caching system that manages both recurrent KDA state and paged KV blocks; prefill/decode disaggregation with NIXL for large-scale deployments; sequence parallelism with custom reduce-scatter/all-gather kernels 1.7×–4.5× faster than NCCL; and multiple fused CUDA/Triton kernels for KDA decode, attention residuals, and LatentMoE tail fusion. The post covers architecture adaptations, deployment tips, benchmark results (0.976 GSM8K, 0.939 GPQA-Diamond), and a roadmap including Decode Context Parallelism and RL training support. NVIDIA Hopper/Blackwell and AMD MI355X are supported at launch. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much does DSpark speculative decoding speed up Kimi K3 inference in vLLM? DSpark speculative decoding gives a 3.14x speedup on single-user requests, raising decode throughput from 118 tok/s to 370 tok/s on 16 NVIDIA GB300 GPUs. It uses a block-diffusion backbone with a low-rank Markov head and confidence head, achieving around 4.73 accepted tokens per step on coding tasks and 2.61 on high-entropy creative writing tasks. _Teams tuning inference latency budgets can track speculative decoding gains like these via daily.dev._ ### How many GPUs are needed to serve Kimi K3 with vLLM? At least one 8x NVIDIA B300 (or GB300) node is required, and 16x B200 GPUs are also supported. AMD MI355X GPUs with ROCm are supported at launch as well. Most production deployments run multi-node setups with expert and data parallelism connected over RDMA or NVLink. _Engineers sizing hardware for massive MoE models can follow deployment guidance like this on daily.dev._ ### Is prefix caching enabled by default for Kimi K3 in vLLM? No, prefix caching is not enabled by default for Kimi K3 while its hybrid-cache design continues to evolve, so the --enable-prefix-caching flag must be passed explicitly. It supports caching over both full-attention KV and recurrent Kimi Delta Attention (KDA) state. _Anyone configuring inference flags for new model architectures can check details like this on daily.dev._ ## Similar posts on daily.dev - [Kimi K3 Performance Optimizations in vLLM: The Road to 2.8× Throughput](https://daily.dev/posts/kimi-k3-performance-optimizations-in-vllm-the-road-to-2-8-throughput-tlqrbg8ni) · vLLM · 0 upvotes · 0 comments --- Tags: [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts), [#kimi](https://daily.dev/tags/kimi) [View this post on daily.dev](https://daily.dev/posts/kimi-k3-is-here-efficient-day-0-support-on-vllm-6alao2vmf) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Kimi K3 Is Here: Efficient Day-0 Support on vLLM","url":"https://daily.dev/posts/kimi-k3-is-here-efficient-day-0-support-on-vllm-6alao2vmf","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/kimi-k3-is-here-efficient-day-0-support-on-vllm-6alao2vmf"},"datePublished":"2026-07-27T15:43:39.824Z","dateModified":"2026-09-14T06:02:17.515Z","description":"vLLM announces day-0 production support for Kimi K3, Moonshot AI's 2.8-trillion-parameter Mixture-of-Experts model with a 1M-token context window and native...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/0c7a0f643ff20d95c7a62fa0bf2fe114?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/0c7a0f643ff20d95c7a62fa0bf2fe114?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"vLLM","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"vLLM","logo":"https://media.daily.dev/image/upload/s--hTxEuls9--/f_auto/v1744613054/logos/vllm","url":"https://daily.dev/sources/vllm"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/kimi-k3-is-here-efficient-day-0-support-on-vllm-6alao2vmf","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference,vllm,mixture-of-experts,kimi","timeRequired":"PT22M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"vLLM","item":"https://daily.dev/sources/vllm"},{"@type":"ListItem","position":3,"name":"Kimi K3 Is Here: Efficient Day-0 Support on vLLM"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/kimi-k3-is-here-efficient-day-0-support-on-vllm-6alao2vmf#faq","mainEntity":[{"@type":"Question","name":"How much does DSpark speculative decoding speed up Kimi K3 inference in vLLM?","acceptedAnswer":{"@type":"Answer","text":"DSpark speculative decoding gives a 3.14x speedup on single-user requests, raising decode throughput from 118 tok/s to 370 tok/s on 16 NVIDIA GB300 GPUs. It uses a block-diffusion backbone with a low-rank Markov head and confidence head, achieving around 4.73 accepted tokens per step on coding tasks and 2.61 on high-entropy creative writing tasks. Teams tuning inference latency budgets can track speculative decoding gains like these via daily.dev."}},{"@type":"Question","name":"How many GPUs are needed to serve Kimi K3 with vLLM?","acceptedAnswer":{"@type":"Answer","text":"At least one 8x NVIDIA B300 (or GB300) node is required, and 16x B200 GPUs are also supported. AMD MI355X GPUs with ROCm are supported at launch as well. Most production deployments run multi-node setups with expert and data parallelism connected over RDMA or NVLink. Engineers sizing hardware for massive MoE models can follow deployment guidance like this on daily.dev."}},{"@type":"Question","name":"Is prefix caching enabled by default for Kimi K3 in vLLM?","acceptedAnswer":{"@type":"Answer","text":"No, prefix caching is not enabled by default for Kimi K3 while its hybrid-cache design continues to evolve, so the --enable-prefix-caching flag must be passed explicitly. It supports caching over both full-attention KV and recurrent Kimi Delta Attention (KDA) state. Anyone configuring inference flags for new model architectures can check details like this on daily.dev."}}]} ```