--- title: A Preview of Production-Scale Kimi K3 Support on vLLM description: vLLM is preparing day-0 open-source serving support for Moonshot AI's Kimi K3, a 2.8-trillion-parameter model with a 1-million-token context window, hybrid... canonical: https://daily.dev/posts/a-preview-of-production-scale-kimi-k3-support-on-vllm-xzp979g1s twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: A Preview of Production-Scale Kimi K3 Support on vLLM | daily.dev og:description: vLLM is preparing day-0 open-source serving support for Moonshot AI's Kimi K3, a 2.8-trillion-parameter model with a 1-million-token context window, hybrid... og:url: https://daily.dev/posts/a-preview-of-production-scale-kimi-k3-support-on-vllm-xzp979g1s og:image: https://api.daily.dev/og/posts/XzP979G1s.png og:image:alt: A Preview of Production-Scale Kimi K3 Support on vLLM og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # A Preview of Production-Scale Kimi K3 Support on vLLM **[vLLM](https://daily.dev/sources/vllm)** · 13 min read · 1 upvotes · 0 comments ## Summary vLLM is preparing day-0 open-source serving support for Moonshot AI's Kimi K3, a 2.8-trillion-parameter model with a 1-million-token context window, hybrid KDA/full-attention, Attention Residuals, and native vision. Key engineering work includes a redesigned prefix caching system that separates physical block size from prefix-match granularity to handle KDA's recurrent state, fused kernels for KDA prefill/decode and AttnRes, MXFP4 MoE support with SiTU activation on both NVIDIA and AMD hardware, and an optimized MLA module for prefill/decode disaggregation. The post also advocates for a model-announce-then-release-weights workflow to give inference engine teams a stable integration window. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does vLLM handle prefix caching for hybrid attention models that use Kimi Delta Attention (KDA)? vLLM separates the physical KDA state-block size from the prefix-match granularity, allowing partial prefix-cache hits even when two requests share most but not all of a large physical state block. When a later request hits a partial block, the cached recurrent state is copied into a private destination via copy-on-write before the request extends it, preserving the shared prefix safely. _Engineers tuning long-context LLM serving costs can follow prefix caching techniques like this on daily.dev._ ### How many experts does Kimi K3's Mixture-of-Experts architecture activate per token? Kimi K3 uses 896 routed experts with 16 active per token, plus shared experts, on top of a 2.8-trillion-parameter total model size. This extreme sparsity avoids activating all parameters per token but places heavy demands on expert routing, dispatch, and communication in the serving stack, which vLLM addresses with optimized grouped top-k routing and MXFP4 quantized execution. _Teams evaluating sparse MoE model deployments can track serving optimizations like this via daily.dev._ ### When will Moonshot AI release the full Kimi K3 model weights? Moonshot AI announced that full Kimi K3 model weights will be released by July 27, 2026, following the model announcement made the prior week. The staggered release lets vLLM, NVIDIA, AMD, and Moonshot AI finish integration, correctness validation, and performance tuning before the weights become publicly downloadable, aiming for day-0 open-source serving support. _Developers planning to deploy a new open-weight model can watch release timelines like this on daily.dev._ ## Similar posts on daily.dev - [Kimi K3 Is Here: Efficient Day-0 Support on vLLM](https://daily.dev/posts/kimi-k3-is-here-efficient-day-0-support-on-vllm-6alao2vmf) · vLLM · 3 upvotes · 0 comments --- Tags: [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm), [#mixture-of-experts](https://daily.dev/tags/mixture-of-experts), [#kimi](https://daily.dev/tags/kimi) [View this post on daily.dev](https://daily.dev/posts/a-preview-of-production-scale-kimi-k3-support-on-vllm-xzp979g1s) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"A Preview of Production-Scale Kimi K3 Support on vLLM","url":"https://daily.dev/posts/a-preview-of-production-scale-kimi-k3-support-on-vllm-xzp979g1s","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/a-preview-of-production-scale-kimi-k3-support-on-vllm-xzp979g1s"},"datePublished":"2026-07-22T08:38:26.219Z","dateModified":"2026-09-14T06:27:04.756Z","description":"vLLM is preparing day-0 open-source serving support for Moonshot AI's Kimi K3, a 2.8-trillion-parameter model with a 1-million-token context window, hybrid...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/1bfeb4b87bfafce53015130d477e87d4?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/1bfeb4b87bfafce53015130d477e87d4?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"vLLM","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"vLLM","logo":"https://media.daily.dev/image/upload/s--hTxEuls9--/f_auto/v1744613054/logos/vllm","url":"https://daily.dev/sources/vllm"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/a-preview-of-production-scale-kimi-k3-support-on-vllm-xzp979g1s","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference,vllm,mixture-of-experts,kimi","timeRequired":"PT13M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"vLLM","item":"https://daily.dev/sources/vllm"},{"@type":"ListItem","position":3,"name":"A Preview of Production-Scale Kimi K3 Support on vLLM"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/a-preview-of-production-scale-kimi-k3-support-on-vllm-xzp979g1s#faq","mainEntity":[{"@type":"Question","name":"How does vLLM handle prefix caching for hybrid attention models that use Kimi Delta Attention (KDA)?","acceptedAnswer":{"@type":"Answer","text":"vLLM separates the physical KDA state-block size from the prefix-match granularity, allowing partial prefix-cache hits even when two requests share most but not all of a large physical state block. When a later request hits a partial block, the cached recurrent state is copied into a private destination via copy-on-write before the request extends it, preserving the shared prefix safely. Engineers tuning long-context LLM serving costs can follow prefix caching techniques like this on daily.dev."}},{"@type":"Question","name":"How many experts does Kimi K3's Mixture-of-Experts architecture activate per token?","acceptedAnswer":{"@type":"Answer","text":"Kimi K3 uses 896 routed experts with 16 active per token, plus shared experts, on top of a 2.8-trillion-parameter total model size. This extreme sparsity avoids activating all parameters per token but places heavy demands on expert routing, dispatch, and communication in the serving stack, which vLLM addresses with optimized grouped top-k routing and MXFP4 quantized execution. Teams evaluating sparse MoE model deployments can track serving optimizations like this via daily.dev."}},{"@type":"Question","name":"When will Moonshot AI release the full Kimi K3 model weights?","acceptedAnswer":{"@type":"Answer","text":"Moonshot AI announced that full Kimi K3 model weights will be released by July 27, 2026, following the model announcement made the prior week. The staggered release lets vLLM, NVIDIA, AMD, and Moonshot AI finish integration, correctness validation, and performance tuning before the weights become publicly downloadable, aiming for day-0 open-source serving support. Developers planning to deploy a new open-weight model can watch release timelines like this on daily.dev."}}]} ```