--- title: Efficient Decode Context Parallelism with vLLM for Long... description: Decode Context Parallelism (DCP) in vLLM addresses a fundamental bottleneck in long-context LLM inference: KV cache replication across GPUs under tensor... canonical: https://daily.dev/posts/efficient-decode-context-parallelism-with-vllm-for-long-context-workloads-rae4d5gxo twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Efficient Decode Context Parallelism with vLLM for Long Context Workloads | daily.dev og:description: Decode Context Parallelism (DCP) in vLLM addresses a fundamental bottleneck in long-context LLM inference: KV cache replication across GPUs under tensor... og:url: https://daily.dev/posts/efficient-decode-context-parallelism-with-vllm-for-long-context-workloads-rae4d5gxo og:image: https://api.daily.dev/og/posts/RAe4D5gXO.png og:image:alt: Efficient Decode Context Parallelism with vLLM for Long Context Workloads og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Efficient Decode Context Parallelism with vLLM for Long Context Workloads **[vLLM](https://daily.dev/sources/vllm)** · 11 min read · 0 upvotes · 0 comments ## Summary Decode Context Parallelism (DCP) in vLLM addresses a fundamental bottleneck in long-context LLM inference: KV cache replication across GPUs under tensor parallelism. Instead of partitioning by attention head (which hits a floor with GQA and is useless for MLA), DCP shards the KV cache along the sequence dimension so each GPU stores only 1/N of every request's KV data. Benchmarks on an 8×B200 node serving Kimi K2.6 show DCP reaching 6,091 tok/s/GPU at concurrency 512 with 82% KV usage, versus a baseline TP ceiling of ~1,863 tok/s/GPU that maxes out memory at concurrency 64. The post explains the AllGather Q → Compute → AllGather+ReduceScatter communication pattern, covers MLA and GQA backend constraints, and provides usage via the `--decode-context-parallel-size` flag. Future work includes better A2A kernels, speculative decoding support, and prefill/decode disaggregation hardening. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does Decode Context Parallelism in vLLM differ from tensor parallelism for long-context inference? Tensor parallelism partitions the KV cache by attention head, hitting a hard floor once GPU count exceeds the number of KV heads — causing full cache replication. DCP instead shards the KV cache along the sequence dimension, so each GPU stores only 1/N of every request's tokens. On an 8×B200 node with Kimi K2.6, this lets DCP reach 6,091 tok/s/GPU at concurrency 512 while TP plateaus at ~1,863 tok/s/GPU and runs out of memory at concurrency 64. _Engineers scaling long-context serving beyond a single TP group track DCP developments on daily.dev._ ### What are the constraints for enabling decode_context_parallel_size in vLLM with MLA models like DeepSeek or Kimi? For MLA models (DeepSeek-V2/V3/R1, Kimi K2.6), MLA compresses all KV into a single latent vector — effectively one KV head — so TP cannot shrink it. DCP sequence-splits that latent instead. The constraints are: tensor_parallel_size >= decode_context_parallel_size, and tensor_parallel_size % decode_context_parallel_size == 0. An optional flag VLLM_DCP_Q_REPLICATE=1 can skip the query all-gather at decode time for MLA. _Developers deploying MLA-based models in production find constraint details like these on daily.dev before they hit runtime errors._ ### What is the communication pattern used by Decode Context Parallelism during the decode phase in vLLM? DCP follows an AllGather Q → Compute → AllGather + ReduceScatter rhythm. First, each GPU all-gathers the full query vector (cheap at decode since it is a single token). Then each GPU runs attention against its local KV cache slice. Finally, partial attention outputs and log-sum-exp values are combined via AllGather and merged using the online-softmax trick, with ReduceScatter returning each GPU its own head-slice of the final output. _Teams optimizing LLM serving latency stay on top of inference kernel changes like these through daily.dev._ ## Similar posts on daily.dev - [Speeding Up Variable-Length Training with Dynamic Context Parallelism and NVIDIA Megatron Core](https://daily.dev/posts/speeding-up-variable-length-training-with-dynamic-context-parallelism-and-nvidia-megatron-core-wpjwnkyfv) · NVIDIA Developer · 1 upvotes · 0 comments - [Designing distributed AI inference: Core concepts and scaling dimensions](https://daily.dev/posts/designing-distributed-ai-inference-core-concepts-and-scaling-dimensions-7bcq8ydb3) · Red Hat Developer · 0 upvotes · 0 comments - [Long-Context Inference at Scale: The Hidden Infrastructure Cost](https://daily.dev/posts/long-context-inference-at-scale-the-hidden-infrastructure-cost-apjpfce7c) · DigitalOcean Community · 0 upvotes · 0 comments - [PD Serving of Qwen3.8-2.4T](https://daily.dev/posts/pd-serving-of-qwen3-8-2-4t-apcbk4sws) · vLLM · 0 upvotes · 0 comments - [Prefill Is Compute-Bound. Decode Is Memory-Bound. Why Your GPU Shouldn’t Do Both.](https://daily.dev/posts/prefill-is-compute-bound-decode-is-memory-bound-why-your-gpu-shouldn-t-do-both--eaagsfeyw) · Towards Data Science · 0 upvotes · 0 comments --- Tags: [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/efficient-decode-context-parallelism-with-vllm-for-long-context-workloads-rae4d5gxo) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Efficient Decode Context Parallelism with vLLM for Long Context Workloads","url":"https://daily.dev/posts/efficient-decode-context-parallelism-with-vllm-for-long-context-workloads-rae4d5gxo","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/efficient-decode-context-parallelism-with-vllm-for-long-context-workloads-rae4d5gxo"},"datePublished":"2026-08-07T19:43:46.365Z","dateModified":"2026-08-07T19:44:21.827Z","description":"Decode Context Parallelism (DCP) in vLLM addresses a fundamental bottleneck in long-context LLM inference: KV cache replication across GPUs under tensor...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/c6bf83b9e49e756a38e11a9c96ba44ec?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/c6bf83b9e49e756a38e11a9c96ba44ec?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"vLLM","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"vLLM","logo":"https://media.daily.dev/image/upload/s--hTxEuls9--/f_auto/v1744613054/logos/vllm","url":"https://daily.dev/sources/vllm"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/efficient-decode-context-parallelism-with-vllm-for-long-context-workloads-rae4d5gxo","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference,vllm","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"vLLM","item":"https://daily.dev/sources/vllm"},{"@type":"ListItem","position":3,"name":"Efficient Decode Context Parallelism with vLLM for Long Context Workloads"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/efficient-decode-context-parallelism-with-vllm-for-long-context-workloads-rae4d5gxo#faq","mainEntity":[{"@type":"Question","name":"How does Decode Context Parallelism in vLLM differ from tensor parallelism for long-context inference?","acceptedAnswer":{"@type":"Answer","text":"Tensor parallelism partitions the KV cache by attention head, hitting a hard floor once GPU count exceeds the number of KV heads — causing full cache replication. DCP instead shards the KV cache along the sequence dimension, so each GPU stores only 1/N of every request's tokens. On an 8×B200 node with Kimi K2.6, this lets DCP reach 6,091 tok/s/GPU at concurrency 512 while TP plateaus at ~1,863 tok/s/GPU and runs out of memory at concurrency 64. Engineers scaling long-context serving beyond a single TP group track DCP developments on daily.dev."}},{"@type":"Question","name":"What are the constraints for enabling decode_context_parallel_size in vLLM with MLA models like DeepSeek or Kimi?","acceptedAnswer":{"@type":"Answer","text":"For MLA models (DeepSeek-V2/V3/R1, Kimi K2.6), MLA compresses all KV into a single latent vector — effectively one KV head — so TP cannot shrink it. DCP sequence-splits that latent instead. The constraints are: tensor_parallel_size >= decode_context_parallel_size, and tensor_parallel_size % decode_context_parallel_size == 0. An optional flag VLLM_DCP_Q_REPLICATE=1 can skip the query all-gather at decode time for MLA. Developers deploying MLA-based models in production find constraint details like these on daily.dev before they hit runtime errors."}},{"@type":"Question","name":"What is the communication pattern used by Decode Context Parallelism during the decode phase in vLLM?","acceptedAnswer":{"@type":"Answer","text":"DCP follows an AllGather Q → Compute → AllGather + ReduceScatter rhythm. First, each GPU all-gathers the full query vector (cheap at decode since it is a single token). Then each GPU runs attention against its local KV cache slice. Finally, partial attention outputs and log-sum-exp values are combined via AllGather and merged using the online-softmax trick, with ReduceScatter returning each GPU its own head-slice of the final output. Teams optimizing LLM serving latency stay on top of inference kernel changes like these through daily.dev."}}]} ```