Decode Context Parallelism (DCP) in vLLM addresses a fundamental bottleneck in long-context LLM inference: KV cache replication across GPUs under tensor parallelism. Instead of partitioning by attention head (which hits a floor with GQA and is useless for MLA), DCP shards the KV cache along the sequence dimension so each GPU stores only 1/N of every request's KV data. Benchmarks on an 8×B200 node serving Kimi K2.6 show DCP reaching 6,091 tok/s/GPU at concurrency 512 with 82% KV usage, versus a baseline TP ceiling of ~1,863 tok/s/GPU that maxes out memory at concurrency 64. The post explains the AllGather Q → Compute → AllGather+ReduceScatter communication pattern, covers MLA and GQA backend constraints, and provides usage via the `--decode-context-parallel-size` flag. Future work includes better A2A kernels, speculative decoding support, and prefill/decode disaggregation hardening.
Table of contents
1. Introduction2. Performance Results3. Challenges of Serving Long Contexts4. What is DCP?5. vLLM Usage6. Future Work7. ConclusionAbout UsQuestions this post answers
How does Decode Context Parallelism in vLLM differ from tensor parallelism for long-context inference?
Tensor parallelism partitions the KV cache by attention head, hitting a hard floor once GPU count exceeds the number of KV heads — causing full cache replication. DCP instead shards the KV cache along the sequence dimension, so each GPU stores only 1/N of every request's tokens. On an 8×B200 node with Kimi K2.6, this lets DCP reach 6,091 tok/s/GPU at concurrency 512 while TP plateaus at ~1,863 tok/s/GPU and runs out of memory at concurrency 64. Engineers scaling long-context serving beyond a single TP group track DCP developments on daily.dev.
What are the constraints for enabling decode_context_parallel_size in vLLM with MLA models like DeepSeek or Kimi?
For MLA models (DeepSeek-V2/V3/R1, Kimi K2.6), MLA compresses all KV into a single latent vector — effectively one KV head — so TP cannot shrink it. DCP sequence-splits that latent instead. The constraints are: tensor_parallel_size >= decode_context_parallel_size, and tensor_parallel_size % decode_context_parallel_size == 0. An optional flag VLLM_DCP_Q_REPLICATE=1 can skip the query all-gather at decode time for MLA. Developers deploying MLA-based models in production find constraint details like these on daily.dev before they hit runtime errors.
What is the communication pattern used by Decode Context Parallelism during the decode phase in vLLM?
DCP follows an AllGather Q → Compute → AllGather + ReduceScatter rhythm. First, each GPU all-gathers the full query vector (cheap at decode since it is a single token). Then each GPU runs attention against its local KV cache slice. Finally, partial attention outputs and log-sum-exp values are combined via AllGather and merged using the online-softmax trick, with ReduceScatter returning each GPU its own head-slice of the final output. Teams optimizing LLM serving latency stay on top of inference kernel changes like these through daily.dev.