--- title: Achieving near-zero-downtime scaling in Elastic Expert... description: vLLM's Elastic EP work reduces GPU scaling downtime from roughly 100 seconds to under 2 seconds for DeepSeek V4 Pro on GB200 GPUs. The approach splits... canonical: https://daily.dev/posts/achieving-near-zero-downtime-scaling-in-elastic-expert-parallelism-1ivb50wuy twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Achieving near-zero-downtime scaling in Elastic Expert Parallelism | daily.dev og:description: vLLM's Elastic EP work reduces GPU scaling downtime from roughly 100 seconds to under 2 seconds for DeepSeek V4 Pro on GB200 GPUs. The approach splits... og:url: https://daily.dev/posts/achieving-near-zero-downtime-scaling-in-elastic-expert-parallelism-1ivb50wuy og:image: https://api.daily.dev/og/posts/1IVB50Wuy.png og:image:alt: Achieving near-zero-downtime scaling in Elastic Expert Parallelism og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Achieving near-zero-downtime scaling in Elastic Expert Parallelism **[vLLM](https://daily.dev/sources/vllm)** · 11 min read · 1 upvotes · 0 comments ## Summary vLLM's Elastic EP work reduces GPU scaling downtime from roughly 100 seconds to under 2 seconds for DeepSeek V4 Pro on GB200 GPUs. The approach splits reconfiguration into asynchronous background preparation and a short commit phase, combined with a new EP communication library called NIXL EP that expands communicator membership in place without invalidating captured CUDA graphs. This preserves existing workers' CUDA graphs across scale-up and scale-down while letting joining workers mask remote peers to independently capture their own graphs. Benchmarks show downtime dropping to 0.82s (16→20 GPUs) and 1.96s (20→16 GPUs), with the service maintaining its p95 TTFT SLO under rising load during scaling. Future work includes integrating Dynamo Snapshot for faster worker restoration and connecting Elastic EP to Dynamo/llm-d orchestration. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much downtime does scaling GPUs in an Elastic EP vLLM deployment cause with NIXL EP compared to a blocking restart? Scaling from 16 to 20 GPUs with NIXL EP and CUDA graph reuse causes only 0.82 seconds of downtime, versus 99.62 seconds for a blocking restart and 58.04 seconds with async preparation alone. Scaling back down from 20 to 16 GPUs takes 1.96 seconds, versus 56.25 seconds blocking and 44.07 seconds with async preparation alone, measured on DeepSeek V4 Pro with GB200 GPUs. _Teams scaling GPU inference clusters track techniques like this through daily.dev to avoid costly serving pauses._ ### Why does changing the number of active ranks in an EP communicator normally invalidate CUDA graphs in vLLM? Switching EP communicators changes the buffers and communication state used by EP communication kernels, and a captured CUDA graph records fixed kernel launch configurations and pointers to input/output tensors. Because the new communicator uses different memory registrations, IPC handles, and RDMA queue pairs, previously captured graphs become invalid and must be recaptured, requiring repeated warmup across all retained workers. _Engineers debugging CUDA graph invalidation during cluster resizing follow deep technical breakdowns like this on daily.dev._ ### What is NIXL EP in vLLM's Elastic Expert Parallelism system? NIXL EP is an expert-parallel communication library built for Elastic EP that adds or removes ranks in place by reusing the same NIXL agent, buffers, and connections instead of creating a new communicator instance. It lets new peers connect while existing workers keep serving, masks joining workers during their own warmup, and keeps captured EP dispatch and combine entries valid so retained workers reuse existing CUDA graphs across reconfiguration. _Developers evaluating elastic GPU scaling libraries compare approaches like NIXL EP through daily.dev._ ## Similar posts on daily.dev - [vLLM Large Scale Serving: DeepSeek @ 2.2k tok/s/H200 with Wide-EP](https://daily.dev/posts/vllm-large-scale-serving-deepseek-2-2k-tok-s-h200-with-wide-ep-c8e1bcy8k) · vLLM · 7 upvotes · 0 comments - [Scaling Large MoE Models with Wide Expert Parallelism on NVL72 Rack Scale Systems](https://daily.dev/posts/scaling-large-moe-models-with-wide-expert-parallelism-on-nvl72-rack-scale-systems-h6ejybkws) · NVIDIA Developer · 1 upvotes · 0 comments - [Driving vLLM WideEP and Large-Scale Serving Toward Maturity on Blackwell \(Part I\)](https://daily.dev/posts/driving-vllm-wideep-and-large-scale-serving-toward-maturity-on-blackwell-part-i--36r4rera9) · vLLM · 1 upvotes · 0 comments - [Accelerate LLM model loading and increase context windows with GPUDirect on Amazon FSx for Lustre and TurboQuant](https://daily.dev/posts/accelerate-llm-model-loading-and-increase-context-windows-with-gpudirect-on-amazon-fsx-for-lustre-an-u0pcfjh9q) · AWS · 0 upvotes · 0 comments --- Tags: [#ai-inference](https://daily.dev/tags/ai-inference), [#vllm](https://daily.dev/tags/vllm) [View this post on daily.dev](https://daily.dev/posts/achieving-near-zero-downtime-scaling-in-elastic-expert-parallelism-1ivb50wuy) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Achieving near-zero-downtime scaling in Elastic Expert Parallelism","url":"https://daily.dev/posts/achieving-near-zero-downtime-scaling-in-elastic-expert-parallelism-1ivb50wuy","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/achieving-near-zero-downtime-scaling-in-elastic-expert-parallelism-1ivb50wuy"},"datePublished":"2026-10-19T20:43:58.684Z","dateModified":"2026-10-19T20:44:25.987Z","description":"vLLM's Elastic EP work reduces GPU scaling downtime from roughly 100 seconds to under 2 seconds for DeepSeek V4 Pro on GB200 GPUs. The approach splits...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/fdb07af10a33b41f8adaa66a96f3f80f?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/fdb07af10a33b41f8adaa66a96f3f80f?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"vLLM","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"vLLM","logo":"https://media.daily.dev/image/upload/s--hTxEuls9--/f_auto/v1744613054/logos/vllm","url":"https://daily.dev/sources/vllm"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/achieving-near-zero-downtime-scaling-in-elastic-expert-parallelism-1ivb50wuy","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-inference,vllm","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"vLLM","item":"https://daily.dev/sources/vllm"},{"@type":"ListItem","position":3,"name":"Achieving near-zero-downtime scaling in Elastic Expert Parallelism"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/achieving-near-zero-downtime-scaling-in-elastic-expert-parallelism-1ivb50wuy#faq","mainEntity":[{"@type":"Question","name":"How much downtime does scaling GPUs in an Elastic EP vLLM deployment cause with NIXL EP compared to a blocking restart?","acceptedAnswer":{"@type":"Answer","text":"Scaling from 16 to 20 GPUs with NIXL EP and CUDA graph reuse causes only 0.82 seconds of downtime, versus 99.62 seconds for a blocking restart and 58.04 seconds with async preparation alone. Scaling back down from 20 to 16 GPUs takes 1.96 seconds, versus 56.25 seconds blocking and 44.07 seconds with async preparation alone, measured on DeepSeek V4 Pro with GB200 GPUs. Teams scaling GPU inference clusters track techniques like this through daily.dev to avoid costly serving pauses."}},{"@type":"Question","name":"Why does changing the number of active ranks in an EP communicator normally invalidate CUDA graphs in vLLM?","acceptedAnswer":{"@type":"Answer","text":"Switching EP communicators changes the buffers and communication state used by EP communication kernels, and a captured CUDA graph records fixed kernel launch configurations and pointers to input/output tensors. Because the new communicator uses different memory registrations, IPC handles, and RDMA queue pairs, previously captured graphs become invalid and must be recaptured, requiring repeated warmup across all retained workers. Engineers debugging CUDA graph invalidation during cluster resizing follow deep technical breakdowns like this on daily.dev."}},{"@type":"Question","name":"What is NIXL EP in vLLM's Elastic Expert Parallelism system?","acceptedAnswer":{"@type":"Answer","text":"NIXL EP is an expert-parallel communication library built for Elastic EP that adds or removes ranks in place by reusing the same NIXL agent, buffers, and connections instead of creating a new communicator instance. It lets new peers connect while existing workers keep serving, masks joining workers during their own warmup, and keeps captured EP dispatch and combine entries valid so retained workers reuse existing CUDA graphs across reconfiguration. Developers evaluating elastic GPU scaling libraries compare approaches like NIXL EP through daily.dev."}}]} ```