--- title: How AI workloads are reshaping Kubernetes autoscaling description: Kubernetes' scheduler and autoscaler historically acted as separate, loosely coordinated control loops, but AI accelerator workloads are exposing the gaps. A... canonical: https://daily.dev/posts/how-ai-workloads-are-reshaping-kubernetes-autoscaling-bsjesle7x twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How AI workloads are reshaping Kubernetes autoscaling | daily.dev og:description: Kubernetes' scheduler and autoscaler historically acted as separate, loosely coordinated control loops, but AI accelerator workloads are exposing the gaps. A... og:url: https://daily.dev/posts/how-ai-workloads-are-reshaping-kubernetes-autoscaling-bsjesle7x og:image: https://api.daily.dev/og/posts/BsJeSLe7X.png og:image:alt: How AI workloads are reshaping Kubernetes autoscaling og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How AI workloads are reshaping Kubernetes autoscaling **[CNCF](https://daily.dev/sources/cncf)** · 8 min read · 18 upvotes · 1 comments ## Summary Kubernetes' scheduler and autoscaler historically acted as separate, loosely coordinated control loops, but AI accelerator workloads are exposing the gaps. A pod can stay pending even when capacity exists, because the node the autoscaler would add isn't the shape the scheduler needs. Dynamic Resource Allocation lets workloads express structured topology and sharing claims instead of bare counts, and partitionable devices plus consumable capacity let several workloads share one accelerator. Workload-aware scheduling (beta, disabled by default) adds gang scheduling for the scheduler, but the autoscaler still reasons pod by pod. A provisioning request mechanism, used by Kueue, bridges this gap today, while a SIG-Autoscaling proposal sketches APIs letting the scheduler and autoscaler communicate directly. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does my GPU pod stay pending in Kubernetes even though the cluster isn't out of capacity? The pod stays pending because the autoscaler and the scheduler disagree about what counts as a usable node. The autoscaler decides whether to add a node without fully understanding the scheduler's placement constraints, such as topology, reservations, or device sharing requirements, so it may provision a node the scheduler won't actually use for that workload. _Teams debugging stuck GPU pods follow Kubernetes autoscaling changes like these on daily.dev._ ### What does Dynamic Resource Allocation (DRA) add to Kubernetes GPU scheduling that plain resource counts didn't support? Dynamic Resource Allocation lets a workload file a structured claim describing device relationships, such as an accelerator paired with the specific network interface feeding it, instead of just requesting a bare count of GPUs. The scheduler reads these claims directly, and DRA-aware autoscalers are beginning to use the same topology information when simulating capacity, though support still varies by implementation. _Engineers planning GPU topology-aware deployments track DRA's rollout progress on daily.dev._ ### Does the Kubernetes autoscaler support gang scheduling for large AI training jobs? Not fully. Workload-aware scheduling, with its Workload and PodGroup APIs, lets the scheduler hold a group's pods until enough can be placed together, but that capability is beta upstream and disabled by default, and the autoscaler itself still reasons pod by pod. As a workaround, the provisioning request mechanism, used by Kueue, lets a workload ask for a whole group's capacity at once, though atomic scale-up remains best effort and depends on provider support. _Those tuning multi-node training jobs watch gang-scheduling support mature on daily.dev._ ## Community discussion Top comments from developers on daily.dev. **@pdfopsdev** · 0 upvotes > ran into this exact gap running a training job on GKE last year, had pods scale up one at a time and just sit there waiting on siblings that never got provisioned. ended up hacking around it with a priority class hack before Kueue was stable enough to trust. ## Similar posts on daily.dev - [AI jobs are waiting while GPUs sit idle. Kubernetes has a new plan.](https://daily.dev/posts/ai-jobs-are-waiting-while-gpus-sit-idle-kubernetes-has-a-new-plan--hwksxjnw5) · The New Stack · 0 upvotes · 0 comments - [GPU Scheduling and Bin-Packing in Kubernetes: Pack More AI onto Every GPU](https://daily.dev/posts/gpu-scheduling-and-bin-packing-in-kubernetes-pack-more-ai-onto-every-gpu-6nuaafnff) · Cast AI · 0 upvotes · 0 comments - [Building an AI factory on Kubernetes](https://daily.dev/posts/building-an-ai-factory-on-kubernetes-m4brxz3ty) · CNCF · 1 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes) [View this post on daily.dev](https://daily.dev/posts/how-ai-workloads-are-reshaping-kubernetes-autoscaling-bsjesle7x) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How AI workloads are reshaping Kubernetes autoscaling","url":"https://daily.dev/posts/how-ai-workloads-are-reshaping-kubernetes-autoscaling-bsjesle7x","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-ai-workloads-are-reshaping-kubernetes-autoscaling-bsjesle7x"},"datePublished":"2026-10-19T11:02:07.180Z","dateModified":"2026-10-19T11:02:44.664Z","description":"Kubernetes' scheduler and autoscaler historically acted as separate, loosely coordinated control loops, but AI accelerator workloads are exposing the gaps. A...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/338cfa88a96c56f7df416aaa5d74d4ec?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/338cfa88a96c56f7df416aaa5d74d4ec?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"CNCF","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"CNCF","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/55536a03390741eebd96d2f9b74d2d8d","url":"https://daily.dev/sources/cncf"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/how-ai-workloads-are-reshaping-kubernetes-autoscaling-bsjesle7x","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":18},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"kubernetes","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"CNCF","item":"https://daily.dev/sources/cncf"},{"@type":"ListItem","position":3,"name":"How AI workloads are reshaping Kubernetes autoscaling"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/how-ai-workloads-are-reshaping-kubernetes-autoscaling-bsjesle7x","comment":[{"@type":"Comment","text":"ran into this exact gap running a training job on GKE last year, had pods scale up one at a time and just sit there waiting on siblings that never got provisioned. ended up hacking around it with a priority class hack before Kueue was stable enough to trust.","datePublished":"2026-10-19T18:05:26.742Z","url":"https://daily.dev/posts/BsJeSLe7X#c-wNIe0uBbK","author":{"@type":"Person","name":"PDFops","url":"https://daily.dev/pdfopsdev","image":"https://media.daily.dev/image/upload/s---8isRBKc--/f_auto/v1782922291/avatars/avatar_orjMeK8QKaaVZwGq7ScPz?_a=BAMAMicg0"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-ai-workloads-are-reshaping-kubernetes-autoscaling-bsjesle7x#faq","mainEntity":[{"@type":"Question","name":"Why does my GPU pod stay pending in Kubernetes even though the cluster isn't out of capacity?","acceptedAnswer":{"@type":"Answer","text":"The pod stays pending because the autoscaler and the scheduler disagree about what counts as a usable node. The autoscaler decides whether to add a node without fully understanding the scheduler's placement constraints, such as topology, reservations, or device sharing requirements, so it may provision a node the scheduler won't actually use for that workload. Teams debugging stuck GPU pods follow Kubernetes autoscaling changes like these on daily.dev."}},{"@type":"Question","name":"What does Dynamic Resource Allocation (DRA) add to Kubernetes GPU scheduling that plain resource counts didn't support?","acceptedAnswer":{"@type":"Answer","text":"Dynamic Resource Allocation lets a workload file a structured claim describing device relationships, such as an accelerator paired with the specific network interface feeding it, instead of just requesting a bare count of GPUs. The scheduler reads these claims directly, and DRA-aware autoscalers are beginning to use the same topology information when simulating capacity, though support still varies by implementation. Engineers planning GPU topology-aware deployments track DRA's rollout progress on daily.dev."}},{"@type":"Question","name":"Does the Kubernetes autoscaler support gang scheduling for large AI training jobs?","acceptedAnswer":{"@type":"Answer","text":"Not fully. Workload-aware scheduling, with its Workload and PodGroup APIs, lets the scheduler hold a group's pods until enough can be placed together, but that capability is beta upstream and disabled by default, and the autoscaler itself still reasons pod by pod. As a workaround, the provisioning request mechanism, used by Kueue, lets a workload ask for a whole group's capacity at once, though atomic scale-up remains best effort and depends on provider support. Those tuning multi-node training jobs watch gang-scheduling support mature on daily.dev."}}]} ```