--- title: "Kubernetes Wasn’t Built for GPUs. Make It Behave" url: https://daily.dev/posts/kubernetes-wasn-t-built-for-gpus-make-it-behave-jreap8at3 source_url: https://cloudnativenow.com/contributed-content/kubernetes-wasnt-built-for-gpus-make-it-behave type: article source: "Cloud Native Now" published: 2026-08-12T13:20:34.816Z updated: 2026-08-12T13:22:02.347Z tags: ["kubernetes", "nvidia", "ai-inference"] reading_time: 10 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Kubernetes Wasn’t Built for GPUs. Make It Behave **[Cloud Native Now](https://daily.dev/sources/cloudnativenow)** · 10 min read · 0 upvotes · 0 comments ## Summary Kubernetes treats GPUs as whole integer resources, so pods requesting nvidia.com/gpu:1 grab an entire card even when using a fraction of VRAM or compute — and autoscaling just multiplies the waste. NVIDIA offers three sharing mechanisms: time-slicing (software round-robin, no isolation), MPS (higher throughput, still no memory isolation), and MIG (hardware-partitioned, isolated slices, Ampere and newer only). LLM serving pods should be treated as stateful because model weights take tens of seconds to minutes to reload into VRAM, so scale-to-zero and Karpenter's GPU-blind consolidation can cause painful cold starts or evict live inference. Dynamic Resource Allocation (DRA) is presented as the long-term fix — its core graduated to GA in Kubernetes v1.34 (released September 1, 2025) and shipped in OpenShift 4.21, but the fine-grained GPU-sharing sub-features remain alpha. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### why does Kubernetes waste GPU capacity when running LLM inference pods Kubernetes schedules GPUs as integer resources via the NVIDIA device plugin, so a pod requesting nvidia.com/gpu:1 claims an entire card regardless of actual need. A 7B parameter model in BF16 might use only a sixth of an 80GB H100's VRAM and around 10% utilization, but still occupies the whole GPU, and autoscaling just multiplies this waste across nodes. _Track how Kubernetes evolves its GPU resource model on daily.dev before over-provisioning your next inference cluster._ ### what is the difference between NVIDIA MIG, MPS, and time-slicing for sharing a GPU in Kubernetes Time-slicing round-robins processes onto a GPU in millisecond quanta with no memory or fault isolation, suitable for dev and low-criticality bursty work. MPS multiplexes CUDA contexts for concurrent kernel execution with higher throughput but still no memory isolation, best for trusted tenants. MIG hardware-partitions a card into up to seven isolated instances with dedicated memory and fault domains, but only works on Ampere-generation GPUs (A100/H100/H200/B-series) and later, requiring static profile planning. _Compare GPU-sharing strategies like MIG, MPS, and time-slicing on daily.dev when picking a multi-tenant serving setup._ ### when did Kubernetes Dynamic Resource Allocation (DRA) reach general availability Core Dynamic Resource Allocation graduated to GA in Kubernetes v1.34, released September 1, 2025, with the stable resource.k8s.io/v1 API group enabled by default. Red Hat shipped it to GA in OpenShift 4.21 in early 2026. However, the sharing-oriented sub-features, including consumable capacity and extended-resource mapping needed for fine-grained GPU fractioning, remained alpha in 1.34. _Follow DRA's rollout stages on daily.dev before betting a GPU platform on features still in alpha._ ## Similar posts on daily.dev - [Deploying GPU workload with Dynamic Resource Allocation](https://daily.dev/posts/deploying-gpu-workload-with-dynamic-resource-allocation-zjseztv9n) · Cast AI · 0 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#nvidia](https://daily.dev/tags/nvidia), [#ai-inference](https://daily.dev/tags/ai-inference) [View this post on daily.dev](https://daily.dev/posts/kubernetes-wasn-t-built-for-gpus-make-it-behave-jreap8at3)