Kubernetes treats GPUs as whole integer resources, so pods requesting nvidia.com/gpu:1 grab an entire card even when using a fraction of VRAM or compute — and autoscaling just multiplies the waste. NVIDIA offers three sharing mechanisms: time-slicing (software round-robin, no isolation), MPS (higher throughput, still no memory isolation), and MIG (hardware-partitioned, isolated slices, Ampere and newer only). LLM serving pods should be treated as stateful because model weights take tens of seconds to minutes to reload into VRAM, so scale-to-zero and Karpenter's GPU-blind consolidation can cause painful cold starts or evict live inference. Dynamic Resource Allocation (DRA) is presented as the long-term fix — its core graduated to GA in Kubernetes v1.34 (released September 1, 2025) and shipped in OpenShift 4.21, but the fine-grained GPU-sharing sub-features remain alpha.
Questions this post answers
why does Kubernetes waste GPU capacity when running LLM inference pods
Kubernetes schedules GPUs as integer resources via the NVIDIA device plugin, so a pod requesting nvidia.com/gpu:1 claims an entire card regardless of actual need. A 7B parameter model in BF16 might use only a sixth of an 80GB H100's VRAM and around 10% utilization, but still occupies the whole GPU, and autoscaling just multiplies this waste across nodes. Track how Kubernetes evolves its GPU resource model on daily.dev before over-provisioning your next inference cluster.
what is the difference between NVIDIA MIG, MPS, and time-slicing for sharing a GPU in Kubernetes
Time-slicing round-robins processes onto a GPU in millisecond quanta with no memory or fault isolation, suitable for dev and low-criticality bursty work. MPS multiplexes CUDA contexts for concurrent kernel execution with higher throughput but still no memory isolation, best for trusted tenants. MIG hardware-partitions a card into up to seven isolated instances with dedicated memory and fault domains, but only works on Ampere-generation GPUs (A100/H100/H200/B-series) and later, requiring static profile planning. Compare GPU-sharing strategies like MIG, MPS, and time-slicing on daily.dev when picking a multi-tenant serving setup.
when did Kubernetes Dynamic Resource Allocation (DRA) reach general availability
Core Dynamic Resource Allocation graduated to GA in Kubernetes v1.34, released September 1, 2025, with the stable resource.k8s.io/v1 API group enabled by default. Red Hat shipped it to GA in OpenShift 4.21 in early 2026. However, the sharing-oriented sub-features, including consumable capacity and extended-resource mapping needed for fine-grained GPU fractioning, remained alpha in 1.34. Follow DRA's rollout stages on daily.dev before betting a GPU platform on features still in alpha.