Karpenter's disruption system uses three mechanisms to recover wasted cluster capacity: consolidation (removes underutilized/empty nodes), drift (replaces nodes whose spec diverges from NodePool/EC2NodeClass), and expiration (TTL-based drain that bypasses disruption budgets). Drift detection is event-driven and typically resolves within ~5 minutes. Disruption budgets limit concurrent node replacements per NodePool, defaulting to 10% if unset; setting nodes to '0' blocks voluntary disruption but NOT expiration. The do-not-disrupt annotation protects individual pods or nodes from voluntary disruption, but a NodePool terminationGracePeriod can override it. For stateful workloads, combining PodDisruptionBudgets, WhenEmpty consolidation policy, and AZ-aware topology constraints prevents EBS reattachment failures and unexpected pod migrations. expireAfter is unsuitable as a primary AMI rotation strategy for workloads with strict maintenance windows — drift-triggered rotation with conservative budgets is safer. Practical YAML examples and kubectl commands for verifying and pausing disruption are included.

16m read timeFrom cast.ai
Post cover image
Table of contents
Key takeawaysThe three disruption methodsWhat drift is and why it mattersControlling disruptionSafe patterns for stateful and critical workloadsScaling disruption management across fleetsFrequently Asked Questions
86 Impressions