--- title: How does the Kubernetes controller manager work? | daily.dev description: Explains how the Kubernetes controller manager achieves self-healing by running many independent reconciliation loops that observe, compare, and act to close... canonical: https://daily.dev/posts/how-does-the-kubernetes-controller-manager-work--zd9mzlihm twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How does the Kubernetes controller manager work? | daily.dev og:description: Explains how the Kubernetes controller manager achieves self-healing by running many independent reconciliation loops that observe, compare, and act to close... og:url: https://daily.dev/posts/how-does-the-kubernetes-controller-manager-work--zd9mzlihm og:image: https://api.daily.dev/og/posts/ZD9mzliHm.png og:image:alt: How does the Kubernetes controller manager work? og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How does the Kubernetes controller manager work? **[LearnKube](https://daily.dev/sources/learnkube)** · 23 min read · 0 upvotes · 0 comments ## Summary Explains how the Kubernetes controller manager achieves self-healing by running many independent reconciliation loops that observe, compare, and act to close gaps between desired and actual state. Covers how Deployment, ReplicaSet, and other controllers use Shared Informers, work queues, and level-triggered reconciliation via client-go, why leader election prevents duplicate work across HA control planes, the real-world timing limits of self-healing (node failure detection at 5 minutes plus volume detach delays), and how custom controllers plus CRDs form operators like cert-manager and Argo CD. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does Kubernetes take over 10 minutes to reschedule a pod after a node fails? Recovery is deliberately slow to avoid unnecessary disruption and data corruption. The node controller waits a default of 5 minutes before marking a node dead, since a brief network hiccup shouldn't trigger mass pod evictions. If the failed node had persistent volumes attached, the attach/detach controller may wait an additional 6 minutes before treating the old attachment as invalid, since mounting a volume on two nodes risks corruption. _daily.dev surfaces practical deep dives like this for engineers designing resilient Kubernetes workloads._ ### What is the difference between Kubernetes automatic recovery and high availability? Automatic recovery means the system eventually fixes itself without human intervention, such as a ReplicaSet controller replacing a crashed pod, but it does not guarantee zero downtime. High availability means users don't experience any disruption during a failure, which requires deliberately designing workloads with multiple replicas, readiness probes, pod disruption budgets, and topology spread constraints. _Engineers weighing resilience trade-offs for their clusters can track patterns like these on daily.dev._ ### How do Kubernetes controllers avoid creating duplicate resources when multiple controller manager instances run in a high-availability control plane? Only one controller manager instance runs the actual controllers at a time, enforced through leader election. The three instances compete for a Lease object in the API server, the winner becomes active while the other two watch passively, and if the leader crashes a standby takes over. This is separate from etcd's own Raft-based leader election and the two do not need to run on the same node. _daily.dev helps engineers debugging HA control plane behavior keep up with these architectural details._ --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes) [View this post on daily.dev](https://daily.dev/posts/how-does-the-kubernetes-controller-manager-work--zd9mzlihm) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How does the Kubernetes controller manager work?","url":"https://daily.dev/posts/how-does-the-kubernetes-controller-manager-work--zd9mzlihm","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-does-the-kubernetes-controller-manager-work--zd9mzlihm"},"datePublished":"2026-08-31T11:20:03.989Z","dateModified":"2026-08-31T11:20:29.829Z","description":"Explains how the Kubernetes controller manager achieves self-healing by running many independent reconciliation loops that observe, compare, and act to close...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d7c216894cbecba8f51b480cfb3e423a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d7c216894cbecba8f51b480cfb3e423a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"LearnKube","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"LearnKube","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/90786df1b5d44abb91a090a0b6c81517","url":"https://daily.dev/sources/learnkube"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-does-the-kubernetes-controller-manager-work--zd9mzlihm","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"kubernetes","timeRequired":"PT23M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"LearnKube","item":"https://daily.dev/sources/learnkube"},{"@type":"ListItem","position":3,"name":"How does the Kubernetes controller manager work?"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-does-the-kubernetes-controller-manager-work--zd9mzlihm#faq","mainEntity":[{"@type":"Question","name":"Why does Kubernetes take over 10 minutes to reschedule a pod after a node fails?","acceptedAnswer":{"@type":"Answer","text":"Recovery is deliberately slow to avoid unnecessary disruption and data corruption. The node controller waits a default of 5 minutes before marking a node dead, since a brief network hiccup shouldn't trigger mass pod evictions. If the failed node had persistent volumes attached, the attach/detach controller may wait an additional 6 minutes before treating the old attachment as invalid, since mounting a volume on two nodes risks corruption. daily.dev surfaces practical deep dives like this for engineers designing resilient Kubernetes workloads."}},{"@type":"Question","name":"What is the difference between Kubernetes automatic recovery and high availability?","acceptedAnswer":{"@type":"Answer","text":"Automatic recovery means the system eventually fixes itself without human intervention, such as a ReplicaSet controller replacing a crashed pod, but it does not guarantee zero downtime. High availability means users don't experience any disruption during a failure, which requires deliberately designing workloads with multiple replicas, readiness probes, pod disruption budgets, and topology spread constraints. Engineers weighing resilience trade-offs for their clusters can track patterns like these on daily.dev."}},{"@type":"Question","name":"How do Kubernetes controllers avoid creating duplicate resources when multiple controller manager instances run in a high-availability control plane?","acceptedAnswer":{"@type":"Answer","text":"Only one controller manager instance runs the actual controllers at a time, enforced through leader election. The three instances compete for a Lease object in the API server, the winner becomes active while the other two watch passively, and if the leader crashes a standby takes over. This is separate from etcd's own Raft-based leader election and the two do not need to run on the same node. daily.dev helps engineers debugging HA control plane behavior keep up with these architectural details."}}]} ```