--- title: The Failover Brownout: Rethinking High Availability in... description: MySQL Group Replication's certification_info memory buffer grows unboundedly when secondary nodes lag behind the primary, because garbage collection is gated... canonical: https://daily.dev/posts/the-failover-brownout-rethinking-high-availability-in-mysql-group-replication-dqnvleduc twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: The Failover Brownout: Rethinking High Availability in MySQL Group Replication | daily.dev og:description: MySQL Group Replication's certification_info memory buffer grows unboundedly when secondary nodes lag behind the primary, because garbage collection is gated... og:url: https://daily.dev/posts/the-failover-brownout-rethinking-high-availability-in-mysql-group-replication-dqnvleduc og:image: https://api.daily.dev/og/posts/dqnVLEduC.png og:image:alt: The Failover Brownout: Rethinking High Availability in MySQL Group Replication og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The Failover Brownout: Rethinking High Availability in MySQL Group Replication **[Planet MySQL](https://daily.dev/sources/planet-mysql)** · 24 min read · 0 upvotes · 0 comments ## Summary MySQL Group Replication's certification_info memory buffer grows unboundedly when secondary nodes lag behind the primary, because garbage collection is gated on a global low watermark that only advances when all nodes have applied transactions. In Kubernetes, this causes OOMKills since memory limits are hard cgroup constraints. During failover, the new primary keeps super_read_only ON until its apply queue drains, creating a 'brownout' write outage proportional to the lag. Flow Control is reactive (1-second polling), uses a gentle PID controller, and cannot prevent queue growth under heavy write spikes. The recommended architectural law for containerized MySQL: aggressively lower group_replication_flow_control_applier_threshold to cap the apply queue, accepting write throttling during spikes rather than risking OOMKill-induced primary crashes. Increasing replica_parallel_workers helps but is limited by workload serializability. The post also compares GR to semi-sync replication and notes that Percona XtraDB Cluster (Galera) enforces harder flow control limits, making it a stronger HA option in Kubernetes. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [The Failover Brownout: Rethinking High Availability in MySQL Group Replication](https://daily.dev/posts/the-failover-brownout-rethinking-high-availability-in-mysql-group-replication-ck8ggeech) · Percona Blog · 4 upvotes · 0 comments - [From Minutes to Seconds: Uber Boosts MySQL Cluster Uptime with Consensus Architecture](https://daily.dev/posts/from-minutes-to-seconds-uber-boosts-mysql-cluster-uptime-with-consensus-architecture-1uhaxiyym) · InfoQ · 0 upvotes · 0 comments - [When failover isn’t safe: Building high-availability PostgreSQL on Kubernetes](https://daily.dev/posts/when-failover-isn-t-safe-building-high-availability-postgresql-on-kubernetes-nljmhojvb) · Datadog · 4 upvotes · 1 comments - [Jepsen: MariaDB Galera Cluster 12.1.2](https://daily.dev/posts/jepsen-mariadb-galera-cluster-12-1-2-rontun0uj) · Lobsters · 0 upvotes · 0 comments - [Quick thoughts on GitHub’s Sept 23 incident](https://daily.dev/posts/quick-thoughts-on-github-s-sept-23-incident-522cyveou) · Surfing Complexity · 16 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#mysql](https://daily.dev/tags/mysql) [View this post on daily.dev](https://daily.dev/posts/the-failover-brownout-rethinking-high-availability-in-mysql-group-replication-dqnvleduc) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"The Failover Brownout: Rethinking High Availability in MySQL Group Replication","url":"https://daily.dev/posts/the-failover-brownout-rethinking-high-availability-in-mysql-group-replication-dqnvleduc","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/the-failover-brownout-rethinking-high-availability-in-mysql-group-replication-dqnvleduc"},"datePublished":"2026-07-09T07:51:55.823Z","dateModified":"2026-07-09T07:53:39.190Z","description":"MySQL Group Replication's certification_info memory buffer grows unboundedly when secondary nodes lag behind the primary, because garbage collection is gated...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f88c334dec1de5a0bdf0b1e47cdd90f6?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f88c334dec1de5a0bdf0b1e47cdd90f6?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Planet MySQL","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Planet MySQL","logo":"https://media.daily.dev/image/upload/s--UKAtmXTq--/f_auto,q_auto/v1780213421/logos/planet-mysql?_a=BAMAMiWQ0","url":"https://daily.dev/sources/planet-mysql"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/the-failover-brownout-rethinking-high-availability-in-mysql-group-replication-dqnvleduc","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"kubernetes,mysql","timeRequired":"PT24M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Planet MySQL","item":"https://daily.dev/sources/planet-mysql"},{"@type":"ListItem","position":3,"name":"The Failover Brownout: Rethinking High Availability in MySQL Group Replication"}]} ```