--- title: "Discord's Global Incident Report" url: https://daily.dev/posts/discord-s-global-incident-report-au2roex5g source_url: https://www.youtube.com/watch?v=uWG4r55PmBc type: video:youtube source: "Gaurav Sen" published: 2026-05-31T21:21:42.223Z updated: 2026-05-31T21:21:57.309Z tags: ["kubernetes", "distributed-systems"] reading_time: 2 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Discord's Global Incident Report **[Gaurav Sen](https://daily.dev/sources/gkcs)** · 2 min read · 0 upvotes · 0 comments ## Summary Discord experienced a major voice outage affecting 17% of global sessions after engineers attempted to vertically scale the session service. A safety check ran longer than the Kubernetes termination grace period, causing pods to be marked dead and triggering millions of simultaneous reconnection attempts — a classic thundering herd. The surge exhausted memory in the US East region, cascaded globally, and overwhelmed the Erlang-based voice connection service, dropping 14 of 15 voice instances. Targeted and full cluster restarts both failed under the connection flood. Recovery was achieved by applying aggressive rate limits and doubling cluster size, with the outage lasting approximately 3 hours. Key takeaways: rate limiting and back-pressure are critical safeguards in distributed systems. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [GitHub has another tough day](https://daily.dev/posts/github-has-another-tough-day-dvefjjxpq) · Surfing Complexity · 18 upvotes · 4 comments - [PagerDuty's Kafka Outage Silences Alerts for Thousands of Companies](https://daily.dev/posts/pagerduty-s-kafka-outage-silences-alerts-for-thousands-of-companies-9f7j3pjbc) · InfoQ · 1 upvotes · 0 comments --- Tags: [#kubernetes](https://daily.dev/tags/kubernetes), [#distributed-systems](https://daily.dev/tags/distributed-systems) [View this post on daily.dev](https://daily.dev/posts/discord-s-global-incident-report-au2roex5g)