--- title: Engineering at Babylist | daily.dev description: Babylist's engineering team traces a mysterious production outage in their Rails-based AI registry consultant, where every server in a Puma fleet would... canonical: https://daily.dev/posts/engineering-at-babylist-eu2yhxdsr twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Engineering at Babylist | daily.dev og:description: Babylist's engineering team traces a mysterious production outage in their Rails-based AI registry consultant, where every server in a Puma fleet would... og:url: https://daily.dev/posts/engineering-at-babylist-eu2yhxdsr og:image: https://api.daily.dev/og/posts/EU2YHxDSR.png og:image:alt: Engineering at Babylist og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Engineering at Babylist **[Ruby Flow](https://daily.dev/sources/rubyflow)** · 6 min read · 0 upvotes · 0 comments ## Summary Babylist's engineering team traces a mysterious production outage in their Rails-based AI registry consultant, where every server in a Puma fleet would silently wedge and stop responding roughly 17-20 hours after each deploy. After two weeks chasing it as a resource leak with worker recycling and added capacity, they reproduced the hang on demand in staging by saturating a single SSE server, then pulled a thread dump showing every worker blocked in ActionController::Live::Buffer#each_chunk. The root cause was a race condition in Rails' Live streaming implementation: when a client disconnects mid-stream, Rails clears the internal queue without pushing the sentinel value that wakes the reader thread, permanently stranding the Puma worker. The team shipped a local patch, filed rails/rails#58047 and #58048 with regression tests, and Aaron Patterson merged the fix, targeted for Rails 8.2. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why do my Rails Puma workers using ActionController::Live for SSE hang forever and never recover? A race condition in ActionController::Live's Buffer means that when a client disconnects mid-stream, the abort path clears the internal queue but never pushes the sentinel value the reader thread waits on, so the reader blocks forever on SizedQueue#pop and the Puma worker is permanently lost. This happens most often on fast client disconnects and accumulates until the fleet runs out of workers. The fix is merged in rails/rails#58048 and targeted for Rails 8.2. _Track fixes like this Rails SSE race condition patch before they cause the same silent hang in production, via daily.dev._ ### How do you debug a Rails server that stops responding with no crash, no restart, and no error logs? Reproduce the failure on demand rather than waiting for it in production: saturate a low-concurrency staging environment (opening enough simultaneous long-lived connections to exhaust its worker pool) to trigger the same hang, then attach to a wedged process and capture a full thread dump. A thread dump showing every worker blocked at the same stack frame pinpoints the exact line causing the deadlock. _Developers chasing silent production hangs use write-ups like this to shortcut the same two-week detective work._ ### What is the fix for the ActionController::Live race condition where aborted SSE streams never wake the reader thread? The fix makes the abort path do what the clean shutdown path already does: empty the queue, then push the sentinel value and wake the reader, instead of only clearing the queue. The reader unblocks, sees the terminator, and unwinds normally; an extra sentinel is harmless because the reader stops at the first one it sees. It shipped as a Rails patch in pull request rails/rails#58048, merged by Aaron Patterson and targeted for Rails 8.2. _Teams running streaming Rails apps can watch daily.dev for the Rails 8.2 release that lands this SSE fix._ ## Similar posts on daily.dev - [Rails Performance Monitoring: What to Watch and Why It Slows Down](https://daily.dev/posts/rails-performance-monitoring-what-to-watch-and-why-it-slows-down-hipvfdnw1) · Last9 · 1 upvotes · 0 comments - [Warming up the Puma master before it forks](https://daily.dev/posts/warming-up-the-puma-master-before-it-forks-ubpjz1eku) · 37signals Dev · 2 upvotes · 0 comments - [Upgrade to Puma 7 and Unlock the Power of Fair Scheduled Keep-alive](https://daily.dev/posts/upgrade-to-puma-7-and-unlock-the-power-of-fair-scheduled-keep-alive-7dsx8uz2e) · Heroku · 0 upvotes · 0 comments - [From slow tests to slow production: Debugging with Stackprof](https://daily.dev/posts/from-slow-tests-to-slow-production-debugging-with-stackprof-r0br7iji6) · RUBYLAND · 0 upvotes · 0 comments - [Background AI: Using Solid Queue for Slow OpenAI API Calls](https://daily.dev/posts/background-ai-using-solid-queue-for-slow-openai-api-calls-jzbzmvjwh) · Ruby Flow · 0 upvotes · 0 comments --- Tags: [#rails](https://daily.dev/tags/rails), [#server-sent-events](https://daily.dev/tags/server-sent-events) [View this post on daily.dev](https://daily.dev/posts/engineering-at-babylist-eu2yhxdsr) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Engineering at Babylist","url":"https://daily.dev/posts/engineering-at-babylist-eu2yhxdsr","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/engineering-at-babylist-eu2yhxdsr"},"datePublished":"2026-08-13T23:59:15.690Z","dateModified":"2026-09-13T19:31:56.988Z","description":"Babylist's engineering team traces a mysterious production outage in their Rails-based AI registry consultant, where every server in a Puma fleet would...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/460bb5a58df84a00981285f25f89d182?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/460bb5a58df84a00981285f25f89d182?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Ruby Flow","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Ruby Flow","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/a21aaa8bcbbf448193a016895c90a0e1","url":"https://daily.dev/sources/rubyflow"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/engineering-at-babylist-eu2yhxdsr","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"rails,server-sent-events","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Ruby Flow","item":"https://daily.dev/sources/rubyflow"},{"@type":"ListItem","position":3,"name":"Engineering at Babylist"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/engineering-at-babylist-eu2yhxdsr#faq","mainEntity":[{"@type":"Question","name":"Why do my Rails Puma workers using ActionController::Live for SSE hang forever and never recover?","acceptedAnswer":{"@type":"Answer","text":"A race condition in ActionController::Live's Buffer means that when a client disconnects mid-stream, the abort path clears the internal queue but never pushes the sentinel value the reader thread waits on, so the reader blocks forever on SizedQueue#pop and the Puma worker is permanently lost. This happens most often on fast client disconnects and accumulates until the fleet runs out of workers. The fix is merged in rails/rails#58048 and targeted for Rails 8.2. Track fixes like this Rails SSE race condition patch before they cause the same silent hang in production, via daily.dev."}},{"@type":"Question","name":"How do you debug a Rails server that stops responding with no crash, no restart, and no error logs?","acceptedAnswer":{"@type":"Answer","text":"Reproduce the failure on demand rather than waiting for it in production: saturate a low-concurrency staging environment (opening enough simultaneous long-lived connections to exhaust its worker pool) to trigger the same hang, then attach to a wedged process and capture a full thread dump. A thread dump showing every worker blocked at the same stack frame pinpoints the exact line causing the deadlock. Developers chasing silent production hangs use write-ups like this to shortcut the same two-week detective work."}},{"@type":"Question","name":"What is the fix for the ActionController::Live race condition where aborted SSE streams never wake the reader thread?","acceptedAnswer":{"@type":"Answer","text":"The fix makes the abort path do what the clean shutdown path already does: empty the queue, then push the sentinel value and wake the reader, instead of only clearing the queue. The reader unblocks, sees the terminator, and unwinds normally; an extra sentinel is harmless because the reader stops at the first one it sees. It shipped as a Rails patch in pull request rails/rails#58048, merged by Aaron Patterson and targeted for Rails 8.2. Teams running streaming Rails apps can watch daily.dev for the Rails 8.2 release that lands this SSE fix."}}]} ```