--- title: "Notes on incidents" url: https://daily.dev/posts/notes-on-incidents-kv7cgliia source_url: https://www.seangoedecke.com/notes-on-incidents type: article source: "Programming Digest" published: 2026-08-10T03:41:07.200Z updated: 2026-08-10T03:41:35.574Z tags: ["infrastructure", "distributed-systems", "company-culture"] reading_time: 6 upvotes: 20 comments: 2 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Notes on incidents **[Programming Digest](https://daily.dev/sources/programmingdigest)** · 6 min read · 20 upvotes · 2 comments ## Summary A collection of practical observations about software incidents: most resolve on their own, most human interventions make things worse, and the first instinct should be to do nothing and observe. Effective incident resolution usually involves simple actions like disabling a feature flag, and success depends more on system knowledge than technical brilliance. The post also covers the political dynamics of incident response — resolving incidents earns goodwill from leadership, but being the go-to incident resolver is not a durable career strategy because executives can't distinguish heroic effort from obvious fixes. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Should I immediately try to fix things when I join an incident call or wait and observe first? The first action in an incident should be nothing. Most incidents resolve on their own — well over half, by one experienced engineer's estimate — and hasty interventions frequently make things worse. Classic examples include clearing a queue that contained critical billing jobs, or forcing redeploys that stress the system more than the original problem. Waiting, observing, and understanding the situation before acting is the safer default. _Engineers navigating on-call decisions track hard-won incident wisdom like this on daily.dev._ ### What kinds of actions actually resolve software incidents in practice? Incident-resolving actions are almost always simple: disabling a problematic feature flag, reverting a recent code change, or wrapping something in a cache. The complexity is not in the fix itself but in knowing which lever to pull — which is why an engineer familiar with the codebase outperforms a group of strong engineers who aren't. The typical pattern is five minutes writing the patch and an hour waiting for review, CI, and deploy. _Developers building on-call intuition for their systems find real-world incident patterns discussed on daily.dev._ ## Community discussion Top comments from developers on daily.dev. **@agustinbarrientos** · 0 upvotes > One person needs to own the change log during an incident. Five fixes at once make it hard to tell which one worked **@trevorsuna** · 0 upvotes > The “observe before acting” point is underrated. During incidents, every speculative change expands the state space and makes the timeline harder to reason about. A short stabilization window, one clear incident commander, and reversible feature-flag moves can prevent well-meaning responders from becoming a second outage. ## Similar posts on daily.dev - [Notes on incidents](https://daily.dev/posts/notes-on-incidents-noabbvg3s) · sean goedecke · 46 upvotes · 1 comments --- Tags: [#infrastructure](https://daily.dev/tags/infrastructure), [#distributed-systems](https://daily.dev/tags/distributed-systems), [#company-culture](https://daily.dev/tags/company-culture) [View this post on daily.dev](https://daily.dev/posts/notes-on-incidents-kv7cgliia)