Untouched tests are half the proof
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Sebastian Bergmann argues that the ideal bug fix only adds a new regression test without modifying any existing tests — the new test proves the bug is fixed, while untouched tests prove nothing else broke. He validates this thesis against PHPUnit's own commit history, finding three patterns when existing tests do change: tests coupled to implementation details (structural tests that break on internal refactors), behavioural changes mislabelled as bug fixes (where the old behaviour was actually documented), and 'neutralised tests' — correct tests silenced under merge pressure with XFAIL markers rather than properly investigated. The analysis of 191 candidate commits shows that genuine rewrites of existing expectations are rare, and each case calls for a specific response: decouple structural tests, communicate behavioural changes properly, or stop silencing failures you don't have time to understand.
