Tailscale traces a series of database corruption incidents affecting its control plane back to a rare data race in SQLite called the WAL-Reset bug, present for at least 16 years. Working with SQLite's core developers over months, they built forensic tooling including a custom virtual filesystem shim to catch the bug in production, leading to a fix in SQLite 3.51.3. The rollout also surfaced a second, unrelated bug involving stale expression indexes and floating-point rounding changes, which caused false corruption alarms and forced SQLite to withdraw version 3.52.0. Tailscale details its aggressive manual checkpointing configuration as the root cause of exposure to the otherwise extremely rare race condition.
Table of contents
Tailscale’s database architectureTrying to find the faultThe transactions that didn’t barkThe writing on the WALThe WAL-Reset bugFixed, with a false alarmParty time!Off the well-trodden pathQuestions this post answers
What is the SQLite WAL-Reset bug and which version fixes it?
The WAL-Reset bug is a rare data race between a checkpoint operation and a write transaction in SQLite that causes pages to be silently lost, corrupting the database file. It had existed in SQLite for at least 16 years but was rare enough to require deliberately triggering it in test environments. SQLite developers fixed it in version 3.51.3, after an initial fix in 3.52.0 was withdrawn due to an unrelated regression. Track SQLite release notes on daily.dev before upgrading production databases past a known corruption fix.
Why was SQLite 3.52.0 withdrawn shortly after release?
SQLite 3.52.0 was withdrawn because, alongside its WAL-Reset bug fix, it changed the rounding behavior of text-to-floating-point conversions, which caused stale expression indexes on computed columns to report false corruption via PRAGMA integrity_check. SQLite developers replaced it with 3.51.3, containing only the WAL-Reset fix, and later added a self-healing index feature in 3.53.0 to prevent the stale index issue. Developers evaluating risky point releases can follow SQLite version changes and regressions on daily.dev.
Why is manually controlling SQLite's checkpoint process risky?
Taking manual control of checkpointing and running it aggressively, as Tailscale did to support fast backups, increases exposure to rare internal race conditions like the WAL-Reset bug because it deviates from SQLite's standard, heavily-tested automatic checkpoint scheduling. Most SQLite deployments never encounter this issue since they rely on the default checkpoint behavior rather than a custom, high-frequency manual trigger. Engineers weighing non-standard database configurations can compare trade-offs and real incidents on daily.dev.