Linear rebuilt the read path for its delta sync system, which lets offline clients catch up by replaying only the sync actions they missed. The previous Postgres-based path degraded in tail latency as workspaces grew, because each query combined ID range scans with array-overlap permission checks that forced Postgres to test and discard large numbers of rows. The new architecture uses Turbopuffer's inverted-index posting lists to represent sync groups and subscriptions as sorted sets of sync action IDs, letting the database intersect these sets directly instead of scanning candidates. A custom CDC pipeline streams committed Postgres sync actions into Turbopuffer with about one second p50 replication latency, and reads are split into a lightweight metadata scan (for filtering) followed by late enrichment (fetching full payloads only for surviving results) from Postgres. To handle replication lag, the freshest sync actions are always served from Postgres directly, with overlapping ranges deduplicated by ID, and Turbopuffer failures fall back to Postgres entirely. The result is flat p95/p99 latency as workspace size and sync volume grow, validated beforehand via shadow-mode comparison against the old path.
Table of contents
What a delta sync query actually does Why the Postgres read path stopped scaling Changing the shape of the query Putting Turbopuffer in the read path Predictable latency at scale Questions this post answers
Why did Linear's Postgres-based delta sync queries get slower as workspaces grew?
Each delta sync request combined a widening sync action ID range with array-overlap predicates checking user permissions and subscriptions, forcing Postgres to test and discard large numbers of candidate rows. This caused increasingly volatile tail latency even when median latency stayed healthy, and adding read replicas did not reduce the per-request intersection work since the query's fundamental shape stayed the same. Teams hitting similar Postgres scaling walls can find architecture writeups like this one on daily.dev.
How does Turbopuffer speed up large permission-based set intersections compared to Postgres array-overlap filters?
Turbopuffer maintains inverted indexes called posting lists, mapping each filterable attribute value to a sorted set of matching document IDs. Instead of scanning and testing candidate rows against permission arrays like Postgres does, Turbopuffer unions the posting lists for a user's sync groups and subscriptions, then intersects those sorted sets directly with the requested ID range, narrowing to the result without discarding rows. Engineers evaluating vector or search databases for permission-heavy queries can track comparisons like this on daily.dev.
How do you serve real-time reads from a secondary index without risking stale data from replication lag?
Serve the most recent slice of data directly from the authoritative source (Postgres) while the secondary index (Turbopuffer) covers the larger historical range behind it, with the two ranges overlapping intentionally. Deduplicate the combined result by ID, and fall back entirely to the authoritative source if the secondary index is unavailable or can't cover the requested range. Developers designing CDC pipelines with eventual consistency can follow patterns like this via daily.dev.
91.5K Impressions1 Comment