Neon's engineering team explains how they implemented working-set-size-aware autoscaling for Postgres. The core challenge: estimating the current working set (pages actively accessed) to keep it in memory, since cache misses over a network are far more costly than local disk reads. A standard HyperLogLog approach over-counted because it accumulated all pages since startup. Their solution replaces HLL register bits with timestamps, enabling arbitrary sliding-window queries via a single data structure. The algorithm then searches for the 'end of a plateau' in working set size as the window grows, using a 5-minute minimum offset to avoid thrashing on bursty workloads and a 1-hour cap for stable ones. Scaling uses projected growth to pre-emptively allocate memory. Results show up to 8x throughput improvement, 100% cache hit rates in batch workloads, and 30% cost savings versus fixed-size compute with equivalent performance.
