--- title: It's the Memory Latency, Stupid | daily.dev description: A database engine's CPU can show 100% usage while doing much less than it could, because it's stalled waiting on memory. Running a GROUP BY count on... canonical: https://daily.dev/posts/it-s-the-memory-latency-stupid-1oknhfr71 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: It's the Memory Latency, Stupid | daily.dev og:description: A database engine's CPU can show 100% usage while doing much less than it could, because it's stalled waiting on memory. Running a GROUP BY count on... og:url: https://daily.dev/posts/it-s-the-memory-latency-stupid-1oknhfr71 og:image: https://api.daily.dev/og/posts/1oKNhfr71.png og:image:alt: It's the Memory Latency, Stupid og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # It's the Memory Latency, Stupid **[Hacker News](https://daily.dev/sources/hn)** · 11 min read · 0 upvotes · 0 comments ## Summary A database engine's CPU can show 100% usage while doing much less than it could, because it's stalled waiting on memory. Running a GROUP BY count on ClickBench's 99-million-row 'hits' dataset with ~17 million unique UserIDs, a naive HashMap approach took 3.9 seconds, with perf showing 50.1% of CPU slots stalled on memory and only 16.1% retiring operations, pulling 4.5GB from DRAM. Splitting the data into 4096 partitions by hashing the key (radix partitioning) before aggregating shrank each HashMap to fit CPU caches, cutting the query to 2.7 seconds, memory stalls to 23.5%, and DRAM traffic to 1.5GB, despite doing more total CPU operations. The Pivot database engine runs the same query in 1848ms on one core and 171ms across 16 cores. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why is my database CPU showing 100% usage but queries still feel slow? The CPU can appear fully busy while actually stalling on memory accesses rather than doing useful work. A Linux CPU counts as busy whenever a task needs CPU time, even if that task is mostly waiting for data from RAM. In one GROUP BY benchmark, perf counters showed 50.1% of CPU slots stalled on memory with only 16.1% actually retiring operations. _Developers chasing phantom 100% CPU usage can find real-world profiling breakdowns like this on daily.dev._ ### How does radix partitioning speed up GROUP BY aggregation in a database engine? Radix partitioning splits rows into thousands of smaller buckets by hashing the group key and using its top bits, then aggregates each bucket separately with its own small hash table. Using 4096 partitions on a 99-million-row GROUP BY query, this shrank each working hash table enough to fit in CPU cache, cutting runtime from 3.9 seconds to 2.7 seconds and reducing DRAM traffic from 4.5GB to 1.5GB. _Engineers tuning aggregation performance can track techniques like radix partitioning on daily.dev._ ### How much slower is a RAM access compared to an L1 cache hit on a modern CPU? A RAM access takes roughly 50 to 100 nanoseconds, while an L1 cache hit takes about 1 nanosecond, making L1 roughly 50 to 100 times faster. Since a modern CPU can execute around 4 to 8 instructions per nanosecond, that single RAM access costs the equivalent of 400 to 800 instructions worth of time. _Developers reasoning about memory latency tradeoffs in performance-critical code can follow deep dives like this on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 3 discussions and 37 comments across hackernews (as of 2026-10-31). **TL;DR:** The discussion mostly drifts from the article's point into broader reflections on mechanical sympathy, cache-aware engineering being a dying skill, and whether such low-level optimization is worth the effort, with some skepticism that the technique shown is truly novel or dramatic. **Sentiment:** 15% positive · 55% mixed · 30% skeptical **The case for** - Several developers share personal anecdotes of achieving large, business-relevant speedups through cache-aware and mechanical-sympathy-driven engineering. - Some argue cache-friendly code tends to also be easier to reason about and debug. **The pushback** - One commenter argues the described approach is essentially just a clustered/locality-sensitive hashmap and not a groundbreaking improvement. - Others note cache-friendliness isn't universally beneficial, since on uncached or low-cache embedded systems (like certain ARM chips) linked-list-style access can be equally or more appropriate. - Some pushback exists that 'premature optimization' concerns are sometimes used to shut down legitimate, benchmark-justified performance work. **By community** - hackernews (mixed): Conversation mostly veers into tangents about mechanical sympathy, game engine homogenization, and pushback against 'premature optimization' culture rather than directly engaging with the post's benchmarks. **Hottest debate:** Whether aggressive cache/memory-aware optimization is underappreciated engineering discipline or sometimes unnecessary premature optimization resisted for good reason. **Open questions** - Is the radix-partitioning approach meaningfully different from simply using a more cache-friendly (clustered) hashmap? - How would these memory-latency-focused techniques translate to constrained embedded systems with little or no cache? **Highlights** > I feel like the solution presented is just a clustered hashmap or “locality sensitive” if you’d like it. Still not orders of magnitude better. > — [drivebyhooting on hackernews](https://news.ycombinator.com/item?id=50320981) > Looking at the specs: 32 KiB of L1, 256 KiB of L2, in-order and partial dual-issue. That's the least forgiving kind of chip when it comes to performance: fast enough to outpace main memory by two orders of magnitude, yet the slightest speed bump (branch misprediction, cache miss) stalls it without any ability to reorder around it. > — [boricj on hackernews](https://news.ycombinator.com/item?id=50320757) > A lot of my poor opinion on developer opinions dates back to a couple projects where Product and Marketing were begging us for substantial performance improvements and the teams pushed back saying this was as fast as you could expect. I got 2x in the first six months on both projects, and delivered 25% per quarter on one for almost 2 years (that maths to around 8x) before I started to scrape the bottom of my barrel. It’s predominantly a way to shut down thinking, which is why it’s so often misquoted. Knuth’s version is more nuanced. Theirs is not at all. As a compromise I got really good at doing optimizations that also improved readability. Things like hoisting and better variable naming combined together can be mistaken for just code cleanup. I also discovered that keyhole optimizations are more palatable. Which is to say, move the slow bit yo a separate function with a single responsibility. Then do the optimization in a separate commit. Then the other devs don’t have to look at your changes and if they turn out to have a bug, we can always revert the change later. They almost never do, but having the option seems to grease the wheels. > — [hinkley on hackernews](https://news.ycombinator.com/item?id=50320844) > I’ve had code review pushback against reorganizing data structures to be more cache-friendly on the grounds that it was “premature optimization” even though I pre-emotively justified the decision with benchmark results indicating that the performance difference was large and business relevant, and addressed a real performance bottleneck that I observed during testing. This was on a team and at a time where Clean Code was very popular. I somewhat wonder if the problem is that, in the 2010s, many people became convinced that certain aesthetic standards were both objectively, provably correct, and more important than anything else. > — [bunderbunder on hackernews · 5 comments](https://news.ycombinator.com/item?id=50319818) > If it has a pittance of cache, then mechanical sympathy might not mean cache-friendliness. Linked lists were mechanically sympathetic on uncached systems - it cost the same to access any byte of memory, so you might as well go directly to the one you wanted. > — [immibis2 on hackernews · 2 comments](https://news.ycombinator.com/item?id=50320495) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=50318720) · 75 points · 36 comments - [hackernews](https://news.ycombinator.com/item?id=50304807) · 2 points · 1 comments - [hackernews](https://news.ycombinator.com/item?id=50291570) · 2 points · 0 comments ## Similar posts on daily.dev - [10X more data, same 4 seconds: single-query scaling in Redpanda SQL on 1TB](https://daily.dev/posts/10x-more-data-same-4-seconds-single-query-scaling-in-redpanda-sql-on-1tb-qj6rt0fu2) · Redpanda · 0 upvotes · 0 comments - [Crunching data? Put that laptop hardware to use](https://daily.dev/posts/crunching-data-put-that-laptop-hardware-to-use-1fcnht8lv) · Simple Thread · 0 upvotes · 0 comments - [No Index GroupBy Optimization](https://daily.dev/posts/no-index-groupby-optimization-mnzlu0nvk) · DoltHub Blog · 17 upvotes · 1 comments --- Tags: [#performance](https://daily.dev/tags/performance), [#database](https://daily.dev/tags/database), [#rust](https://daily.dev/tags/rust) [View this post on daily.dev](https://daily.dev/posts/it-s-the-memory-latency-stupid-1oknhfr71) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"It's the Memory Latency, Stupid","url":"https://daily.dev/posts/it-s-the-memory-latency-stupid-1oknhfr71","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/it-s-the-memory-latency-stupid-1oknhfr71"},"datePublished":"2026-10-31T20:50:43.482Z","dateModified":"2026-10-31T22:51:29.785Z","description":"A database engine's CPU can show 100% usage while doing much less than it could, because it's stalled waiting on memory. Running a GROUP BY count on...","image":"https://media.daily.dev/image/upload/s--0_ODbtD2--/f_auto/v1722860399/public/Placeholder%2008","thumbnailUrl":"https://media.daily.dev/image/upload/s--0_ODbtD2--/f_auto/v1722860399/public/Placeholder%2008","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/it-s-the-memory-latency-stupid-1oknhfr71","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"performance,database,rust","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"It's the Memory Latency, Stupid"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/it-s-the-memory-latency-stupid-1oknhfr71#faq","mainEntity":[{"@type":"Question","name":"Why is my database CPU showing 100% usage but queries still feel slow?","acceptedAnswer":{"@type":"Answer","text":"The CPU can appear fully busy while actually stalling on memory accesses rather than doing useful work. A Linux CPU counts as busy whenever a task needs CPU time, even if that task is mostly waiting for data from RAM. In one GROUP BY benchmark, perf counters showed 50.1% of CPU slots stalled on memory with only 16.1% actually retiring operations. Developers chasing phantom 100% CPU usage can find real-world profiling breakdowns like this on daily.dev."}},{"@type":"Question","name":"How does radix partitioning speed up GROUP BY aggregation in a database engine?","acceptedAnswer":{"@type":"Answer","text":"Radix partitioning splits rows into thousands of smaller buckets by hashing the group key and using its top bits, then aggregates each bucket separately with its own small hash table. Using 4096 partitions on a 99-million-row GROUP BY query, this shrank each working hash table enough to fit in CPU cache, cutting runtime from 3.9 seconds to 2.7 seconds and reducing DRAM traffic from 4.5GB to 1.5GB. Engineers tuning aggregation performance can track techniques like radix partitioning on daily.dev."}},{"@type":"Question","name":"How much slower is a RAM access compared to an L1 cache hit on a modern CPU?","acceptedAnswer":{"@type":"Answer","text":"A RAM access takes roughly 50 to 100 nanoseconds, while an L1 cache hit takes about 1 nanosecond, making L1 roughly 50 to 100 times faster. Since a modern CPU can execute around 4 to 8 instructions per nanosecond, that single RAM access costs the equivalent of 400 to 800 instructions worth of time. Developers reasoning about memory latency tradeoffs in performance-critical code can follow deep dives like this on daily.dev."}}]} ```