--- title: How we saved 100 terabytes of memory by optimizing... description: Cloudflare details five successive Rust-level memory optimizations applied to the DNS cache layer of Big Pineapple, the platform behind 1.1.1.1, Gateway DNS,... canonical: https://daily.dev/posts/how-we-saved-100-terabytes-of-memory-by-optimizing-1-1-1-1-s-dns-cache-udwq79lxw twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache | daily.dev og:description: Cloudflare details five successive Rust-level memory optimizations applied to the DNS cache layer of Big Pineapple, the platform behind 1.1.1.1, Gateway DNS,... og:url: https://daily.dev/posts/how-we-saved-100-terabytes-of-memory-by-optimizing-1-1-1-1-s-dns-cache-udwq79lxw og:image: https://api.daily.dev/og/posts/UdwQ79LxW.png og:image:alt: How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache **[Cloudflare](https://daily.dev/sources/cloudflare)** · 13 min read · 68 upvotes · 6 comments ## Summary Cloudflare details five successive Rust-level memory optimizations applied to the DNS cache layer of Big Pineapple, the platform behind 1.1.1.1, Gateway DNS, DNS Firewall, and AS112. Changes include replacing Vec/String with Box<[T]>/Box to drop capacity fields, merging record lists into offset-based storage, dropping redundant owner names, boxing large enum variants, and finally storing record data in raw wire format. Together these cut per-entry memory footprint by 56% (953 to 420 bytes), reduced per-entry allocations by 58%, increased cache insert throughput by 43%, and reduced lookup latency by 19%. In production, this freed roughly 100 terabytes of memory across Cloudflare's fleet, with p99 per-instance memory dropping from 9.3 GB to 5.3 GB. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I reduce memory usage of a Rust struct that stores Vec and String fields which are never resized after creation? Replace Vec and String fields with Box<[T]> and Box once the data is immutable. Vec and String carry an extra 8-byte capacity field and often over-allocate heap space for future growth; Box<[T]> and Box drop the capacity field entirely and only allocate exactly what's needed. In Cloudflare's DNS cache, converting 8 such fields per entry saved 64 bytes per entry plus eliminated wasted heap space, totaling over 15 terabytes at 250 billion cache entries. _daily.dev surfaces engineering deep dives like this for teams tuning Rust memory layouts at scale._ ### Why does a Rust enum take up as much memory as its largest variant, and how do I fix that for a mix of small and rare large variants? Rust enums are sum types sized to fit the largest variant plus a tag, so a rarely-used large variant inflates memory for every instance. Boxing the larger variants moves their data to the heap, shrinking the enum itself to just a tag plus an 8-byte pointer for those variants. In one DNS record enum, this cut a 144-byte enum down to 24 bytes, saving 120 bytes per A/AAAA record while adding only a heap allocation for the rare 136-byte NAPTR variant. _developers optimizing Rust data structures can track patterns like this via daily.dev._ ### What memory and latency gains resulted from optimizing Cloudflare's 1.1.1.1 DNS cache entry layout? Five successive Rust memory layout changes cut per-entry cache footprint from 953 bytes to 420 bytes, a 56% reduction, and per-entry allocations dropped from 1.1 KB to 461 bytes (58% reduction). Cache insert throughput rose 43% to 893,000 entries per second, and lookup latency fell 19% to 670 ns. In production, aggregate resident memory across the fleet dropped by roughly 100 terabytes after the rollout completed on July 6, 2026. _daily.dev helps engineers follow real-world benchmarks behind infrastructure performance work like this._ ## Community take How the wider developer community reacted, aggregated from 2 discussions and 479 comments across hackernews, lobsters (as of 2026-09-03). **TL;DR:** The technical write-up itself drew little direct engineering critique; instead the discussion mostly branched into a long debate about when premature optimization is justified versus building fast and optimizing later, plus tangents on enterprise software and housing prices. **Sentiment:** 15% positive · 55% mixed · 30% skeptical **The case for** - Some argue the optimizations were relatively cheap given LLM-assisted profiling tools now make this kind of tuning low-effort. - A few note that choosing better data structures upfront (Box<[T]> vs Vec) isn't really 'premature optimization' since it costs little extra effort even early on. **The pushback** - Several commenters questioned why such 'trivial' inefficiencies (like an unused Vec capacity field) weren't caught in original design review. - Others countered that focusing on correctness, throughput, and shipping fast was the right early priority, and RAM savings are a minor concern compared to bigger architectural risks. - Some pointed out that framing memory savings as impressive is partly a promotion/narrative exercise rather than a hard engineering necessity. - One thread argued the real cost tradeoff is opportunity cost — engineering time spent optimizing vs. building revenue-generating features. **By community** - hackernews (heated): Discussion split sharply between those praising 'ship first, optimize later' philosophy and those baffled that basic data-structure inefficiencies persisted for years, with side-arguments about premature optimization, enterprise software culture, and even unrelated housing-price tangents. **Hottest debate:** Whether choosing wasteful-but-simple data structures early on was justified pragmatism or an avoidable oversight that should have been caught in initial design. **Open questions** - Would using more efficient data structures from the start have meaningfully delayed 1.1.1.1's time to market? - How much does cross-instance cache deduplication versus per-instance memory optimization actually matter for Cloudflare's overall resource usage? **Highlights** > > Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec Were there no design discussions/reviews when the system was setup to catch trivial things like this? > — [eviks on hackernews · 6 comments](https://news.ycombinator.com/item?id=49468379) > One of the "evils" of premature optimization is how much time you spend on the optimization vs. the benefit you get from it. If your goal is correctness and shipping fast and you're not memory constrained then spending time using the least amount of memory is a waste of time specifically because you want to ship fast. Another interesting thing that happens is you don't necessarily know what form your actual optimizations will need to take. Later when your systems grow you discover the suboptimal parts you hadn't optimized for. Very early on at Cloudflare I worked on part of the DNS infrastructure that took DNS records from the UI and got them in a state for actual authoritative serving. The system had been constructed anticipating Cloudflare having millions of customers with unique domains, but it had not been constructed for a single customer with a single domain with millions of records. This caused a periodic slow down in DNS record updating while the system churned on that one customer. In a different job I worked on a piece of optimization software that needed to keep track of "node" A is reachable from node "B". This had been implemented as a matrix (literally a malloced NxN matrix of ints storing 0 or 1) which worked really well for small systems. But you'd be out of memory really fast on a large project. I replaced the matrix with a hash table and all was good because the matrix was actually really sparse. > — [jgrahamc on hackernews · 1 comments](https://news.ycombinator.com/item?id=49468539) > Imagine you're an engineer at cloudflare, an 8 year old (at the time of launch of 1.1.1.1) company. The company is wildly popular and any service launched is going to have a lot of traffic and a lot of attacks right away. Any problems with it are going to embarass the company a lot. You're tasked with making a DNS caching recursive resolver that can operate at a large scale and will be run on thousands of servers each of which has a lot of GBs of ram. You are given some period of time to build this and make it production ready. How do you spend your time: * Focusing on making sure that the resolver works correctly? * Focusing on make sure that it actually provides improved DNS performance for internet users? * Handles an very large number of record requests/s? * Saves a few GB of ram per server? There are tradeoffs to consider. RAM is cheap, even at today's prices RAM is not the most expensive thing that can go wrong in such a scenario. Having the responses be slow or incorrect is a far more expensive problem. A good engineer would pick a simple data structure that has the right shape but might not be optimal in footprint to focus on correctness and response time. The few extra GBs of RAM per server can be dealt with later. When building things at scale you want to make sure it works correctly, fails correctly, and does the thing quickly before worrying about reducing resource consumption. I've never seen a project fail on Vec vs Box<[T]> memory differeneces, or even on a few GBs of RAM usage per instance. I have seen them fail on "one wierd corner case of correctness" though, and on poorly thought through failure modes. > — [sophacles on hackernews · 1 comments](https://news.ycombinator.com/item?id=49472018) > > The biggest performance gains cloudflare can provide in Web and DNS cache come from a cache hit. Using twice as much ram per cache entry makes the cache half as large, assuming your cache is bounded by ram, unless the queried, unexpired result set is less than the ram budget (which I would tend to doubt... lots of randomized queries out there; maybe I'm wrong if the cache size dropped). When you're storing billions of records, it makes sense to spend a few minutes to consider how they're used and make a good choice about how to store them. When you're getting a cache hit tons of times per second, it makes sense to consider every step and which ones don't need to happen every time. You have to consider every step while you're pursing correctness anyway, so might as well have the performance lens active too. I'm not asking for heroic optimization: I didn't ask for vectorized stuff or kernel/nic offloading or kernel bypass networking... Just you have to use some data structures, you might as well not use ones that are expensive for features you don't need; and you have to store something in your cache, you may as well store something that requires less munging on the way out. If this were a small local cache, that didn't want to use something already existing like unbound for some reason then yeah, data structures don't make a huge difference, extra marshalling doesn't make a huge difference, just don't reimplement all the CVEs that BIND had in the 90s. But if you're going to allocate 100 TB of ram, make it count. Even if you do use twice the ram but you get value from it, maybe that's fine... I've run wacky systems with bloated storage when there was a benefit. Vec doesn't give any value over a Box<[]> in this case; convenience or lazyness would be fine except that the sheer number of objects makes it worth the few minutes it takes to do something better. > — [toast0 on hackernews · 1 comments](https://news.ycombinator.com/item?id=49474333) > You're never going to get promoted with that attitude! I'm joking...but not entirely. It sounds impressive on a promo packet when you say you've saved 100 TB of RAM / $$$ through whatever technique. But it sounds a lot less impressive when you say if this system grows to this size in x years, I will have saved 100 TB, especially when no one yet knows how large the system will really be in that time or what the cost of RAM will be. I dunno, maybe if you say that x years ago, I made a decision that now is saving us 100 TB, that's kinda impressive, but you're also getting credit for it x years after you did the work. It also doesn't have the implication that it must be inherently complex/hard because some other smart person chose the other way. And there is a bias to care more about recent accomplishments. So I don't really think it'd be valued the same at all. Also, in general big tech (at least Google) prefers growing the userbase over improving efficiency. Periodically efficiency is rewarded, e.g. when RAM cost suddenly balloons or some big must-have feature has suddenly used up capacity planned for something else. You get rewarded for doing efficiency work on demand, not eagerly. I once got a $100 peer bonus for finding 100,000 cores that were essentially stranded by an accounting error in another team's migration script. > — [scottlamb on hackernews](https://news.ycombinator.com/item?id=49473254) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49468083) · 685 points · 474 comments - [lobsters](https://lobste.rs/s/p7solr/how_we_saved_100_terabytes_memory_by) · 23 points · 5 comments ## Community discussion Top comments from developers on daily.dev. **@talhasiddique7** · 1 upvotes > recommeneded - Everyone should read **@agustinbarrientos** · 1 upvotes > Thanks for publishing the per-step benchmarks, because they show which layout changes earned their complexity. Raw DNS wire storage looks less like premature optimization when lookup latency improves alongside memory use. **@jenueldev** · 1 upvotes > The allocation count is the detail I’d carry into other systems: cutting it by 58% likely explains why both inserts and lookups improved instead of memory savings trading off against speed. Publishing every intermediate benchmark also makes the complexity budget unusually easy to evaluate. **@mikhailmakeev** · 1 upvotes > The per-step benchmarks are the part I'd steal. We went after the same symptom last week and it turned out to be nowhere near our data structures: two collectors kept climbing into cgroup OOM, and /proc//smaps put the memory in glibc arena heaps, one per parser thread, with lxml fragmenting each one. Setting MALLOC_ARENA_MAX to 2 took the growth from 19.7 MB an hour to 1.8 and the kills to zero, without touching a line of code. I read smaps first now, because from the outside a leak and fragmentation look the same. --- Tags: [#rust](https://daily.dev/tags/rust), [#cloudflare](https://daily.dev/tags/cloudflare), [#dns](https://daily.dev/tags/dns) [View this post on daily.dev](https://daily.dev/posts/how-we-saved-100-terabytes-of-memory-by-optimizing-1-1-1-1-s-dns-cache-udwq79lxw) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache","url":"https://daily.dev/posts/how-we-saved-100-terabytes-of-memory-by-optimizing-1-1-1-1-s-dns-cache-udwq79lxw","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-we-saved-100-terabytes-of-memory-by-optimizing-1-1-1-1-s-dns-cache-udwq79lxw"},"datePublished":"2026-08-27T17:08:05.286Z","dateModified":"2026-09-03T13:23:16.604Z","description":"Cloudflare details five successive Rust-level memory optimizations applied to the DNS cache layer of Big Pineapple, the platform behind 1.1.1.1, Gateway DNS,...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d53e38da3a9bfd91909492035012ed11?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/d53e38da3a9bfd91909492035012ed11?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Cloudflare","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Cloudflare","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/38522e1d11354cd6b7af66f9d4316735","url":"https://daily.dev/sources/cloudflare"},"commentCount":5,"discussionUrl":"https://daily.dev/posts/how-we-saved-100-terabytes-of-memory-by-optimizing-1-1-1-1-s-dns-cache-udwq79lxw","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":68},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":5}],"keywords":"rust,cloudflare,dns","timeRequired":"PT13M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Cloudflare","item":"https://daily.dev/sources/cloudflare"},{"@type":"ListItem","position":3,"name":"How we saved 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/how-we-saved-100-terabytes-of-memory-by-optimizing-1-1-1-1-s-dns-cache-udwq79lxw","comment":[{"@type":"Comment","text":"recommeneded - veryone should read","datePublished":"2026-08-29T09:18:53.000Z","url":"https://daily.dev/posts/UdwQ79LxW#c-HAwdLpKxp","author":{"@type":"Person","name":"Talha Siddique","url":"https://daily.dev/talhasiddique7","image":"https://media.daily.dev/image/upload/s--ICFWp8AE--/f_auto/v1776346634/avatars/avatar_6b53uw8lPdfVKdrwHcxDO?_a=BAMAMiWQ0"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1}},{"@type":"Comment","text":"Thanks for publishing the per-step benchmarks, because they show which layout changes earned their complexity. Raw DNS wire storage looks less like premature optimization when lookup latency improves alongside memory use.","datePublished":"2026-09-02T01:09:12.857Z","url":"https://daily.dev/posts/UdwQ79LxW#c-YCXiu0f3y","author":{"@type":"Person","name":"Agustin Barrientos","url":"https://daily.dev/agustinbarrientos","image":"https://media.daily.dev/image/upload/s--5ayxQnqn--/f_auto/v1788281802/avatars/avatar_wQYYVe5Tbj0NJ7C7qPoa8?_a=BAMAMicg0"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1}},{"@type":"Comment","text":"The allocation count is the detail I’d carry into other systems: cutting it by 58% likely explains why both inserts and lookups improved instead of memory savings trading off against speed. Publishing every intermediate benchmark also makes the complexity budget unusually easy to evaluate.","datePublished":"2026-09-02T05:54:37.872Z","url":"https://daily.dev/posts/UdwQ79LxW#c-M86mfG6Ji","author":{"@type":"Person","name":"Jenuel Oras Ganawed","url":"https://daily.dev/jenueldev","image":"https://media.daily.dev/image/upload/s--BadmiNyj--/f_auto/v1787625077/avatars/avatar_JJtZr4Sjm?_a=BAMAMicg0"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1}},{"@type":"Comment","text":"The per-step benchmarks are the part I’d steal. We went after the same symptom last week and it turned out to be nowhere near our data structures: two collectors kept climbing into cgroup OOM, and /proc/<pid>/smaps put the memory in glibc arena heaps, one per parser thread, with lxml fragmenting each one. Setting MALLOC_ARENA_MAX to 2 took the growth from 19.7 MB an hour to 1.8 and the kills to zero, without touching a line of code. I read smaps first now, because from the outside a leak and fragmentation look the same.","datePublished":"2026-09-02T15:00:47.923Z","url":"https://daily.dev/posts/UdwQ79LxW#c-cCMelKbyE","author":{"@type":"Person","name":"Mikhail Makeev","url":"https://daily.dev/mikhailmakeev","image":"https://lh3.googleusercontent.com/a/ACg8ocIjcvDU1gmJ9-KJDeTYqY2rMdl5QxcWYtLTGSXrj8ubF4nwar_w=s96-c"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-we-saved-100-terabytes-of-memory-by-optimizing-1-1-1-1-s-dns-cache-udwq79lxw#faq","mainEntity":[{"@type":"Question","name":"How can I reduce memory usage of a Rust struct that stores Vec and String fields which are never resized after creation?","acceptedAnswer":{"@type":"Answer","text":"Replace Vec and String fields with Box<[T]> and Box once the data is immutable. Vec and String carry an extra 8-byte capacity field and often over-allocate heap space for future growth; Box<[T]> and Box drop the capacity field entirely and only allocate exactly what's needed. In Cloudflare's DNS cache, converting 8 such fields per entry saved 64 bytes per entry plus eliminated wasted heap space, totaling over 15 terabytes at 250 billion cache entries. daily.dev surfaces engineering deep dives like this for teams tuning Rust memory layouts at scale."}},{"@type":"Question","name":"Why does a Rust enum take up as much memory as its largest variant, and how do I fix that for a mix of small and rare large variants?","acceptedAnswer":{"@type":"Answer","text":"Rust enums are sum types sized to fit the largest variant plus a tag, so a rarely-used large variant inflates memory for every instance. Boxing the larger variants moves their data to the heap, shrinking the enum itself to just a tag plus an 8-byte pointer for those variants. In one DNS record enum, this cut a 144-byte enum down to 24 bytes, saving 120 bytes per A/AAAA record while adding only a heap allocation for the rare 136-byte NAPTR variant. developers optimizing Rust data structures can track patterns like this via daily.dev."}},{"@type":"Question","name":"What memory and latency gains resulted from optimizing Cloudflare's 1.1.1.1 DNS cache entry layout?","acceptedAnswer":{"@type":"Answer","text":"Five successive Rust memory layout changes cut per-entry cache footprint from 953 bytes to 420 bytes, a 56% reduction, and per-entry allocations dropped from 1.1 KB to 461 bytes (58% reduction). Cache insert throughput rose 43% to 893,000 entries per second, and lookup latency fell 19% to 670 ns. In production, aggregate resident memory across the fleet dropped by roughly 100 terabytes after the rollout completed on July 6, 2026. daily.dev helps engineers follow real-world benchmarks behind infrastructure performance work like this."}}]} ```