--- title: Cloudflare shaved 100TB of RAM by obsessing over Rust... description: Cloudflare detailed a five-step Rust memory optimization effort on Big Pineapple, the DNS cache layer behind 1.1.1.1, Gateway DNS, and DNS Firewall. Changes... canonical: https://daily.dev/posts/cloudflare-shaved-100tb-of-ram-by-obsessing-over-rust-struct-layouts-u9rnns8af twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Cloudflare shaved 100TB of RAM by obsessing over Rust struct layouts | daily.dev og:description: Cloudflare detailed a five-step Rust memory optimization effort on Big Pineapple, the DNS cache layer behind 1.1.1.1, Gateway DNS, and DNS Firewall. Changes... og:url: https://daily.dev/posts/cloudflare-shaved-100tb-of-ram-by-obsessing-over-rust-struct-layouts-u9rnns8af og:image: https://api.daily.dev/og/posts/U9rnns8Af.png og:image:alt: Cloudflare shaved 100TB of RAM by obsessing over Rust struct layouts og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Cloudflare shaved 100TB of RAM by obsessing over Rust struct layouts **[Trends](https://daily.dev/sources/trends)** · 2 min read · 20 upvotes · 6 comments ## Summary Cloudflare detailed a five-step Rust memory optimization effort on Big Pineapple, the DNS cache layer behind 1.1.1.1, Gateway DNS, and DNS Firewall. Changes included swapping Vec/String for Box<[T]>/Box, merging DNS record lists into offset-based structs, deduplicating owner names, boxing large enum variants, and storing record data in raw wire format. Per-entry footprint dropped from 953 to 420 bytes (56% reduction), allocations fell 58%, cache insert throughput rose 43%, lookup latency improved 19%, and P99 per-instance memory dropped from 9.3GB to 5.3GB, totaling roughly 100TB freed fleet-wide. ## Content Cloudflare just published a detailed breakdown of how they squeezed 100 terabytes of memory out of their DNS cache layer, and the engineering is worth paying attention to. The target was Big Pineapple, the platform behind 1.1.1.1, Gateway DNS, and DNS Firewall. The team ran five successive Rust-level optimizations, each one finding waste the previous pass left behind. **What they actually changed:** First, they swapped `Vec` and `String` for `Box<[T]>` and `Box`. A `Vec` carries three fields: pointer, length, and capacity. Once you're done building it, that capacity field is dead weight. Boxed slices drop it. Then they collapsed the answer/authority/additional record lists into a single offset-based struct instead of three separate allocations. After that, they noticed owner names were being stored redundantly — the cache key already has that information, so they dropped it from the entries entirely. Large enum variants were next. DNS record types vary wildly in size; NAPTR records are huge. When you put them in an enum, Rust pads every variant to match the largest one. Boxing the big variants fixes that. Finally, they switched to storing record data in raw wire format rather than parsed structs, which cut both memory and serialization overhead. **The numbers:** Per-entry footprint dropped from 953 bytes to 420 bytes — a 56% reduction. Per-entry allocations fell 58%. Cache insert throughput went from 625K to nearly 900K entries per second (up 43%). Lookup latency improved 19%. P99 per-instance memory dropped from 9.3 GB to 5.3 GB. Across Cloudflare's entire fleet, that adds up to roughly 100 terabytes freed. The interesting thing here isn't just the scale — it's the method. None of these are exotic tricks. Boxed slices, deduplication, enum boxing: these are standard Rust patterns that show up in any serious performance discussion. What's notable is that a production system at this scale had all five of these inefficiencies sitting in the same hot path, and fixing them sequentially produced compounding gains. If you're working on anything that holds large in-memory datasets, the post is worth reading carefully. The specific numbers will differ, but the audit process translates directly. ## Questions this post answers ### How did Cloudflare reduce memory usage in its DNS cache by using Rust struct layout optimizations? Cloudflare applied five sequential optimizations to its Big Pineapple DNS cache: replacing Vec and String with Box<[T]> and Box to drop the unused capacity field, merging answer/authority/additional record lists into one offset-based struct, deduplicating owner names already present in the cache key, boxing large enum variants like NAPTR records to avoid padding, and storing record data in raw wire format instead of parsed structs. _Anyone tightening memory in a Rust service can track patterns like these on daily.dev._ ### Why does an enum in Rust take up more memory than expected when one variant is much larger than the others? Rust pads every enum variant to match the size of the largest variant, so a small variant still consumes as much memory as the biggest one. In Cloudflare's DNS record enum, huge NAPTR records forced all other record type variants to pad up to that size; boxing the large variants removed this waste and was one of five changes that helped cut per-entry memory from 953 to 420 bytes. _Developers debugging bloated Rust structs can find write-ups like this via daily.dev._ ## Community take How the wider developer community reacted, aggregated from 2 discussions and 479 comments across hackernews, lobsters (as of 2026-09-03). **TL;DR:** The technical write-up itself drew little direct engineering critique; instead the discussion mostly branched into a long debate about when premature optimization is justified versus building fast and optimizing later, plus tangents on enterprise software and housing prices. **Sentiment:** 15% positive · 55% mixed · 30% skeptical **The case for** - Some argue the optimizations were relatively cheap given LLM-assisted profiling tools now make this kind of tuning low-effort. - A few note that choosing better data structures upfront (Box<[T]> vs Vec) isn't really 'premature optimization' since it costs little extra effort even early on. **The pushback** - Several commenters questioned why such 'trivial' inefficiencies (like an unused Vec capacity field) weren't caught in original design review. - Others countered that focusing on correctness, throughput, and shipping fast was the right early priority, and RAM savings are a minor concern compared to bigger architectural risks. - Some pointed out that framing memory savings as impressive is partly a promotion/narrative exercise rather than a hard engineering necessity. - One thread argued the real cost tradeoff is opportunity cost — engineering time spent optimizing vs. building revenue-generating features. **By community** - hackernews (heated): Discussion split sharply between those praising 'ship first, optimize later' philosophy and those baffled that basic data-structure inefficiencies persisted for years, with side-arguments about premature optimization, enterprise software culture, and even unrelated housing-price tangents. **Hottest debate:** Whether choosing wasteful-but-simple data structures early on was justified pragmatism or an avoidable oversight that should have been caught in initial design. **Open questions** - Would using more efficient data structures from the start have meaningfully delayed 1.1.1.1's time to market? - How much does cross-instance cache deduplication versus per-instance memory optimization actually matter for Cloudflare's overall resource usage? **Highlights** > > Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec Were there no design discussions/reviews when the system was setup to catch trivial things like this? > — [eviks on hackernews · 6 comments](https://news.ycombinator.com/item?id=49468379) > One of the "evils" of premature optimization is how much time you spend on the optimization vs. the benefit you get from it. If your goal is correctness and shipping fast and you're not memory constrained then spending time using the least amount of memory is a waste of time specifically because you want to ship fast. Another interesting thing that happens is you don't necessarily know what form your actual optimizations will need to take. Later when your systems grow you discover the suboptimal parts you hadn't optimized for. Very early on at Cloudflare I worked on part of the DNS infrastructure that took DNS records from the UI and got them in a state for actual authoritative serving. The system had been constructed anticipating Cloudflare having millions of customers with unique domains, but it had not been constructed for a single customer with a single domain with millions of records. This caused a periodic slow down in DNS record updating while the system churned on that one customer. In a different job I worked on a piece of optimization software that needed to keep track of "node" A is reachable from node "B". This had been implemented as a matrix (literally a malloced NxN matrix of ints storing 0 or 1) which worked really well for small systems. But you'd be out of memory really fast on a large project. I replaced the matrix with a hash table and all was good because the matrix was actually really sparse. > — [jgrahamc on hackernews · 1 comments](https://news.ycombinator.com/item?id=49468539) > Imagine you're an engineer at cloudflare, an 8 year old (at the time of launch of 1.1.1.1) company. The company is wildly popular and any service launched is going to have a lot of traffic and a lot of attacks right away. Any problems with it are going to embarass the company a lot. You're tasked with making a DNS caching recursive resolver that can operate at a large scale and will be run on thousands of servers each of which has a lot of GBs of ram. You are given some period of time to build this and make it production ready. How do you spend your time: * Focusing on making sure that the resolver works correctly? * Focusing on make sure that it actually provides improved DNS performance for internet users? * Handles an very large number of record requests/s? * Saves a few GB of ram per server? There are tradeoffs to consider. RAM is cheap, even at today's prices RAM is not the most expensive thing that can go wrong in such a scenario. Having the responses be slow or incorrect is a far more expensive problem. A good engineer would pick a simple data structure that has the right shape but might not be optimal in footprint to focus on correctness and response time. The few extra GBs of RAM per server can be dealt with later. When building things at scale you want to make sure it works correctly, fails correctly, and does the thing quickly before worrying about reducing resource consumption. I've never seen a project fail on Vec vs Box<[T]> memory differeneces, or even on a few GBs of RAM usage per instance. I have seen them fail on "one wierd corner case of correctness" though, and on poorly thought through failure modes. > — [sophacles on hackernews · 1 comments](https://news.ycombinator.com/item?id=49472018) > > The biggest performance gains cloudflare can provide in Web and DNS cache come from a cache hit. Using twice as much ram per cache entry makes the cache half as large, assuming your cache is bounded by ram, unless the queried, unexpired result set is less than the ram budget (which I would tend to doubt... lots of randomized queries out there; maybe I'm wrong if the cache size dropped). When you're storing billions of records, it makes sense to spend a few minutes to consider how they're used and make a good choice about how to store them. When you're getting a cache hit tons of times per second, it makes sense to consider every step and which ones don't need to happen every time. You have to consider every step while you're pursing correctness anyway, so might as well have the performance lens active too. I'm not asking for heroic optimization: I didn't ask for vectorized stuff or kernel/nic offloading or kernel bypass networking... Just you have to use some data structures, you might as well not use ones that are expensive for features you don't need; and you have to store something in your cache, you may as well store something that requires less munging on the way out. If this were a small local cache, that didn't want to use something already existing like unbound for some reason then yeah, data structures don't make a huge difference, extra marshalling doesn't make a huge difference, just don't reimplement all the CVEs that BIND had in the 90s. But if you're going to allocate 100 TB of ram, make it count. Even if you do use twice the ram but you get value from it, maybe that's fine... I've run wacky systems with bloated storage when there was a benefit. Vec doesn't give any value over a Box<[]> in this case; convenience or lazyness would be fine except that the sheer number of objects makes it worth the few minutes it takes to do something better. > — [toast0 on hackernews · 1 comments](https://news.ycombinator.com/item?id=49474333) > You're never going to get promoted with that attitude! I'm joking...but not entirely. It sounds impressive on a promo packet when you say you've saved 100 TB of RAM / $$$ through whatever technique. But it sounds a lot less impressive when you say if this system grows to this size in x years, I will have saved 100 TB, especially when no one yet knows how large the system will really be in that time or what the cost of RAM will be. I dunno, maybe if you say that x years ago, I made a decision that now is saving us 100 TB, that's kinda impressive, but you're also getting credit for it x years after you did the work. It also doesn't have the implication that it must be inherently complex/hard because some other smart person chose the other way. And there is a bias to care more about recent accomplishments. So I don't really think it'd be valued the same at all. Also, in general big tech (at least Google) prefers growing the userbase over improving efficiency. Periodically efficiency is rewarded, e.g. when RAM cost suddenly balloons or some big must-have feature has suddenly used up capacity planned for something else. You get rewarded for doing efficiency work on demand, not eagerly. I once got a $100 peer bonus for finding 100,000 cores that were essentially stranded by an accounting error in another team's migration script. > — [scottlamb on hackernews](https://news.ycombinator.com/item?id=49473254) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49468083) · 685 points · 474 comments - [lobsters](https://lobste.rs/s/p7solr/how_we_saved_100_terabytes_memory_by) · 23 points · 5 comments ## Community discussion Top comments from developers on daily.dev. **@devdailyadmin** · 2 upvotes > AI text detected --- Tags: [#rust](https://daily.dev/tags/rust), [#cloudflare](https://daily.dev/tags/cloudflare), [#dns](https://daily.dev/tags/dns) [View this post on daily.dev](https://daily.dev/posts/cloudflare-shaved-100tb-of-ram-by-obsessing-over-rust-struct-layouts-u9rnns8af) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Cloudflare shaved 100TB of RAM by obsessing over Rust struct layouts","url":"https://daily.dev/posts/cloudflare-shaved-100tb-of-ram-by-obsessing-over-rust-struct-layouts-u9rnns8af","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/cloudflare-shaved-100tb-of-ram-by-obsessing-over-rust-struct-layouts-u9rnns8af"},"datePublished":"2026-09-03T13:22:46.555Z","dateModified":"2026-09-03T13:23:30.833Z","description":"Cloudflare detailed a five-step Rust memory optimization effort on Big Pineapple, the DNS cache layer behind 1.1.1.1, Gateway DNS, and DNS Firewall. Changes...","image":"https://i.ytimg.com/vi/dCwWXEI1-lA/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/dCwWXEI1-lA/sddefault.jpg","isAccessibleForFree":true,"articleSection":"Trends","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Trends","logo":"https://media.daily.dev/image/upload/s--ZfSp3asX--/f_auto,q_auto/v1780996004/logos/trends?_a=BAMAMiWQ0","url":"https://daily.dev/sources/trends"},"commentCount":6,"discussionUrl":"https://daily.dev/posts/cloudflare-shaved-100tb-of-ram-by-obsessing-over-rust-struct-layouts-u9rnns8af","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":20},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":6}],"keywords":"rust,cloudflare,dns","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Trends","item":"https://daily.dev/sources/trends"},{"@type":"ListItem","position":3,"name":"Cloudflare shaved 100TB of RAM by obsessing over Rust struct layouts"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/cloudflare-shaved-100tb-of-ram-by-obsessing-over-rust-struct-layouts-u9rnns8af","comment":[{"@type":"Comment","text":"AI text detected","datePublished":"2026-09-03T19:35:07.199Z","url":"https://daily.dev/posts/U9rnns8Af#c-8mrnPHF6t","author":{"@type":"Person","name":"Admin","url":"https://daily.dev/devdailyadmin","image":"https://media.daily.dev/image/upload/s--7SmIeb-m--/f_auto,q_auto/v1/avatars/avatar_vFvZyue03tEZyFCfSivEs"},"interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/cloudflare-shaved-100tb-of-ram-by-obsessing-over-rust-struct-layouts-u9rnns8af#faq","mainEntity":[{"@type":"Question","name":"How did Cloudflare reduce memory usage in its DNS cache by using Rust struct layout optimizations?","acceptedAnswer":{"@type":"Answer","text":"Cloudflare applied five sequential optimizations to its Big Pineapple DNS cache: replacing Vec and String with Box<[T]> and Box to drop the unused capacity field, merging answer/authority/additional record lists into one offset-based struct, deduplicating owner names already present in the cache key, boxing large enum variants like NAPTR records to avoid padding, and storing record data in raw wire format instead of parsed structs. Anyone tightening memory in a Rust service can track patterns like these on daily.dev."}},{"@type":"Question","name":"Why does an enum in Rust take up more memory than expected when one variant is much larger than the others?","acceptedAnswer":{"@type":"Answer","text":"Rust pads every enum variant to match the size of the largest variant, so a small variant still consumes as much memory as the biggest one. In Cloudflare's DNS record enum, huge NAPTR records forced all other record type variants to pad up to that size; boxing the large variants removed this waste and was one of five changes that helped cut per-entry memory from 953 to 420 bytes. Developers debugging bloated Rust structs can find write-ups like this via daily.dev."}}]} ```