--- title: Remote Cache CDC: Reusing Bytes | daily.dev description: BuildBuddy has implemented Content-Defined Chunking (CDC) in its remote cache to dramatically reduce redundant data transfers during Bazel builds. Instead of... canonical: https://daily.dev/posts/remote-cache-cdc-reusing-bytes-b2fv2gke6 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Remote Cache CDC: Reusing Bytes | daily.dev og:description: BuildBuddy has implemented Content-Defined Chunking (CDC) in its remote cache to dramatically reduce redundant data transfers during Bazel builds. Instead of... og:url: https://daily.dev/posts/remote-cache-cdc-reusing-bytes-b2fv2gke6 og:image: https://api.daily.dev/og/posts/b2fV2Gke6.png og:image:alt: Remote Cache CDC: Reusing Bytes og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Remote Cache CDC: Reusing Bytes **[BuildBuddy Blog](https://daily.dev/sources/buildbuddy)** · 14 min read · 0 upvotes · 0 comments ## Summary BuildBuddy has implemented Content-Defined Chunking (CDC) in its remote cache to dramatically reduce redundant data transfers during Bazel builds. Instead of re-uploading entire large outputs (binaries, test binaries, archives) when only a small portion changes, CDC splits files into content-addressed chunks using a rolling hash algorithm. Only changed chunks are uploaded or downloaded. In production, CDC has skipped ~300 TiB of duplicate chunk uploads over two weeks, with ~85% deduplication on eligible writes and 40% less data uploaded in benchmarks. The implementation spans three layers: new SplitBlob/SpliceBlob Remote APIs, Bazel's combined cache path (available in Bazel 8.7 and 9.1+), and BuildBuddy's server and executor infrastructure. Opt in with --experimental_remote_cache_chunking. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How do I enable content-defined chunking for Bazel remote cache uploads? Use Bazel 8.7 or 9.1+ and pass the flag --experimental_remote_cache_chunking when building with a CDC-enabled remote cache such as BuildBuddy. This enables client-side chunked upload and download paths so only changed chunks of large outputs are transferred instead of whole blobs, cutting network usage on repeated builds. _Teams tuning Bazel remote cache performance track flag and version changes like this via daily.dev._ ### What performance improvement does content-defined chunking provide for Bazel remote caching? Benchmarking 50 commits of the BuildBuddy repository showed about 40% less data uploaded and a 40% smaller disk cache when content-defined chunking was enabled, along with faster builds. In production, roughly 85% of written bytes across eligible large-output cache writes were deduplicated, with a two-week window skipping about 300 TiB of duplicate chunk uploads, peaking over 4 TiB per hour. _Engineers deciding whether to adopt CDC for build caching follow benchmarks like this on daily.dev._ ### What size threshold and file types work best for content-defined chunking in remote build caches? BuildBuddy applies chunking to blobs larger than 2 MiB, though in one test only about 4.2% of objects exceeded that threshold. CDC works best on large, byte-stable outputs like linking and packaging results; compressed or randomized formats such as tar.gz archives and Docker image layers are less chunkable because small input changes rewrite more of the compressed byte stream. _Developers optimizing build artifact caching weigh trade-offs like these through daily.dev._ ## Similar posts on daily.dev - [Accessing external resources reliably with Bazel](https://daily.dev/posts/accessing-external-resources-reliably-with-bazel-1skvfsyoj) · Planet Haskell · 0 upvotes · 0 comments - [The physics of build systems](https://daily.dev/posts/the-physics-of-build-systems-jp9ocfyl5) · Tuist · 0 upvotes · 0 comments - [The continuous integration platform for the World’s best engineering teams - CircleCI](https://daily.dev/posts/the-continuous-integration-platform-for-the-world-s-best-engineering-teams---circleci-khswu38xi) · Product Hunt · 0 upvotes · 0 comments - [How Pinterest Transfers Hundreds of Terabytes of Data With CDC](https://daily.dev/posts/how-pinterest-transfers-hundreds-of-terabytes-of-data-with-cdc-x1wxpwc1y) · ByteByteGo · 39 upvotes · 0 comments --- Tags: [#build-systems](https://daily.dev/tags/build-systems), [#bazel](https://daily.dev/tags/bazel) [View this post on daily.dev](https://daily.dev/posts/remote-cache-cdc-reusing-bytes-b2fv2gke6) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Remote Cache CDC: Reusing Bytes","url":"https://daily.dev/posts/remote-cache-cdc-reusing-bytes-b2fv2gke6","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/remote-cache-cdc-reusing-bytes-b2fv2gke6"},"datePublished":"2026-07-26T07:49:22.832Z","dateModified":"2026-09-13T20:55:23.764Z","description":"BuildBuddy has implemented Content-Defined Chunking (CDC) in its remote cache to dramatically reduce redundant data transfers during Bazel builds. Instead of...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7492b735b3fb78676bdcc1e2e87a44fe?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/7492b735b3fb78676bdcc1e2e87a44fe?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"BuildBuddy Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"BuildBuddy Blog","logo":"https://media.daily.dev/image/upload/s---X2Wa4pv--/c_limit,w_256/f_auto,q_auto/v1785052099/logos/buildbuddy?_a=BAMAMicg0","url":"https://daily.dev/sources/buildbuddy"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/remote-cache-cdc-reusing-bytes-b2fv2gke6","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"build-systems,bazel","timeRequired":"PT14M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"BuildBuddy Blog","item":"https://daily.dev/sources/buildbuddy"},{"@type":"ListItem","position":3,"name":"Remote Cache CDC: Reusing Bytes"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/remote-cache-cdc-reusing-bytes-b2fv2gke6#faq","mainEntity":[{"@type":"Question","name":"How do I enable content-defined chunking for Bazel remote cache uploads?","acceptedAnswer":{"@type":"Answer","text":"Use Bazel 8.7 or 9.1+ and pass the flag --experimental_remote_cache_chunking when building with a CDC-enabled remote cache such as BuildBuddy. This enables client-side chunked upload and download paths so only changed chunks of large outputs are transferred instead of whole blobs, cutting network usage on repeated builds. Teams tuning Bazel remote cache performance track flag and version changes like this via daily.dev."}},{"@type":"Question","name":"What performance improvement does content-defined chunking provide for Bazel remote caching?","acceptedAnswer":{"@type":"Answer","text":"Benchmarking 50 commits of the BuildBuddy repository showed about 40% less data uploaded and a 40% smaller disk cache when content-defined chunking was enabled, along with faster builds. In production, roughly 85% of written bytes across eligible large-output cache writes were deduplicated, with a two-week window skipping about 300 TiB of duplicate chunk uploads, peaking over 4 TiB per hour. Engineers deciding whether to adopt CDC for build caching follow benchmarks like this on daily.dev."}},{"@type":"Question","name":"What size threshold and file types work best for content-defined chunking in remote build caches?","acceptedAnswer":{"@type":"Answer","text":"BuildBuddy applies chunking to blobs larger than 2 MiB, though in one test only about 4.2% of objects exceeded that threshold. CDC works best on large, byte-stable outputs like linking and packaging results; compressed or randomized formats such as tar.gz archives and Docker image layers are less chunkable because small input changes rewrite more of the compressed byte stream. Developers optimizing build artifact caching weigh trade-offs like these through daily.dev."}}]} ```