--- title: Git at any scale · Cursor | daily.dev description: A deep architectural history of hosting Git at scale, from Git's packfile-based design and its inherent conflict with distribution, through Google's failed... canonical: https://daily.dev/posts/git-at-any-scale-cursor-xz70xrmlg twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Git at any scale · Cursor | daily.dev og:description: A deep architectural history of hosting Git at scale, from Git's packfile-based design and its inherent conflict with distribution, through Google's failed... og:url: https://daily.dev/posts/git-at-any-scale-cursor-xz70xrmlg og:image: https://api.daily.dev/og/posts/XZ70XrMLg.png og:image:alt: Git at any scale · Cursor og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Git at any scale · Cursor **[Programming Digest](https://daily.dev/sources/programmingdigest)** · 26 min read · 0 upvotes · 0 comments ## Summary A deep architectural history of hosting Git at scale, from Git's packfile-based design and its inherent conflict with distribution, through Google's failed distributed-hash-table approach (JGit/DHT), GitHub's early filesystem-distribution failures (NFS, GFS, DRBD), and the eventual industry-standard Spokes architecture (three-phase commit consensus across NVMe-backed replicas). The piece then details Cursor's new system, Continuity, which replaces quorum-based consensus with a write-ahead log stored in S3, using rendezvous hashing for stateless routing, gossip-based optimistic replication verified via S3 ETags, and primary-driven compaction. Continuity reportedly sustains 120 pushes/s on S3 Standard and 300+ pushes/s on S3 Express One Zone with linear read scaling across 100 replicas, and underlies Cursor's new Git hosting product, Origin. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does GitHub's Spokes system keep Git repository replicas consistent? Spokes uses a consensus-based architecture where an orchestrator fans out pushed packfiles to all replicas simultaneously, then synchronizes the reference transaction using three-phase commit (3PC) so a push is only accepted once a majority of nodes acknowledge it. This keeps all replicas fully consistent, letting reads be routed safely to any single replica, but limits scalability since 3PC latency is bound by the slowest node in the cluster. _Teams evaluating Git hosting architectures can follow deep infrastructure breakdowns like this on daily.dev._ ### Why is it hard to distribute Git objects across a distributed key-value store or DHT? A Git repository's objects form a directed acyclic graph, so operations like listing commits require walking the graph step by step, and each fetch depends on the value of the previous one. Storing objects in a distributed hash table, as attempted at Google with JGit, made round trips prohibitively expensive and left git clone performance too slow because the Git network protocol still requires packfiles to be sent over the wire. _Engineers designing distributed storage for graph-like data can track these tradeoffs on daily.dev._ ### What push throughput can a write-ahead-log-based Git storage system achieve using S3? Using S3 Standard, a WAL-based Git storage system can sustain up to 120 pushes per second while compacting and replicating data to all nodes. Deploying on S3 Express One Zone, which has much lower PUT latency, raises ingestion to more than 300 pushes per second, at which point the bottleneck becomes the speed at which Git can compact on-disk data rather than S3 latency itself. _Developers benchmarking storage backends for high-throughput Git hosting can follow this kind of data on daily.dev._ --- Tags: [#github](https://daily.dev/tags/github), [#git](https://daily.dev/tags/git), [#distributed-systems](https://daily.dev/tags/distributed-systems), [#aws-s3](https://daily.dev/tags/aws-s3) [View this post on daily.dev](https://daily.dev/posts/git-at-any-scale-cursor-xz70xrmlg) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Git at any scale · Cursor","url":"https://daily.dev/posts/git-at-any-scale-cursor-xz70xrmlg","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/git-at-any-scale-cursor-xz70xrmlg"},"datePublished":"2026-08-30T18:45:29.272Z","dateModified":"2026-09-01T19:10:06.215Z","description":"A deep architectural history of hosting Git at scale, from Git's packfile-based design and its inherent conflict with distribution, through Google's failed...","image":"https://ptht05hbb1ssoooe.public.blob.vercel-storage.com/assets/blog/blog-demo-animation-2ntJbzDU6xSEoHctbbXf2VUO4b2nxK.gif","thumbnailUrl":"https://ptht05hbb1ssoooe.public.blob.vercel-storage.com/assets/blog/blog-demo-animation-2ntJbzDU6xSEoHctbbXf2VUO4b2nxK.gif","isAccessibleForFree":true,"articleSection":"Programming Digest","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Programming Digest","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/4637f2287b44420aa9a44f432116043d","url":"https://daily.dev/sources/programmingdigest"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/git-at-any-scale-cursor-xz70xrmlg","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"github,git,distributed-systems,aws-s3","timeRequired":"PT26M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Programming Digest","item":"https://daily.dev/sources/programmingdigest"},{"@type":"ListItem","position":3,"name":"Git at any scale · Cursor"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/git-at-any-scale-cursor-xz70xrmlg#faq","mainEntity":[{"@type":"Question","name":"How does GitHub's Spokes system keep Git repository replicas consistent?","acceptedAnswer":{"@type":"Answer","text":"Spokes uses a consensus-based architecture where an orchestrator fans out pushed packfiles to all replicas simultaneously, then synchronizes the reference transaction using three-phase commit (3PC) so a push is only accepted once a majority of nodes acknowledge it. This keeps all replicas fully consistent, letting reads be routed safely to any single replica, but limits scalability since 3PC latency is bound by the slowest node in the cluster. Teams evaluating Git hosting architectures can follow deep infrastructure breakdowns like this on daily.dev."}},{"@type":"Question","name":"Why is it hard to distribute Git objects across a distributed key-value store or DHT?","acceptedAnswer":{"@type":"Answer","text":"A Git repository's objects form a directed acyclic graph, so operations like listing commits require walking the graph step by step, and each fetch depends on the value of the previous one. Storing objects in a distributed hash table, as attempted at Google with JGit, made round trips prohibitively expensive and left git clone performance too slow because the Git network protocol still requires packfiles to be sent over the wire. Engineers designing distributed storage for graph-like data can track these tradeoffs on daily.dev."}},{"@type":"Question","name":"What push throughput can a write-ahead-log-based Git storage system achieve using S3?","acceptedAnswer":{"@type":"Answer","text":"Using S3 Standard, a WAL-based Git storage system can sustain up to 120 pushes per second while compacting and replicating data to all nodes. Deploying on S3 Express One Zone, which has much lower PUT latency, raises ingestion to more than 300 pushes per second, at which point the bottleneck becomes the speed at which Git can compact on-disk data rather than S3 latency itself. Developers benchmarking storage backends for high-throughput Git hosting can follow this kind of data on daily.dev."}}]} ```