--- title: WAL + S3: Lakebase storage for the era of agents | daily.dev description: Neon explains the architecture behind Lakebase Postgres, a design that treats the Postgres write-ahead log (WAL) as the source of truth rather than a recovery... canonical: https://daily.dev/posts/wal-s3-lakebase-storage-for-the-era-of-agents-4trqspanr twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: WAL + S3: Lakebase storage for the era of agents | daily.dev og:description: Neon explains the architecture behind Lakebase Postgres, a design that treats the Postgres write-ahead log (WAL) as the source of truth rather than a recovery... og:url: https://daily.dev/posts/wal-s3-lakebase-storage-for-the-era-of-agents-4trqspanr og:image: https://api.daily.dev/og/posts/4trQsPAnr.png og:image:alt: WAL + S3: Lakebase storage for the era of agents og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # WAL + S3: Lakebase storage for the era of agents **[Neon](https://daily.dev/sources/neontech)** · 16 min read · 25 upvotes · 2 comments ## Summary Neon explains the architecture behind Lakebase Postgres, a design that treats the Postgres write-ahead log (WAL) as the source of truth rather than a recovery mechanism. By splitting compute (standard Postgres execution) from storage (safekeepers for WAL replication, pageservers that materialize pages, and S3 for immutable history), the system makes the entire transaction timeline addressable by LSN. This enables cheap branching, instant point-in-time restore, time-travel queries, replica creation without data copying, and scale-to-zero compute, all without Postgres ever reading directly from object storage during query execution. The post also covers the persistent binary search tree data structure used to quickly locate the right storage layer for a given key and LSN, and introduces LTAP (Lake Transactional/Analytical Processing), where one columnar copy in object storage serves both transactional and analytical workloads. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does Neon's Lakebase Postgres let you branch a terabyte-size database in seconds? Branching creates a pointer to a specific LSN in the write-ahead log rather than copying pages. The new branch diverges from that point using copy-on-write semantics, so writes are stored as deltas against the parent. This means a branch of a terabyte-size database is created in seconds and costs nothing until data actually changes, with no additional load on the parent. _Teams evaluating fast database branching for agent workflows can follow architecture deep dives like this on daily.dev._ ### Why doesn't reading from object storage make Postgres slow in a lakebase architecture? Object storage is never read directly by query execution. The compute node checks RAM first, then local NVMe cache, and only crosses the network to the pageserver on a local miss; the pageserver is the only component that reads from object storage, and only when reconstructing a page version it doesn't already have materialized. Commits also never wait on object storage, since a transaction is acknowledged once a quorum of safekeepers has the WAL record. _Developers weighing object storage under OLTP workloads can track these architecture trade-offs on daily.dev._ ### How does Lakebase Postgres find the correct historical page version quickly across millions of layers? It builds a persistent binary search tree of layer coverage, inserting layers in LSN order from the bottom up so each insert only touches nodes along one path and shares the rest of the tree with prior versions. Every intermediate root represents the coverage at a different LSN, so a historical read costs the same single lookup as a current one, regardless of how much history has accumulated. _Engineers designing time-travel or point-in-time query systems can follow deep dives like this via daily.dev._ ## Community discussion Top comments from developers on daily.dev. **@trevorsuna** · 0 upvotes > Treating WAL as the timeline and object storage as immutable history makes branching and point-in-time recovery feel like native storage operations instead of expensive copies. The tradeoff I’d watch closely is how safekeeper and pageserver lag affects recovery guarantees during a regional failure. **@agustinbarrientos** · 0 upvotes > Each agent branch should record its parent LSN and schema diff so reviewers can reconstruct which database state produced the final patch ## Similar posts on daily.dev - [From monolith to Lakebase to LTAP: rethinking the database from storage up](https://daily.dev/posts/from-monolith-to-lakebase-to-ltap-rethinking-the-database-from-storage-up-gsup5t8nr) · databricks · 6 upvotes · 0 comments - [Inside branch-based restores in Lakebase Postgres](https://daily.dev/posts/inside-branch-based-restores-in-lakebase-postgres-udhwx0m8z) · Neon · 1 upvotes · 1 comments - [Lakebase Postgres branch-based restores for fast recovery at scale](https://daily.dev/posts/lakebase-postgres-branch-based-restores-for-fast-recovery-at-scale-gvio7wglc) · databricks · 0 upvotes · 0 comments --- Tags: [#postgresql](https://daily.dev/tags/postgresql) [View this post on daily.dev](https://daily.dev/posts/wal-s3-lakebase-storage-for-the-era-of-agents-4trqspanr) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"WAL + S3: Lakebase storage for the era of agents","url":"https://daily.dev/posts/wal-s3-lakebase-storage-for-the-era-of-agents-4trqspanr","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/wal-s3-lakebase-storage-for-the-era-of-agents-4trqspanr"},"datePublished":"2026-08-24T15:15:38.935Z","dateModified":"2026-09-14T08:21:55.727Z","description":"Neon explains the architecture behind Lakebase Postgres, a design that treats the Postgres write-ahead log (WAL) as the source of truth rather than a recovery...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/8ce1758c7ab0ee0c6c0d3a77a754d3d1?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/8ce1758c7ab0ee0c6c0d3a77a754d3d1?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Neon","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Neon","logo":"https://media.daily.dev/image/upload/s--65EDghou--/f_auto/v1724339118/logos/neontech","url":"https://daily.dev/sources/neontech"},"commentCount":2,"discussionUrl":"https://daily.dev/posts/wal-s3-lakebase-storage-for-the-era-of-agents-4trqspanr","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":25},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":2}],"keywords":"postgresql","timeRequired":"PT16M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Neon","item":"https://daily.dev/sources/neontech"},{"@type":"ListItem","position":3,"name":"WAL + S3: Lakebase storage for the era of agents"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/wal-s3-lakebase-storage-for-the-era-of-agents-4trqspanr","comment":[{"@type":"Comment","text":"Treating WAL as the timeline and object storage as immutable history makes branching and point-in-time recovery feel like native storage operations instead of expensive copies. The tradeoff I’d watch closely is how safekeeper and pageserver lag affects recovery guarantees during a regional failure.","datePublished":"2026-08-25T14:50:40.620Z","url":"https://daily.dev/posts/4trQsPAnr#c-zLORUu9Dd","author":{"@type":"Person","name":"Trevor Suna","url":"https://daily.dev/trevorsuna","image":"https://media.daily.dev/image/upload/s--dZ7gXxpp--/f_auto/v1784081551/avatars/avatar_EMoP47rpuw8DNjhp6R1b6?_a=BAMAMicg0"}},{"@type":"Comment","text":"Each agent branch should record its parent LSN and schema diff so reviewers can reconstruct which database state produced the final patch","datePublished":"2026-08-27T00:33:20.867Z","url":"https://daily.dev/posts/4trQsPAnr#c-aPyFteP5d","author":{"@type":"Person","name":"Agustin Barrientos","url":"https://daily.dev/agustinbarrientos","image":"https://media.daily.dev/image/upload/s--5ayxQnqn--/f_auto/v1788281802/avatars/avatar_wQYYVe5Tbj0NJ7C7qPoa8?_a=BAMAMicg0"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/wal-s3-lakebase-storage-for-the-era-of-agents-4trqspanr#faq","mainEntity":[{"@type":"Question","name":"How does Neon's Lakebase Postgres let you branch a terabyte-size database in seconds?","acceptedAnswer":{"@type":"Answer","text":"Branching creates a pointer to a specific LSN in the write-ahead log rather than copying pages. The new branch diverges from that point using copy-on-write semantics, so writes are stored as deltas against the parent. This means a branch of a terabyte-size database is created in seconds and costs nothing until data actually changes, with no additional load on the parent. Teams evaluating fast database branching for agent workflows can follow architecture deep dives like this on daily.dev."}},{"@type":"Question","name":"Why doesn't reading from object storage make Postgres slow in a lakebase architecture?","acceptedAnswer":{"@type":"Answer","text":"Object storage is never read directly by query execution. The compute node checks RAM first, then local NVMe cache, and only crosses the network to the pageserver on a local miss; the pageserver is the only component that reads from object storage, and only when reconstructing a page version it doesn't already have materialized. Commits also never wait on object storage, since a transaction is acknowledged once a quorum of safekeepers has the WAL record. Developers weighing object storage under OLTP workloads can track these architecture trade-offs on daily.dev."}},{"@type":"Question","name":"How does Lakebase Postgres find the correct historical page version quickly across millions of layers?","acceptedAnswer":{"@type":"Answer","text":"It builds a persistent binary search tree of layer coverage, inserting layers in LSN order from the bottom up so each insert only touches nodes along one path and shares the rest of the tree with prior versions. Every intermediate root represents the coverage at a different LSN, so a historical read costs the same single lookup as a current one, regardless of how much history has accumulated. Engineers designing time-travel or point-in-time query systems can follow deep dives like this via daily.dev."}}]} ```