--- title: DuckLake: The Lakehouse That's Just SQL and Parquet description: A deep-dive podcast interview with DuckLake's lead developer Pedro Holanda and Guillermo Sanchez Dionis covers the design of DuckLake, an open table format... canonical: https://daily.dev/posts/ducklake-the-lakehouse-that-s-just-sql-and-parquet-vyaa3jmht twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: DuckLake: The Lakehouse That's Just SQL and Parquet | daily.dev og:description: A deep-dive podcast interview with DuckLake's lead developer Pedro Holanda and Guillermo Sanchez Dionis covers the design of DuckLake, an open table format... og:url: https://daily.dev/posts/ducklake-the-lakehouse-that-s-just-sql-and-parquet-vyaa3jmht og:image: https://api.daily.dev/og/posts/vyaa3jMHt.png og:image:alt: DuckLake: The Lakehouse That's Just SQL and Parquet og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # DuckLake: The Lakehouse That's Just SQL and Parquet **[Planet Python](https://daily.dev/sources/planetpython)** · [@mkennedy](https://daily.dev/mkennedy) · 94 min read · 1 upvotes · 0 comments ## Summary A deep-dive podcast interview with DuckLake's lead developer Pedro Holanda and Guillermo Sanchez Dionis covers the design of DuckLake, an open table format that stores metadata in a real SQL database instead of JSON/manifest files like Iceberg or Delta. Topics include the three-part architecture (storage, catalog, compute), choosing a catalog (DuckDB for quick starts, Postgres for production, the new Quack client-server protocol for high contention handling 200 tx/sec vs Postgres's 5), data inlining to avoid small files, the 'frozen DuckLake' pattern for serverless read-only lakes, DuckLake 1.0's production readiness, and DuckDB's origin story at CWI. It also touches on columnar storage, vectorized execution, zero-copy data sharing with NumPy, and upcoming DuckDB 2.0 features like asynchronous I/O. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How does DuckLake handle metadata differently from Apache Iceberg? DuckLake stores all metadata in a real SQL database (DuckDB, Postgres, MySQL, or the Quack protocol) rather than in JSON, Avro, manifest lists, and manifest files like Iceberg. This means an engine asks the catalog one SQL question to learn which Parquet files to read, instead of needing three or four file format readers and several round trips before reaching real data. _Engineers weighing lakehouse formats can track how DuckLake and Iceberg trade-offs evolve via daily.dev._ ### How many transactions per second can DuckLake handle with the Quack catalog versus Postgres under heavy contention? With around 20 concurrent writers under heavy contention, a Postgres catalog manages about five transactions per second, while Quack, DuckDB's new client-server protocol, manages around 200 transactions per second. Quack achieves this because conflict retries happen server-side instead of bouncing back to the client to recompute and resend queries. _Teams evaluating catalog choices for high-concurrency data lakes can follow protocol benchmarks like this on daily.dev._ ### What is the 'frozen DuckLake' pattern and when should I use it? It is a serverless data lake pattern where an in-process DuckDB file serves as the catalog: you pull the file locally, write new Parquet data to S3 through it, then upload the updated DuckDB catalog file back to S3. Any number of read-only clients can then attach to that catalog file with no Postgres or Quack server running, making it well suited to batch workloads like one big nightly load. _Developers designing low-cost batch data pipelines can keep up with patterns like this on daily.dev._ ## Similar posts on daily.dev - [DuckLake Architecture Deep Dive](https://daily.dev/posts/ducklake-architecture-deep-dive-yuky5mgag) · MotherDuck · 0 upvotes · 0 comments --- Tags: [#data-engineering](https://daily.dev/tags/data-engineering), [#duckdb](https://daily.dev/tags/duckdb), [#apache-iceberg](https://daily.dev/tags/apache-iceberg) [View this post on daily.dev](https://daily.dev/posts/ducklake-the-lakehouse-that-s-just-sql-and-parquet-vyaa3jmht) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"DuckLake: The Lakehouse That's Just SQL and Parquet","url":"https://daily.dev/posts/ducklake-the-lakehouse-that-s-just-sql-and-parquet-vyaa3jmht","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/ducklake-the-lakehouse-that-s-just-sql-and-parquet-vyaa3jmht"},"datePublished":"2026-09-10T16:49:27.983Z","dateModified":"2026-09-11T13:00:30.881Z","description":"A deep-dive podcast interview with DuckLake's lead developer Pedro Holanda and Guillermo Sanchez Dionis covers the design of DuckLake, an open table format...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6efe8ca52e4d3cffbd1f5b434300303b?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/6efe8ca52e4d3cffbd1f5b434300303b?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Planet Python","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Person","name":"Michael Kennedy","url":"https://daily.dev/mkennedy","image":"https://media.daily.dev/image/upload/s--CAIUuwxP--/f_auto,q_auto/v1701901463/avatars/avatar_5Q1yteeM08cxesQG2actL","description":"Python enthusiast. Host of @talkpython and @pythonbytes podcasts, founder of Talk Python Training.","interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"EndorseAction"},"userInteractionCount":30}},"commentCount":0,"discussionUrl":"https://daily.dev/posts/ducklake-the-lakehouse-that-s-just-sql-and-parquet-vyaa3jmht","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"data-engineering,duckdb,apache-iceberg","timeRequired":"PT94M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Planet Python","item":"https://daily.dev/sources/planetpython"},{"@type":"ListItem","position":3,"name":"DuckLake: The Lakehouse That's Just SQL and Parquet"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/ducklake-the-lakehouse-that-s-just-sql-and-parquet-vyaa3jmht#faq","mainEntity":[{"@type":"Question","name":"How does DuckLake handle metadata differently from Apache Iceberg?","acceptedAnswer":{"@type":"Answer","text":"DuckLake stores all metadata in a real SQL database (DuckDB, Postgres, MySQL, or the Quack protocol) rather than in JSON, Avro, manifest lists, and manifest files like Iceberg. This means an engine asks the catalog one SQL question to learn which Parquet files to read, instead of needing three or four file format readers and several round trips before reaching real data. Engineers weighing lakehouse formats can track how DuckLake and Iceberg trade-offs evolve via daily.dev."}},{"@type":"Question","name":"How many transactions per second can DuckLake handle with the Quack catalog versus Postgres under heavy contention?","acceptedAnswer":{"@type":"Answer","text":"With around 20 concurrent writers under heavy contention, a Postgres catalog manages about five transactions per second, while Quack, DuckDB's new client-server protocol, manages around 200 transactions per second. Quack achieves this because conflict retries happen server-side instead of bouncing back to the client to recompute and resend queries. Teams evaluating catalog choices for high-concurrency data lakes can follow protocol benchmarks like this on daily.dev."}},{"@type":"Question","name":"What is the 'frozen DuckLake' pattern and when should I use it?","acceptedAnswer":{"@type":"Answer","text":"It is a serverless data lake pattern where an in-process DuckDB file serves as the catalog: you pull the file locally, write new Parquet data to S3 through it, then upload the updated DuckDB catalog file back to S3. Any number of read-only clients can then attach to that catalog file with no Postgres or Quack server running, making it well suited to batch workloads like one big nightly load. Developers designing low-cost batch data pipelines can keep up with patterns like this on daily.dev."}}]} ```