DuckLake is a new open table format from the DuckDB team that stores lakehouse metadata in a standard SQL database instead of the complex file-based metadata systems used by Apache Iceberg and Delta Lake, while still keeping table data as Parquet files on blob storage. By moving catalog, schema, statistics, and snapshot tracking into relational tables with pure-SQL transactions, DuckLake claims simpler deployment, ACID multi-table transactions, faster query planning, fewer small files, and support for millions of snapshots. It ships as the free, open-source ducklake DuckDB extension starting with DuckDB v1.3.0, supports catalog databases like PostgreSQL, SQLite, MySQL, and MotherDuck, and writes data files compatible with Apache Iceberg for metadata-only migration. The extension is currently marked experimental.
Table of contents
BackgroundIceberg and DeltaCatalogsA Database You Say?DuckLakeThe ducklake DuckDB ExtensionInstallationUsageSummaryQuestions this post answers
What is DuckLake and how does it differ from Apache Iceberg?
DuckLake is an open table format that stores all lakehouse metadata (schemas, snapshots, statistics, file lists) in a standard SQL database rather than in JSON/Avro catalog files like Iceberg. Data itself is still stored as Parquet files on blob storage. This reduces the sequential file I/O needed for reads, avoids small-file proliferation, and lets any ACID SQL database with primary keys serve as the catalog. Teams weighing Iceberg against newer lakehouse formats can track format comparisons like this on daily.dev.
Which SQL databases can be used as the catalog store for DuckLake?
DuckLake can use a local DuckDB file for testing, or an external centralized database including PostgreSQL, SQLite, MySQL, and MotherDuck. The only requirements are support for ACID transactions and primary key constraints; a PostgreSQL-backed catalog can already scale to hundreds of terabytes and thousands of compute nodes. Engineers picking a catalog backend for their data lake can follow setup guidance like this on daily.dev.
How do I start using DuckLake with DuckDB?
Install the ducklake extension via 'INSTALL ducklake;' in DuckDB, available starting from DuckDB release v1.3.0 (codename Ossivalis). Then attach a lake with 'ATTACH \'ducklake:metadata.ducklake\' AS my_ducklake;', which creates a metadata file and a matching folder for Parquet data files, after which standard SQL commands like CREATE TABLE and INSERT work directly against it. Developers standing up their first lakehouse can find hands-on setup walkthroughs like this on daily.dev.