DuckLake 0.2 adds several new capabilities to the open table format standard: relative schema/table/data paths for structured file layout and prefix-based access control, name mapping to register existing Parquet files without field ids, scoped settings at the schema/table level instead of only globally, and year/month/day/hour partition transforms. A migration guide with SQL statements is provided for upgrading from v0.1 to v0.2, and the DuckDB ducklake extension applies these automatically.

4m read timeFrom ducklake.select
Post cover image
Table of contents
New FeaturesMigration Guide

Questions this post answers

How do I migrate a DuckLake table from version 0.1 to 0.2?

Migration requires running a set of ALTER TABLE and CREATE TABLE SQL statements against the DuckLake metadata tables: adding path and path_is_relative columns to ducklake_schema and ducklake_table, adding scope and scope_id columns to ducklake_metadata, adding a mapping_id column to ducklake_data_file, creating ducklake_column_mapping and ducklake_name_mapping tables, updating partition column ids, and finally setting the version metadata value to 0.2. The DuckDB ducklake extension runs these automatically. daily.dev surfaces release notes like this so lakehouse upgrades don't catch you off guard.

How can I add an existing Parquet file to a DuckLake table without field ids?

DuckLake 0.2 introduces name mapping, which lets Parquet files lacking field identifiers be registered using the ducklake_add_data_files function, for example CALL ducklake_add_data_files('my_ducklake', 'people', 'people.parquet'). Each registered file gets an optional mapping_id that maps column names to field ids on a per-file basis, so files from other writers can be added without restriction and still support schema evolution and change feeds. developers integrating external parquet files into a lakehouse can track feature updates like this on daily.dev.

How are data and delete file paths organized in DuckLake 0.2 compared to earlier versions?

DuckLake 0.2 introduces a three-layer relative path structure: data paths are relative to the table path, table paths are relative to the schema path, and schema paths are relative to the global data path. Previously all paths were relative only to the global data path. This structured layout, with schema and table names as default subdirectories, enables prefix-based access control at the object store level. daily.dev helps engineers keep up with lakehouse storage layout changes like these.

2 Impressions