Delta Lake 0.4.0 adds Python APIs for DML and utility operations, letting developers update, delete, and merge data in Delta tables and run vacuum/history commands from Python. It also introduces Convert-to-Delta, a reversible in-place conversion of Parquet tables to Delta Lake without rewriting data, and adds SQL support for vacuum and history utility operations in Apache Spark.
Questions this post answers
What new features were added in Delta Lake 0.4.0?
Delta Lake 0.4.0 adds Python APIs for DML and utility operations, letting developers update, delete, and merge data in Delta tables and run vacuum and history commands directly from Python. It also introduces Convert-to-Delta, a reversible in-place conversion from Parquet to Delta Lake tables, and adds SQL support for vacuum and history utility operations in Spark. Teams tracking Delta Lake and Spark ecosystem updates can follow releases like this on daily.dev.
How do I convert an existing Parquet table to Delta Lake without rewriting the data?
Use the Convert-to-Delta feature introduced in Delta Lake 0.4.0, which converts a Parquet table in place to a Delta Lake table without rewriting any data. The process is reversible, so a table can be converted to Delta Lake, modified with operations like delete or merge, and then converted back to Parquet if needed. Developers weighing Parquet to Delta Lake migrations can find release details like this on daily.dev.