Percona tested feeding an InnoDB MySQL primary's row-based replication into a replica where tables use the experimental DuckDB storage engine, aiming to get an analytics-ready column store that stays in sync automatically. The first run silently lost data: because replicas write their own position into InnoDB system tables alongside DuckDB data, every applied transaction spans two engines and triggers a two-phase commit, and a bug in the engine's prepare/commit logic dropped the DuckDB writes while reporting success. The fix landed in v0.2.3, after which INSERTs, UPDATEs, DELETEs, all column types, DDL, transactions, bulk loads, graceful restarts, and SIGKILL crash recovery all replicate correctly and match the primary byte-for-byte. The project remains an experiment, not production software, and is available under GPLv2 on GitHub.
Table of contents
Why replicate into DuckDBThe setupWhat we tested, and howThe bug: multi-engine transactions lost dataThe fixWhat works, and what doesn’t yetWhere it standsQuestions this post answers
Why did my DuckDB MySQL storage engine replica silently lose data during replication even though replication showed no errors?
Because every replicated transaction touches two storage engines at once: InnoDB for the replica's own position (mysql.slave_worker_info, GTID) and DuckDB for the actual row data, which forces MySQL into a two-phase commit. A bug in the DuckDB engine's prepare step cleared the per-connection transaction state before commit, so commit found nothing to commit and silently dropped the DuckDB rows while the position still advanced and COMMIT reported success. This was fixed in v0.2.3 of the ducksdb-mysql-engine project. Teams evaluating experimental storage engines for MySQL replicas can track fixes like this one on daily.dev.
Can I replicate from a normal InnoDB MySQL primary into a replica with DuckDB-engine tables for analytics?
Yes, as of v0.2.3 of the Percona-Lab ducksdb-mysql-engine project, an InnoDB primary can feed row-based replication into a replica where large tables are marked ENGINE=DuckDB, giving an analytics-ready column store that stays in sync via normal MySQL replication. Testing confirmed inserts, updates, deletes, all common column types, DDL, transactions, bulk loads, graceful restarts, and SIGKILL crash recovery all replicate correctly, though it remains an experiment rather than production-ready software. Developers weighing analytics replicas against export pipelines can follow experiments like this on daily.dev.
What do I need to set up manually when creating DuckDB tables on a MySQL replica for replication?
You must pre-create the tables as ENGINE=DuckDB on the replica yourself, because a CREATE TABLE with ENGINE=InnoDB on the primary is logged into the binlog with that engine word intact and the replica will run it verbatim, producing an InnoDB table rather than a DuckDB one. There is no automatic engine mapping. Also add a primary key on the replica table, since UPDATE and DELETE row events locate rows by their old image and the engine needs that key. Engineers setting up cross-engine replication can check practical setup notes like these on daily.dev.