--- title: The DuckDB MySQL engine at 500 GB | daily.dev description: Percona benchmarked a custom MySQL storage engine backed by DuckDB against InnoDB and native DuckDB using TPC-H at scale factor 500 (~500 GB, 3 billion... canonical: https://daily.dev/posts/the-duckdb-mysql-engine-at-500-gb-baqjll9pk twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: The DuckDB MySQL engine at 500 GB | daily.dev og:description: Percona benchmarked a custom MySQL storage engine backed by DuckDB against InnoDB and native DuckDB using TPC-H at scale factor 500 (~500 GB, 3 billion... og:url: https://daily.dev/posts/the-duckdb-mysql-engine-at-500-gb-baqjll9pk og:image: https://api.daily.dev/og/posts/baqJlL9PK.png og:image:alt: The DuckDB MySQL engine at 500 GB og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The DuckDB MySQL engine at 500 GB **[Planet MySQL](https://daily.dev/sources/planet-mysql)** · 5 min read · 0 upvotes · 0 comments ## Summary Percona benchmarked a custom MySQL storage engine backed by DuckDB against InnoDB and native DuckDB using TPC-H at scale factor 500 (~500 GB, 3 billion lineitem rows) on an 80-core, 187 GB RAM server. The DuckDB engine completed all 22 TPC-H queries in ~186 seconds total; InnoDB finished only 18 of 22 in over 28 hours, with 4 queries never completing. Data load was 25x faster with the DuckDB engine (36 minutes vs 15+ hours), and storage footprint was 5x smaller (132 GB vs 673 GB). Query results matched native DuckDB on 21 of 22 queries. The engine and benchmark harness are available on GitHub under GPLv2, and a Docker image is provided for testing. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much faster is the DuckDB MySQL storage engine than InnoDB for loading large TPC-H datasets? It loads data about 25.5 times faster than InnoDB. In a 500 GB, three-billion-row TPC-H scale factor 500 test, the DuckDB engine's COPY fast path loaded all data in 36 minutes 5 seconds, versus 15 hours 21 minutes for InnoDB's bulk LOAD DATA, because InnoDB inserts row by row while building the primary key. _Teams weighing MySQL storage engines for analytics can track benchmarks like this one on daily.dev._ ### How much smaller is a DuckDB-backed MySQL table compared to the same data stored in InnoDB? The DuckDB engine's file is about 5 times smaller than the equivalent InnoDB tables. At TPC-H scale factor 500, 500 GB of raw CSV compressed to 132.4 GB in the DuckDB engine's columnar file, while InnoDB's tables ballooned to 673.2 GB, 135% of the raw CSV size, mainly due to its 446 GB lineitem index-carrying table. _Anyone sizing storage for analytical MySQL workloads can follow engine comparisons like this on daily.dev._ ### Can a MySQL storage engine finish TPC-H analytical queries that InnoDB cannot complete? Yes, an experimental MySQL storage engine built on DuckDB finished all 22 TPC-H queries at scale factor 500 in about 186 seconds total, while InnoDB failed to complete 4 of the 22 queries within a two-hour cap each and spent over 28 hours on the 18 it did finish. Output correctness was verified against native DuckDB, with 21 of 22 results matching exactly. _Developers evaluating analytical extensions for MySQL can follow real benchmark results like this on daily.dev._ ## Similar posts on daily.dev - [Running DuckDB as a MySQL 9.7 storage engine](https://daily.dev/posts/running-duckdb-as-a-mysql-9-7-storage-engine-hhxnvb0s1) · Percona Blog · 0 upvotes · 0 comments - [DuckDB Speed on MySQL with dbtrail, Without a New Storage Engine](https://daily.dev/posts/duckdb-speed-on-mysql-with-dbtrail-without-a-new-storage-engine-0gt0zkgml) · Planet MySQL · 1 upvotes · 0 comments - [Trying DuckDB for Magento Analytics: An Experiment with One Million Orders](https://daily.dev/posts/trying-duckdb-for-magento-analytics-an-experiment-with-one-million-orders-xunzodi5k) · MariaDB.org · 0 upvotes · 0 comments - [SQLite vs DuckDB on the same $16 box: every cliff moved 100x · Traceway](https://daily.dev/posts/sqlite-vs-duckdb-on-the-same-16-box-every-cliff-moved-100x-traceway-smgqxesze) · Hacker News · 0 upvotes · 0 comments - [Replicating from InnoDB into a DuckDB storage engine](https://daily.dev/posts/replicating-from-innodb-into-a-duckdb-storage-engine-pye2iwehn) · Planet MySQL · 1 upvotes · 0 comments --- Tags: [#mysql](https://daily.dev/tags/mysql), [#duckdb](https://daily.dev/tags/duckdb) [View this post on daily.dev](https://daily.dev/posts/the-duckdb-mysql-engine-at-500-gb-baqjll9pk) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"The DuckDB MySQL engine at 500 GB","url":"https://daily.dev/posts/the-duckdb-mysql-engine-at-500-gb-baqjll9pk","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/the-duckdb-mysql-engine-at-500-gb-baqjll9pk"},"datePublished":"2026-08-07T12:53:11.281Z","dateModified":"2026-09-14T09:31:34.444Z","description":"Percona benchmarked a custom MySQL storage engine backed by DuckDB against InnoDB and native DuckDB using TPC-H at scale factor 500 (~500 GB, 3 billion...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f6ef770a537a9682d161b27d1b96f748?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f6ef770a537a9682d161b27d1b96f748?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Planet MySQL","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Planet MySQL","logo":"https://media.daily.dev/image/upload/s--UKAtmXTq--/f_auto,q_auto/v1780213421/logos/planet-mysql?_a=BAMAMiWQ0","url":"https://daily.dev/sources/planet-mysql"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/the-duckdb-mysql-engine-at-500-gb-baqjll9pk","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"mysql,duckdb","timeRequired":"PT5M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Planet MySQL","item":"https://daily.dev/sources/planet-mysql"},{"@type":"ListItem","position":3,"name":"The DuckDB MySQL engine at 500 GB"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/the-duckdb-mysql-engine-at-500-gb-baqjll9pk#faq","mainEntity":[{"@type":"Question","name":"How much faster is the DuckDB MySQL storage engine than InnoDB for loading large TPC-H datasets?","acceptedAnswer":{"@type":"Answer","text":"It loads data about 25.5 times faster than InnoDB. In a 500 GB, three-billion-row TPC-H scale factor 500 test, the DuckDB engine's COPY fast path loaded all data in 36 minutes 5 seconds, versus 15 hours 21 minutes for InnoDB's bulk LOAD DATA, because InnoDB inserts row by row while building the primary key. Teams weighing MySQL storage engines for analytics can track benchmarks like this one on daily.dev."}},{"@type":"Question","name":"How much smaller is a DuckDB-backed MySQL table compared to the same data stored in InnoDB?","acceptedAnswer":{"@type":"Answer","text":"The DuckDB engine's file is about 5 times smaller than the equivalent InnoDB tables. At TPC-H scale factor 500, 500 GB of raw CSV compressed to 132.4 GB in the DuckDB engine's columnar file, while InnoDB's tables ballooned to 673.2 GB, 135% of the raw CSV size, mainly due to its 446 GB lineitem index-carrying table. Anyone sizing storage for analytical MySQL workloads can follow engine comparisons like this on daily.dev."}},{"@type":"Question","name":"Can a MySQL storage engine finish TPC-H analytical queries that InnoDB cannot complete?","acceptedAnswer":{"@type":"Answer","text":"Yes, an experimental MySQL storage engine built on DuckDB finished all 22 TPC-H queries at scale factor 500 in about 186 seconds total, while InnoDB failed to complete 4 of the 22 queries within a two-hour cap each and spent over 28 hours on the 18 it did finish. Output correctness was verified against native DuckDB, with 21 of 22 results matching exactly. Developers evaluating analytical extensions for MySQL can follow real benchmark results like this on daily.dev."}}]} ```