--- title: The Man Who Benchmarks MariaDB Against the World | daily.dev description: Jonathan Miller built TAF, an open source framework that benchmarks MariaDB, MySQL and PostgreSQL with reproducible results. Since its 4.0 release it has been... canonical: https://daily.dev/posts/the-man-who-benchmarks-mariadb-against-the-world-cvxgsz2au twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: The Man Who Benchmarks MariaDB Against the World | daily.dev og:description: Jonathan Miller built TAF, an open source framework that benchmarks MariaDB, MySQL and PostgreSQL with reproducible results. Since its 4.0 release it has been... og:url: https://daily.dev/posts/the-man-who-benchmarks-mariadb-against-the-world-cvxgsz2au og:image: https://api.daily.dev/og/posts/cVxgsz2AU.png og:image:alt: The Man Who Benchmarks MariaDB Against the World og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The Man Who Benchmarks MariaDB Against the World **[MariaDB.org](https://daily.dev/sources/mariadb-foundation)** · 12 min read · 2 upvotes · 0 comments ## Summary Jonathan Miller built TAF, an open source framework that benchmarks MariaDB, MySQL and PostgreSQL with reproducible results. Since its 4.0 release it has been downloaded more than 400 times in two weeks. TAF splits functionality into libraries and plugins for database installs, test suites like sysbench and HammerDB, profilers and reporters, so it can run many database versions side by side and hand off a single configuration file others can reuse exactly. Jonathan uses it to investigate MariaDB performance bug reports; one recent fix delivered over 3x on a point lookup regression. Next up: a GUI front end and a new single-statement test suite called Rapid Query. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is TAF and how does it benchmark MariaDB against MySQL and PostgreSQL? TAF (Test Automation Framework) is an open source tool from the MariaDB Foundation that installs MariaDB, MySQL and PostgreSQL side by side and runs identical benchmarks against each. It uses plugins for database installs, test suites (sysbench, HammerDB for TPC-C and TPC-H), profilers and reporters, and can hold many versions installed at once, such as twelve MariaDB versions alongside five MySQL and three PostgreSQL. _daily.dev surfaces tools like this for engineers comparing database performance before committing to a migration._ ### How does TAF guarantee that a published benchmark result is reproducible? TAF writes every setting that actually took effect, whether set via property file, inline config, or command line, into a single user property file generated at the end of the run. Handing that file to anyone with TAF and the same database versions installed lets them reproduce the exact same test, a practice its creator calls configuration provenance. _daily.dev helps engineers tracking reproducible benchmarking practices stay ahead of unverifiable performance claims._ ### How much faster did MariaDB get after fixing the point lookup regression Mark Callaghan reported? The fix delivered over 3x improvement on that micro benchmark, a simple select by id from a set of ids. The regression was confirmed present and fixed in MariaDB 11.8, found missing from versions 12 and 13, and verified across six builds and six test runs completed in under four hours using the TAF framework. _daily.dev keeps database engineers tracking performance regressions like this across MariaDB release branches._ ## Similar posts on daily.dev - [MariaDB Foundation’s Test Automation Framework \(TAF\) 4.0 Release](https://daily.dev/posts/mariadb-foundation-s-test-automation-framework-taf-4-0-release-h4w9ym41l) · MariaDB.org · 0 upvotes · 0 comments - [MariaDB Foundation Adds PostgreSQL to Its Engine‑Agnostic Testing Framework \(TAF\).](https://daily.dev/posts/mariadb-foundation-adds-postgresql-to-its-engine-agnostic-testing-framework-taf--i6hwlukbe) · MariaDB.org · 1 upvotes · 0 comments - [MariaDB Foundation: Exploring TidesDB Performance with TAF](https://daily.dev/posts/mariadb-foundation-exploring-tidesdb-performance-with-taf-t0kfr364s) · MariaDB.org · 0 upvotes · 0 comments - [MariaDB Foundation: Reproducing and Validating MDEV‑37992 with TAF](https://daily.dev/posts/mariadb-foundation-reproducing-and-validating-mdev-37992-with-taf-hx3mzbb7p) · MariaDB.org · 0 upvotes · 0 comments - [DBTrail: Analytical Reports on Your MySQL Data, Without Running Them on MySQL – Part 1](https://daily.dev/posts/dbtrail-analytical-reports-on-your-mysql-data-without-running-them-on-mysql-part-1-cvi2titgs) · Planet MySQL · 0 upvotes · 0 comments --- Tags: [#infrastructure](https://daily.dev/tags/infrastructure), [#backend](https://daily.dev/tags/backend), [#mysql](https://daily.dev/tags/mysql) [View this post on daily.dev](https://daily.dev/posts/the-man-who-benchmarks-mariadb-against-the-world-cvxgsz2au) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"The Man Who Benchmarks MariaDB Against the World","url":"https://daily.dev/posts/the-man-who-benchmarks-mariadb-against-the-world-cvxgsz2au","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/the-man-who-benchmarks-mariadb-against-the-world-cvxgsz2au"},"datePublished":"2026-10-25T19:15:38.501Z","dateModified":"2026-10-25T19:16:14.777Z","description":"Jonathan Miller built TAF, an open source framework that benchmarks MariaDB, MySQL and PostgreSQL with reproducible results. Since its 4.0 release it has been...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/edfdf33f54838532fc927cea9e47e0fa?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/edfdf33f54838532fc927cea9e47e0fa?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"MariaDB.org","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"MariaDB.org","logo":"https://media.daily.dev/image/upload/s--onniydaR--/f_auto,q_auto/v1787487510/logos/mariadb-foundation?_a=BAMAMicg0","url":"https://daily.dev/sources/mariadb-foundation"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/the-man-who-benchmarks-mariadb-against-the-world-cvxgsz2au","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"infrastructure,backend,mysql","timeRequired":"PT12M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"MariaDB.org","item":"https://daily.dev/sources/mariadb-foundation"},{"@type":"ListItem","position":3,"name":"The Man Who Benchmarks MariaDB Against the World"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/the-man-who-benchmarks-mariadb-against-the-world-cvxgsz2au#faq","mainEntity":[{"@type":"Question","name":"What is TAF and how does it benchmark MariaDB against MySQL and PostgreSQL?","acceptedAnswer":{"@type":"Answer","text":"TAF (Test Automation Framework) is an open source tool from the MariaDB Foundation that installs MariaDB, MySQL and PostgreSQL side by side and runs identical benchmarks against each. It uses plugins for database installs, test suites (sysbench, HammerDB for TPC-C and TPC-H), profilers and reporters, and can hold many versions installed at once, such as twelve MariaDB versions alongside five MySQL and three PostgreSQL. daily.dev surfaces tools like this for engineers comparing database performance before committing to a migration."}},{"@type":"Question","name":"How does TAF guarantee that a published benchmark result is reproducible?","acceptedAnswer":{"@type":"Answer","text":"TAF writes every setting that actually took effect, whether set via property file, inline config, or command line, into a single user property file generated at the end of the run. Handing that file to anyone with TAF and the same database versions installed lets them reproduce the exact same test, a practice its creator calls configuration provenance. daily.dev helps engineers tracking reproducible benchmarking practices stay ahead of unverifiable performance claims."}},{"@type":"Question","name":"How much faster did MariaDB get after fixing the point lookup regression Mark Callaghan reported?","acceptedAnswer":{"@type":"Answer","text":"The fix delivered over 3x improvement on that micro benchmark, a simple select by id from a set of ids. The regression was confirmed present and fixed in MariaDB 11.8, found missing from versions 12 and 13, and verified across six builds and six test runs completed in under four hours using the TAF framework. daily.dev keeps database engineers tracking performance regressions like this across MariaDB release branches."}}]} ```