--- title: Updated PDS-H benchmark results | daily.dev description: A rerun of the PDS-H (TPC-H derived) benchmark suite compares Polars against pandas, Modin, Dask, PySpark, and DuckDB across query performance, both with and... canonical: https://daily.dev/posts/updated-pds-h-benchmark-results-yfzo0qf61 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Updated PDS-H benchmark results | daily.dev og:description: A rerun of the PDS-H (TPC-H derived) benchmark suite compares Polars against pandas, Modin, Dask, PySpark, and DuckDB across query performance, both with and... og:url: https://daily.dev/posts/updated-pds-h-benchmark-results-yfzo0qf61 og:image: https://api.daily.dev/og/posts/yFzO0qf61.png og:image:alt: Updated PDS-H benchmark results og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Updated PDS-H benchmark results **[Polars Blog](https://daily.dev/sources/polars)** · 5 min read · 0 upvotes · 0 comments ## Summary A rerun of the PDS-H (TPC-H derived) benchmark suite compares Polars against pandas, Modin, Dask, PySpark, and DuckDB across query performance, both with and without disk I/O, using scale factor 10 on a Google Cloud VM. Polars comes out fastest among dataframe libraries, with DuckDB close behind as a SQL engine. pandas struggles on most queries, Modin is inconsistent and hits memory issues, and Dask/PySpark trail Polars and DuckDB despite being built for distributed systems. An addendum reruns the tests on a smaller machine with similar results, except Modin fails to complete any queries there. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Which dataframe library performs best on TPC-H style benchmarks, Polars or pandas? Polars is the fastest among dataframe libraries in PDS-H benchmark testing, a TPC-H derived benchmark suite. pandas performs poorly on most queries, while DuckDB, a SQL engine, shows comparable performance to Polars. Dask improves significantly over pandas but still trails Polars and DuckDB, and PySpark performs similarly to Dask on a single machine. _Compare dataframe library benchmarks like this on daily.dev before picking a query engine for your data pipeline._ ### Why was vaex excluded from a dataframe library benchmark comparison? Vaex was excluded from the PDS-H benchmark comparison because it is no longer being maintained. The benchmark instead compared Polars against pandas, Modin, Dask, PySpark, and DuckDB using the latest versions of each library at scale factor 10 (about 10GB of data) on a Google Cloud c3-highmem-22 VM. _Track which data tools remain actively maintained by following ecosystem updates on daily.dev._ ### Why did Modin fail to complete benchmark queries on smaller machines? Modin, which uses a Ray backend, was unable to complete any queries when memory was constrained to 22GB on a smaller c3-highcpu-22 machine with 44GB total memory, compared to running fine with 90GB allocated on a larger 178GB machine. This suggests Modin's memory requirements scale poorly relative to data size versus other dataframe libraries. _Weigh memory requirements alongside speed when comparing dataframe libraries on daily.dev._ ## Similar posts on daily.dev - [Polars vs DuckDB vs Pandas: Benchmark on 33 Million Rows](https://daily.dev/posts/polars-vs-duckdb-vs-pandas-benchmark-on-33-million-rows-p8chasjqs) · Medium · 0 upvotes · 0 comments - [Pandas Should Go Extinct](https://daily.dev/posts/pandas-should-go-extinct-1jqzeauiq) · Lobsters · 0 upvotes · 0 comments - [650GB of Data \(Delta Lake on S3\). Polars vs DuckDB vs Daft vs Spark.](https://daily.dev/posts/650gb-of-data-delta-lake-on-s3-polars-vs-duckdb-vs-daft-vs-spark--bbkmmear9) · Hacker News · 1 upvotes · 0 comments - [Crunching data? Put that laptop hardware to use](https://daily.dev/posts/crunching-data-put-that-laptop-hardware-to-use-1fcnht8lv) · Simple Thread · 0 upvotes · 0 comments - [Release of Polars 2.0](https://daily.dev/posts/release-of-polars-2-0-84cwf427y) · Polars Blog · 3 upvotes · 0 comments --- Tags: [#pandas](https://daily.dev/tags/pandas), [#duckdb](https://daily.dev/tags/duckdb), [#pyspark](https://daily.dev/tags/pyspark), [#polars](https://daily.dev/tags/polars) [View this post on daily.dev](https://daily.dev/posts/updated-pds-h-benchmark-results-yfzo0qf61) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Updated PDS-H benchmark results","url":"https://daily.dev/posts/updated-pds-h-benchmark-results-yfzo0qf61","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/updated-pds-h-benchmark-results-yfzo0qf61"},"datePublished":"2026-08-16T08:53:13.638Z","dateModified":"2026-08-16T08:54:32.510Z","description":"A rerun of the PDS-H (TPC-H derived) benchmark suite compares Polars against pandas, Modin, Dask, PySpark, and DuckDB across query performance, both with and...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5434e7bec80a8efdf8c46b2f6f76f432?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5434e7bec80a8efdf8c46b2f6f76f432?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Polars Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Polars Blog","logo":"https://media.daily.dev/image/upload/s--xo60W3UW--/c_limit,w_256/f_auto,q_auto/v1786870390/logos/polars?_a=BAMAMicg0","url":"https://daily.dev/sources/polars"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/updated-pds-h-benchmark-results-yfzo0qf61","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"pandas,duckdb,pyspark,polars","timeRequired":"PT5M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Polars Blog","item":"https://daily.dev/sources/polars"},{"@type":"ListItem","position":3,"name":"Updated PDS-H benchmark results"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/updated-pds-h-benchmark-results-yfzo0qf61#faq","mainEntity":[{"@type":"Question","name":"Which dataframe library performs best on TPC-H style benchmarks, Polars or pandas?","acceptedAnswer":{"@type":"Answer","text":"Polars is the fastest among dataframe libraries in PDS-H benchmark testing, a TPC-H derived benchmark suite. pandas performs poorly on most queries, while DuckDB, a SQL engine, shows comparable performance to Polars. Dask improves significantly over pandas but still trails Polars and DuckDB, and PySpark performs similarly to Dask on a single machine. Compare dataframe library benchmarks like this on daily.dev before picking a query engine for your data pipeline."}},{"@type":"Question","name":"Why was vaex excluded from a dataframe library benchmark comparison?","acceptedAnswer":{"@type":"Answer","text":"Vaex was excluded from the PDS-H benchmark comparison because it is no longer being maintained. The benchmark instead compared Polars against pandas, Modin, Dask, PySpark, and DuckDB using the latest versions of each library at scale factor 10 (about 10GB of data) on a Google Cloud c3-highmem-22 VM. Track which data tools remain actively maintained by following ecosystem updates on daily.dev."}},{"@type":"Question","name":"Why did Modin fail to complete benchmark queries on smaller machines?","acceptedAnswer":{"@type":"Answer","text":"Modin, which uses a Ray backend, was unable to complete any queries when memory was constrained to 22GB on a smaller c3-highcpu-22 machine with 44GB total memory, compared to running fine with 90GB allocated on a larger 178GB machine. This suggests Modin's memory requirements scale poorly relative to data size versus other dataframe libraries. Weigh memory requirements alongside speed when comparing dataframe libraries on daily.dev."}}]} ```