<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/benchmarking-polars-and-spark-tflcauzdc" -->

---
title: Benchmarking Polars and Spark | daily.dev
description: Polars published its first distributed engine benchmarks, comparing against PySpark on the PDS-H benchmark (derived from TPC-H) at scale factor 1000 (~1TB...
canonical: https://daily.dev/posts/benchmarking-polars-and-spark-tflcauzdc
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: Benchmarking Polars and Spark | daily.dev
og:description: Polars published its first distributed engine benchmarks, comparing against PySpark on the PDS-H benchmark (derived from TPC-H) at scale factor 1000 (~1TB...
og:url: https://daily.dev/posts/benchmarking-polars-and-spark-tflcauzdc
og:image: https://api.daily.dev/og/posts/tFLcaUzDc.png
og:image:alt: Benchmarking Polars and Spark
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Benchmarking Polars and Spark

**[Polars Blog](https://daily.dev/sources/polars)** · 4 min read · 0 upvotes · 0 comments

## Summary

Polars published its first distributed engine benchmarks, comparing against PySpark on the PDS-H benchmark (derived from TPC-H) at scale factor 1000 (~1TB uncompressed). On a single node (m8id.32xlarge), Polars 1.41.1 was ~6.4x faster on average than PySpark 4.0.1, with per-query speedups up to 38x. On a 32-node distributed cluster with equivalent total resources, Polars-cloud 0.8.0 was ~3.2x faster on average, ranging up to 7.7x per query. The team notes single-node and distributed results aren't directly comparable due to network I/O differences, and promises a follow-up comparing Polars single-node vs distributed performance.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://pola.rs/posts/polars-pyspark-benchmarks>

## Questions this post answers

### How much faster is Polars than PySpark on a single node according to recent benchmarks?

Polars is about 6.4x faster than PySpark on total runtime on a single node, with per-query speedups ranging from 3x to 38x. This was measured using Polars 1.41.1 against PySpark 4.0.1 on the PDS-H benchmark (derived from TPC-H) at scale factor 1000, roughly 1TB of uncompressed data, run on an m8id.32xlarge instance with 128 vCPUs and 512GB RAM.

_Comparing dataframe engines like Polars and Spark gets easier when tracking benchmark results on daily.dev._

### How does Polars distributed performance compare to PySpark on a cluster?

Polars-cloud is about 3.2x faster than PySpark on total runtime in a distributed setup, with per-query speedups ranging from 1.6x to 7.7x. The test used Polars-cloud 0.8.0 versus PySpark 4.0.1 across 32 m8id.xlarge nodes (4 vCPUs, 16GB RAM each) with the same total resources as the single-node test, at scale factor 1000 on the PDS-H benchmark.

_Engineers weighing distributed dataframe tools can follow benchmark comparisons like this on daily.dev._

### What is the PDS-H benchmark and how does it relate to TPC-H?

PDS-H, or Polars Decision Support, is an open implementation derived from the TPC-H benchmark that uses the same data model, generation methodology, and query workload to measure analytical query performance. It is not officially audited or certified as TPC-H, so its results are meant for comparative evaluation within PDS-H itself rather than direct comparison to published TPC-H benchmark numbers.

_Understanding benchmark methodology helps developers evaluate dataframe tools fairly, a topic worth following on daily.dev._

## Similar posts on daily.dev

- [650GB of Data \(Delta Lake on S3\). Polars vs DuckDB vs Daft vs Spark.](https://daily.dev/posts/650gb-of-data-delta-lake-on-s3-polars-vs-duckdb-vs-daft-vs-spark--bbkmmear9) · Hacker News · 1 upvotes · 0 comments
- [Release of Polars 2.0](https://daily.dev/posts/release-of-polars-2-0-84cwf427y) · Polars Blog · 1 upvotes · 0 comments
- [Polars vs DuckDB vs Pandas: Benchmark on 33 Million Rows](https://daily.dev/posts/polars-vs-duckdb-vs-pandas-benchmark-on-33-million-rows-p8chasjqs) · Medium · 0 upvotes · 0 comments
- [Decathlon Switches to Polars to Optimize Data Pipelines and Infrastructure Costs](https://daily.dev/posts/decathlon-switches-to-polars-to-optimize-data-pipelines-and-infrastructure-costs-5gcd7b1tn) · InfoQ · 70 upvotes · 0 comments

---

Tags: [#distributed-systems](https://daily.dev/tags/distributed-systems), [#data-engineering](https://daily.dev/tags/data-engineering), [#apache-spark](https://daily.dev/tags/apache-spark), [#polars](https://daily.dev/tags/polars)

[View this post on daily.dev](https://daily.dev/posts/benchmarking-polars-and-spark-tflcauzdc)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"Benchmarking Polars and Spark","url":"https://daily.dev/posts/benchmarking-polars-and-spark-tflcauzdc","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/benchmarking-polars-and-spark-tflcauzdc"},"datePublished":"2026-08-16T08:53:14.671Z","dateModified":"2026-08-16T08:53:57.029Z","description":"Polars published its first distributed engine benchmarks, comparing against PySpark on the PDS-H benchmark (derived from TPC-H) at scale factor 1000 (~1TB...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5434e7bec80a8efdf8c46b2f6f76f432?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5434e7bec80a8efdf8c46b2f6f76f432?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Polars Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Polars Blog","logo":"https://media.daily.dev/image/upload/s--xo60W3UW--/c_limit,w_256/f_auto,q_auto/v1786870390/logos/polars?_a=BAMAMicg0","url":"https://daily.dev/sources/polars"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/benchmarking-polars-and-spark-tflcauzdc","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"distributed-systems,data-engineering,apache-spark,polars","timeRequired":"PT4M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Polars Blog","item":"https://daily.dev/sources/polars"},{"@type":"ListItem","position":3,"name":"Benchmarking Polars and Spark"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/benchmarking-polars-and-spark-tflcauzdc#faq","mainEntity":[{"@type":"Question","name":"How much faster is Polars than PySpark on a single node according to recent benchmarks?","acceptedAnswer":{"@type":"Answer","text":"Polars is about 6.4x faster than PySpark on total runtime on a single node, with per-query speedups ranging from 3x to 38x. This was measured using Polars 1.41.1 against PySpark 4.0.1 on the PDS-H benchmark (derived from TPC-H) at scale factor 1000, roughly 1TB of uncompressed data, run on an m8id.32xlarge instance with 128 vCPUs and 512GB RAM. Comparing dataframe engines like Polars and Spark gets easier when tracking benchmark results on daily.dev."}},{"@type":"Question","name":"How does Polars distributed performance compare to PySpark on a cluster?","acceptedAnswer":{"@type":"Answer","text":"Polars-cloud is about 3.2x faster than PySpark on total runtime in a distributed setup, with per-query speedups ranging from 1.6x to 7.7x. The test used Polars-cloud 0.8.0 versus PySpark 4.0.1 across 32 m8id.xlarge nodes (4 vCPUs, 16GB RAM each) with the same total resources as the single-node test, at scale factor 1000 on the PDS-H benchmark. Engineers weighing distributed dataframe tools can follow benchmark comparisons like this on daily.dev."}},{"@type":"Question","name":"What is the PDS-H benchmark and how does it relate to TPC-H?","acceptedAnswer":{"@type":"Answer","text":"PDS-H, or Polars Decision Support, is an open implementation derived from the TPC-H benchmark that uses the same data model, generation methodology, and query workload to measure analytical query performance. It is not officially audited or certified as TPC-H, so its results are meant for comparative evaluation within PDS-H itself rather than direct comparison to published TPC-H benchmark numbers. Understanding benchmark methodology helps developers evaluate dataframe tools fairly, a topic worth following on daily.dev."}}]}
```