--- title: Polars in Aggregate: Faster CSV writer, dead expression... description: A roundup of recent Polars changes spanning versions 0.20.17 through 0.20.31, covering a ~40% faster CSV writer thanks to per-datatype serializers, a new query... canonical: https://daily.dev/posts/polars-in-aggregate-faster-csv-writer-dead-expression-elimination-and-more-1sanf5rgr twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Polars in Aggregate: Faster CSV writer, dead expression elimination, and more | daily.dev og:description: A roundup of recent Polars changes spanning versions 0.20.17 through 0.20.31, covering a ~40% faster CSV writer thanks to per-datatype serializers, a new query... og:url: https://daily.dev/posts/polars-in-aggregate-faster-csv-writer-dead-expression-elimination-and-more-1sanf5rgr og:image: https://api.daily.dev/og/posts/1sAnf5rgr.png og:image:alt: Polars in Aggregate: Faster CSV writer, dead expression elimination, and more og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Polars in Aggregate: Faster CSV writer, dead expression elimination, and more **[Polars Blog](https://daily.dev/sources/polars)** · 8 min read · 0 upvotes · 0 comments ## Summary A roundup of recent Polars changes spanning versions 0.20.17 through 0.20.31, covering a ~40% faster CSV writer thanks to per-datatype serializers, a new query optimization that clusters and deduplicates sequential with_columns calls, easier ad-hoc SQL querying via new df.sql() and pl.sql() methods, native (PyArrow-free) export to PyTorch and JAX tensors, common subexpression elimination across full query plans, more ergonomic struct field manipulation with with_fields and pl.field, and an early look at a redesign of the streaming engine. The post closes with open engineering roles at Polars Cloud. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much faster is the Polars CSV writer after the per-datatype serializer optimization in version 0.20.30? The Polars CSV writer became about 40% faster after PR #15576 introduced a dedicated serializer per datatype, reducing function call overhead and type conversion. Benchmarked on 15 parquet files (~800MB) of NYC Yellow Tripdata, writing took 8.975s in version 0.20.30 versus 14.684s in version 0.20.19. The result is roughly 45x faster than pandas and 3.5x faster than PyArrow. _Track performance-focused library releases like this on daily.dev before picking your dataframe tooling._ ### What does the cluster with_columns optimization do in Polars starting with PR #16274? It automatically merges sequential with_columns calls into a single call and removes column assignments that are never used, cutting down redundant evaluations and preserving parallelism. For example, four chained with_columns calls with an overwritten intermediate column get collapsed into one call with only the final, used expressions retained. _Developers optimizing lazy query pipelines can follow changes like this via daily.dev._ ### How do I run ad-hoc SQL queries on a Polars DataFrame or LazyFrame without a separate SQL context setup? Use the new .sql() method directly on a DataFrame or LazyFrame, or the pl.sql() function under the polars namespace to query across multiple DataFrames using variable names from the global namespace. For example, df.sql("SELECT a, b FROM self") runs SQL inline, and pl.sql() can join df1 and df2 by referencing them by name in the query string. _Anyone weighing SQL versus DataFrame APIs can keep up with these interface changes on daily.dev._ ## Similar posts on daily.dev - [I Rewrote a Real Data Workflow in Polars. Pandas Didn’t Stand a Chance.](https://daily.dev/posts/i-rewrote-a-real-data-workflow-in-polars-pandas-didn-t-stand-a-chance--dpixpj7fx) · Towards Data Science · 0 upvotes · 0 comments - [Announcing Polars 1.44](https://daily.dev/posts/announcing-polars-1-44-nyx9iebfn) · Polars Blog · 1 upvotes · 0 comments - [Polars Built a GPU Hook for NVIDIA. It Works on the Apple Silicon GPU Too](https://daily.dev/posts/polars-built-a-gpu-hook-for-nvidia-it-works-on-the-apple-silicon-gpu-too-n91scp2q0) · Medium · 0 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#sql](https://daily.dev/tags/sql), [#polars](https://daily.dev/tags/polars) [View this post on daily.dev](https://daily.dev/posts/polars-in-aggregate-faster-csv-writer-dead-expression-elimination-and-more-1sanf5rgr) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Polars in Aggregate: Faster CSV writer, dead expression elimination, and more","url":"https://daily.dev/posts/polars-in-aggregate-faster-csv-writer-dead-expression-elimination-and-more-1sanf5rgr","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/polars-in-aggregate-faster-csv-writer-dead-expression-elimination-and-more-1sanf5rgr"},"datePublished":"2026-08-16T08:53:15.090Z","dateModified":"2026-08-16T08:53:47.789Z","description":"A roundup of recent Polars changes spanning versions 0.20.17 through 0.20.31, covering a ~40% faster CSV writer thanks to per-datatype serializers, a new query...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5434e7bec80a8efdf8c46b2f6f76f432?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/5434e7bec80a8efdf8c46b2f6f76f432?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Polars Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Polars Blog","logo":"https://media.daily.dev/image/upload/s--xo60W3UW--/c_limit,w_256/f_auto,q_auto/v1786870390/logos/polars?_a=BAMAMicg0","url":"https://daily.dev/sources/polars"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/polars-in-aggregate-faster-csv-writer-dead-expression-elimination-and-more-1sanf5rgr","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,sql,polars","timeRequired":"PT8M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Polars Blog","item":"https://daily.dev/sources/polars"},{"@type":"ListItem","position":3,"name":"Polars in Aggregate: Faster CSV writer, dead expression elimination, and more"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/polars-in-aggregate-faster-csv-writer-dead-expression-elimination-and-more-1sanf5rgr#faq","mainEntity":[{"@type":"Question","name":"How much faster is the Polars CSV writer after the per-datatype serializer optimization in version 0.20.30?","acceptedAnswer":{"@type":"Answer","text":"The Polars CSV writer became about 40% faster after PR #15576 introduced a dedicated serializer per datatype, reducing function call overhead and type conversion. Benchmarked on 15 parquet files (~800MB) of NYC Yellow Tripdata, writing took 8.975s in version 0.20.30 versus 14.684s in version 0.20.19. The result is roughly 45x faster than pandas and 3.5x faster than PyArrow. Track performance-focused library releases like this on daily.dev before picking your dataframe tooling."}},{"@type":"Question","name":"What does the cluster with_columns optimization do in Polars starting with PR #16274?","acceptedAnswer":{"@type":"Answer","text":"It automatically merges sequential with_columns calls into a single call and removes column assignments that are never used, cutting down redundant evaluations and preserving parallelism. For example, four chained with_columns calls with an overwritten intermediate column get collapsed into one call with only the final, used expressions retained. Developers optimizing lazy query pipelines can follow changes like this via daily.dev."}},{"@type":"Question","name":"How do I run ad-hoc SQL queries on a Polars DataFrame or LazyFrame without a separate SQL context setup?","acceptedAnswer":{"@type":"Answer","text":"Use the new .sql() method directly on a DataFrame or LazyFrame, or the pl.sql() function under the polars namespace to query across multiple DataFrames using variable names from the global namespace. For example, df.sql(\"SELECT a, b FROM self\") runs SQL inline, and pl.sql() can join df1 and df2 by referencing them by name in the query string. Anyone weighing SQL versus DataFrame APIs can keep up with these interface changes on daily.dev."}}]} ```