--- title: Asynchronous I/O in DuckDB: Work, Thread, Work | daily.dev description: DuckDB v2.0 (scheduled for fall 2026) introduces asynchronous I/O for Parquet and CSV files, targeting remote storage scenarios like EC2/S3 setups where... canonical: https://daily.dev/posts/asynchronous-i-o-in-duckdb-work-thread-work-p1obh9kxz twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Asynchronous I/O in DuckDB: Work, Thread, Work | daily.dev og:description: DuckDB v2.0 (scheduled for fall 2026) introduces asynchronous I/O for Parquet and CSV files, targeting remote storage scenarios like EC2/S3 setups where... og:url: https://daily.dev/posts/asynchronous-i-o-in-duckdb-work-thread-work-p1obh9kxz og:image: https://api.daily.dev/og/posts/p1OBh9KxZ.png og:image:alt: Asynchronous I/O in DuckDB: Work, Thread, Work og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Asynchronous I/O in DuckDB: Work, Thread, Work **[DuckDB](https://daily.dev/sources/duckdb)** · 17 min read · 3 upvotes · 0 comments ## Summary DuckDB v2.0 (scheduled for fall 2026) introduces asynchronous I/O for Parquet and CSV files, targeting remote storage scenarios like EC2/S3 setups where synchronous reads leave network bandwidth underutilized. The implementation uses two thread pools — REGULAR workers for computation and ASYNC threads (defaulting to 4× system threads, capped at 256) for blocking I/O — combined with a read-ahead queue and memory-governed backlog control. Benchmarks on TPC-H Q6 at SF100 show Parquet queries running ~3× faster on S3 (8.2s → 2.8s), with a tuned configuration reaching ~3.7× speedup by saturating a 25 Gbit/s network. CSV sees the most dramatic gain: ~20× faster (878s → 45s) due to its row-oriented nature requiring far more data transfer. Cold local SSD reads improve ~1.5×. Concurrent query workloads jump from ~6 average busy cores to ~48 out of 64, nearly eliminating idle wait time. The read-ahead depth is configurable via `read_ahead_depth`, and memory pressure automatically throttles prefetching. Future plans include async reads for JSON and DuckDB-native formats, plus potential io_uring integration. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much faster is DuckDB v2.0 at reading Parquet files from S3 compared to synchronous I/O in v1.5.5? Asynchronous I/O in DuckDB v2.0.0-dev makes TPC-H Query 6 at SF100 run almost 3x faster on a remote Parquet file stored on S3, dropping the mean runtime from 8.230 seconds in v1.5.5 to 2.844 seconds. With tuned settings (async_threads=48, read-ahead capped at 64 jobs, adjusted HTTP retry settings), the runtime falls further to 2.227 seconds, about 3.7x faster than v1.5.5, by keeping the 25 Gbit/s network link saturated. _Track DuckDB's async I/O rollout on daily.dev when planning S3-backed analytics performance upgrades._ ### How do I configure read-ahead depth for asynchronous I/O in DuckDB? Use the read_ahead_depth configuration option, set via SET read_ahead_depth = N. A value of -1 (the default) means unlimited depth bounded by the temporary memory manager's budget, a positive N caps read-ahead at N jobs with no memory budget, and 0 disables read-ahead so each scan task only schedules I/O for its own job, behaving like synchronous access. _Developers tuning DuckDB memory and throughput tradeoffs follow config changes like this via daily.dev._ ### Does asynchronous I/O help when DuckDB reads Parquet files from local disk instead of remote storage? Yes, but the improvement is much smaller than with remote storage. On a MacBook Pro with an Apple M4 Max chip, cold local disk reads of an SF100 Parquet file were about 1.5x faster with asynchronous I/O in DuckDB v2.0.0-dev (0.883 seconds) versus synchronous I/O in v1.5.5 (1.321 seconds), because SSDs have much lower latency and higher bandwidth than EC2-to-S3 network access; hot cached runs show negligible difference. _Engineers weighing local versus remote DuckDB workloads compare these performance details on daily.dev._ ## Similar posts on daily.dev - [Why DuckDB 2.0 is faster](https://daily.dev/posts/why-duckdb-2-0-is-faster-miuefalpu) · MotherDuck · 99 upvotes · 6 comments - [Self-hosting DuckDB: the road to production](https://daily.dev/posts/self-hosting-duckdb-the-road-to-production-crsdobrlt) · MotherDuck · 1 upvotes · 0 comments - [Why DuckDB is my first choice for data processing](https://daily.dev/posts/why-duckdb-is-my-first-choice-for-data-processing-sx7gglwxx) · Hacker News · 1 upvotes · 0 comments - [Run SQL queries on local parquet and delta files using DuckDB](https://daily.dev/posts/run-sql-queries-on-local-parquet-and-delta-files-using-duckdb-0qdy9r9y9) · The Art of Simplicity · 0 upvotes · 0 comments --- Tags: [#database](https://daily.dev/tags/database), [#big-data](https://daily.dev/tags/big-data), [#duckdb](https://daily.dev/tags/duckdb) [View this post on daily.dev](https://daily.dev/posts/asynchronous-i-o-in-duckdb-work-thread-work-p1obh9kxz) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Asynchronous I/O in DuckDB: Work, Thread, Work","url":"https://daily.dev/posts/asynchronous-i-o-in-duckdb-work-thread-work-p1obh9kxz","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/asynchronous-i-o-in-duckdb-work-thread-work-p1obh9kxz"},"datePublished":"2026-07-31T09:08:09.867Z","dateModified":"2026-09-13T19:54:49.884Z","description":"DuckDB v2.0 (scheduled for fall 2026) introduces asynchronous I/O for Parquet and CSV files, targeting remote storage scenarios like EC2/S3 setups where...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/55d463b57fe39b89b4268b27e7b00132?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/55d463b57fe39b89b4268b27e7b00132?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"DuckDB","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"DuckDB","logo":"https://media.daily.dev/image/upload/s--3luD3Wei--/f_auto/v1714800008/logos/duckdb","url":"https://daily.dev/sources/duckdb"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/asynchronous-i-o-in-duckdb-work-thread-work-p1obh9kxz","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":3},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"database,big-data,duckdb","timeRequired":"PT17M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"DuckDB","item":"https://daily.dev/sources/duckdb"},{"@type":"ListItem","position":3,"name":"Asynchronous I/O in DuckDB: Work, Thread, Work"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/asynchronous-i-o-in-duckdb-work-thread-work-p1obh9kxz#faq","mainEntity":[{"@type":"Question","name":"How much faster is DuckDB v2.0 at reading Parquet files from S3 compared to synchronous I/O in v1.5.5?","acceptedAnswer":{"@type":"Answer","text":"Asynchronous I/O in DuckDB v2.0.0-dev makes TPC-H Query 6 at SF100 run almost 3x faster on a remote Parquet file stored on S3, dropping the mean runtime from 8.230 seconds in v1.5.5 to 2.844 seconds. With tuned settings (async_threads=48, read-ahead capped at 64 jobs, adjusted HTTP retry settings), the runtime falls further to 2.227 seconds, about 3.7x faster than v1.5.5, by keeping the 25 Gbit/s network link saturated. Track DuckDB's async I/O rollout on daily.dev when planning S3-backed analytics performance upgrades."}},{"@type":"Question","name":"How do I configure read-ahead depth for asynchronous I/O in DuckDB?","acceptedAnswer":{"@type":"Answer","text":"Use the read_ahead_depth configuration option, set via SET read_ahead_depth = N. A value of -1 (the default) means unlimited depth bounded by the temporary memory manager's budget, a positive N caps read-ahead at N jobs with no memory budget, and 0 disables read-ahead so each scan task only schedules I/O for its own job, behaving like synchronous access. Developers tuning DuckDB memory and throughput tradeoffs follow config changes like this via daily.dev."}},{"@type":"Question","name":"Does asynchronous I/O help when DuckDB reads Parquet files from local disk instead of remote storage?","acceptedAnswer":{"@type":"Answer","text":"Yes, but the improvement is much smaller than with remote storage. On a MacBook Pro with an Apple M4 Max chip, cold local disk reads of an SF100 Parquet file were about 1.5x faster with asynchronous I/O in DuckDB v2.0.0-dev (0.883 seconds) versus synchronous I/O in v1.5.5 (1.321 seconds), because SSDs have much lower latency and higher bandwidth than EC2-to-S3 network access; hot cached runs show negligible difference. Engineers weighing local versus remote DuckDB workloads compare these performance details on daily.dev."}}]} ```