ClickHouse Cloud introduces multi-stage distributed query execution, a new execution model that repartitions intermediate data between query stages using exchange operators (Shuffle, Broadcast, Gather, Scatter). This removes two key bottlenecks in the previous parallel replicas model: every node having to build a full right-side hash table for joins, and high-cardinality aggregations being bottlenecked on a single coordinator. The system uses streaming TCP-based exchanges by default, with a persisted mode writing to shared object storage for fault tolerance. TPC-H SF100 benchmarks on 8 nodes show near-linear scaling for scan-heavy queries (7.4× for Q01) and 2–4.5× speedups for multi-join queries, with a 2.1× total improvement. The feature is currently experimental and available via private preview in ClickHouse Cloud. A cost-based optimizer is planned to further improve plan selection.

16m read timeFrom clickhouse.com
Post cover image
Table of contents
Scaling one query across many nodes #Why existing distributed execution was not enough #Introducing multi-stage distributed execution #TPC-H benchmark results for multi-stage distributed query execution #What’s next? #How can I use multi-stage distributed query execution? #
1.5K Impressions