Have your Iceberg Cubed, Not Sorted: Meet Qbeast, the OTree Spatial Index — Jack Vanlightly
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Qbeast, a Barcelona-based startup, introduces the OTree multidimensional spatial index for open table formats like Apache Iceberg and Delta Lake. Unlike traditional partitioning and sort-order clustering strategies, the OTree organizes data into adaptive hypercubes that subdivide automatically based on actual data distribution. Each row is mapped to a point in normalized multidimensional space, and cubes split as they fill up, creating finer granularity in dense regions and coarser granularity in sparse ones. This approach solves key problems with conventional clustering: partition granularity issues, imbalanced partitions, changing distributions, and constant layout drift requiring compaction. The OTree is stored as lightweight metadata (Puffin file in Iceberg, Delta Log tags in Delta Lake) and is invisible to query engines, which continue using standard column statistics. Writers optionally use the Qbeast module to maintain spatial locality from the first write. The result is a self-correcting, globally consistent layout index that sits between the rigidity of B-tree clustered indexes and the loose metadata approach of standard OTFs.