How CockroachDB Built Vector Indexing at Scale

CockroachDB's engineering team built C-SPANN, a novel vector index designed to satisfy six strict architectural constraints of a distributed SQL database: no central coordinator, no large in-memory caches, minimal network hops, sharding compatibility, no hot spots, and real-time incremental updates. C-SPANN combines ideas from Microsoft's SPANN, SPFresh, and Google's ScaNN, using a hierarchical K-means tree stored as ordinary key-value table data inside CockroachDB. This design lets the index inherit existing distributed machinery — range splitting, rebalancing, replication — for free. Vectors are compressed using RaBitQ (94% size reduction) with a reranking step to recover accuracy. Multi-tenancy is handled via prefix columns, maintaining separate K-means trees per user and supporting geo-partitioning. The tradeoff: C-SPANN loses on raw latency vs. specialized in-memory vector databases but wins on transactional consistency, multi-tenant isolation, and multi-region deployment.