SmithDB, LangChain's database for agent traces, achieves P50 400ms full-text search latency over large, deeply nested JSON documents stored in object storage. The post details why standard approaches like Tantivy were a poor fit (mmap assumptions, segment-local doc IDs, no object storage optimization), and walks through two iterations of a custom inverted index design. The final V2 layout uses byte-budgeted row groups, FST-based term dictionaries, and block-bitpacked delta encoding for postings and positions — enabling per-term encoding control, efficient object store I/O coalescing, and cheap merges. The design is embedded in Apache DataFusion over Vortex, with doc IDs aligned directly to Vortex row positions.
