Exact nearest neighbor search breaks down at scale due to the curse of dimensionality, making approximate nearest neighbor (ANN) indexes essential for production vector search. HNSW builds a multilayer proximity graph enabling fast greedy traversal, achieving recall@10 of 0.99 at 15,000 QPS in Elasticsearch with float32 vectors. DiskBBQ, Elasticsearch's disk-based binary quantization, compresses vectors 32x by storing them on disk while keeping the HNSW graph in memory, reaching 0.97 recall@10 at 55,000 QPS — a 3.7x throughput gain for a 0.02 recall reduction. The post covers HNSW parameters (m, ef_construction, num_candidates), scalar and product quantization trade-offs, when to use LSH vs HNSW, and concrete Elasticsearch mapping and query examples for dense_vector and semantic_text field types. A decision table guides configuration choices from under 100K to over 1B vectors.
