Pure vector search in RAG pipelines fails on queries requiring exact token matches — like specific error codes, version numbers, or feature flag names — because embedding models are approximation engines that collapse small but critical distinctions. The solution is hybrid retrieval: running BM25 and vector search in parallel, then fusing their ranked lists using Reciprocal Rank Fusion (RRF), which avoids score normalization by operating on rank positions alone. The article walks through three query types (semantic, exact-match, hybrid) with concrete RRF score tables showing where each retriever succeeds and fails. It also covers production implementation in Elasticsearch using the native RRF retriever API, tuning parameters (rank constant k, kNN candidates), and an optional cross-encoder reranking stage for final relevance gains on a small candidate set.

18m read timeFrom infoq.com
Post cover image
Table of contents
Where Vector-Only RAG Pipelines BreakThe Problem Is That Embeddings Are Approximation EnginesBM25 Provides Precision Where Embeddings ApproximateHybrid Search with Reciprocal Rank FusionProduction Implementation of ElasticsearchHybrid Query with RRFConclusionAbout the Author
39 Impressions