A deep-dive into anchor detection for enterprise RAG systems, presenting a three-stage pipeline: parallel keyword and embedding detectors on two structured tables (line_df and toc_df), followed by a single LLM arbiter call that ranks candidates with auditable reasoning. The article argues keyword detection should always run (free, deterministic), embeddings are optional parallel signals for vocabulary mismatch, and BM25 underperforms custom co-occurrence scoring on enterprise documents where the correct answer mentions a keyword once alongside a specific value. Three cross-table composition patterns are detailed: reason-then-match (two LLM calls for very large docs), section-weighted match (zero LLM, cheap pre-filter), and hybrid embedding with section-title boosting. Line-level cosine similarity is shown to expose which lines drive page-level embedding scores, enabling auditable re-ranking. Cross-encoder rerankers are positioned as remediation for weak upstream retrieval, not a default pipeline stage.

33m read timeFrom towardsdatascience.com
Post cover image
Table of contents
1. The anchor-detection pipeline2. Filtering on toc_df3. Filtering on line_df4. Combining the two tables5. Conclusion6. Sources and further reading
114 Impressions