RAG pipelines have many tunable parameters—chunk size, overlap, top-k, retrieval method—and picking them by intuition leads to poor results, especially with small models. AutoRAG addresses this by running fast, deterministic retrieval sweeps across many configuration combinations, scoring each without any LLM calls using context recall, MRR, and context word count. A demo using a fictional bank card dispute knowledge base shows that AutoRAG's winning config (chunk 150, overlap 30, top-k 3, dense) achieves the same recall as a naive baseline while cutting context size by 88%, from 1,367 to 165 words. The approach scales from a local Ollama setup to Red Hat OpenShift AI in production with no code changes, simply swapping the backend. Importantly, AutoRAG improves retrieval quality but does not make a 1B-parameter model fully reliable on its own.
