--- title: A RAG That Says “Not in This Document” Has to Show Four... description: A practical framework for making RAG systems' "not found in this document" answers trustworthy rather than a dead end. The approach breaks the enterprise RAG... canonical: https://daily.dev/posts/a-rag-that-says-not-in-this-document-has-to-show-four-kinds-of-evidence-wvk7r4oy5 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: A RAG That Says “Not in This Document” Has to Show Four Kinds of Evidence | daily.dev og:description: A practical framework for making RAG systems' "not found in this document" answers trustworthy rather than a dead end. The approach breaks the enterprise RAG... og:url: https://daily.dev/posts/a-rag-that-says-not-in-this-document-has-to-show-four-kinds-of-evidence-wvk7r4oy5 og:image: https://api.daily.dev/og/posts/Wvk7R4oy5.png og:image:alt: A RAG That Says “Not in This Document” Has to Show Four Kinds of Evidence og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # A RAG That Says “Not in This Document” Has to Show Four Kinds of Evidence **[Towards Data Science](https://daily.dev/sources/tds)** · 13 min read · 1 upvotes · 0 comments ## Summary A practical framework for making RAG systems' "not found in this document" answers trustworthy rather than a dead end. The approach breaks the enterprise RAG pipeline into four bricks (parsing, question parsing, retrieval, generation), each producing a piece of evidence: parse coverage stats, an expert-validated keyword set, an exhaustive concept sweep (not top-k), and a structured no-answer response with justification and closest mentions. Demonstrated on a real case asking a World Bank commodity report about global AI electricity consumption, where the system correctly returns no answer but explains why. Also covers edge cases like partial answers, ambiguous questions, and out-of-scope queries, proposing a three-sibling schema (yes/partial/no) for RAG outputs. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can a RAG system prove that an answer really isn't in the document instead of just saying it can't find one? A defensible no-answer requires evidence from four pipeline stages: a parse coverage report (pages parsed, images OCR'd, unresolved cross-references), an expert-validated keyword set covering all synonyms and variants of the concept, a full sweep across every page (not just top-k) checking for zero, scattered, or co-located hits, and a generation step that names the closest mentions found and explains why each fails to answer the question. _daily.dev surfaces engineering deep dives like this for teams hardening RAG pipelines against silent failures._ ### Why does retrieval need to sweep every page instead of returning top-k results when checking if an answer is absent from a document? Top-k retrieval only proves the answer isn't among the highest-ranked pages, not that it's absent from the whole document. A no-answer claim needs a full sweep returning every page and line where any keyword variant appears, producing a DataFrame of evidence rows; zero rows for a concept is strong evidence of absence, while co-located rows require the model to judge whether they actually answer the question. _developers designing RAG retrieval logic can follow ongoing coverage of these patterns on daily.dev._ ## Similar posts on daily.dev - [10 Common RAG Mistakes We Keep Seeing in Production](https://daily.dev/posts/10-common-rag-mistakes-we-keep-seeing-in-production-dfkintj3j) · Towards Data Science · 1 upvotes · 0 comments - [One RAG Pipeline, Four Very Different PDFs: Same Four Bricks, Every Answer Typed and Cited](https://daily.dev/posts/one-rag-pipeline-four-very-different-pdfs-same-four-bricks-every-answer-typed-and-cited-usztm5tae) · Towards Data Science · 1 upvotes · 0 comments - [Debugging Our Docs RAG, Part 1: Evaluating a Production RAG System](https://daily.dev/posts/debugging-our-docs-rag-part-1-evaluating-a-production-rag-system-nspolfjkp) · dltHub · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#prompt-engineering](https://daily.dev/tags/prompt-engineering), [#rag](https://daily.dev/tags/rag) [View this post on daily.dev](https://daily.dev/posts/a-rag-that-says-not-in-this-document-has-to-show-four-kinds-of-evidence-wvk7r4oy5) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"A RAG That Says “Not in This Document” Has to Show Four Kinds of Evidence","url":"https://daily.dev/posts/a-rag-that-says-not-in-this-document-has-to-show-four-kinds-of-evidence-wvk7r4oy5","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/a-rag-that-says-not-in-this-document-has-to-show-four-kinds-of-evidence-wvk7r4oy5"},"datePublished":"2026-09-02T14:51:05.974Z","dateModified":"2026-09-14T08:09:16.235Z","description":"A practical framework for making RAG systems' \"not found in this document\" answers trustworthy rather than a dead end. The approach breaks the enterprise RAG...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/717218601ab3576c12a16102e96dc425?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/717218601ab3576c12a16102e96dc425?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Towards Data Science","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Towards Data Science","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/tds","url":"https://daily.dev/sources/tds"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/a-rag-that-says-not-in-this-document-has-to-show-four-kinds-of-evidence-wvk7r4oy5","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,prompt-engineering,rag","timeRequired":"PT13M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Towards Data Science","item":"https://daily.dev/sources/tds"},{"@type":"ListItem","position":3,"name":"A RAG That Says “Not in This Document” Has to Show Four Kinds of Evidence"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/a-rag-that-says-not-in-this-document-has-to-show-four-kinds-of-evidence-wvk7r4oy5#faq","mainEntity":[{"@type":"Question","name":"How can a RAG system prove that an answer really isn't in the document instead of just saying it can't find one?","acceptedAnswer":{"@type":"Answer","text":"A defensible no-answer requires evidence from four pipeline stages: a parse coverage report (pages parsed, images OCR'd, unresolved cross-references), an expert-validated keyword set covering all synonyms and variants of the concept, a full sweep across every page (not just top-k) checking for zero, scattered, or co-located hits, and a generation step that names the closest mentions found and explains why each fails to answer the question. daily.dev surfaces engineering deep dives like this for teams hardening RAG pipelines against silent failures."}},{"@type":"Question","name":"Why does retrieval need to sweep every page instead of returning top-k results when checking if an answer is absent from a document?","acceptedAnswer":{"@type":"Answer","text":"Top-k retrieval only proves the answer isn't among the highest-ranked pages, not that it's absent from the whole document. A no-answer claim needs a full sweep returning every page and line where any keyword variant appears, producing a DataFrame of evidence rows; zero rows for a concept is strong evidence of absence, while co-located rows require the model to judge whether they actually answer the question. developers designing RAG retrieval logic can follow ongoing coverage of these patterns on daily.dev."}}]} ```