--- title: Medium | daily.dev description: A developer built a fully local RAG assistant that handles mixed document types (PDF, DOCX, CSV, Excel, images via OCR, URLs, raw text) without any cloud API.... canonical: https://daily.dev/posts/medium-kz1oa1nub twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Medium | daily.dev og:description: A developer built a fully local RAG assistant that handles mixed document types (PDF, DOCX, CSV, Excel, images via OCR, URLs, raw text) without any cloud API.... og:url: https://daily.dev/posts/medium-kz1oa1nub og:image: https://api.daily.dev/og/posts/kz1OA1nUb.png og:image:alt: Medium og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Medium **[Medium](https://daily.dev/sources/medium_js)** · 10 min read · 1 upvotes · 0 comments ## Summary A developer built a fully local RAG assistant that handles mixed document types (PDF, DOCX, CSV, Excel, images via OCR, URLs, raw text) without any cloud API. The system uses Ollama for local LLM and embedding inference, LangChain for document loading and retrieval plumbing, and FAISS as the vector store. Key engineering decisions include: overlapping 800-character chunks with metadata for citations, an over-fetch-then-rerank retrieval pipeline (fetch 24, filter by relevance score, keep top 6), history-aware query rewriting for follow-up questions, and a self-check loop that verifies answers are grounded in retrieved context before returning them. All parameters are configurable via a .env file, and the index persists to disk. The project is open source on GitHub. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I make a RAG system handle follow-up questions that lack context, like 'what about custom orders?' Rewrite the question using chat history before embedding it, so a vague follow-up like 'what about custom orders?' becomes a self-contained query such as 'What is the return policy for custom orders as mentioned in the documents?' This rewritten question is what gets embedded and searched, letting vector retrieval work even though it has no memory of its own. _daily.dev surfaces practical RAG techniques like history-aware query rewriting for developers building conversational document assistants._ ### What chunk size and overlap should I use when splitting documents for a RAG pipeline with FAISS 800-character chunks with 150 characters of overlap, using a recursive splitter that prefers breaking on paragraphs and sentences before cutting mid-word, worked well for a local FAISS-based assistant. The overlap prevents sentences that straddle a chunk boundary from being split ineffectively across two chunks, trading a small amount of extra storage for better retrieval recall. _developers tuning retrieval quality can track chunking and overlap tips like these on daily.dev._ ### How do you stop a RAG assistant from confidently making up answers not supported by retrieved documents Add a self-check loop after generation: a second model call verifies whether the answer is actually grounded in the retrieved chunks, and if not, the prompt is tightened and the answer regenerated, capped at two attempts to avoid endless loops. Generation temperature is kept low, around 0.1, to favor faithfulness over creativity, and the system returns its best attempt with a caveat if grounding still fails. _anyone building trustworthy document Q&A tools can follow grounding techniques like this self-check loop via daily.dev._ ## Similar posts on daily.dev - [Local RAG Without the Cloud: Private Document AI Setup](https://daily.dev/posts/local-rag-without-the-cloud-private-document-ai-setup-mqpsbepfw) · SitePoint · 2 upvotes · 2 comments - [How to Build a RAG Q&A AI Agent for Your Documents Using LangChain v1](https://daily.dev/posts/how-to-build-a-rag-q-a-ai-agent-for-your-documents-using-langchain-v1-e6ncpwv5h) · freeCodeCamp · 4 upvotes · 0 comments - [Vector Search Using Ollama for Retrieval-Augmented Generation \(RAG\)](https://daily.dev/posts/vector-search-using-ollama-for-retrieval-augmented-generation-rag--z8lvz9yho) · PyImageSearch · 0 upvotes · 0 comments - [From zero to a RAG system: successes and failures](https://daily.dev/posts/from-zero-to-a-rag-system-successes-and-failures-kqbclvwwu) · Hacker News · 0 upvotes · 0 comments - [Untitled](https://daily.dev/posts/untitled-1xksj4ty0) · SitePoint · 0 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#rag](https://daily.dev/tags/rag), [#langchain](https://daily.dev/tags/langchain), [#ollama](https://daily.dev/tags/ollama) [View this post on daily.dev](https://daily.dev/posts/medium-kz1oa1nub) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Medium","url":"https://daily.dev/posts/medium-kz1oa1nub","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/medium-kz1oa1nub"},"datePublished":"2026-07-18T05:17:51.381Z","dateModified":"2026-09-14T07:37:40.437Z","description":"A developer built a fully local RAG assistant that handles mixed document types (PDF, DOCX, CSV, Excel, images via OCR, URLs, raw text) without any cloud API....","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a790b3e0d6a701785fc6ebe1ae05f5bc?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a790b3e0d6a701785fc6ebe1ae05f5bc?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/medium-kz1oa1nub","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"python,rag,langchain,ollama","timeRequired":"PT10M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"Medium"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/medium-kz1oa1nub#faq","mainEntity":[{"@type":"Question","name":"How can I make a RAG system handle follow-up questions that lack context, like 'what about custom orders?'","acceptedAnswer":{"@type":"Answer","text":"Rewrite the question using chat history before embedding it, so a vague follow-up like 'what about custom orders?' becomes a self-contained query such as 'What is the return policy for custom orders as mentioned in the documents?' This rewritten question is what gets embedded and searched, letting vector retrieval work even though it has no memory of its own. daily.dev surfaces practical RAG techniques like history-aware query rewriting for developers building conversational document assistants."}},{"@type":"Question","name":"What chunk size and overlap should I use when splitting documents for a RAG pipeline with FAISS","acceptedAnswer":{"@type":"Answer","text":"800-character chunks with 150 characters of overlap, using a recursive splitter that prefers breaking on paragraphs and sentences before cutting mid-word, worked well for a local FAISS-based assistant. The overlap prevents sentences that straddle a chunk boundary from being split ineffectively across two chunks, trading a small amount of extra storage for better retrieval recall. developers tuning retrieval quality can track chunking and overlap tips like these on daily.dev."}},{"@type":"Question","name":"How do you stop a RAG assistant from confidently making up answers not supported by retrieved documents","acceptedAnswer":{"@type":"Answer","text":"Add a self-check loop after generation: a second model call verifies whether the answer is actually grounded in the retrieved chunks, and if not, the prompt is tightened and the answer regenerated, capped at two attempts to avoid endless loops. Generation temperature is kept low, around 0.1, to favor faithfulness over creativity, and the system returns its best attempt with a caveat if grounding still fails. anyone building trustworthy document Q&A tools can follow grounding techniques like this self-check loop via daily.dev."}}]} ```