--- title: How I Boosted a Local SQL Agent’s Accuracy from 13% to... description: A developer shares how they improved a local SQL agent's accuracy from 13% to 72% for a private agricultural management system running on a consumer GPU... canonical: https://daily.dev/posts/how-i-boosted-a-local-sql-agent-s-accuracy-from-13-to-72-in-production-whtfqeqxg twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How I Boosted a Local SQL Agent’s Accuracy from 13% to 72% in Production | daily.dev og:description: A developer shares how they improved a local SQL agent's accuracy from 13% to 72% for a private agricultural management system running on a consumer GPU... og:url: https://daily.dev/posts/how-i-boosted-a-local-sql-agent-s-accuracy-from-13-to-72-in-production-whtfqeqxg og:image: https://api.daily.dev/og/posts/WHtfQeqXg.png og:image:alt: How I Boosted a Local SQL Agent’s Accuracy from 13% to 72% in Production og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How I Boosted a Local SQL Agent’s Accuracy from 13% to 72% in Production **[Medium](https://daily.dev/sources/medium_js)** · 5 min read · 1 upvotes · 0 comments ## Summary A developer shares how they improved a local SQL agent's accuracy from 13% to 72% for a private agricultural management system running on a consumer GPU (NVIDIA RTX 3070). Key improvements included replacing a ReAct agent with a 5-step deterministic pipeline featuring domain classification, hybrid BM25+MMR schema retrieval, dynamic few-shot SQL examples, LLM self-correction on errors, and PostgreSQL materialized views. A hybrid RAG pipeline combining BM25 (70%) and MMR (30%) achieved 94% retrieval accuracy for unstructured documentation. A critical VRAM bug caused by inconsistent Ollama context window sizes was identified using Langfuse observability, and standardizing num_ctx=8192 across all components reduced per-turn latency from 67 seconds to 2-4 seconds. The post concludes that architectural engineering outperformed simply scaling model size (7B to 14B only reached 37% accuracy with 10x latency increase). ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does a pure ReAct agent perform badly with a 7B local LLM on text-to-SQL tasks? A pure ReAct agent with schema inspection tools achieved only 13% accuracy on a real-world agricultural Golden Set when using a 7B model, because the model entered infinite reasoning loops, selected irrelevant tables, and generated invalid SQL columns by failing to inspect the schema before executing queries. Replacing it with a deterministic 5-step pipeline (domain classification, schema retrieval and pruning, dynamic few-shot examples, self-correcting generation, and materialized views) raised accuracy to 72%. _Anyone weighing agent frameworks against controlled pipelines for small local models can compare real approaches on daily.dev._ ### Why is my Ollama model reloading into VRAM repeatedly and causing latency spikes? Ollama treats each distinct num_ctx (context window) value as a separate model instance, so if different parts of an application initialize the same model with varying context sizes, it repeatedly unloads and reloads the weights in VRAM on every state transition. This caused a 30-second delay per call in one deployment; standardizing num_ctx to 8192 across all components fixed it, stabilizing latency to 2-4 seconds per turn. _Developers debugging local LLM latency issues can track fixes like this on daily.dev before hitting the same wall._ ### Does scaling a local LLM from 7B to 14B parameters improve text-to-SQL accuracy? Not necessarily: scaling from a 7B to a 14B parameter model only reached 37% accuracy while increasing latency by 10x, compared to 72% accuracy achieved on the 7B model after re-engineering the context and pipeline architecture. This suggests architecture improvements matter more than raw model size for structured query generation on consumer hardware. _Teams deciding between bigger models or better pipelines can weigh trade-offs like this via daily.dev._ ## Similar posts on daily.dev - [How we made a SQL query optimization agent 59% more accurate using autoresearch and LLM Observability](https://daily.dev/posts/how-we-made-a-sql-query-optimization-agent-59-more-accurate-using-autoresearch-and-llm-observabilit-gosjqm3v3) · Datadog · 1 upvotes · 0 comments - [Can a Local LLM Run My AI Assistant?](https://daily.dev/posts/can-a-local-llm-run-my-ai-assistant--k8pjpamza) · Towards Data Science · 0 upvotes · 0 comments - [The Agentic Analytics Benchmark: Measuring model accuracy and efficiency in analytical agents](https://daily.dev/posts/the-agentic-analytics-benchmark-measuring-model-accuracy-and-efficiency-in-analytical-agents-81a3conc5) · ClickHouse · 0 upvotes · 0 comments - [I almost upgraded my GPU to run larger local LLMs, but this 8B model proved I didn't have to](https://daily.dev/posts/i-almost-upgraded-my-gpu-to-run-larger-local-llms-but-this-8b-model-proved-i-didn-t-have-to-gc24e505l) · XDA Developers · 1 upvotes · 0 comments --- Tags: [#postgresql](https://daily.dev/tags/postgresql), [#rag](https://daily.dev/tags/rag), [#langchain](https://daily.dev/tags/langchain), [#local-ai](https://daily.dev/tags/local-ai) [View this post on daily.dev](https://daily.dev/posts/how-i-boosted-a-local-sql-agent-s-accuracy-from-13-to-72-in-production-whtfqeqxg) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How I Boosted a Local SQL Agent’s Accuracy from 13% to 72% in Production","url":"https://daily.dev/posts/how-i-boosted-a-local-sql-agent-s-accuracy-from-13-to-72-in-production-whtfqeqxg","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-i-boosted-a-local-sql-agent-s-accuracy-from-13-to-72-in-production-whtfqeqxg"},"datePublished":"2026-07-26T22:00:15.163Z","dateModified":"2026-09-14T06:26:38.892Z","description":"A developer shares how they improved a local SQL agent's accuracy from 13% to 72% for a private agricultural management system running on a consumer GPU...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/c851e245c62f17a42288134d8bf9c1cf?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/c851e245c62f17a42288134d8bf9c1cf?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-i-boosted-a-local-sql-agent-s-accuracy-from-13-to-72-in-production-whtfqeqxg","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"postgresql,rag,langchain,local-ai","timeRequired":"PT5M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"How I Boosted a Local SQL Agent’s Accuracy from 13% to 72% in Production"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-i-boosted-a-local-sql-agent-s-accuracy-from-13-to-72-in-production-whtfqeqxg#faq","mainEntity":[{"@type":"Question","name":"Why does a pure ReAct agent perform badly with a 7B local LLM on text-to-SQL tasks?","acceptedAnswer":{"@type":"Answer","text":"A pure ReAct agent with schema inspection tools achieved only 13% accuracy on a real-world agricultural Golden Set when using a 7B model, because the model entered infinite reasoning loops, selected irrelevant tables, and generated invalid SQL columns by failing to inspect the schema before executing queries. Replacing it with a deterministic 5-step pipeline (domain classification, schema retrieval and pruning, dynamic few-shot examples, self-correcting generation, and materialized views) raised accuracy to 72%. Anyone weighing agent frameworks against controlled pipelines for small local models can compare real approaches on daily.dev."}},{"@type":"Question","name":"Why is my Ollama model reloading into VRAM repeatedly and causing latency spikes?","acceptedAnswer":{"@type":"Answer","text":"Ollama treats each distinct num_ctx (context window) value as a separate model instance, so if different parts of an application initialize the same model with varying context sizes, it repeatedly unloads and reloads the weights in VRAM on every state transition. This caused a 30-second delay per call in one deployment; standardizing num_ctx to 8192 across all components fixed it, stabilizing latency to 2-4 seconds per turn. Developers debugging local LLM latency issues can track fixes like this on daily.dev before hitting the same wall."}},{"@type":"Question","name":"Does scaling a local LLM from 7B to 14B parameters improve text-to-SQL accuracy?","acceptedAnswer":{"@type":"Answer","text":"Not necessarily: scaling from a 7B to a 14B parameter model only reached 37% accuracy while increasing latency by 10x, compared to 72% accuracy achieved on the 7B model after re-engineering the context and pipeline architecture. This suggests architecture improvements matter more than raw model size for structured query generation on consumer hardware. Teams deciding between bigger models or better pipelines can weigh trade-offs like this via daily.dev."}}]} ```