--- title: Preloading Knowledge Into a Model Instead of Retrieving It description: Covers two related topics on efficient LLM inference. First, a newsletter promotes a three-part RAG systems course on preloading knowledge into a model's KV... canonical: https://daily.dev/posts/preloading-knowledge-into-a-model-instead-of-retrieving-it-nzwsexx0x twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Preloading Knowledge Into a Model Instead of Retrieving It | daily.dev og:description: Covers two related topics on efficient LLM inference. First, a newsletter promotes a three-part RAG systems course on preloading knowledge into a model's KV... og:url: https://daily.dev/posts/preloading-knowledge-into-a-model-instead-of-retrieving-it-nzwsexx0x og:image: https://api.daily.dev/og/posts/NzWSExX0X.png og:image:alt: Preloading Knowledge Into a Model Instead of Retrieving It og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Preloading Knowledge Into a Model Instead of Retrieving It **[Daily Dose of Data Science \| Avi Chawla \| Substack](https://daily.dev/sources/dailydoseofds)** · 14 min read · 1 upvotes · 0 comments ## Summary Covers two related topics on efficient LLM inference. First, a newsletter promotes a three-part RAG systems course on preloading knowledge into a model's KV cache instead of retrieving it per query, covering naive preloading, compressed/modular/trained preloading, cost economics, and effective context length limits. Second, a sponsored deep dive covers TrueForge, an open-source MIT-licensed agent harness from TrueFoundry that reduces agent token usage by deferring tool schema loading, offloading large tool responses to disk, using subagents for context isolation, running data joins via generated code (Code Mode), and compacting conversation history. Benchmarks against Claude Managed Agents and deepagents-cli show TrueForge reaching similar task success using roughly 25-40% of the tokens and finishing faster. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much can an agent harness reduce token usage compared to a naive implementation running the same model? TrueForge, an open-source MIT-licensed agent harness, reached the same task success as Claude Managed Agents and deepagents on 14 DevRev Enterprise-Bench tasks while using about 40% of Claude Managed Agents' tokens and under a quarter of deepagents' tokens. It also finished runs in about 40 minutes versus 63 and 64 minutes for the other two harnesses, using the same underlying model. _Anyone chasing runaway agent costs can track harness engineering approaches like this through daily.dev._ ### Why do most published KV cache compression methods not work for preloading a knowledge base into a model? Most published KV cache compression methods require the query at compression time, which defeats the purpose of preloading since preloading needs to process the corpus once before any query arrives. This query-agnostic requirement rules out most existing compression techniques, forcing systems that want to preload knowledge to use different approaches like modular or trained preloading instead. _Teams evaluating RAG caching strategies can follow this kind of infrastructure tradeoff on daily.dev._ ### What is Code Mode in an AI agent harness and how does it avoid exposing credentials to generated code? Code Mode lets an agent write a script that calls multiple MCP tools and joins their results in code rather than passing all the raw data through the model, so only the final output enters the model's context. Credentials never reach the sandbox: a call_tool request routes back through the harness on the server, which applies the stored credentials and returns results to the sandbox running the generated code. _Developers securing agent tool access can keep up with runtime patterns like this via daily.dev._ ## Similar posts on daily.dev - [The Hands-on AI Engineer Playbook to Build RAG Apps for Production](https://daily.dev/posts/the-hands-on-ai-engineer-playbook-to-build-rag-apps-for-production-rsram2ncn) · Daily Dose of Data Science \| Avi Chawla \| Substack · 0 upvotes · 0 comments - [Harnesses: Eager vs. Just-in-Time](https://daily.dev/posts/harnesses-eager-vs-just-in-time-qfsluogey) · Medium · 1 upvotes · 0 comments - [A Guide to AI Inference Engineering](https://daily.dev/posts/a-guide-to-ai-inference-engineering-dlkwzf3be) · ByteByteGo · 4 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents), [#mcp](https://daily.dev/tags/mcp), [#rag](https://daily.dev/tags/rag), [#ai-inference](https://daily.dev/tags/ai-inference) [View this post on daily.dev](https://daily.dev/posts/preloading-knowledge-into-a-model-instead-of-retrieving-it-nzwsexx0x) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Preloading Knowledge Into a Model Instead of Retrieving It","url":"https://daily.dev/posts/preloading-knowledge-into-a-model-instead-of-retrieving-it-nzwsexx0x","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/preloading-knowledge-into-a-model-instead-of-retrieving-it-nzwsexx0x"},"datePublished":"2026-08-24T17:18:56.280Z","dateModified":"2026-09-13T19:33:31.391Z","description":"Covers two related topics on efficient LLM inference. First, a newsletter promotes a three-part RAG systems course on preloading knowledge into a model's KV...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e32f3e744a8e04c7627ccf21ac4cfa25?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e32f3e744a8e04c7627ccf21ac4cfa25?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Daily Dose of Data Science | Avi Chawla | Substack","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Daily Dose of Data Science | Avi Chawla | Substack","logo":"https://media.daily.dev/image/upload/s--4IHQgTOw--/f_auto/v1710503712/logos/dailydoseofds","url":"https://daily.dev/sources/dailydoseofds"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/preloading-knowledge-into-a-model-instead-of-retrieving-it-nzwsexx0x","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,ai-agents,mcp,rag,ai-inference","timeRequired":"PT14M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Daily Dose of Data Science | Avi Chawla | Substack","item":"https://daily.dev/sources/dailydoseofds"},{"@type":"ListItem","position":3,"name":"Preloading Knowledge Into a Model Instead of Retrieving It"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/preloading-knowledge-into-a-model-instead-of-retrieving-it-nzwsexx0x#faq","mainEntity":[{"@type":"Question","name":"How much can an agent harness reduce token usage compared to a naive implementation running the same model?","acceptedAnswer":{"@type":"Answer","text":"TrueForge, an open-source MIT-licensed agent harness, reached the same task success as Claude Managed Agents and deepagents on 14 DevRev Enterprise-Bench tasks while using about 40% of Claude Managed Agents' tokens and under a quarter of deepagents' tokens. It also finished runs in about 40 minutes versus 63 and 64 minutes for the other two harnesses, using the same underlying model. Anyone chasing runaway agent costs can track harness engineering approaches like this through daily.dev."}},{"@type":"Question","name":"Why do most published KV cache compression methods not work for preloading a knowledge base into a model?","acceptedAnswer":{"@type":"Answer","text":"Most published KV cache compression methods require the query at compression time, which defeats the purpose of preloading since preloading needs to process the corpus once before any query arrives. This query-agnostic requirement rules out most existing compression techniques, forcing systems that want to preload knowledge to use different approaches like modular or trained preloading instead. Teams evaluating RAG caching strategies can follow this kind of infrastructure tradeoff on daily.dev."}},{"@type":"Question","name":"What is Code Mode in an AI agent harness and how does it avoid exposing credentials to generated code?","acceptedAnswer":{"@type":"Answer","text":"Code Mode lets an agent write a script that calls multiple MCP tools and joins their results in code rather than passing all the raw data through the model, so only the final output enters the model's context. Credentials never reach the sandbox: a call_tool request routes back through the harness on the server, which applies the stored credentials and returns results to the sandbox running the generated code. Developers securing agent tool access can keep up with runtime patterns like this via daily.dev."}}]} ```