Covers two related topics on efficient LLM inference. First, a newsletter promotes a three-part RAG systems course on preloading knowledge into a model's KV cache instead of retrieving it per query, covering naive preloading, compressed/modular/trained preloading, cost economics, and effective context length limits. Second, a sponsored deep dive covers TrueForge, an open-source MIT-licensed agent harness from TrueFoundry that reduces agent token usage by deferring tool schema loading, offloading large tool responses to disk, using subagents for context isolation, running data joins via generated code (Code Mode), and compacting conversation history. Benchmarks against Claude Managed Agents and deepagents-cli show TrueForge reaching similar task success using roughly 25-40% of the tokens and finishing faster.
Table of contents
Preloading knowledge into a model instead of retrieving itHow to cut agent tokens by 2.7x (using an open harness)Questions this post answers
Why does an AI agent using the same model sometimes cost three times more to run the same task?
The cost difference comes from the harness code wrapped around the model, not the model itself. If a tool returns a large payload, like a 50,000-token JSON response, and nothing removes it from the conversation, the model re-reads it on every subsequent step, so one response read fifteen more times becomes 800,000 billed input tokens for a single tool call. Anyone debugging runaway agent costs can follow harness engineering discussions like this on daily.dev.
How much can an open-source agent harness reduce LLM token usage compared to Claude Managed Agents or deepagents-cli?
TrueForge, an MIT-licensed open agent harness from TrueFoundry, reached the same task success rate using about 40% of the tokens Claude Managed Agents used and under a quarter of what deepagents-cli used, on 14 tasks from DevRev's Enterprise-Bench. It also finished in about 40 minutes per run versus 63 and 64 minutes for the other two. Developers weighing agent runtimes can track harness benchmarks like this one on daily.dev.
How did LangChain improve deepagents-cli's score on Terminal Bench 2.0 without changing the model?
LangChain raised deepagents-cli from 52.8% to 66.5% on Terminal Bench 2.0 while keeping the model pinned to gpt-5.2-codex, moving it from outside the top 30 to rank 5. The improvement came entirely from changes to the harness layer controlling context, tool calls, and state, not from the underlying model. Teams tuning agent performance beyond the model can follow harness-level benchmarks on daily.dev.