Every AI coding tool we tried had the same assumption: send as much context as possible.

In our production codebase, that meant sending 45,000 tokens per query — even when only ~5,000 were actually useful. We didn’t notice how inefficient this was until we saw the cost and latency impact.

We tried improving prompts and tweaking model settings, but nothing addressed the core problem:
we were optimising the model, not the context.

So we built a local retrieval layer between the codebase and the agent.

Instead of sending full files, we:

Structured code using AST-aware chunks (tree-sitter)
Combined vector search with keyword matching for better retrieval
Used a lightweight relationship layer to follow execution across files
The result: 👉 94% reduction in tokens
👉 faster responses
👉 more accurate outputs

The hardest problem wasn’t retrieval — it was knowing when retrieval was wrong.
We experimented with LLM-based scoring and threshold tuning, but a simple heuristic ended up working best.

Everything runs locally, with no data leaving the machine, and one index supports multiple AI tools.

In this talk, I’ll walk through:

What we got wrong initially
Why context matters more than model tuning
The architecture behind the system
Real benchmarks and trade-offs
The key takeaway: 👉 The biggest optimisation in AI coding isn’t the model — it’s the context.

Speakers:
- Rajkumar Sakthivel (Tesco): Rajkumar Sakthivel builds LLM infrastructure at scale and co-created Code Context Engine after his team's AI coding bill jumped from £15 to £200 in a single month.
  X/Twitter: https://x.com/rajkumarsakthi
  LinkedIn: https://www.linkedin.com/in/rajkumar-sakthivel/
  GitHub: https://github.com/rajkumarsakthivel

AI Engineer

A developer at Tesco shares how he and a colleague built a local code index to drastically reduce AI coding tool costs. They discovered that 90% of AI costs come from input tokens, not output, and that most tools send far more context than needed. Their solution: a local search layer that breaks code into meaningful chunks, runs hybrid semantic and keyword search, scores results with a simple formula, and sends only relevant snippets to the AI. Tested on FastAPI's codebase, it reduced tokens per query from 83K to 4.9K — a 94% reduction — while maintaining 90% recall accuracy. The tool also creates a shared index across multiple AI coding tools (Claude Code, Cursor, Copilot) so context is learned once and reused. It's free, open source, and called CCE.

We Cut 94% of AI Coding Tokens With a Local Code Index - Rajkumar Sakthivel, Tesco