Thinking of ACE? We Can Do It with Fewer Tokens
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
IBM Research compares ALTK-Evolve and ACE (Agentic Context Engineering), two systems that let LLM agents learn from their own past trajectories without weight updates. Both systems avoid compressing learned lessons into summaries, instead maintaining countable guidelines or playbooks. The key difference is delivery: ACE injects its full playbook on every inference step, while ALTK-Evolve selectively retrieves only the guidelines relevant to the current task and model capability. On the AppWorld benchmark with 168 tasks, ALTK-Evolve achieves 89.3 TGC vs ACE's 80.4 on DeepSeek-V3.2 at ~40% of ACE's token cost (263K vs 634K tokens/task). On the weaker gpt-oss-120b model, accuracy is roughly tied (56.0 vs 54.8 TGC) but ALTK-Evolve uses about one-seventh the tokens (116K vs 777K). The post argues that calibrated delivery — matching the volume of injected guidelines to what a given model can actually absorb — is the key to both efficiency and accuracy gains.
Table of contents
What we agree onWhere we differWhy it mattersSame lessons, different deliveryLinked artifacts / referencesMethod notesQuestions this post answers
How does ALTK-Evolve compare to ACE in token usage and accuracy on the AppWorld benchmark?
On AppWorld test_normal (168 tasks) with DeepSeek-V3.2, ALTK-Evolve achieves 89.3 TGC vs ACE's 80.4 while using 263K tokens per task compared to ACE's 634K — roughly 40% of the cost. On the weaker gpt-oss-120b model, accuracy is nearly tied (56.0 vs 54.8 TGC) but ALTK-Evolve uses only 116K tokens versus ACE's 777K, about one-seventh the cost. Teams optimizing LLM agent inference costs track benchmarks like these on daily.dev.
What is the difference between ACE and ALTK-Evolve in how they deliver learned lessons to an agent at inference time?
ACE injects its full playbook on every inference step regardless of model or task. ALTK-Evolve treats delivery as a dial: it sends a small fixed core of high-support guidelines plus a handful selected per task via cosine or LLM-guided retrieval, or the full consolidated set when the model has sufficient capacity. This selective delivery is the primary source of ALTK-Evolve's token savings. Developers building agentic systems find the latest research on memory and retrieval strategies on daily.dev.
Why does injecting fewer guidelines help weaker LLMs in agentic memory systems?
A large context overwhelms a weaker model rather than helping it. On gpt-oss-120b, ACE's full playbook injection (777K tokens/task) gives it an edge on Easy and Medium tasks but falls behind on Hard tasks, where curated retrieval of ~29 guidelines (116K tokens/task) lets the model focus on the right lesson rather than wade through all of them — and Hard tasks decide the aggregate score. Engineers choosing between agentic memory architectures for different model tiers follow this research on daily.dev.