The Hidden Cache You’re Paying For But Not Using
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Prompt caching stores the computed key/value attention state for a prefix of your input and bills re-reads at roughly one-tenth the normal input price. Agent loops are the ideal workload because they re-send a growing transcript every turn, making costs quadratic without caching. The correct pattern uses two cache_control breakpoints: one static on tools + system, one rolling on the last message. Anthropic requires explicit breakpoints while OpenRouter caches OpenAI/DeepSeek/Gemini automatically but still requires explicit markers for Anthropic-backed routes. The failure mode is silent — a plain string in the system field or a client that only reads prompt_tokens means you pay full price while believing you're optimized. A worked cost model shows a ~78% reduction in input costs for a 12-turn agent run. Key observability advice: instrument cache_read_input_tokens, keep your prefix byte-stable, and verify the number increases on turn two.