--- title: Why Model Routing Backfires and How to Build Agents That... description: Model routing can reduce AI inference costs by 40–85%, but switching models mid-session in agentic workflows destroys prompt cache savings, often costing more... canonical: https://daily.dev/posts/why-model-routing-backfires-and-how-to-build-agents-that-don-t-burn-your-budget-i0hiciec3 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Why Model Routing Backfires and How to Build Agents That Don’t Burn Your Budget | daily.dev og:description: Model routing can reduce AI inference costs by 40–85%, but switching models mid-session in agentic workflows destroys prompt cache savings, often costing more... og:url: https://daily.dev/posts/why-model-routing-backfires-and-how-to-build-agents-that-don-t-burn-your-budget-i0hiciec3 og:image: https://api.daily.dev/og/posts/I0HICIEC3.png og:image:alt: Why Model Routing Backfires and How to Build Agents That Don’t Burn Your Budget og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Why Model Routing Backfires and How to Build Agents That Don’t Burn Your Budget **[JFrog](https://daily.dev/sources/jfrog)** · 7 min read · 0 upvotes · 0 comments ## Summary Model routing can reduce AI inference costs by 40–85%, but switching models mid-session in agentic workflows destroys prompt cache savings, often costing more than no routing at all. Each provider's KV cache is model-specific, so a mid-session switch forces the new model to reprocess the entire conversation history at full input token rates. Common anti-patterns include enabling auto-routing mid-session via gateways like LiteLLM or OpenRouter, modifying tools or system prompts during a session, and using a separate model for history compression. Better approaches include keeping sessions short, using an orchestrator-subagent architecture where each subagent starts fresh with only the context it needs, locking session configuration at start, and summarizing state explicitly before any unavoidable model switch. The post also promotes JFrog Boost, a CLI tool that compacts terminal output to reduce context bloat in agentic coding loops. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Token Spend Out of Control? The Case for Smarter Routing](https://daily.dev/posts/token-spend-out-of-control-the-case-for-smarter-routing-jxkbskjha) · ByteByteGo · 3 upvotes · 0 comments - [Model Routing Is Simple. Until It Isn’t.](https://daily.dev/posts/model-routing-is-simple-until-it-isn-t--hk5ukjdm8) · Hugging Face · 0 upvotes · 0 comments - [LLM Routing Can Cost More Than Not Routing](https://daily.dev/posts/llm-routing-can-cost-more-than-not-routing-hujhqshkk) · Daily Dose of Data Science \| Avi Chawla \| Substack · 1 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#ai-gateway](https://daily.dev/tags/ai-gateway), [#ai-inference](https://daily.dev/tags/ai-inference), [#context-engineering](https://daily.dev/tags/context-engineering), [#llm](https://daily.dev/tags/llm) [View this post on daily.dev](https://daily.dev/posts/why-model-routing-backfires-and-how-to-build-agents-that-don-t-burn-your-budget-i0hiciec3) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Why Model Routing Backfires and How to Build Agents That Don’t Burn Your Budget","url":"https://daily.dev/posts/why-model-routing-backfires-and-how-to-build-agents-that-don-t-burn-your-budget-i0hiciec3","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/why-model-routing-backfires-and-how-to-build-agents-that-don-t-burn-your-budget-i0hiciec3"},"datePublished":"2026-07-23T17:17:45.729Z","dateModified":"2026-10-14T02:27:16.801Z","description":"Model routing can reduce AI inference costs by 40–85%, but switching models mid-session in agentic workflows destroys prompt cache savings, often costing more...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a1ca3099df1a8342de9ee2e608cab57e?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a1ca3099df1a8342de9ee2e608cab57e?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"JFrog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"JFrog","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/b11bf37102384ac9983be701b2cf7cd5","url":"https://daily.dev/sources/jfrog"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/why-model-routing-backfires-and-how-to-build-agents-that-don-t-burn-your-budget-i0hiciec3","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,ai-gateway,ai-inference,context-engineering,llm","timeRequired":"PT7M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"JFrog","item":"https://daily.dev/sources/jfrog"},{"@type":"ListItem","position":3,"name":"Why Model Routing Backfires and How to Build Agents That Don’t Burn Your Budget"}]} ```