--- title: Designing agents that outlive their models | daily.dev description: A proposed architecture for AI agents aims to keep them functioning across changes to the underlying model, rather than being tightly bound to one. Key... canonical: https://daily.dev/posts/designing-agents-that-outlive-their-models-xwvpunadl twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Designing agents that outlive their models | daily.dev og:description: A proposed architecture for AI agents aims to keep them functioning across changes to the underlying model, rather than being tightly bound to one. Key... og:url: https://daily.dev/posts/designing-agents-that-outlive-their-models-xwvpunadl og:image: https://api.daily.dev/og/posts/XwvPuNAdL.png og:image:alt: Designing agents that outlive their models og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Designing agents that outlive their models **[Collections](https://daily.dev/sources/collections)** · 4 min read · 1 upvotes · 0 comments ## Summary A proposed architecture for AI agents aims to keep them functioning across changes to the underlying model, rather than being tightly bound to one. Key principles include a stable interface independent of the model, memory and state stored outside the model, continuous knowledge accumulation across sessions, and designing explicitly for model handoff. The core argument: an agent tightly coupled to a single model is really just a wrapper, not a persistent agent. ## Content ## The core problem: short benchmarks hide long-run failures A Microsoft paper on agent reliability found something that should give builders pause: agents that look nearly perfect on short tasks fall apart as workflows get longer. On ToolQA, models that were close to flawless on 2-4 step runs dropped to 0-33% success by 16 steps. The reason is straightforward and a little uncomfortable. Every step has some probability of going wrong. Those probabilities compound. A 95% per-step success rate sounds solid until you run 16 steps and realize you're flipping a coin on whether the whole thing completes. Across 9 models tested, success rates declined consistently as dependent steps increased. One thing that didn't help: trimming context. Shortening the history actually made things worse, so blindly cutting tokens isn't a reliability fix. The practical takeaway: a benchmark pass rate is not proof an agent is production-ready. Test at the workflow lengths you actually expect to run in production, measure per-step reliability, and add checkpoints before a bad step poisons everything downstream. --- ## Five design patterns for long-running agents **1. Structure prompts for caching** Keep system instructions and tool definitions at the top of your prompt, and push dynamic memories and runtime data to the end. This makes prompt caching possible, which cuts both cost and latency on repeated calls. **2. Process memory updates asynchronously** Always reply to the user first, then handle memory writes and learning updates in the background. The harness should never slow down user interactions. **3. Preserve environment state between sessions** Don't rebuild agent environments from scratch each turn. Preserve files, installed tools, and unfinished work, then reattach agents to user-scoped environments. Rebuilding is wasteful and introduces failure points. **4. Use structured sub-agent statuses** Sub-agents should return explicit statuses: `completed`, `timed_out`, `halted`, `awaiting_approval`. Returning ambiguous prose creates real risk - a parent agent can misread a failure description as a success signal. **5. Layer security checks in order of cost** Apply cheap, auditable checks first (normalize addresses and commands), then policy rules, then human approval last. Assume safeguards will eventually be bypassed. Isolate credentials and restrict network access accordingly. --- ## What durable infrastructure actually looks like A reference implementation for a personal-loan underwriting agent illustrates what production-grade persistence requires. It uses Temporal for control-flow execution and Databricks Lakebase Postgres for queryable operational state. Temporal's Event History handles workflow replay and recovery after worker crashes - if a worker dies mid-run, the workflow picks up where it left off. Lakebase stores the application-facing state (run status, messages, evidence, review decisions) that a FastAPI layer can query with SQL. A synced table exposes Unity Catalog policy data to the running agent, and Change Data Feed can propagate Lakebase changes back to Delta history tables for audit. Because Temporal and Lakebase don't share a transaction, the implementation requires careful attention to deterministic identifiers, guarded upserts, retry policies, and idempotency patterns. Deployment considerations include OAuth credential rotation, autoscaling, and Temporal Cloud pricing. The broader point from practitioners building these systems: the most valuable agent workloads - the ones driving the most revenue - are ones that maintain state and learnings over weeks or months. That's not a throwaway sandbox problem. It's closer to running a persistent Linux environment with security, rollback, and production-grade reliability baked in from the start. --- ## What this means for builders The pattern across all of this is the same: agents fail in ways that only show up at scale and over time. Short benchmarks, ephemeral environments, and ambiguous status signals all hide problems that compound into production failures. The fixes aren't exotic - structured statuses, async memory, cached prompts, durable state, layered security. But they require treating agent infrastructure as a first-class engineering problem rather than an afterthought bolted onto a model that passed a benchmark. ## Questions this post answers ### Why do AI agents that pass benchmarks fail in production on long multi-step tasks? Per-step error rates compound across a workflow, so an agent with a 95% success rate per step can still fail most of the time over many steps. A Microsoft study found models nearly flawless on 2-4 step ToolQA tasks dropped to 0-33% success by 16 steps across 9 models tested, showing benchmark pass rates don't predict reliability at production workflow lengths. _daily.dev surfaces engineering analysis like this for teams validating agents before shipping them to production._ ### Does shortening an AI agent's context window improve its reliability on long tasks? No, trimming context actually made agent performance worse rather than better. This means blindly cutting tokens to save cost or improve speed is not a valid reliability fix for long-running agent workflows, and reliability problems need to be addressed through better checkpointing and step-level testing instead. _track findings like this on daily.dev before assuming a shortcut will fix agent reliability issues._ ### What technology stack can provide durable state and crash recovery for long-running AI agent workflows? A reference implementation for a loan-underwriting agent pairs Temporal for control-flow execution and crash recovery with Databricks Lakebase Postgres for queryable operational state. Temporal's Event History replays workflows after worker crashes, while Lakebase stores run status, messages, and review decisions queryable via SQL through a FastAPI layer, with Change Data Feed syncing changes to Delta tables for audit. _developers architecting durable agent infrastructure can find implementation patterns like this on daily.dev._ ## Similar posts on daily.dev - [Long-running Agents](https://daily.dev/posts/long-running-agents-irtro0veg) · Addy Osmani · 78 upvotes · 3 comments - [How to Build Reliable AI Agents: 5 Engineering Patterns](https://daily.dev/posts/how-to-build-reliable-ai-agents-5-engineering-patterns-gpdk9dujm) · Salesforce Engineering · 10 upvotes · 0 comments - [Long-running Agents](https://daily.dev/posts/long-running-agents-5udbvhegl) · Elevate · 33 upvotes · 0 comments - [5 Resilience Patterns for AI Agents](https://daily.dev/posts/5-resilience-patterns-for-ai-agents-oiijblpah) · The T-Shaped Dev · 1 upvotes · 0 comments - [The Multi-Agent Trap](https://daily.dev/posts/the-multi-agent-trap-gcmbn0u9m) · Towards Data Science · 1 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents), [#architecture](https://daily.dev/tags/architecture), [#mcp](https://daily.dev/tags/mcp) [View this post on daily.dev](https://daily.dev/posts/designing-agents-that-outlive-their-models-xwvpunadl) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Designing agents that outlive their models","url":"https://daily.dev/posts/designing-agents-that-outlive-their-models-xwvpunadl","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/designing-agents-that-outlive-their-models-xwvpunadl"},"datePublished":"2026-09-04T08:53:50.991Z","dateModified":"2026-09-13T19:34:03.970Z","description":"A proposed architecture for AI agents aims to keep them functioning across changes to the underlying model, rather than being tightly bound to one. Key...","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/designing-agents-that-outlive-their-models-xwvpunadl","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,ai-agents,architecture,mcp","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"Designing agents that outlive their models"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/designing-agents-that-outlive-their-models-xwvpunadl#faq","mainEntity":[{"@type":"Question","name":"Why do AI agents that pass benchmarks fail in production on long multi-step tasks?","acceptedAnswer":{"@type":"Answer","text":"Per-step error rates compound across a workflow, so an agent with a 95% success rate per step can still fail most of the time over many steps. A Microsoft study found models nearly flawless on 2-4 step ToolQA tasks dropped to 0-33% success by 16 steps across 9 models tested, showing benchmark pass rates don't predict reliability at production workflow lengths. daily.dev surfaces engineering analysis like this for teams validating agents before shipping them to production."}},{"@type":"Question","name":"Does shortening an AI agent's context window improve its reliability on long tasks?","acceptedAnswer":{"@type":"Answer","text":"No, trimming context actually made agent performance worse rather than better. This means blindly cutting tokens to save cost or improve speed is not a valid reliability fix for long-running agent workflows, and reliability problems need to be addressed through better checkpointing and step-level testing instead. track findings like this on daily.dev before assuming a shortcut will fix agent reliability issues."}},{"@type":"Question","name":"What technology stack can provide durable state and crash recovery for long-running AI agent workflows?","acceptedAnswer":{"@type":"Answer","text":"A reference implementation for a loan-underwriting agent pairs Temporal for control-flow execution and crash recovery with Databricks Lakebase Postgres for queryable operational state. Temporal's Event History replays workflows after worker crashes, while Lakebase stores run status, messages, and review decisions queryable via SQL through a FastAPI layer, with Change Data Feed syncing changes to Delta tables for audit. developers architecting durable agent infrastructure can find implementation patterns like this on daily.dev."}}]} ```