--- title: "Agent drift is real and your unit tests won’t catch it" url: https://daily.dev/posts/agent-drift-is-real-and-your-unit-tests-won-t-catch-it-uhezym5n4 source_url: https://allthingsopen.org/articles/agent-drift-open-source-arksim-ai-agent-testing type: article source: "All Things Open" published: 2026-05-26T07:17:55.462Z updated: 2026-05-26T07:18:26.578Z tags: ["python", "llm"] reading_time: 11 upvotes: 0 comments: 0 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Agent drift is real and your unit tests won’t catch it **[All Things Open](https://daily.dev/sources/allthingsopen)** · 11 min read · 0 upvotes · 0 comments ## Summary Agent drift describes the gap between how an AI agent behaves in testing versus production, where real users apply pressure that exposes scope creep, hallucination, and policy violations. ArkSim is an open source Python tool that addresses this by running profile-driven synthetic users against your agent in multi-turn conversations, then scoring each turn across helpfulness, coherence, relevance, faithfulness, and goal completion. The tool supports Python and HTTP connectors, generates HTML reports with named failure types, and integrates into CI/CD via threshold gates that exit with code 1 on failure. Three bundled examples cover scope enforcement (insurance), faithfulness under changing data (e-commerce), and goal completion (open-ended assistants). ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Test Agents Using Agents](https://daily.dev/posts/test-agents-using-agents-lya7ynb5u) · Daily Dose of Data Science \| Avi Chawla \| Substack · 2 upvotes · 0 comments - [Announcing User Simulation in ADK Evaluation](https://daily.dev/posts/announcing-user-simulation-in-adk-evaluation-hgcgnjkrw) · Google Developers · 0 upvotes · 0 comments --- Tags: [#python](https://daily.dev/tags/python), [#llm](https://daily.dev/tags/llm) [View this post on daily.dev](https://daily.dev/posts/agent-drift-is-real-and-your-unit-tests-won-t-catch-it-uhezym5n4)