--- title: "Three Years of Building Agents in Production (Part 1)" url: https://daily.dev/posts/three-years-of-building-agents-in-production-part-1--d2wjasjzx source_url: https://www.mabl.com/blog/three-years-of-building-agents-in-production-part-1 type: article source: "Mabl Engineering Blog" published: 2026-07-01T17:00:08.138Z updated: 2026-07-01T18:22:44.260Z tags: ["testing", "ai-agents", "rag", "google-gemini"] reading_time: 11 upvotes: 3 comments: 1 language: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Three Years of Building Agents in Production (Part 1) **[Mabl Engineering Blog](https://daily.dev/sources/mabl)** · 11 min read · 3 upvotes · 1 comments ## Summary A mabl engineer shares three years of lessons building AI agents for software testing in production. Starting with failed PaLM experiments in 2023, the team learned to work with LLM strengths rather than against them, pivoting to LLM-as-a-judge for GenAI Assertions, implementing RAG with Gemini 2 for context memory, and building cross-platform state management. Key insights include: early agents couldn't reliably navigate simple UIs, semantic grouping worked better than word similarity matching, static test datasets don't scale for probabilistic AI features, and rigid Chain-of-Thought prompts backfired when upgrading to Gemini 2.5 because the model reasoned about the CoT structure instead of the actual problem. The authoring agent was rewritten five times. The post ends with the team recognizing they needed a unified agent framework rather than isolated smart features. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community discussion Top comments from developers on daily.dev. **@kartiknvj** · 0 upvotes > The CoT regression on the Gemini 2.5 upgrade is a sharp catch, since baking reasoning steps into the prompt quietly turns into a liability once the newer model wants to reason its own way. I have had hardcoded scaffolding age badly across model swaps too, which is why I treat prompt structure as versioned per model now. Curious how you caught it, was it eval scores drifting or someone reading transcripts? ## Similar posts on daily.dev - [Three Years of Building Agents in Production \(Part 2\)](https://daily.dev/posts/three-years-of-building-agents-in-production-part-2--9uvhtnda1) · Mabl Engineering Blog · 2 upvotes · 1 comments - [Why I Stopped Using One Agent and Built a Multi-Agent Pipeline Instead](https://daily.dev/posts/why-i-stopped-using-one-agent-and-built-a-multi-agent-pipeline-instead-mkq8bhaeb) · Towards Data Science · 1 upvotes · 0 comments --- Tags: [#testing](https://daily.dev/tags/testing), [#ai-agents](https://daily.dev/tags/ai-agents), [#rag](https://daily.dev/tags/rag), [#google-gemini](https://daily.dev/tags/google-gemini) [View this post on daily.dev](https://daily.dev/posts/three-years-of-building-agents-in-production-part-1--d2wjasjzx)