Three Years of Building Agents in Production (Part 2)

A mabl engineer shares lessons from three years building AI agents in production for software testing. After a successful demo exposed infrastructure fragility, the team paused feature development for a winter sprint to build a unified Agent Framework. The framework rests on three pillars: shared context compaction strategies to manage LLM token limits, composable shared tool definitions with a message handling layer that catches hallucinations and enforces self-correction loops, and evals-as-code with LLM-as-a-judge suites to prevent regressions. The result is an ecosystem of six agents (authoring, recovery, results analysis, failure diagnosis, application modeling, session review) that can be assembled rather than hand-crafted. Key insight: scaling AI is not about better prompts but about returning to foundational software engineering practices — reusability, error handling, and standardized testing become more critical, not less, when non-deterministic tools are involved.