--- title: The Future of Evals: From LLM as a Judge to Agent as a... description: Aparna Dhinakaran from Arize AI argues that traditional LLM-as-a-judge evaluation methods are insufficient for modern agentic AI systems. As agents have grown... canonical: https://daily.dev/posts/the-future-of-evals-from-llm-as-a-judge-to-agent-as-a-judge-aparna-dhinakaran-arize-ai-7ecochtih twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: The Future of Evals: From LLM as a Judge to Agent as a Judge — Aparna Dhinakaran, Arize AI | daily.dev og:description: Aparna Dhinakaran from Arize AI argues that traditional LLM-as-a-judge evaluation methods are insufficient for modern agentic AI systems. As agents have grown... og:url: https://daily.dev/posts/the-future-of-evals-from-llm-as-a-judge-to-agent-as-a-judge-aparna-dhinakaran-arize-ai-7ecochtih og:image: https://api.daily.dev/og/posts/7ecoChtiH.png og:image:alt: The Future of Evals: From LLM as a Judge to Agent as a Judge — Aparna Dhinakaran, Arize AI og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # The Future of Evals: From LLM as a Judge to Agent as a Judge — Aparna Dhinakaran, Arize AI **[AI Engineer](https://daily.dev/sources/aidotengineer)** · 6 min read · 1 upvotes · 1 comments ## Summary Aparna Dhinakaran from Arize AI argues that traditional LLM-as-a-judge evaluation methods are insufficient for modern agentic AI systems. As agents have grown more complex — with tool calls, reasoning, memory, and long-horizon tasks — their failure modes have become fundamentally different. Fixed rubrics can't catch issues like agents looping on the same tool call or taking inefficient trajectories. The proposed solution is 'Agent as a Judge': using a long-running agent to dynamically analyze traces, discover failure patterns, and even auto-generate fixes via PRs. Arize has released a product called Signal that implements this approach. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community discussion Top comments from developers on daily.dev. **@kartiknvj** · 0 upvotes > Agent-as-judge is the missing piece. Static rubrics cannot catch an agent looping on the same tool call. [https://news.google.com/swg/ui/v1/serviceiframe?_=1789587315318&publicationId=publication-id-free](https://news.google.com/swg/ui/v1/serviceiframe?_=1789587315318&publicationId=publication-id-free) ## Similar posts on daily.dev - [Evaluating Agents via LLM-as-a-Judge in Langfuse](https://daily.dev/posts/evaluating-agents-via-llm-as-a-judge-in-langfuse-3scbxjmmw) · Niklas Heidloff · 1 upvotes · 0 comments - [Beyond LLM-as-a-judge: Establishing LLM evaluations as a foundation for trustworthy agentic AI systems](https://daily.dev/posts/beyond-llm-as-a-judge-establishing-llm-evaluations-as-a-foundation-for-trustworthy-agentic-ai-syste-m8kymtr9x) · Dynatrace · 1 upvotes · 0 comments - [The Roadmap to Mastering AI Agent Evaluation](https://daily.dev/posts/the-roadmap-to-mastering-ai-agent-evaluation-mb2gy2ef7) · Machine Learning Mastery · 3 upvotes · 1 comments - [Building Custom LLM Judges for AI Agent Accuracy](https://daily.dev/posts/building-custom-llm-judges-for-ai-agent-accuracy-phlvzjknk) · databricks · 55 upvotes · 2 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents), [#observability](https://daily.dev/tags/observability) [View this post on daily.dev](https://daily.dev/posts/the-future-of-evals-from-llm-as-a-judge-to-agent-as-a-judge-aparna-dhinakaran-arize-ai-7ecochtih) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"The Future of Evals: From LLM as a Judge to Agent as a Judge — Aparna Dhinakaran, Arize AI","url":"https://daily.dev/posts/the-future-of-evals-from-llm-as-a-judge-to-agent-as-a-judge-aparna-dhinakaran-arize-ai-7ecochtih","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/the-future-of-evals-from-llm-as-a-judge-to-agent-as-a-judge-aparna-dhinakaran-arize-ai-7ecochtih"},"datePublished":"2026-07-24T20:00:25.914Z","dateModified":"2026-07-24T21:01:33.899Z","description":"Aparna Dhinakaran from Arize AI argues that traditional LLM-as-a-judge evaluation methods are insufficient for modern agentic AI systems. As agents have grown...","image":"https://i.ytimg.com/vi/q2JrUKBMf0w/sddefault.jpg","thumbnailUrl":"https://i.ytimg.com/vi/q2JrUKBMf0w/sddefault.jpg","isAccessibleForFree":true,"articleSection":"AI Engineer","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"AI Engineer","logo":"https://media.daily.dev/image/upload/s--u5PucxNT--/f_auto/v1724338940/logos/aidotengineer","url":"https://daily.dev/sources/aidotengineer"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/the-future-of-evals-from-llm-as-a-judge-to-agent-as-a-judge-aparna-dhinakaran-arize-ai-7ecochtih","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"llm,ai-agents,observability","timeRequired":"PT6M","video":{"@type":"VideoObject","name":"The Future of Evals: From LLM as a Judge to Agent as a Judge — Aparna Dhinakaran, Arize AI","description":"Aparna Dhinakaran from Arize AI argues that traditional LLM-as-a-judge evaluation methods are insufficient for modern agentic AI systems. As agents have grown...","thumbnailUrl":"https://i.ytimg.com/vi/q2JrUKBMf0w/sddefault.jpg","uploadDate":"2026-07-24T20:00:25.914Z","duration":"PT6M","url":"https://api.daily.dev/r/7ecoChtiH","embedUrl":"https://www.youtube.com/embed/q2JrUKBMf0w"}} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"AI Engineer","item":"https://daily.dev/sources/aidotengineer"},{"@type":"ListItem","position":3,"name":"The Future of Evals: From LLM as a Judge to Agent as a Judge — Aparna Dhinakaran, Arize AI"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/the-future-of-evals-from-llm-as-a-judge-to-agent-as-a-judge-aparna-dhinakaran-arize-ai-7ecochtih","comment":[{"@type":"Comment","text":"Agent-as-judge is the missing piece. Static rubrics cannot catch an agent looping on the same tool call. https://news.google.com/swg/ui/v1/serviceiframe?_=1789587315318&publicationId=publication-id-free","datePublished":"2026-09-16T19:35:18.792Z","url":"https://daily.dev/posts/7ecoChtiH#c-GEu8wkIqw","author":{"@type":"Person","name":"kartik-nvjk","url":"https://daily.dev/kartiknvj","image":"https://media.daily.dev/image/upload/s--3gGgsVCw--/f_auto/v1781456774/avatars/avatar_TvTVeiMdkRCqWUDullFmy?_a=BAMAMiWQ0"}}]} ```