--- title: Automated agent triage with Agent Tracing and Claude... description: Sentry built a Claude Routine that connects through the Sentry MCP server to automatically triage roughly 800 overnight conversations from Seer, its AI agent... canonical: https://daily.dev/posts/automated-agent-triage-with-agent-tracing-and-claude-routines-tlqwdoxc4 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Automated agent triage with Agent Tracing and Claude Routines | daily.dev og:description: Sentry built a Claude Routine that connects through the Sentry MCP server to automatically triage roughly 800 overnight conversations from Seer, its AI agent... og:url: https://daily.dev/posts/automated-agent-triage-with-agent-tracing-and-claude-routines-tlqwdoxc4 og:image: https://api.daily.dev/og/posts/TLQwDoxC4.png og:image:alt: Automated agent triage with Agent Tracing and Claude Routines og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Automated agent triage with Agent Tracing and Claude Routines **[Sentry Engineering](https://daily.dev/sources/sentry)** · 6 min read · 0 upvotes · 0 comments ## Summary Sentry built a Claude Routine that connects through the Sentry MCP server to automatically triage roughly 800 overnight conversations from Seer, its AI agent for error triage and fixing. The routine pulls aggregate stats, samples errored tool spans and full conversations, checks whether the agent's verdict matches its reasoning, searches Linear for existing tickets, and files new ones for unresolved issues. It surfaced problems like malformed repo_name values causing repeated 'repository not found' retries and unexplained 30-56s hangs before internal errors. Sampling both successful and failed conversations proved important, since some conversations completed without tool errors yet still reached wrong conclusions. Setting it up just requires Agent Tracing, a Claude Routine prompt, and the Sentry MCP connector enabled. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I automatically triage overnight AI agent conversation logs and file bug tickets without manual review? A Claude Routine connected through the Sentry MCP can pull aggregate conversation and tool-error stats, sample spans and full conversations for reasoning quality, and search an existing Linear project before filing new tickets only for untracked issues. The prompt stays simple, telling Claude to inspect the last 24 hours of conversations, check for tool errors, sample successes too, and log findings in Linear. _Teams automating agent triage workflows can find similar Sentry and Claude Routine writeups on daily.dev._ ### Why do AI agents sometimes fail on tool calls like searching a code repository? One observed cause is malformed repo_name values, such as a monorepo subpath mistakenly appended to the real slug (like foo/bar/applications/integrations instead of foo/bar), which triggers a repository not found error until the agent retries. In one sampled period, about 21% of conversations (83 of 400 sampled) hit at least one tool error, mostly self-corrected on retry. _Debugging flaky agent tool calls gets easier when developers compare real-world failure patterns shared on daily.dev._ ### Why is it important to sample successful AI agent conversations, not just ones with tool errors, when auditing agent quality? A conversation can complete with zero tool errors and still reach the wrong final verdict, a failure mode that only surfaces when the reasoning is checked against the outcome directly. Sampling only failed runs misses this class of silent errors, so auditing both successful and failed conversations was found to matter more than expected for catching real quality issues. _Anyone building QA processes for AI agents can track practical lessons like this through daily.dev._ ## Similar posts on daily.dev - [How we built an automated debugging workflow at Sentry](https://daily.dev/posts/how-we-built-an-automated-debugging-workflow-at-sentry-cpnywm3s2) · Sentry Engineering · 1 upvotes · 0 comments - [Sentry Launches Seer Agent, Enabling Developers to Investigate Any Production Problem in Plain Language](https://daily.dev/posts/sentry-launches-seer-agent-enabling-developers-to-investigate-any-production-problem-in-plain-langu-n3n5pqxaj) · SD Times · 1 upvotes · 0 comments - [Sentry’s Seer Agent lets developers debug production issues in natural language](https://daily.dev/posts/sentry-s-seer-agent-lets-developers-debug-production-issues-in-natural-language-urdz0d6wz) · The New Stack · 1 upvotes · 0 comments - [Seer, the Sentry MCP and CLI, or your coding agent](https://daily.dev/posts/seer-the-sentry-mcp-and-cli-or-your-coding-agent-bktdrm9tf) · Sentry Engineering · 1 upvotes · 0 comments - [How to Investigate a Production Incident Using an AI Agent \(AppSignal MCP\)](https://daily.dev/posts/how-to-investigate-a-production-incident-using-an-ai-agent-appsignal-mcp--5snliyioe) · AppSignal · 0 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#observability](https://daily.dev/tags/observability), [#mcp](https://daily.dev/tags/mcp), [#claude](https://daily.dev/tags/claude) [View this post on daily.dev](https://daily.dev/posts/automated-agent-triage-with-agent-tracing-and-claude-routines-tlqwdoxc4) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Automated agent triage with Agent Tracing and Claude Routines","url":"https://daily.dev/posts/automated-agent-triage-with-agent-tracing-and-claude-routines-tlqwdoxc4","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/automated-agent-triage-with-agent-tracing-and-claude-routines-tlqwdoxc4"},"datePublished":"2026-08-13T17:18:17.640Z","dateModified":"2026-09-14T06:29:41.627Z","description":"Sentry built a Claude Routine that connects through the Sentry MCP server to automatically triage roughly 800 overnight conversations from Seer, its AI agent...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b40212e3652be84a84ba22636b2e5ca3?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b40212e3652be84a84ba22636b2e5ca3?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Sentry Engineering","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Sentry Engineering","logo":"https://media.daily.dev/image/upload/s--qqXwMThM--/f_auto/v1710505439/logos/sentry","url":"https://daily.dev/sources/sentry"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/automated-agent-triage-with-agent-tracing-and-claude-routines-tlqwdoxc4","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,observability,mcp,claude","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Sentry Engineering","item":"https://daily.dev/sources/sentry"},{"@type":"ListItem","position":3,"name":"Automated agent triage with Agent Tracing and Claude Routines"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/automated-agent-triage-with-agent-tracing-and-claude-routines-tlqwdoxc4#faq","mainEntity":[{"@type":"Question","name":"How can I automatically triage overnight AI agent conversation logs and file bug tickets without manual review?","acceptedAnswer":{"@type":"Answer","text":"A Claude Routine connected through the Sentry MCP can pull aggregate conversation and tool-error stats, sample spans and full conversations for reasoning quality, and search an existing Linear project before filing new tickets only for untracked issues. The prompt stays simple, telling Claude to inspect the last 24 hours of conversations, check for tool errors, sample successes too, and log findings in Linear. Teams automating agent triage workflows can find similar Sentry and Claude Routine writeups on daily.dev."}},{"@type":"Question","name":"Why do AI agents sometimes fail on tool calls like searching a code repository?","acceptedAnswer":{"@type":"Answer","text":"One observed cause is malformed repo_name values, such as a monorepo subpath mistakenly appended to the real slug (like foo/bar/applications/integrations instead of foo/bar), which triggers a repository not found error until the agent retries. In one sampled period, about 21% of conversations (83 of 400 sampled) hit at least one tool error, mostly self-corrected on retry. Debugging flaky agent tool calls gets easier when developers compare real-world failure patterns shared on daily.dev."}},{"@type":"Question","name":"Why is it important to sample successful AI agent conversations, not just ones with tool errors, when auditing agent quality?","acceptedAnswer":{"@type":"Answer","text":"A conversation can complete with zero tool errors and still reach the wrong final verdict, a failure mode that only surfaces when the reasoning is checked against the outcome directly. Sampling only failed runs misses this class of silent errors, so auditing both successful and failed conversations was found to matter more than expected for catching real quality issues. Anyone building QA processes for AI agents can track practical lessons like this through daily.dev."}}]} ```