Traditional APM tools fall short for AI agents because a single agent execution spans multiple model calls, tool invocations, sub-agent handoffs, and reasoning loops. Effective agent observability requires both aggregate dashboards and detailed traces. The OpenTelemetry gen_ai semantic conventions provide a standardized way to instrument agent systems, producing structured spans for model calls, tool executions, and agent lifecycle events. Key metrics to track include agent error rate, tool failure rate, latency percentiles, token usage per model, cost per user/tier, and cache hit rate. Sentry auto-instruments 10+ frameworks (OpenAI, Anthropic, LangChain, LangGraph, Pydantic AI, etc.) with minimal setup. Best practices include sampling AI traces at 100%, tracking cost by user not just model, monitoring tool reliability separately, and connecting AI monitoring to full-stack APM so upstream issues like slow database queries appear in the same trace as agent behavior.
