--- title: Trace Is Evals: Data Engineering for Agent Trace... description: AI agent teams need full execution traces, not just pass/fail results, to understand why agents succeed, fail, or waste tokens. The piece argues that 'trace is... canonical: https://daily.dev/posts/trace-is-evals-data-engineering-for-agent-trace-analysis-and-attribution-focxrnoyx twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Trace Is Evals: Data Engineering for Agent Trace Analysis and Attribution | daily.dev og:description: AI agent teams need full execution traces, not just pass/fail results, to understand why agents succeed, fail, or waste tokens. The piece argues that 'trace is... og:url: https://daily.dev/posts/trace-is-evals-data-engineering-for-agent-trace-analysis-and-attribution-focxrnoyx og:image: https://api.daily.dev/og/posts/FOCxRnOyx.png og:image:alt: Trace Is Evals: Data Engineering for Agent Trace Analysis and Attribution og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Trace Is Evals: Data Engineering for Agent Trace Analysis and Attribution **[Databend Blog](https://daily.dev/sources/databend)** · 11 min read · 0 upvotes · 0 comments ## Summary AI agent teams need full execution traces, not just pass/fail results, to understand why agents succeed, fail, or waste tokens. The piece argues that 'trace is evals' and walks through why agent traces differ from traditional distributed tracing: they are long-running, nested, schema-drifting JSON that can span hundreds of MB. An internal experiment comparing Claude Code, Evot, and Pi on the same task (all using DeepSeek V4 Pro) shows execution time varies wildly due to harness-model fit rather than raw model capability. The piece then describes what a data layer needs (native JSON storage, cleaning, hot-field acceleration, full-text search, path-level governance, cheap object storage, incremental computation) and presents Databend's architecture combining VARIANT storage, accelerated columns, full-text indexes, and Stream/Task for incremental aggregation as a foundation for evaluation, replay, attribution, and training data generation. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why did Claude Code take much longer to complete a task when using a third-party model instead of its own model? The slowdown came from harness-model mismatch rather than weaker model capability. In an internal experiment, Claude Code took about 15 minutes on a task using DeepSeek V4 Pro, but only about 3 minutes using its own Opus 4.6 model, because Claude Code's system prompts, tool names, and execution conventions are optimized specifically for models it was designed around. _daily.dev surfaces this kind of harness-versus-model tradeoff for teams picking an agent stack._ ### How is tracing AI agent execution different from traditional distributed tracing? Agent traces run for minutes to hours instead of seconds, evolve as spans are appended over time, and originate from prompts, tool calls, and model responses rather than stable SDK instrumentation. A single agent trace can range from hundreds of KB to hundreds of MB, with deeply nested and schema-drifting JSON, unlike the relatively stable schemas of traditional service traces. _Engineers designing agent observability pipelines track these distinctions on daily.dev._ ### What capabilities does a data warehouse need to support AI agent trace analysis? It needs native JSON storage so traces aren't forced into rigid schemas, strong JSON cleaning functions, fast access to hot fields like trace_id and model, full-text search across messages and errors, path-level governance for sensitive data, low-cost object storage for retention, and high-throughput ingestion with incremental aggregation instead of full historical rescans. _daily.dev helps teams evaluating data infrastructure choices for agent workloads stay informed._ ## Similar posts on daily.dev - [How we Build Evals for Deep Agents](https://daily.dev/posts/how-we-build-evals-for-deep-agents-jnpktgn8x) · LangChain · 19 upvotes · 0 comments - [When AI agent traces become application data](https://daily.dev/posts/when-ai-agent-traces-become-application-data-nb2duz3uk) · The New Stack · 1 upvotes · 1 comments - [Improve AI agent quality with Bits Evals](https://daily.dev/posts/improve-ai-agent-quality-with-bits-evals-111o9kt2x) · Datadog · 1 upvotes · 0 comments - [Data Engineering in the AI Era - Taming Fragmented Agent Trace Pipelines with a Unified Lakehouse](https://daily.dev/posts/data-engineering-in-the-ai-era---taming-fragmented-agent-trace-pipelines-with-a-unified-lakehouse-62xenewdp) · Databend Blog · 0 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents), [#observability](https://daily.dev/tags/observability) [View this post on daily.dev](https://daily.dev/posts/trace-is-evals-data-engineering-for-agent-trace-analysis-and-attribution-focxrnoyx) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Trace Is Evals: Data Engineering for Agent Trace Analysis and Attribution","url":"https://daily.dev/posts/trace-is-evals-data-engineering-for-agent-trace-analysis-and-attribution-focxrnoyx","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/trace-is-evals-data-engineering-for-agent-trace-analysis-and-attribution-focxrnoyx"},"datePublished":"2026-08-16T08:53:59.409Z","dateModified":"2026-08-16T08:57:32.462Z","description":"AI agent teams need full execution traces, not just pass/fail results, to understand why agents succeed, fail, or waste tokens. The piece argues that 'trace is...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/01a1a520d584384557ee0cec09b0d28a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/01a1a520d584384557ee0cec09b0d28a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Databend Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Databend Blog","logo":"https://media.daily.dev/image/upload/s--Vkqa36Ma--/f_auto,q_auto/v1786870423/logos/databend?_a=BAMAMicg0","url":"https://daily.dev/sources/databend"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/trace-is-evals-data-engineering-for-agent-trace-analysis-and-attribution-focxrnoyx","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,ai-agents,observability","timeRequired":"PT11M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Databend Blog","item":"https://daily.dev/sources/databend"},{"@type":"ListItem","position":3,"name":"Trace Is Evals: Data Engineering for Agent Trace Analysis and Attribution"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/trace-is-evals-data-engineering-for-agent-trace-analysis-and-attribution-focxrnoyx#faq","mainEntity":[{"@type":"Question","name":"Why did Claude Code take much longer to complete a task when using a third-party model instead of its own model?","acceptedAnswer":{"@type":"Answer","text":"The slowdown came from harness-model mismatch rather than weaker model capability. In an internal experiment, Claude Code took about 15 minutes on a task using DeepSeek V4 Pro, but only about 3 minutes using its own Opus 4.6 model, because Claude Code's system prompts, tool names, and execution conventions are optimized specifically for models it was designed around. daily.dev surfaces this kind of harness-versus-model tradeoff for teams picking an agent stack."}},{"@type":"Question","name":"How is tracing AI agent execution different from traditional distributed tracing?","acceptedAnswer":{"@type":"Answer","text":"Agent traces run for minutes to hours instead of seconds, evolve as spans are appended over time, and originate from prompts, tool calls, and model responses rather than stable SDK instrumentation. A single agent trace can range from hundreds of KB to hundreds of MB, with deeply nested and schema-drifting JSON, unlike the relatively stable schemas of traditional service traces. Engineers designing agent observability pipelines track these distinctions on daily.dev."}},{"@type":"Question","name":"What capabilities does a data warehouse need to support AI agent trace analysis?","acceptedAnswer":{"@type":"Answer","text":"It needs native JSON storage so traces aren't forced into rigid schemas, strong JSON cleaning functions, fast access to hot fields like trace_id and model, full-text search across messages and errors, path-level governance for sensitive data, low-cost object storage for retention, and high-throughput ingestion with incremental aggregation instead of full historical rescans. daily.dev helps teams evaluating data infrastructure choices for agent workloads stay informed."}}]} ```