--- title: Making agentic token costs visible in production | daily.dev description: Token costs in agentic AI systems compound across tool definitions, session history, and retrieval loops in ways that are hard to detect without proper... canonical: https://daily.dev/posts/making-agentic-token-costs-visible-in-production-fejyo9qbg twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Making agentic token costs visible in production | daily.dev og:description: Token costs in agentic AI systems compound across tool definitions, session history, and retrieval loops in ways that are hard to detect without proper... og:url: https://daily.dev/posts/making-agentic-token-costs-visible-in-production-fejyo9qbg og:image: https://api.daily.dev/og/posts/FeJyo9qbG.png og:image:alt: Making agentic token costs visible in production og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Making agentic token costs visible in production **[Datadog](https://daily.dev/sources/datadog)** · 10 min read · 2 upvotes · 0 comments ## Summary Token costs in agentic AI systems compound across tool definitions, session history, and retrieval loops in ways that are hard to detect without proper instrumentation. Token usage per request more than doubled year-over-year for median customers. Key cost drivers include loading full tool catalogs on every call, unbounded session history growth in multi-turn agents, and redundant retrieval loops. Practical mitigations include trimming tool catalogs to task-relevant subsets, using progressive schema discovery, enabling prompt caching, implementing sliding window or summarization-based history compression, and deduplicating retrieved chunks. Visibility requires span-level traces capturing input/output token splits per LLM call, tool invocation, and retrieval step — not just aggregate dashboards. Once visible, token budgets should be assigned per agent or service with anomaly-detection monitors, and constraints encoded directly into agent configuration. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How can I reduce token costs from large tool catalogs in an agentic AI system using MCP or function calling? Load only the tool subset relevant to the current user intent instead of registering the full catalog on every call, or use progressive discovery by loading tool names and descriptions first and fetching full schemas on demand. Prompt caching also helps for tools that must load every call, since it avoids reprocessing unchanged schema tokens, though dynamic pruning mid-session can invalidate the key-value cache and offset savings. _Teams tuning agent tool catalogs for cost can track these techniques as they build out on daily.dev._ ### Should I use a sliding window or summarization to control token growth from session history in a multi-turn agent? It depends on whether early context stays relevant: a task-focused agent where early turns become irrelevant quickly suits a sliding window that keeps only the N most recent messages, while a coding assistant that must remember earlier decisions is better served by history summarization, which compresses older turns instead of dropping them outright. The signal that either is working is a token growth curve that flattens over a session. _Engineers weighing windowing versus summarization for agent memory can keep tabs on these tradeoffs via daily.dev._ ### How much did token usage per request grow year over year for AI agents according to Datadog's monitoring data? Token usage per request more than doubled for median customers year over year and quadrupled for 90th-percentile power users among Datadog-monitored services, based on Datadog's State of AI Engineering research. This growth is attributed to sprawling tool sets and larger context windows used during agent development bleeding into production costs. _Anyone budgeting for rising agentic token spend can follow this kind of cost data on daily.dev._ ## Similar posts on daily.dev - [Identifying Token Costs Hiding in Your Agentic Loop](https://daily.dev/posts/identifying-token-costs-hiding-in-your-agentic-loop-xdycs70yb) · Machine Learning Mastery · 3 upvotes · 1 comments - [AI Cost Visibility: How to Track and Optimize Token Spend Before the Invoice Arrives](https://daily.dev/posts/ai-cost-visibility-how-to-track-and-optimize-token-spend-before-the-invoice-arrives-133h7kulz) · Telerik · 1 upvotes · 1 comments - [Why your AI pipeline costs 10x more after the demo](https://daily.dev/posts/why-your-ai-pipeline-costs-10x-more-after-the-demo-loavt2qiu) · The New Stack · 0 upvotes · 0 comments - [AI cost observability: A practical guide to understanding and managing LLM spend](https://daily.dev/posts/ai-cost-observability-a-practical-guide-to-understanding-and-managing-llm-spend-8pc0rjuej) · portkey · 0 upvotes · 0 comments - [Why token cost is hidden by design in most agent UIs](https://daily.dev/posts/why-token-cost-is-hidden-by-design-in-most-agent-uis-hekzq90vu) · Medium · 1 upvotes · 0 comments --- Tags: [#devops](https://daily.dev/tags/devops), [#ai-agents](https://daily.dev/tags/ai-agents), [#finops](https://daily.dev/tags/finops), [#llm-observability](https://daily.dev/tags/llm-observability) [View this post on daily.dev](https://daily.dev/posts/making-agentic-token-costs-visible-in-production-fejyo9qbg) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Making agentic token costs visible in production","url":"https://daily.dev/posts/making-agentic-token-costs-visible-in-production-fejyo9qbg","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/making-agentic-token-costs-visible-in-production-fejyo9qbg"},"datePublished":"2026-07-11T14:57:06.646Z","dateModified":"2026-09-14T06:11:41.384Z","description":"Token costs in agentic AI systems compound across tool definitions, session history, and retrieval loops in ways that are hard to detect without proper...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/ac26510548aa180d701ef8e5149dd6e9?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/ac26510548aa180d701ef8e5149dd6e9?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Datadog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Datadog","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/8f9d7e3e4bd64bd7b9db9b2ea83e0c6a","url":"https://daily.dev/sources/datadog"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/making-agentic-token-costs-visible-in-production-fejyo9qbg","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"devops,ai-agents,finops,llm-observability","timeRequired":"PT10M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Datadog","item":"https://daily.dev/sources/datadog"},{"@type":"ListItem","position":3,"name":"Making agentic token costs visible in production"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/making-agentic-token-costs-visible-in-production-fejyo9qbg#faq","mainEntity":[{"@type":"Question","name":"How can I reduce token costs from large tool catalogs in an agentic AI system using MCP or function calling?","acceptedAnswer":{"@type":"Answer","text":"Load only the tool subset relevant to the current user intent instead of registering the full catalog on every call, or use progressive discovery by loading tool names and descriptions first and fetching full schemas on demand. Prompt caching also helps for tools that must load every call, since it avoids reprocessing unchanged schema tokens, though dynamic pruning mid-session can invalidate the key-value cache and offset savings. Teams tuning agent tool catalogs for cost can track these techniques as they build out on daily.dev."}},{"@type":"Question","name":"Should I use a sliding window or summarization to control token growth from session history in a multi-turn agent?","acceptedAnswer":{"@type":"Answer","text":"It depends on whether early context stays relevant: a task-focused agent where early turns become irrelevant quickly suits a sliding window that keeps only the N most recent messages, while a coding assistant that must remember earlier decisions is better served by history summarization, which compresses older turns instead of dropping them outright. The signal that either is working is a token growth curve that flattens over a session. Engineers weighing windowing versus summarization for agent memory can keep tabs on these tradeoffs via daily.dev."}},{"@type":"Question","name":"How much did token usage per request grow year over year for AI agents according to Datadog's monitoring data?","acceptedAnswer":{"@type":"Answer","text":"Token usage per request more than doubled for median customers year over year and quadrupled for 90th-percentile power users among Datadog-monitored services, based on Datadog's State of AI Engineering research. This growth is attributed to sprawling tool sets and larger context windows used during agent development bleeding into production costs. Anyone budgeting for rising agentic token spend can follow this kind of cost data on daily.dev."}}]} ```