--- title: How Microsoft Ships AI Agents at Enterprise Scale description: Microsoft VP Marco Casalaina shares lessons from running AI agents at enterprise scale on Microsoft Foundry, which serves 80,000+ enterprises and 20M+... canonical: https://daily.dev/posts/how-microsoft-ships-ai-agents-at-enterprise-scale-5adiyz7mp twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How Microsoft Ships AI Agents at Enterprise Scale | daily.dev og:description: Microsoft VP Marco Casalaina shares lessons from running AI agents at enterprise scale on Microsoft Foundry, which serves 80,000+ enterprises and 20M+... og:url: https://daily.dev/posts/how-microsoft-ships-ai-agents-at-enterprise-scale-5adiyz7mp og:image: https://api.daily.dev/og/posts/5ADIYz7MP.png og:image:alt: How Microsoft Ships AI Agents at Enterprise Scale og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How Microsoft Ships AI Agents at Enterprise Scale **[ByteByteGo](https://daily.dev/sources/bytebytego)** · 21 min read · 2 upvotes · 0 comments ## Summary Microsoft VP Marco Casalaina shares lessons from running AI agents at enterprise scale on Microsoft Foundry, which serves 80,000+ enterprises and 20M+ Microsoft 365 Copilot users. Key insights cover why prototypes fail in production, the five-layer agent harness (inference, runtime, observability, identity, context), the 'retrieval-as-a-subagent' pattern that replaces classic one-shot RAG with iterative multi-source retrieval, how agents need their own enterprise identities and audit trails, and rubric-based continuous evaluation with an auto-improvement loop. The core lesson: the harness around the model matters as much as the model itself. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why does classic RAG fail for production AI agents? Classic retrieval-augmented generation is a one-shot pattern that embeds a question, searches a single index, and returns top-k results with no way to recover from a bad retrieval. It breaks when the question is ambiguous, the data spans multiple sources, or the first search comes back empty, and the agent's overall quality collapses since it has no recovery path. _Teams choosing a retrieval architecture for agents can track these design tradeoffs on daily.dev._ ### What is rubric-based evaluation for AI agents? Rubric-based evaluation scores an agent against specific, use-case-tied yes-or-no checks about behaviors it should exhibit, rather than relying only on generic metrics like groundedness or task completion. For example, a restaurant-booking agent might be checked on whether it asks for missing details or verifies availability before confirming, revealing failures that generic success metrics miss. _Engineers building agent eval pipelines can follow evolving practices like this on daily.dev._ ### How does Microsoft Foundry give AI agents their own identity for auditability? Foundry extends Entra, Microsoft's enterprise identity platform, to treat agents as their own class of principal alongside employees, giving each agent a named directory entry, role assignments, and its own audit trail. This avoids anonymous or borrowed-identity actions, which are unworkable for regulated enterprises needing traceable accountability for agent actions. _Developers designing accountable, auditable agent actions can follow this pattern on daily.dev._ ## Similar posts on daily.dev - [Microsoft Foundry Now Builds, Deploys, and Governs AI Agents End to End.](https://daily.dev/posts/microsoft-foundry-now-builds-deploys-and-governs-ai-agents-end-to-end--it7nmfneb) · ITNEXT · 1 upvotes · 0 comments - [AI alone won’t change your business. The system running it will.](https://daily.dev/posts/ai-alone-won-t-change-your-business-the-system-running-it-will--hwgfpoqlc) · Microsoft · 1 upvotes · 0 comments - [How enterprise leaders are scaling AI agents across their organization](https://daily.dev/posts/how-enterprise-leaders-are-scaling-ai-agents-across-their-organization-gaahtnpqz) · databricks · 0 upvotes · 0 comments - [Building the enterprise environment for agentic AI](https://daily.dev/posts/building-the-enterprise-environment-for-agentic-ai-ymntx9dtl) · MIT Technology Review · 0 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents), [#azure](https://daily.dev/tags/azure), [#rag](https://daily.dev/tags/rag) [View this post on daily.dev](https://daily.dev/posts/how-microsoft-ships-ai-agents-at-enterprise-scale-5adiyz7mp) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How Microsoft Ships AI Agents at Enterprise Scale","url":"https://daily.dev/posts/how-microsoft-ships-ai-agents-at-enterprise-scale-5adiyz7mp","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-microsoft-ships-ai-agents-at-enterprise-scale-5adiyz7mp"},"datePublished":"2026-07-13T15:05:56.719Z","dateModified":"2026-09-14T09:15:44.789Z","description":"Microsoft VP Marco Casalaina shares lessons from running AI agents at enterprise scale on Microsoft Foundry, which serves 80,000+ enterprises and 20M+...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e0922359d52bda7c15299db3d433baf9?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e0922359d52bda7c15299db3d433baf9?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"ByteByteGo","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"ByteByteGo","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/35be29234ee14d01a9cd049c52e12753","url":"https://daily.dev/sources/bytebytego"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-microsoft-ships-ai-agents-at-enterprise-scale-5adiyz7mp","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"llm,ai-agents,azure,rag","timeRequired":"PT21M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"ByteByteGo","item":"https://daily.dev/sources/bytebytego"},{"@type":"ListItem","position":3,"name":"How Microsoft Ships AI Agents at Enterprise Scale"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-microsoft-ships-ai-agents-at-enterprise-scale-5adiyz7mp#faq","mainEntity":[{"@type":"Question","name":"Why does classic RAG fail for production AI agents?","acceptedAnswer":{"@type":"Answer","text":"Classic retrieval-augmented generation is a one-shot pattern that embeds a question, searches a single index, and returns top-k results with no way to recover from a bad retrieval. It breaks when the question is ambiguous, the data spans multiple sources, or the first search comes back empty, and the agent's overall quality collapses since it has no recovery path. Teams choosing a retrieval architecture for agents can track these design tradeoffs on daily.dev."}},{"@type":"Question","name":"What is rubric-based evaluation for AI agents?","acceptedAnswer":{"@type":"Answer","text":"Rubric-based evaluation scores an agent against specific, use-case-tied yes-or-no checks about behaviors it should exhibit, rather than relying only on generic metrics like groundedness or task completion. For example, a restaurant-booking agent might be checked on whether it asks for missing details or verifies availability before confirming, revealing failures that generic success metrics miss. Engineers building agent eval pipelines can follow evolving practices like this on daily.dev."}},{"@type":"Question","name":"How does Microsoft Foundry give AI agents their own identity for auditability?","acceptedAnswer":{"@type":"Answer","text":"Foundry extends Entra, Microsoft's enterprise identity platform, to treat agents as their own class of principal alongside employees, giving each agent a named directory entry, role assignments, and its own audit trail. This avoids anonymous or borrowed-identity actions, which are unworkable for regulated enterprises needing traceable accountability for agent actions. Developers designing accountable, auditable agent actions can follow this pattern on daily.dev."}}]} ```