--- title: Microsoft Open Sources Evals for Agent Interop Starter... description: Microsoft has open-sourced Evals for Agent Interop, a starter kit for evaluating AI agents in realistic enterprise scenarios. It includes curated scenarios,... canonical: https://daily.dev/posts/microsoft-open-sources-evals-for-agent-interop-starter-kit-to-benchmark-enterprise-ai-agents-oht1huo7t twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Microsoft Open Sources Evals for Agent Interop Starter Kit to Benchmark Enterprise AI Agents | daily.dev og:description: Microsoft has open-sourced Evals for Agent Interop, a starter kit for evaluating AI agents in realistic enterprise scenarios. It includes curated scenarios,... og:url: https://daily.dev/posts/microsoft-open-sources-evals-for-agent-interop-starter-kit-to-benchmark-enterprise-ai-agents-oht1huo7t og:image: https://api.daily.dev/og/posts/OHt1HUo7T.png og:image:alt: Microsoft Open Sources Evals for Agent Interop Starter Kit to Benchmark Enterprise AI Agents og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Microsoft Open Sources Evals for Agent Interop Starter Kit to Benchmark Enterprise AI Agents **[InfoQ](https://daily.dev/sources/infoq)** · 2 min read · 0 upvotes · 0 comments ## Summary Microsoft has open-sourced Evals for Agent Interop, a starter kit for evaluating AI agents in realistic enterprise scenarios. It includes curated scenarios, representative datasets, and an evaluation harness that measures schema adherence, tool call correctness, and AI judge assessments for qualities like coherence and helpfulness. Initially focused on email and calendar interactions, the kit ships with declarative JSON evaluation specs and a leaderboard concept for comparing agents built on different stacks. Deployed via Docker Compose, developers can clone the repo, run baseline evaluations, and customize rubrics for their specific workflows. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [Microsoft open sources AI evaluation framework for enterprise agents](https://daily.dev/posts/microsoft-open-sources-ai-evaluation-framework-for-enterprise-agents-yivfz7bll) · InfoWorld · 1 upvotes · 0 comments - [Put your AI to the Test with Microsoft.Extensions.AI.Evaluation](https://daily.dev/posts/put-your-ai-to-the-test-with-microsoft-extensions-ai-evaluation-4lc8ewfkp) · DevBlogs · 0 upvotes · 0 comments --- Tags: [#microsoft](https://daily.dev/tags/microsoft), [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents), [#docker](https://daily.dev/tags/docker) [View this post on daily.dev](https://daily.dev/posts/microsoft-open-sources-evals-for-agent-interop-starter-kit-to-benchmark-enterprise-ai-agents-oht1huo7t) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Microsoft Open Sources Evals for Agent Interop Starter Kit to Benchmark Enterprise AI Agents","url":"https://daily.dev/posts/microsoft-open-sources-evals-for-agent-interop-starter-kit-to-benchmark-enterprise-ai-agents-oht1huo7t","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/microsoft-open-sources-evals-for-agent-interop-starter-kit-to-benchmark-enterprise-ai-agents-oht1huo7t"},"datePublished":"2026-02-27T08:05:27.186Z","dateModified":"2026-08-24T07:03:56.230Z","description":"Microsoft has open-sourced Evals for Agent Interop, a starter kit for evaluating AI agents in realistic enterprise scenarios. It includes curated scenarios,...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b9071b5d8f0da65911e4d08809e12e4a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b9071b5d8f0da65911e4d08809e12e4a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"InfoQ","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"InfoQ","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/afc3bced3e1e4b188dd9127017a60e0c","url":"https://daily.dev/sources/infoq"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/microsoft-open-sources-evals-for-agent-interop-starter-kit-to-benchmark-enterprise-ai-agents-oht1huo7t","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"microsoft,llm,ai-agents,docker","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"InfoQ","item":"https://daily.dev/sources/infoq"},{"@type":"ListItem","position":3,"name":"Microsoft Open Sources Evals for Agent Interop Starter Kit to Benchmark Enterprise AI Agents"}]} ```