--- title: Microsoft’s AI Coding Agent Testing Tool is Open Source description: Microsoft open-sourced Scope, a self-hosted testing platform for evaluating coding agents like GitHub Copilot and Claude Code under the MIT license. It runs... canonical: https://daily.dev/posts/microsoft-s-ai-coding-agent-testing-tool-is-open-source-02yqtjm0d twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Microsoft’s AI Coding Agent Testing Tool is Open Source | daily.dev og:description: Microsoft open-sourced Scope, a self-hosted testing platform for evaluating coding agents like GitHub Copilot and Claude Code under the MIT license. It runs... og:url: https://daily.dev/posts/microsoft-s-ai-coding-agent-testing-tool-is-open-source-02yqtjm0d og:image: https://api.daily.dev/og/posts/02YQtJM0d.png og:image:alt: Microsoft’s AI Coding Agent Testing Tool is Open Source og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Microsoft’s AI Coding Agent Testing Tool is Open Source **[It's Foss](https://daily.dev/sources/itsfoss)** · 3 min read · 1 upvotes · 1 comments ## Summary Microsoft open-sourced Scope, a self-hosted testing platform for evaluating coding agents like GitHub Copilot and Claude Code under the MIT license. It runs tasks through different agent setups and scores outcomes using an LLM-based Judge against criteria organized as a dependency graph, keeping logs, snapshots, and tool-call traces for debugging. Teams maintaining SDKs, docs, or MCP servers can use it to spot where agents stumble, swap configurations, and rerun tasks for comparison. There's no hosted version; local setup requires Docker, Node 22, pnpm, mkcert, GitHub CLI, a GitHub Copilot entitlement, and model provider credentials, with Kubernetes/Helm available for shared deployments. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is Microsoft's Scope tool for testing AI coding agents? Scope is an open-source, MIT-licensed platform Microsoft built to evaluate how coding agents like GitHub Copilot and Claude Code perform on tasks. It runs the same task through different agents, scores results with an LLM-based Judge against dependency-graph criteria, and keeps logs, snapshots, and tool-call traces so failed checks can be traced back through each step. Human review of conclusions is still recommended. _Teams comparing agent reliability can track tools like Scope on daily.dev as the space evolves._ ### What do I need installed to self-host Microsoft Scope locally? A local Scope setup requires Docker with Compose v2, Node 22, pnpm, mkcert, and the GitHub CLI. The default worker also needs an active GitHub Copilot entitlement, and the Judge service needs credentials for a model provider. There is no hosted version, so running it yourself is the only option, though Kubernetes with a Helm chart supports shared deployments. _Developers evaluating self-hosted dev tooling can follow setup requirements like these on daily.dev._ ### How does Scope score whether an AI coding agent succeeded at a task? Success criteria form a dependency graph where one check can gate another, such as verifying a Node.js server returns JSON from a GET /health route. An LLM-based Judge service scores each run against those criteria, though Microsoft notes it can make mistakes and important conclusions still warrant human review. Versioned profiles let users swap agents, models, or MCP servers and rerun for comparison. _Anyone building agent evaluation workflows can keep tabs on tools like this via daily.dev._ ## Community discussion Top comments from developers on daily.dev. **@ahmetozel** · 0 upvotes > The dependency graph for success criteria is useful because a missing prerequisite should change how later checks are interpreted. If the agent never starts the server, a downstream /health response check is blocked by that failure; it is not independent evidence of another defect. > > I would keep those states distinct in comparisons between versioned profiles and attach the failed prerequisite to each blocked criterion. For executable requirements such as status code and JSON shape, a deterministic probe can also provide the Judge with observed facts. That makes the trace explain where the... ## Similar posts on daily.dev - [Building AI agents with the GitHub Copilot SDK](https://daily.dev/posts/building-ai-agents-with-the-github-copilot-sdk-tauv44gc4) · InfoWorld · 1 upvotes · 0 comments - [You don't need to build your own agent harness](https://daily.dev/posts/you-don-t-need-to-build-your-own-agent-harness-lqcnyprbc) · The Art of Simplicity · 1 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#github](https://daily.dev/tags/github), [#mcp](https://daily.dev/tags/mcp), [#claude-code](https://daily.dev/tags/claude-code) [View this post on daily.dev](https://daily.dev/posts/microsoft-s-ai-coding-agent-testing-tool-is-open-source-02yqtjm0d) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Microsoft’s AI Coding Agent Testing Tool is Open Source","url":"https://daily.dev/posts/microsoft-s-ai-coding-agent-testing-tool-is-open-source-02yqtjm0d","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/microsoft-s-ai-coding-agent-testing-tool-is-open-source-02yqtjm0d"},"datePublished":"2026-10-20T08:48:29.565Z","dateModified":"2026-10-20T08:49:31.376Z","description":"Microsoft open-sourced Scope, a self-hosted testing platform for evaluating coding agents like GitHub Copilot and Claude Code under the MIT license. It runs...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f550d1854bd645eb5c8626af7869350a?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f550d1854bd645eb5c8626af7869350a?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"It's Foss","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"It's Foss","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/582e3faed71842bfb9d3840a787d6700","url":"https://daily.dev/sources/itsfoss"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/microsoft-s-ai-coding-agent-testing-tool-is-open-source-02yqtjm0d","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"ai-agents,github,mcp,claude-code","timeRequired":"PT3M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"It's Foss","item":"https://daily.dev/sources/itsfoss"},{"@type":"ListItem","position":3,"name":"Microsoft’s AI Coding Agent Testing Tool is Open Source"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/microsoft-s-ai-coding-agent-testing-tool-is-open-source-02yqtjm0d","comment":[{"@type":"Comment","text":"The dependency graph for success criteria is useful because a missing prerequisite should change how later checks are interpreted. If the agent never starts the server, a downstream /health response check is blocked by that failure; it is not independent evidence of another defect.\nI would keep those states distinct in comparisons between versioned profiles and attach the failed prerequisite to each blocked criterion. For executable requirements such as status code and JSON shape, a deterministic probe can also provide the Judge with observed facts. That makes the trace explain where the workflow first diverged, rather than just how many checks ended red.","datePublished":"2026-10-20T09:01:55.218Z","url":"https://daily.dev/posts/02YQtJM0d#c-9BYYtgAcQ","author":{"@type":"Person","name":"Ahmet Özel","url":"https://daily.dev/ahmetozel","image":"https://avatars.githubusercontent.com/u/70992231?v=4"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/microsoft-s-ai-coding-agent-testing-tool-is-open-source-02yqtjm0d#faq","mainEntity":[{"@type":"Question","name":"What is Microsoft's Scope tool for testing AI coding agents?","acceptedAnswer":{"@type":"Answer","text":"Scope is an open-source, MIT-licensed platform Microsoft built to evaluate how coding agents like GitHub Copilot and Claude Code perform on tasks. It runs the same task through different agents, scores results with an LLM-based Judge against dependency-graph criteria, and keeps logs, snapshots, and tool-call traces so failed checks can be traced back through each step. Human review of conclusions is still recommended. Teams comparing agent reliability can track tools like Scope on daily.dev as the space evolves."}},{"@type":"Question","name":"What do I need installed to self-host Microsoft Scope locally?","acceptedAnswer":{"@type":"Answer","text":"A local Scope setup requires Docker with Compose v2, Node 22, pnpm, mkcert, and the GitHub CLI. The default worker also needs an active GitHub Copilot entitlement, and the Judge service needs credentials for a model provider. There is no hosted version, so running it yourself is the only option, though Kubernetes with a Helm chart supports shared deployments. Developers evaluating self-hosted dev tooling can follow setup requirements like these on daily.dev."}},{"@type":"Question","name":"How does Scope score whether an AI coding agent succeeded at a task?","acceptedAnswer":{"@type":"Answer","text":"Success criteria form a dependency graph where one check can gate another, such as verifying a Node.js server returns JSON from a GET /health route. An LLM-based Judge service scores each run against those criteria, though Microsoft notes it can make mistakes and important conclusions still warrant human review. Versioned profiles let users swap agents, models, or MCP servers and rerun for comparison. Anyone building agent evaluation workflows can keep tabs on tools like this via daily.dev."}}]} ```