--- title: MirrorCode: What's the largest software project AI can... description: MirrorCode is a new benchmark from Epoch AI (co-developed with METR) that tests AI models on long-horizon coding tasks by having them reimplement entire... canonical: https://daily.dev/posts/mirrorcode-what-s-the-largest-software-project-ai-can-complete-on-its-own-epoch-ai-xgockomvb twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: MirrorCode: What's the largest software project AI can complete on its own? | Epoch AI | daily.dev og:description: MirrorCode is a new benchmark from Epoch AI (co-developed with METR) that tests AI models on long-horizon coding tasks by having them reimplement entire... og:url: https://daily.dev/posts/mirrorcode-what-s-the-largest-software-project-ai-can-complete-on-its-own-epoch-ai-xgockomvb og:image: https://api.daily.dev/og/posts/xgoCkomvB.png og:image:alt: MirrorCode: What's the largest software project AI can complete on its own? | Epoch AI og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # MirrorCode: What's the largest software project AI can complete on its own? | Epoch AI **[Hacker News](https://daily.dev/sources/hn)** · 4 min read · 0 upvotes · 0 comments ## Summary MirrorCode is a new benchmark from Epoch AI (co-developed with METR) that tests AI models on long-horizon coding tasks by having them reimplement entire programs end-to-end without access to the original source code. Unlike most benchmarks capped at $1–10 inference budgets, MirrorCode allows serious compute spend — one task cost $2,600 and ran for 19 days. The benchmark covers 25 target programs across Unix utilities, bioinformatics, cryptography, compression, and more, evaluated in multiple languages including Go and Ada. Claude Opus 4.7 successfully reimplemented gotree, a ~16,000-line Go bioinformatics toolkit, in 14 hours for $251 — a task estimated to take a human 2–17 weeks. The benchmark is cheat-resistant via sandboxing and held-out tests. 22 of 25 target programs are open-sourced, with 3 held as a private test set. Data contamination from pretraining is acknowledged as a caveat but is argued not to dominate results. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is the MirrorCode benchmark for AI coding? MirrorCode is a benchmark created by Epoch AI with METR that tests whether AI models can reimplement an entire program end-to-end without access to the original source code, matching outputs on held-out end-to-end tests. It spans 25 target programs across Unix utilities, bioinformatics, interpreters, cryptography, and compression, evaluated across six programming languages with large inference budgets rather than the $1-10 caps used in typical coding benchmarks. _Track new long-horizon coding benchmarks like this one on daily.dev as agentic capabilities evolve._ ### How well did Claude Opus 4.7 perform on the MirrorCode gotree reimplementation task? Claude Opus 4.7 reimplemented gotree, a bioinformatics toolkit with roughly 16,000 lines of Go code and 40+ commands, in 14 hours at a cost of $251. The best-scoring implementation passed 2000 of 2001 tests, missing only a niche edge-case for date annotation manipulation, a task estimated to take a human engineer without AI assistance 2 to 17 weeks. _Developers gauging how far AI coding agents can scale can follow benchmark results like this on daily.dev._ ### Why do most AI coding benchmarks limit inference spending to $1-10 per task? Most existing software engineering benchmarks cap inference spending around $1-10 per task even when a task would take a human weeks, which limits how much long-horizon work an AI model can attempt. MirrorCode instead provides a much larger inference budget, with one task costing $2,600 for a single run and requiring the AI to work for 19 days without human intervention, to better measure long-horizon coding capability. _Anyone comparing coding benchmark methodologies can follow how inference budgets shape reported AI capability on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 2 discussions and 81 comments across hackernews (as of 2026-09-14). **TL;DR:** The community is genuinely interested in MirrorCode's benchmark design and impressed by individual AI coding feats, but heavily skeptical that autonomous AI coding scales well in practice — most commenters share experiences of architectural drift, code bloat, and the need for constant human guidance on real projects. **Sentiment:** 30% positive · 50% mixed · 20% skeptical **The case for** - The benchmark is cheat-resistant via sandboxing and held-out tests, which commenters see as a meaningful advance in eval design. - Showing AIs the list of test inputs while keeping some hidden is praised as a clever way to define task scope fairly. - Several commenters report impressive real-world results when humans provide architecture guidance and good test harnesses alongside AI. - The reimplementation framing (with a reference binary as oracle) is noted as a smart way to give agents a definitive end-to-end signal to home in on. **The pushback** - Reimplementing existing software doesn't generalize well to building novel software where requirements are unclear and no reference implementation exists. - Data contamination from pretraining is acknowledged but commenters feel it may still significantly inflate results. - Commenters widely report that unguided AI agents produce bloated, duplicated, architecturally incoherent code that degrades over time. - The benchmark only covers plain-text output programs, excluding the vast majority of consumer-facing applications. - The benchmark measures one-shot completion, not long-term maintainability — commenters argue the harder question is whether AI can maintain coherent architecture over months of changes. - AI agents tend to fix symptoms rather than root causes, looping on the same class of bugs without recognizing when a larger refactor is needed. **By community** - hackernews (mixed): Commenters are impressed by the benchmark's design rigor but broadly skeptical that autonomous AI coding translates to real-world projects, citing consistent experiences of architectural decay, code bloat, and the indispensability of human oversight. **Hottest debate:** Whether the benchmark's reimplementation framing (with a reference binary as oracle) meaningfully predicts AI capability on novel software development, or whether it's a fundamentally different and easier problem. **Open questions** - What is the largest software system an AI can maintain with coherent architecture over a multi-year horizon, not just build once? - How much do pretraining data contamination and model familiarity with target programs inflate MirrorCode scores? - Can AI agents reliably handle projects where requirements are ambiguous and no reference implementation exists? - What harness design or workflow structure best prevents architectural degradation in long-running AI coding projects? **Highlights** > Hi HN! I'm the creator of MirrorCode, a benchmark of long-horizon SWE tasks. In a MirrorCode task, AI models are tasked with reimplementing an entire program end-to-end, without access to the original source code. AI-generated solutions must match the original program’s stdout and stderr exactly on end-to-end tests. MirrorCode’s 25 target programs span different areas of computing: Unix utilities, data serialization and query tools, bioinformatics, interpreters, static analysis, cryptography, and compression. IMO, the key tension in eval design is now: how to make tasks that are difficult for AI, but actually fair? My default expectation in 2026 when I see a coding eval with low scores is that the tasks turn out to be impossible. In MirrorCode, we leverage reimplementation of existing software as the raw material to make tasks hard. But we also manually select each of the 25 target programs, and carefully collect the end-to-end tests, so that the task is actually achievable (if extremely difficult). It's very easy for MirrorCode-style tasks to degenerate into absurd reverse-engineering: stuff you'd never do as a human engineer. e.g. one of our target programs is the brotli compression library. But we explicitly test the decompression path only (and call the task "brotlid") RFC 7932 only defines decompression: given compressed bytes, there's one correct output. In the other direction, there's no unique mapping. So the output of a compressor is governed by thousands of internal efficiency heuristics (e.g. how far back to look for repeated strings). Replicating those perfectly from a black box is extreme and pointless reverse-engineering. An aspect of MirrorCode I’m proud of: it’s the first benchmark I know of to implement a serious “security mindset”. The field fatalistically treats AI cheating as an unwinnable game of whack-a-mole. I think this is completely wrong. My views: (1) in 2026, cheat-proofing must be a first-class consideration in benchmark design, not an afterthought (2) most benchmarks CAN be made secure, by using known primitives like containers The problem is that benchmark authors aren’t even trying to be secure against human-level attackers, which AIs are. Another key differentiator of MirrorCode: AIs are clearly told the scope of the task, rather than having to guess what they'll secretly be tested on. We clarify task scope by showing AIs a list of test inputs (while also keeping some hidden to prevent AIs from cheating with a lookup table). If this sounds too easy or like giving them the answer, I assure you, you're thinking about it wrong. e.g. perfectly matching gotree's behaviour does not become easy because I show you 1,899 inputs (!) that your program will be tested on. Meanwhile, if you don't show test cases and just say "reimplement all of gotree" the task is dominated by an impossible guessing-game: what's actually in scope? The Nexus format was only loosely specified in its original publication (which we give to AIs). Among other omissions, documentation does not mention that Nexus files may contain comments. Nexus comments are between brackets [], so they may appear in the middle of a line of data, unlike code comments. gotree _generally_ handles them without complaint, but errors with comments in certain locations. I personally think guessing that comments exist, and all the ways comments must be handled, despite comments not even being mentioned in documentation, is functionally impossible. So the benchmark becomes dominated by the guessing-game, rather than actual implementation. Human engineers gradually learn the scope of inputs a program should support thanks to external feedback from users of the program, they don't just think them all up in a vacuum before publication. Yet that's what some benchmarks test. We’re releasing MirrorCode as open-source. > — [tadamcz on hackernews](https://news.ycombinator.com/item?id=48691420) > Very cool, but I'm skeptical. Over the last year, every Cursor/Claude/Codex project I've worked on devolves into shitty, grafted-on, duct-tape and bandaid'ed architecture, with "fixes" just working around broken code, and tons of duplication that lead to whack-a-mole. The system works at first, but then bugs start piling up and eventually take over. Everyone I talk to shares the same experience. Not to mention that if I, human overseer, don't course correct it many times a day, it goes off into Neverland. Yet I see comments and articles like above every week, where someone says "Yeah, I just left my agent run for a few nights, and then it made this awesome tool." There is a huge gap here, but I can't pinpoint it yet. Either I (and everyone I talk to) is Doing It Wrong, or your code is actually flawed at its core and bugs will take exponentially longer to fix and never converge, or... I just need to let my agent run for 3 months?? > — [khazhoux on hackernews · 7 comments](https://news.ycombinator.com/item?id=49158971) > What size project an AI can complete on its own is IMHO not an interesting question. The AIs and harnesses are getting good enough that quite large things can be built, primarily as a function of cost. The interesting question in my view is what is the largest system an AI can maintain over a long time horizon -- while keeping a coherent architecture that adapts and responds well to change. All the while remaining performant, scalable, and understandable to humans. I am unaware of what is the currently longest-lived vibe coded project but it cannot be more than a few years old. So to my mind, we don't have very much real data yet. > — [\_doctor\_love on hackernews · 2 comments](https://news.ycombinator.com/item?id=49159556) > IME you can get good results if: 1. You have very good tests. The agent can write them, but you have to prompt it to do that, and some human input may be needed here. 2. You have a good, modular architecture. Again, this is something that needs to be prompted and/or developed up front. Agent work can scale if you divide and conquer, so that any given task has a well-defined scope, limiting the amount of context needed to handle it well. If you let an agent just build a big pile of code, you’ll have issues. Once you have the above, you’re more likely to be able to just let the agent run. This is why you hear about things like SQLite being ported to Rust (by multiple groups!) It has a comprehensive unit test suite and a good existing architecture. Such porting projects are benefitting from already having done the upfront work that I mentioned, plus they have the original source code as a reference implementation. Because of that, most of what I do involves a lot of interaction with the agent. That’s because I’m typically working on complex existing systems, that don’t necessarily have the above properties, and agents aren’t at the point where they can handle that autonomously. > — [antonvs on hackernews · 1 comments](https://news.ycombinator.com/item?id=49159527) > It really depends on how you define "complete". It's possible the answer in "not large at all" if we're looking for high-quality, reliable, user-friendly software that works near perfectly after being written, tested, deployed completely by AI with no human intervention. People who try to get AI to do a lot of un-manned work seem to have a bit of a bias toward viewing LLM work as "good" or "good enough" in a way that they wouldn't have accepted in the pre-AI era. > — [vcryan on hackernews](https://news.ycombinator.com/item?id=49163857) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49157786) · 67 points · 79 comments - [hackernews](https://news.ycombinator.com/item?id=48691419) · 4 points · 2 comments ## Similar posts on daily.dev - [Measuring AI Ability to Complete Long Software Tasks](https://daily.dev/posts/measuring-ai-ability-to-complete-long-software-tasks-zt241vimh) · Metadata · 1 upvotes · 0 comments - [Building a C compiler with a team of parallel Claudes](https://daily.dev/posts/building-a-c-compiler-with-a-team-of-parallel-claudes-gdolsqfbx) · Hacker News · 0 upvotes · 0 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#claude](https://daily.dev/tags/claude) [View this post on daily.dev](https://daily.dev/posts/mirrorcode-what-s-the-largest-software-project-ai-can-complete-on-its-own-epoch-ai-xgockomvb) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"MirrorCode: What's the largest software project AI can complete on its own? | Epoch AI","url":"https://daily.dev/posts/mirrorcode-what-s-the-largest-software-project-ai-can-complete-on-its-own-epoch-ai-xgockomvb","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/mirrorcode-what-s-the-largest-software-project-ai-can-complete-on-its-own-epoch-ai-xgockomvb"},"datePublished":"2026-08-03T17:22:46.895Z","dateModified":"2026-09-14T07:53:22.908Z","description":"MirrorCode is a new benchmark from Epoch AI (co-developed with METR) that tests AI models on long-horizon coding tasks by having them reimplement entire...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/0687ae58106a2579aedda140610b3352?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/0687ae58106a2579aedda140610b3352?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/mirrorcode-what-s-the-largest-software-project-ai-can-complete-on-its-own-epoch-ai-xgockomvb","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,claude","timeRequired":"PT4M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"MirrorCode: What's the largest software project AI can complete on its own? | Epoch AI"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/mirrorcode-what-s-the-largest-software-project-ai-can-complete-on-its-own-epoch-ai-xgockomvb#faq","mainEntity":[{"@type":"Question","name":"What is the MirrorCode benchmark for AI coding?","acceptedAnswer":{"@type":"Answer","text":"MirrorCode is a benchmark created by Epoch AI with METR that tests whether AI models can reimplement an entire program end-to-end without access to the original source code, matching outputs on held-out end-to-end tests. It spans 25 target programs across Unix utilities, bioinformatics, interpreters, cryptography, and compression, evaluated across six programming languages with large inference budgets rather than the $1-10 caps used in typical coding benchmarks. Track new long-horizon coding benchmarks like this one on daily.dev as agentic capabilities evolve."}},{"@type":"Question","name":"How well did Claude Opus 4.7 perform on the MirrorCode gotree reimplementation task?","acceptedAnswer":{"@type":"Answer","text":"Claude Opus 4.7 reimplemented gotree, a bioinformatics toolkit with roughly 16,000 lines of Go code and 40+ commands, in 14 hours at a cost of $251. The best-scoring implementation passed 2000 of 2001 tests, missing only a niche edge-case for date annotation manipulation, a task estimated to take a human engineer without AI assistance 2 to 17 weeks. Developers gauging how far AI coding agents can scale can follow benchmark results like this on daily.dev."}},{"@type":"Question","name":"Why do most AI coding benchmarks limit inference spending to $1-10 per task?","acceptedAnswer":{"@type":"Answer","text":"Most existing software engineering benchmarks cap inference spending around $1-10 per task even when a task would take a human weeks, which limits how much long-horizon work an AI model can attempt. MirrorCode instead provides a much larger inference budget, with one task costing $2,600 for a single run and requiring the AI to work for 19 days without human intervention, to better measure long-horizon coding capability. Anyone comparing coding benchmark methodologies can follow how inference budgets shape reported AI capability on daily.dev."}}]} ```