<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/how-the-claude-code-team-thinks-about-verification-fwye26i9d" -->

---
title: How the Claude Code team thinks about verification
description: Verification loops let the Claude Code team run many Claude agents in parallel instead of checking every change by hand. About 90% of their code is written and...
canonical: https://daily.dev/posts/how-the-claude-code-team-thinks-about-verification-fwye26i9d
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: How the Claude Code team thinks about verification | daily.dev
og:description: Verification loops let the Claude Code team run many Claude agents in parallel instead of checking every change by hand. About 90% of their code is written and...
og:url: https://daily.dev/posts/how-the-claude-code-team-thinks-about-verification-fwye26i9d
og:image: https://api.daily.dev/og/posts/FwYE26i9d.png
og:image:alt: How the Claude Code team thinks about verification
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# How the Claude Code team thinks about verification

**[ClaudeDevs](https://daily.dev/sources/claudedevs)** · 11 min read · 0 upvotes · 0 comments

## Summary

Verification loops let the Claude Code team run many Claude agents in parallel instead of checking every change by hand. About 90% of their code is written and reviewed by Claude. A separate grader checking work against a rubric improved task success by up to 10 points over standard prompting. The team recommends three test types: end-to-end tests of real user flows, property tests that generate thousands of random inputs with tools like fast-check, Hypothesis or proptest, and differential tests comparing old versus new systems during migrations. They keep a 39-item reliability checklist, gate commits with a /code-review or /verify requirement, and stop Claude after three failed fix attempts to bring in a human.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://claude.dev/blog/how-the-claude-code-team-thinks-about-verification>

## Questions this post answers

### What types of tests work best for verifying AI coding agent output?

End-to-end tests are the highest priority because they are hard to fool and have clear pass or fail criteria. Property tests, using libraries like fast-check for JS/TS, Hypothesis for Python, or proptest for Rust, generate thousands of random inputs to catch edge cases agents miss. Differential tests compare old versus new system outputs on the same inputs, which is especially useful during large-scale migrations.

_Teams building agent-driven pipelines can track emerging testing patterns like these on daily.dev._

### How much does adding a grader rubric improve an AI coding agent's task success rate?

A separate grader that checks an agent's work against a rubric improved task success by up to 10 points over a standard prompting loop in internal testing around structured file generation, with the largest gains showing up on the hardest problems.

_Anyone tuning agent verification loops can follow results like this on daily.dev before adopting them._

### How does the Claude Code team prevent AI agents from gaming their own code review checks?

Review rules live in a REVIEW.md file read only from the base or default branch, never from the pull request's own changes, so a change cannot modify the check that judges it. Reviews are also tied to specific commits; if someone pushes new code while a review runs, that review result is discarded as no longer useful.

_Developers designing tamper-resistant review gates can track these safeguards on daily.dev._

## Community take

How the wider developer community reacted, aggregated from 2 discussions and 146 comments across hackernews (as of 2026-11-17).

**TL;DR:** Discussion is dominated by skepticism that this is unsustainable, token-maximizing advice from a company that profits from token usage, with heavy debate over whether cheaper deterministic tools (Playwright, linters, scripts) would achieve the same verification results for far less cost.

**Sentiment:** 15% positive · 25% mixed · 60% skeptical

**The case for**

- Some argue tokens are already cheap and getting cheaper, making token-heavy verification loops a reasonable tradeoff for the productivity gained.
- A few commenters value capturing invariants in natural language as more decoupled from implementation than hardcoded deterministic tests.
- Some see value in learning from how a team operating at extreme PR-merge scale handles verification, even skeptically.

**The pushback**

- Many argue deterministic tools like Playwright, linters, and static analysis would achieve equal or better verification for near-zero marginal cost compared to token-burning agent-driven checks.
- Commenters see an inherent conflict of interest: a company that sells tokens recommending token-heavy practices.
- Several doubt the claimed scale (2000+ PRs/month) reflects meaningful, well-reviewed work rather than low-quality 'slop'.
- Some worry that at this scale no human can meaningfully review code, which is itself seen as risky.
- A few note Claude Code itself has reliability and bugginess issues, undermining the credibility of its team's verification advice.

**By community**

- hackernews (skeptical): Most commenters frame the verification approach as an expensive, self-serving justification for heavy token usage, arguing cheaper deterministic tooling already solves the same problems, though a vocal minority defends the practices as reasonable given falling inference costs.

**Hottest debate:** Whether spending large amounts of tokens on agent-driven verification is justified versus relying on cheap, deterministic tools like Playwright and linters.

**Open questions**

- Is the claimed 2000+ PRs/month actually producing quality, well-understood work or just high-volume low-value output?
- How will per-task verification costs scale as token usage and model demand grow, and will 'near-free' tokens remain true?
- Can deterministic tooling fully replace agent-driven verification, or are there cases where natural-language invariants are genuinely more valuable?

**Highlights**

> This article feels like the result of having unlimited tokens at your disposal. It’s got some very expensive practices, where near-free practices would produce the same or better results. There’s a long section in the middle where the article describes per-change verification via skill files and “ask Claude to use the chrome-for-Claude tool” when checking if changes have introduced regressions. But there’s already tools which do this without spending tokens tokens. Playwright will achieve equal quality but deterministic results across the same verification attempts. It is write-once-use-often, so has a marginal cost near zero.
> — [OtherShrezzing on hackernews · 16 comments](https://news.ycombinator.com/item?id=50530087)

> It's not the per-run costs I worry about, it's the costs of churn. As soon as Oracle Tests fail, the LLM has three options: 1. Rewrite the test - This undoes the benefit of the test in the first place  2. Implement workarounds and quirks to match the oracle, often at the expense of recreating buggy behaviour  3. Implement both corrected behaviour and quirks-mode behaviour to meet the oracle, raise further jobs to formalise the switch over and change of behaviour. ( And this "change of behaviour" might be a few pixels out of place! Or it could be a much more significant change. When trying to meet a goal, LLMs will often treat these with the same zeal. ) If you're working interactively, you can catch it and steer it to what's best for the feature. If you're fully automating development, then you're at the mercy of whatever the model does at that moment. Oracle-based testing has its place, but I don't think it's good for large scale automated testing. Also, you're talking about $500/release 2-3 times a week. That's fine, that's less than the cost of an employee. Anthropic and others are talking about how to scale to merge 2000-3000 PRs per week. At that scale, that cost per merge is $6M/month, surely a cost you'd want to minimise if you can cut them by half with better type-checking, static analysis, and other cheap deterministic tests, even if that's still a small cost of the whole project. If nothing else, it's pure waste. Our planet is (mostly metaphorically) burning, and I believe that more efficient IT processes and development practices would go a long way to helping mitigate some of that.
> — [xnorswap on hackernews](https://news.ycombinator.com/item?id=50530702)

> At 2000 PR/mo, assuming a "standard" 8 hour per day, 5 days a week workweek with no time factored for lunch, meetings, breaks, or PTO... literally all they do for 8 hours a day, 5 days a week, is submitting PRs. That works out to roughly 5 minutes to slop, review, and document a single PR, approx 92 PRs per work day. That tells me they aren't doing shit.
> — [rolosa on hackernews · 2 comments](https://news.ycombinator.com/item?id=50531344)

> Its almost become a meme at this point: well actually you can achieve the same using X so why waste tokens?! The reason you should use Claude and not playwright always is because the the invariant you wanna capture is at the natural language level. Like imagine I wanted to verify that Claude Code works and doesn't break. I can describe the invariant I want in natural language. Like: make sure that subagents don't block the original thread. Doing this in playwright means that you have already bought into some previously existing architecture which means your tests are now correlated to your code. Having the invariants written in natural language allows you to decouple code vs the invariant. I expect a lot of greybeard conservative programmers to keep opposing using AI here until eventually, not using AI will be seen as more risky.
> — [simianwords on hackernews · 2 comments](https://news.ycombinator.com/item?id=50530219)

> It's really interesting to me to contrast the code and publications of deepseek and anthropic over the last year. Anthropic's stuff has some ideas that can be extended or applied but everything they do assumes basically unlimited frontier model inference and so is nearly unusable to me in the form they publish it. Where deepseek prices inference the way systems programmers price latency. As a resource to be spent carefully towards specific goals, with systems in place to ensure you're within budget and getting the value you expect from it. And then this also has second order effects on how each company treats human attention. Having fable xhigh do 3-4 passes over a change can legitimately reduce the need for human attention on it in a way that deepseek flash just can't really. So then they price operator attention in a similar way. With ant nearly considering human intervention a failure mode, while ds again prices it as a budget to be designed within. And neither one is "right" speaking generally. I have my aesthetic preferences and my own constraints that make one a more appealing approach personally. But both are building towards the goals they have with consideration of the resources at their disposal, it's good engineering either way.
> — [dcss\_gardener on hackernews](https://news.ycombinator.com/item?id=50532354)

**Source threads**

- [hackernews](https://news.ycombinator.com/item?id=50529612) · 118 points · 146 comments
- [hackernews](https://news.ycombinator.com/item?id=50525965) · 1 points · 0 comments

## Similar posts on daily.dev

- [How to Improve Claude Code Performance with Automated Testing](https://daily.dev/posts/how-to-improve-claude-code-performance-with-automated-testing-xkejpz4rd) · Towards Data Science · 0 upvotes · 0 comments
- [Claude Code's output got dramatically better when I stopped letting it decide when it was done](https://daily.dev/posts/claude-code-s-output-got-dramatically-better-when-i-stopped-letting-it-decide-when-it-was-done-mln6fekdq) · XDA Developers · 0 upvotes · 1 comments
- [I'm Building Agents That Run While I Sleep](https://daily.dev/posts/i-m-building-agents-that-run-while-i-sleep-psrvg5tk0) · Hacker News · 2 upvotes · 0 comments
- [How to Make Claude Code Validate its own Work](https://daily.dev/posts/how-to-make-claude-code-validate-its-own-work-iqvgculyl) · Towards Data Science · 0 upvotes · 0 comments

---

Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#testing](https://daily.dev/tags/testing), [#claude-code](https://daily.dev/tags/claude-code), [#code-review](https://daily.dev/tags/code-review)

[View this post on daily.dev](https://daily.dev/posts/how-the-claude-code-team-thinks-about-verification-fwye26i9d)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"How the Claude Code team thinks about verification","url":"https://daily.dev/posts/how-the-claude-code-team-thinks-about-verification-fwye26i9d","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-the-claude-code-team-thinks-about-verification-fwye26i9d"},"datePublished":"2026-11-16T20:37:33.310Z","dateModified":"2026-11-17T16:08:28.580Z","description":"Verification loops let the Claude Code team run many Claude agents in parallel instead of checking every change by hand. About 90% of their code is written and...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/1b670d9016aea5c9c6dd7e5f59c91fdc?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/1b670d9016aea5c9c6dd7e5f59c91fdc?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"ClaudeDevs","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"ClaudeDevs","logo":"https://media.daily.dev/image/upload/s--wBREB-hc--/f_auto,q_auto/v1787418470/logos/claudedevs?_a=BAMAMicg0","url":"https://daily.dev/sources/claudedevs"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-the-claude-code-team-thinks-about-verification-fwye26i9d","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,testing,claude-code,code-review","timeRequired":"PT11M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"ClaudeDevs","item":"https://daily.dev/sources/claudedevs"},{"@type":"ListItem","position":3,"name":"How the Claude Code team thinks about verification"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-the-claude-code-team-thinks-about-verification-fwye26i9d#faq","mainEntity":[{"@type":"Question","name":"What types of tests work best for verifying AI coding agent output?","acceptedAnswer":{"@type":"Answer","text":"End-to-end tests are the highest priority because they are hard to fool and have clear pass or fail criteria. Property tests, using libraries like fast-check for JS/TS, Hypothesis for Python, or proptest for Rust, generate thousands of random inputs to catch edge cases agents miss. Differential tests compare old versus new system outputs on the same inputs, which is especially useful during large-scale migrations. Teams building agent-driven pipelines can track emerging testing patterns like these on daily.dev."}},{"@type":"Question","name":"How much does adding a grader rubric improve an AI coding agent's task success rate?","acceptedAnswer":{"@type":"Answer","text":"A separate grader that checks an agent's work against a rubric improved task success by up to 10 points over a standard prompting loop in internal testing around structured file generation, with the largest gains showing up on the hardest problems. Anyone tuning agent verification loops can follow results like this on daily.dev before adopting them."}},{"@type":"Question","name":"How does the Claude Code team prevent AI agents from gaming their own code review checks?","acceptedAnswer":{"@type":"Answer","text":"Review rules live in a REVIEW.md file read only from the base or default branch, never from the pull request's own changes, so a change cannot modify the check that judges it. Reviews are also tied to specific commits; if someone pushes new code while a review runs, that review result is discarded as no longer useful. Developers designing tamper-resistant review gates can track these safeguards on daily.dev."}}]}
```

