--- title: Evaluating code review agents with ReviewBench | daily.dev description: LangChain built ReviewBench, an internal benchmark for evaluating code review agents using real PR feedback from their LangSmith mono-repo. Rather than... canonical: https://daily.dev/posts/evaluating-code-review-agents-with-reviewbench-uliiub1oa twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Evaluating code review agents with ReviewBench | daily.dev og:description: LangChain built ReviewBench, an internal benchmark for evaluating code review agents using real PR feedback from their LangSmith mono-repo. Rather than... og:url: https://daily.dev/posts/evaluating-code-review-agents-with-reviewbench-uliiub1oa og:image: https://api.daily.dev/og/posts/UliiUB1oA.png og:image:alt: Evaluating code review agents with ReviewBench og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Evaluating code review agents with ReviewBench **[LangChain](https://daily.dev/sources/langchain)** · 6 min read · 1 upvotes · 0 comments ## Summary LangChain built ReviewBench, an internal benchmark for evaluating code review agents using real PR feedback from their LangSmith mono-repo. Rather than synthetic bugs, it uses curated comments from trusted reviewers, filtered through an LLM gate and manual review to isolate substantive, verifiable defects. Tasks are written in Harbor format and scored on F1 combining coverage (did the agent find the real issue?) and precision (were its findings valid?). Current models with a basic harness recover only ~30% of baseline issues. A key finding: prompting strategy matters significantly — adding a structured review prompt to one model (Luna) substantially improved its score without changing tools or the model itself, suggesting that how an agent approaches review is as important as which model powers it. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How well do current LLM-based code review agents catch real reviewer-level issues in pull requests? Current models running under a basic agent harness recover only about 30% of curated baseline issues drawn from real trusted-reviewer PR feedback. Agents tend to submit valid findings but miss many issues that require reconstructing implicit system context, such as missing tenant checks on database queries or API-parity regressions during endpoint migrations, rather than just scanning changed lines. _daily.dev surfaces benchmarks like this for teams deciding whether an ai code review agent is ready for their workflow._ ### Does a more structured prompting strategy improve an LLM code review agent's ability to find bugs, without adding new tools? Yes, a structured review prompt improved Luna's score from a weak baseline to 0.32 on a 20-task slice, higher than static-review Opus and Kimi runs on the same tasks, without giving the model any new tools or the ability to run code. The prompt instructed the agent to identify what changed, trace dependent behavior, and validate findings against callers, tests, and related implementations before submitting. _engineers tuning code review agents can track how prompting strategy shifts real-world results via daily.dev._ ### How is the ReviewBench benchmark for code review agents constructed and scored? ReviewBench has 59 tasks covering 64 baseline issues, built by filtering real reviewer comments from merged PRs in a LangSmith mono-repo through an LLM gate and manual review, then converting them into Harbor-format tasks with frozen PR context served by a local GitHub stub. Each task is scored on coverage and precision using an LLM-as-judge, combined into an F1 score. _daily.dev helps developers evaluating code review agents keep up with how benchmarks like this are built and scored._ ## Similar posts on daily.dev - [How Similarweb Evaluates Agent Reports with LangSmith](https://daily.dev/posts/how-similarweb-evaluates-agent-reports-with-langsmith-brldcch8t) · LangChain · 1 upvotes · 0 comments - [One Week of Building and Reviewing Code With LLM Agents](https://daily.dev/posts/one-week-of-building-and-reviewing-code-with-llm-agents-0bpygawo1) · RUBYLAND · 0 upvotes · 0 comments - [What we learned running the industry’s first AI code review benchmark](https://daily.dev/posts/what-we-learned-running-the-industry-s-first-ai-code-review-benchmark-rrpbuegj7) · Dev Interrupted · 0 upvotes · 0 comments - [How We Benchmark Deep Agents](https://daily.dev/posts/how-we-benchmark-deep-agents-nppzx6htk) · LangChain · 1 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#code-review](https://daily.dev/tags/code-review), [#langchain](https://daily.dev/tags/langchain) [View this post on daily.dev](https://daily.dev/posts/evaluating-code-review-agents-with-reviewbench-uliiub1oa) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Evaluating code review agents with ReviewBench","url":"https://daily.dev/posts/evaluating-code-review-agents-with-reviewbench-uliiub1oa","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/evaluating-code-review-agents-with-reviewbench-uliiub1oa"},"datePublished":"2026-07-31T17:03:02.891Z","dateModified":"2026-09-13T20:19:23.992Z","description":"LangChain built ReviewBench, an internal benchmark for evaluating code review agents using real PR feedback from their LangSmith mono-repo. Rather than...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3c3f06dc051466b319f35949d32727cf?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/3c3f06dc051466b319f35949d32727cf?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"LangChain","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"LangChain","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/0f4f2629e0724a849823f8cd0d913e13","url":"https://daily.dev/sources/langchain"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/evaluating-code-review-agents-with-reviewbench-uliiub1oa","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,code-review,langchain","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"LangChain","item":"https://daily.dev/sources/langchain"},{"@type":"ListItem","position":3,"name":"Evaluating code review agents with ReviewBench"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/evaluating-code-review-agents-with-reviewbench-uliiub1oa#faq","mainEntity":[{"@type":"Question","name":"How well do current LLM-based code review agents catch real reviewer-level issues in pull requests?","acceptedAnswer":{"@type":"Answer","text":"Current models running under a basic agent harness recover only about 30% of curated baseline issues drawn from real trusted-reviewer PR feedback. Agents tend to submit valid findings but miss many issues that require reconstructing implicit system context, such as missing tenant checks on database queries or API-parity regressions during endpoint migrations, rather than just scanning changed lines. daily.dev surfaces benchmarks like this for teams deciding whether an ai code review agent is ready for their workflow."}},{"@type":"Question","name":"Does a more structured prompting strategy improve an LLM code review agent's ability to find bugs, without adding new tools?","acceptedAnswer":{"@type":"Answer","text":"Yes, a structured review prompt improved Luna's score from a weak baseline to 0.32 on a 20-task slice, higher than static-review Opus and Kimi runs on the same tasks, without giving the model any new tools or the ability to run code. The prompt instructed the agent to identify what changed, trace dependent behavior, and validate findings against callers, tests, and related implementations before submitting. engineers tuning code review agents can track how prompting strategy shifts real-world results via daily.dev."}},{"@type":"Question","name":"How is the ReviewBench benchmark for code review agents constructed and scored?","acceptedAnswer":{"@type":"Answer","text":"ReviewBench has 59 tasks covering 64 baseline issues, built by filtering real reviewer comments from merged PRs in a LangSmith mono-repo through an LLM gate and manual review, then converting them into Harbor-format tasks with frozen PR context served by a local GitHub stub. Each task is scored on coverage and precision using an LLM-as-judge, combined into an F1 score. daily.dev helps developers evaluating code review agents keep up with how benchmarks like this are built and scored."}}]} ```