--- title: How wide should an agent think? | daily.dev description: An in-depth analysis of 'search breadth' in AI coding/reasoning agents - how many alternative solutions an agent generates before committing. Drawing on... canonical: https://daily.dev/posts/how-wide-should-an-agent-think--xv4fqdzr7 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: How wide should an agent think? | daily.dev og:description: An in-depth analysis of 'search breadth' in AI coding/reasoning agents - how many alternative solutions an agent generates before committing. Drawing on... og:url: https://daily.dev/posts/how-wide-should-an-agent-think--xv4fqdzr7 og:image: https://api.daily.dev/og/posts/Xv4FQdZr7.png og:image:alt: How wide should an agent think? og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # How wide should an agent think? **[Medium](https://daily.dev/sources/medium_js)** · 38 min read · 0 upvotes · 0 comments ## Summary An in-depth analysis of 'search breadth' in AI coding/reasoning agents - how many alternative solutions an agent generates before committing. Drawing on academic papers (Inside the Scaffold, Large Language Monkeys, AI Agents That Matter, Agentless, and others), it lays out five levels of breadth from single-trajectory ReAct loops to full MCTS, and argues the selector (how you pick among candidates) is the real bottleneck, not the generation itself. Key findings include: 100x more samples only bought 0.91 points on MATH (no executable reward) versus 40 points on SWE-bench (executable tests exist); realized accuracy tends to land at about half the gap between random baseline and oracle ceiling; temperature-0 sampling produces identical candidates unless diversity is injected at the input; model critics are unreliable as gates (barely above chance on objective correctness) but useful as comparators; and at matched token budgets, breadth's advantage on agentic coding tasks is only about three points and shrinking with newer models. It closes with practical rules: match search level to available reward, choose selection vs. merge deliberately, fix diversity at branching rather than at merge, and treat breadth as a scaffolding investment (context/filesystem isolation, compaction, budget enforcement) rather than a prompting trick. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Does increasing sample size (best-of-N) actually improve accuracy on math versus coding tasks? Gains depend heavily on whether a selector can verify candidates. On MATH with Llama-3-8B-Instruct, a hundredfold increase in samples bought only 0.91 points of shippable accuracy against a 57-point gap to the theoretical ceiling, because no executable check exists. On SWE-bench Lite with DeepSeek-Coder-V2-Instruct, repeated sampling raised accuracy from 15.9% to 56% at 250 samples, since generated tests let a selector recognize correct patches. _Weighing sampling strategies for a coding agent is easier when you track results like this on daily.dev._ ### What is the ranking of selector types by reliability when picking among multiple AI-generated code patches? In matched comparisons with the same generator and candidates, reliability increases in this order: a model judging its own output, majority vote, a learned critic, and executed tests. Agentless's selector cascade (normalize, majority vote, regression tests, reproduction tests) showed reproduction tests executed against real behavior gave the largest single gain, confirming that execution beats prediction. _Developers choosing a verification strategy for agentic code review can track this comparison on daily.dev._ ### How much does the choice of scaffold or harness affect an AI coding agent's benchmark score on the same underlying model? Harness design can swing scores by more than tenfold on a frozen model. OpenAI reported GPT-4's SWE-bench Lite score ranged from 2.7% with an early retrieval-based scaffold to 28.3% with a stronger one. SWE-agent's own ablations showed one model spanning 10.3% to 18.0% across interface variants, and a bad search interface (12.0%) scored worse than no search interface at all (15.7%). _Engineers tuning agent harnesses rather than swapping models can follow findings like this on daily.dev._ ## Similar posts on daily.dev - [Measuring What Matters with Jules](https://daily.dev/posts/measuring-what-matters-with-jules-bwyrmpwn7) · Google Developers · 1 upvotes · 0 comments - [The Complete Guide to Tool Selection in AI Agents](https://daily.dev/posts/the-complete-guide-to-tool-selection-in-ai-agents-nnbhdagkm) · Machine Learning Mastery · 1 upvotes · 0 comments - [The Missing Layer in Many AI Agents: Search Freshness](https://daily.dev/posts/the-missing-layer-in-many-ai-agents-search-freshness-6nvslssye) · Medium · 2 upvotes · 0 comments - [Building AI agents that overcome the complexity cliff](https://daily.dev/posts/building-ai-agents-that-overcome-the-complexity-cliff-rcglfmwep) · Temporal · 1 upvotes · 0 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents) [View this post on daily.dev](https://daily.dev/posts/how-wide-should-an-agent-think--xv4fqdzr7) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"How wide should an agent think?","url":"https://daily.dev/posts/how-wide-should-an-agent-think--xv4fqdzr7","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/how-wide-should-an-agent-think--xv4fqdzr7"},"datePublished":"2026-08-24T05:20:18.922Z","dateModified":"2026-09-14T08:50:30.555Z","description":"An in-depth analysis of 'search breadth' in AI coding/reasoning agents - how many alternative solutions an agent generates before committing. Drawing on...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/36740211dece7dfad44e3e298973c3a4?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/36740211dece7dfad44e3e298973c3a4?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Medium","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Medium","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/medium","url":"https://daily.dev/sources/medium_js"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/how-wide-should-an-agent-think--xv4fqdzr7","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,llm,ai-agents","timeRequired":"PT38M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Medium","item":"https://daily.dev/sources/medium_js"},{"@type":"ListItem","position":3,"name":"How wide should an agent think?"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/how-wide-should-an-agent-think--xv4fqdzr7#faq","mainEntity":[{"@type":"Question","name":"Does increasing sample size (best-of-N) actually improve accuracy on math versus coding tasks?","acceptedAnswer":{"@type":"Answer","text":"Gains depend heavily on whether a selector can verify candidates. On MATH with Llama-3-8B-Instruct, a hundredfold increase in samples bought only 0.91 points of shippable accuracy against a 57-point gap to the theoretical ceiling, because no executable check exists. On SWE-bench Lite with DeepSeek-Coder-V2-Instruct, repeated sampling raised accuracy from 15.9% to 56% at 250 samples, since generated tests let a selector recognize correct patches. Weighing sampling strategies for a coding agent is easier when you track results like this on daily.dev."}},{"@type":"Question","name":"What is the ranking of selector types by reliability when picking among multiple AI-generated code patches?","acceptedAnswer":{"@type":"Answer","text":"In matched comparisons with the same generator and candidates, reliability increases in this order: a model judging its own output, majority vote, a learned critic, and executed tests. Agentless's selector cascade (normalize, majority vote, regression tests, reproduction tests) showed reproduction tests executed against real behavior gave the largest single gain, confirming that execution beats prediction. Developers choosing a verification strategy for agentic code review can track this comparison on daily.dev."}},{"@type":"Question","name":"How much does the choice of scaffold or harness affect an AI coding agent's benchmark score on the same underlying model?","acceptedAnswer":{"@type":"Answer","text":"Harness design can swing scores by more than tenfold on a frozen model. OpenAI reported GPT-4's SWE-bench Lite score ranged from 2.7% with an early retrieval-based scaffold to 28.3% with a stronger one. SWE-agent's own ablations showed one model spanning 10.3% to 18.0% across interface variants, and a bad search interface (12.0%) scored worse than no search interface at all (15.7%). Engineers tuning agent harnesses rather than swapping models can follow findings like this on daily.dev."}}]} ```