--- title: advanced-context-engineering-for-coding-agents/benchmarki... description: A hands-on evaluation of Claude Opus 5, Opus 4.8, and Sonnet 5 on SlopCodeBench — a long-horizon coding benchmark where models must evolve a codebase... canonical: https://daily.dev/posts/advanced-context-engineering-for-coding-agents-benchmarking-opus-5-on-slop-code-bench-md-at-main-h-l5bkkqkom twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: advanced-context-engineering-for-coding-agents/benchmarking-opus-5-on-slop-code-bench.md at main · humanlayer/advanced-context-engineering-for-coding-agents | daily.dev og:description: A hands-on evaluation of Claude Opus 5, Opus 4.8, and Sonnet 5 on SlopCodeBench — a long-horizon coding benchmark where models must evolve a codebase... og:url: https://daily.dev/posts/advanced-context-engineering-for-coding-agents-benchmarking-opus-5-on-slop-code-bench-md-at-main-h-l5bkkqkom og:image: https://api.daily.dev/og/posts/L5BkKQkOm.png og:image:alt: advanced-context-engineering-for-coding-agents/benchmarking-opus-5-on-slop-code-bench.md at main · humanlayer/advanced-context-engineering-for-coding-agents og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # advanced-context-engineering-for-coding-agents/benchmarking-opus-5-on-slop-code-bench.md at main · humanlayer/advanced-context-engineering-for-coding-agents **[Hacker News](https://daily.dev/sources/hn)** · 17 min read · 2 upvotes · 0 comments ## Summary A hands-on evaluation of Claude Opus 5, Opus 4.8, and Sonnet 5 on SlopCodeBench — a long-horizon coding benchmark where models must evolve a codebase incrementally across multiple checkpoints without knowing future requirements upfront. Opus 5 achieved a 24% strict pass rate on a 17-checkpoint subset (3 problems: easy, medium, hard), compared to 6% for the other two models. No model completed any challenge without defects. Code quality metrics showed all models accumulate complexity, verbosity, and duplication over time, with Opus 5 writing 5x more functions than Opus 4.8. The author argues this benchmark provides real signal that current frontier models cannot reliably run 'lights-off' on real-world iterative software engineering tasks without human steering. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 1 discussion and 68 comments across hackernews (as of 2026-07-28). **TL;DR:** The community sees Opus 5 as a modest, incremental improvement over Opus 4.8 at best — useful for cost/speed tradeoffs at medium effort — but many find it overconfident, verbose, and prone to generating bloated 'slop' code compared to Fable or Sol, which remain preferred daily drivers for serious coding tasks. **Sentiment:** 20% positive · 45% mixed · 35% skeptical **The case for** - Opus 5 medium offers a better cost/speed tradeoff, with some users replacing Opus 4.8 xhigh with Opus 5 medium and getting comparable or better results. - Opus 5 is noted as useful for general knowledge work and visual design tasks at lower cost than Fable. - Lower effort levels (medium/low) reportedly prevent Opus 5 from overthinking, which can improve output quality. - The SlopCodeBench approach using deterministic scores and longitudinal maintainability signals is praised as a meaningful step beyond typical benchmarks. **The pushback** - Opus 5 is widely described as overconfident and prone to generating verbose, over-abstracted 'slop' code rather than clean, maintainable solutions. - Many commenters feel Opus 5 is not a meaningful improvement over Opus 4.8, and some prefer 4.8 for its less pedantic tone. - Agentic/automated pipelines with Opus 5 reportedly enter feedback loops where pedantic reviews balloon simple scripts into complex messes. - Models in general (and Opus 5 specifically) are criticized for not knowing how to simplify code, since most RL/benchmarks don't penalize complexity. - Fable is considered the gold standard for code quality but was not included in this benchmark run, limiting its usefulness for power users. - The benchmark only covers a small subset of problems and effort levels, and lacks Fable/Sol comparisons, making conclusions preliminary. **By community** - hackernews (mixed): Commenters are split between finding Opus 5 a useful cost/speed upgrade at medium effort and criticizing it as a slop-prone, overconfident step sideways from Opus 4.8, with broad consensus that Fable and Sol remain superior for serious coding. **Hottest debate:** Whether Opus 5's tendency toward verbose, pedantic code is an inherent model regression or a harness/prompt engineering problem that can be mitigated. **Open questions** - How does Opus 5 perform at high vs. xhigh effort levels on this benchmark, and is medium truly the sweet spot? - How do Fable and Sol compare on SlopCodeBench, especially for longitudinal maintainability? - Would interleaving periodic 'refactor turns' into the benchmark better capture real-world code quality degradation? - Can prompting or harness constraints reliably prevent Opus 5's slop accumulation in agentic pipelines? - Does changing the order of checkpoint implementation reveal which checkpoints are inherently harder vs. model-dependent failures? **Highlights** > We've been running automated code reviews on claude with a bunch of skills/subagents with different specialties. Any review feedback is then fed back into claude to fix. Since switching to Opus 5 I've noticed the reviews are overly pedantic, and that leads to feedback loops where each fix generates more feedback, which requires more fixes, i.e. slop. I had, for example, a simple SQL migration script with a single CREATE TABLE. After a few rounds of review, it ballooned into a complicated 200 line script. I'm not ready to blame Opus 5 for being stupid. Perhaps we have a prompt buried somewhere that's essentially asking it to be pedantic, and it's just obeying the prompt. > — [patwolf on hackernews · 1 comments](https://news.ycombinator.com/item?id=49078388) > i laughed at the pelican bit its good yes the labs will always prioritize the vibeslop dopamine casino as far as I can tell - making the models useful and addictive for unsophisticated users, sometimes at the expense or at the very least at the ignorance of the needs of power users > — [dhorthy on hackernews](https://news.ycombinator.com/item?id=49078447) > Nice! I actually ran across this paper+benchmark recently, too. It's the first I've found that start to aim at some of the non-functional and longitudinal requirements that I think have always been an important part of writing production code. It's especially relevant now that models are good enough to solve ~most point-in-time problems. Some relevant but disconnected thoughts: - deterministic scores are so nice - what "maintainable" is is probably some high dimensional space described by these signals; it'd probably require some human labeling to figure out where this space is - another signal I've been thinking about and I'm seeing increasingly get brought up is the state space of a system; I'm seeing formal methods pop up a lot recently > — [Vgoose on hackernews · 2 comments](https://news.ycombinator.com/item?id=49077756) > Great writeup. The excessive function thing has always driven me crazy; I guard against this explicitly in Claude.md. I have found that models are generally poor at managing refactors / complexity while also implementing new features. But I’ve had some success with a semi-lights-off approach where you decompose it and prompt the model adversarially in a second pass to look for new rough edges and areas of complexity or refactors that might simplify the codebase. So I’d be very curious to see this benchmark but with something like a periodic “refactor turn” interleaved in. Also eager to see Fable benchmarked; anecdotally that was the only model whose code I felt I could actually trust to not review closely. > — [piazz on hackernews](https://news.ycombinator.com/item?id=49079484) > my issue with frontier code is that it uses a model judge for quality whereas slop code bench forces a model to grapple with its own garbage code in order to receive a functionality reward > — [dhorthy on hackernews](https://news.ycombinator.com/item?id=49078496) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=49076391) · 67 points · 68 comments ## Similar posts on daily.dev - [Claude did best on a new benchmark for agents that build agents. It still passed fewer than a quarter of the tests.](https://daily.dev/posts/claude-did-best-on-a-new-benchmark-for-agents-that-build-agents-it-still-passed-fewer-than-a-quarte-rrx7ttpl4) · The New Stack · 2 upvotes · 0 comments - [Claude Opus 4.5, and why evaluating new LLMs is increasingly difficult](https://daily.dev/posts/claude-opus-4-5-and-why-evaluating-new-llms-is-increasingly-difficult-xmamztmxo) · Simon Willison · 3 upvotes · 0 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#ai-agents](https://daily.dev/tags/ai-agents), [#claude](https://daily.dev/tags/claude) [View this post on daily.dev](https://daily.dev/posts/advanced-context-engineering-for-coding-agents-benchmarking-opus-5-on-slop-code-bench-md-at-main-h-l5bkkqkom) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"advanced-context-engineering-for-coding-agents/benchmarking-opus-5-on-slop-code-bench.md at main · humanlayer/advanced-context-engineering-for-coding-agents","url":"https://daily.dev/posts/advanced-context-engineering-for-coding-agents-benchmarking-opus-5-on-slop-code-bench-md-at-main-h-l5bkkqkom","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/advanced-context-engineering-for-coding-agents-benchmarking-opus-5-on-slop-code-bench-md-at-main-h-l5bkkqkom"},"datePublished":"2026-07-28T00:19:46.820Z","dateModified":"2026-07-28T04:53:23.284Z","description":"A hands-on evaluation of Claude Opus 5, Opus 4.8, and Sonnet 5 on SlopCodeBench — a long-horizon coding benchmark where models must evolve a codebase...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f628dee7738c5c2f966051f0c2c2f12f?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f628dee7738c5c2f966051f0c2c2f12f?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/advanced-context-engineering-for-coding-agents-benchmarking-opus-5-on-slop-code-bench-md-at-main-h-l5bkkqkom","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,ai-agents,claude","timeRequired":"PT17M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"advanced-context-engineering-for-coding-agents/benchmarking-opus-5-on-slop-code-bench.md at main · humanlayer/advanced-context-engineering-for-coding-agents"}]} ```