--- title: Show HN: Parsimony – an open-source benchmark for how... description: Parsimony is an open-source benchmark that measures how much code AI coding agents add when solving tasks, rather than just whether they solve them. It parses... canonical: https://daily.dev/posts/show-hn-parsimony-an-open-source-benchmark-for-how-much-code-ai-models-add-qkwmhwo3k twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Show HN: Parsimony – an open-source benchmark for how much code AI models add | daily.dev og:description: Parsimony is an open-source benchmark that measures how much code AI coding agents add when solving tasks, rather than just whether they solve them. It parses... og:url: https://daily.dev/posts/show-hn-parsimony-an-open-source-benchmark-for-how-much-code-ai-models-add-qkwmhwo3k og:image: https://api.daily.dev/og/posts/QkwMHWO3k.png og:image:alt: Show HN: Parsimony – an open-source benchmark for how much code AI models add og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Show HN: Parsimony – an open-source benchmark for how much code AI models add **[Hacker News](https://daily.dev/sources/hn)** · 5 min read · 0 upvotes · 0 comments ## Summary Parsimony is an open-source benchmark that measures how much code AI coding agents add when solving tasks, rather than just whether they solve them. It parses patches into 'coding units' (statements, calls, comparisons, names, literals), subtracting deleted code from added code, and ranks models by mean net units added on tasks they solved. Data is pulled from DeepSWE (Python, JavaScript, TypeScript, Go), SWE-bench Verified, SWE-bench Live Lite, and SWEeper-Bench PHP, covering roughly two dozen models including Gemini, Claude, GPT, Grok, Kimi, DeepSeek, GLM, and Qwen variants. The project notes that smaller changes aren't automatically better (units measure size, not readability), and that models writing more code also tend to solve more tasks, showing a moderate correlation between verbosity and solve rate. Users can add their own benchmark data via a provided template. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### What is the Parsimony benchmark for AI coding agents? Parsimony is an open-source benchmark measuring how much code AI models add when completing coding tasks, rather than just whether they solve them. It counts 'coding units' such as statements, calls, comparisons, names, and literals in patches, subtracting deleted code from added code to get a net units figure, then ranks models by mean net units added on tasks they solved. _Developers comparing AI coding agents on code verbosity can track benchmarks like this on daily.dev._ ### Is there a correlation between how much code an AI coding agent writes and its task solve rate? Yes, models that write more code also tend to solve more tasks, with a Spearman correlation of roughly 0.33 to 0.60 observed on benchmark boards featuring 26 or more models. This suggests verbosity and capability are linked, so writing less code is not automatically a sign of a better or more efficient model, since smaller changes don't guarantee better readability or design. _Teams weighing AI agent verbosity against solve rate can follow this kind of analysis on daily.dev._ ### What datasets and tasks does the Parsimony benchmark use to evaluate AI coding agents? It draws on DeepSWE Python (34 tasks), JavaScript (5 tasks), TypeScript (35 tasks), and Go (34 tasks) suites, each with four attempts per task, plus SWE-bench Verified (500 tasks, one attempt), SWE-bench Live Lite (300 tasks), and SWEeper-Bench PHP (128 tasks). Every model runs through the same agent, mini-SWE-agent, with pass/fail taken from DeepSWE's published results rather than rerun tests. _Engineers picking an AI coding agent across languages can keep an eye on multi-language benchmarks via daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 22 comments across hackernews (as of 2026-10-14). **TL;DR:** Commenters welcome a benchmark that tracks AI-generated code verbosity, but many push back on the idea that 'less code' equals 'better code,' citing readability and complexity concerns with the top-ranked model. **Sentiment:** 15% positive · 45% mixed · 40% skeptical **The case for** - Some see value in measuring and reducing the verbosity LLMs introduce into codebases. - A benchmark like this could help surface complexity/readability issues that raw solve-rate metrics miss. **The pushback** - The model that 'wins' on low verbosity is seen by some as producing unreadable, overly terse code. - Terseness can be just as bad as verbosity, so fewer lines isn't necessarily better. - Code unit count doesn't capture whether a change is actually easier to review or maintain. - Lack of abstraction in LLM-generated code causes reliability problems regardless of code size. **By community** - hackernews (mixed): People appreciate the goal of measuring verbosity but are split on whether the current leaderboard results reflect good code quality, with several arguing terse code can be worse than verbose code. **Hottest debate:** Whether minimal/terse code (as produced by the top-ranked model) is actually preferable to more verbose but readable code. **Open questions** - Are there practical techniques to reduce complexity and verbosity in agent-generated code? - Does less code generated actually translate into less reviewer effort? - Could cyclomatic complexity be added as a readability metric alongside code unit counts? **Highlights** > On the one hand, I absolutely agree that we should try to measure and reduce verbosity that LLMs are slopping into our codebases. On the other hand, the fact that Astra is "winning" this bench is not great signal for me, since imo Astra produces basically unreadable code. Astra plays code golf. My own revealed preference is that I'd rather have the excessive comments and tests of Claude rather than the inscrutability of Astra. > — [fastball on hackernews · 2 comments](https://news.ycombinator.com/item?id=50078803) > Yeah the issue is that while code that’s too verbose is bad, code that’s too terse is as well. Astra codes like it has to pay by the keystroke (although in a way, it indeed does) > — [demibabs on hackernews](https://news.ycombinator.com/item?id=50080796) > LLM systems start to Frankenstein things together at a very modest amount of code. The lack of abstractions cause huge and glaring problems for anything that needs to run predictably. Abstraction is an issue for every AI coding agent. They are painfully useful, but need constant badgering in the right direction. > — [nostrebored on hackernews · 1 comments](https://news.ycombinator.com/item?id=50082379) > On a related note: Would love to see cyclomatic complexity as a supporting metric for readability. I increasingly find AI-adjacent and AI-written code suffering from this. > — [smilekzs on hackernews](https://news.ycombinator.com/item?id=50081658) > Nice project! But I wonder does less coding actually means less work for whoever reviews it? > — [chengjunsuger on hackernews](https://news.ycombinator.com/item?id=50082128) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=50077619) · 63 points · 22 comments ## Similar posts on daily.dev - [Measuring What Matters with Jules](https://daily.dev/posts/measuring-what-matters-with-jules-bwyrmpwn7) · Google Developers · 1 upvotes · 0 comments - [Benchmarking Coding Agents on Databricks’ Multi-Million Line Codebase](https://daily.dev/posts/benchmarking-coding-agents-on-databricks-multi-million-line-codebase-qjzvv6o07) · databricks · 1 upvotes · 0 comments - [Most coding agent benchmarks skip large-scale refactoring. Not this one.](https://daily.dev/posts/most-coding-agent-benchmarks-skip-large-scale-refactoring-not-this-one--p55v6sihn) · The New Stack · 0 upvotes · 0 comments - [Separating signal from noise in coding evaluations](https://daily.dev/posts/separating-signal-from-noise-in-coding-evaluations-swoud4dtk) · Hacker News · 0 upvotes · 0 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents) [View this post on daily.dev](https://daily.dev/posts/show-hn-parsimony-an-open-source-benchmark-for-how-much-code-ai-models-add-qkwmhwo3k) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Show HN: Parsimony – an open-source benchmark for how much code AI models add","url":"https://daily.dev/posts/show-hn-parsimony-an-open-source-benchmark-for-how-much-code-ai-models-add-qkwmhwo3k","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/show-hn-parsimony-an-open-source-benchmark-for-how-much-code-ai-models-add-qkwmhwo3k"},"datePublished":"2026-10-14T04:39:58.559Z","dateModified":"2026-10-14T05:04:12.612Z","description":"Parsimony is an open-source benchmark that measures how much code AI coding agents add when solving tasks, rather than just whether they solve them. It parses...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e35708a94f32d848967fe0ed9777108b?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/e35708a94f32d848967fe0ed9777108b?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/show-hn-parsimony-an-open-source-benchmark-for-how-much-code-ai-models-add-qkwmhwo3k","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,llm,ai-agents","timeRequired":"PT5M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Show HN: Parsimony – an open-source benchmark for how much code AI models add"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/show-hn-parsimony-an-open-source-benchmark-for-how-much-code-ai-models-add-qkwmhwo3k#faq","mainEntity":[{"@type":"Question","name":"What is the Parsimony benchmark for AI coding agents?","acceptedAnswer":{"@type":"Answer","text":"Parsimony is an open-source benchmark measuring how much code AI models add when completing coding tasks, rather than just whether they solve them. It counts 'coding units' such as statements, calls, comparisons, names, and literals in patches, subtracting deleted code from added code to get a net units figure, then ranks models by mean net units added on tasks they solved. Developers comparing AI coding agents on code verbosity can track benchmarks like this on daily.dev."}},{"@type":"Question","name":"Is there a correlation between how much code an AI coding agent writes and its task solve rate?","acceptedAnswer":{"@type":"Answer","text":"Yes, models that write more code also tend to solve more tasks, with a Spearman correlation of roughly 0.33 to 0.60 observed on benchmark boards featuring 26 or more models. This suggests verbosity and capability are linked, so writing less code is not automatically a sign of a better or more efficient model, since smaller changes don't guarantee better readability or design. Teams weighing AI agent verbosity against solve rate can follow this kind of analysis on daily.dev."}},{"@type":"Question","name":"What datasets and tasks does the Parsimony benchmark use to evaluate AI coding agents?","acceptedAnswer":{"@type":"Answer","text":"It draws on DeepSWE Python (34 tasks), JavaScript (5 tasks), TypeScript (35 tasks), and Go (34 tasks) suites, each with four attempts per task, plus SWE-bench Verified (500 tasks, one attempt), SWE-bench Live Lite (300 tasks), and SWEeper-Bench PHP (128 tasks). Every model runs through the same agent, mini-SWE-agent, with pass/fail taken from DeepSWE's published results rather than rerun tests. Engineers picking an AI coding agent across languages can keep an eye on multi-language benchmarks via daily.dev."}}]} ```