--- title: 13 Models and 4 Agents on SWE Tasks: Go, Java, Python,... description: SWE-rebench is a continuously evolving and decontaminated benchmark for evaluating LLMs on software engineering tasks. The leaderboard ranks models and agents... canonical: https://daily.dev/posts/13-models-and-4-agents-on-swe-tasks-go-java-python-rust-ts-ithomlqxu twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: 13 Models and 4 Agents on SWE Tasks: Go, Java, Python, Rust, TS | daily.dev og:description: SWE-rebench is a continuously evolving and decontaminated benchmark for evaluating LLMs on software engineering tasks. The leaderboard ranks models and agents... og:url: https://daily.dev/posts/13-models-and-4-agents-on-swe-tasks-go-java-python-rust-ts-ithomlqxu og:image: https://api.daily.dev/og/posts/itHomLQxU.png og:image:alt: 13 Models and 4 Agents on SWE Tasks: Go, Java, Python, Rust, TS og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # 13 Models and 4 Agents on SWE Tasks: Go, Java, Python, Rust, TS **[Hacker News](https://daily.dev/sources/hn)** · 6 min read · 2 upvotes · 0 comments ## Summary SWE-rebench is a continuously evolving and decontaminated benchmark for evaluating LLMs on software engineering tasks. The leaderboard ranks models and agents by resolved rate, Pass@5, cost per problem, and tokens per problem. Top performers include Fable 5 (64.5% resolved), Grok 4.5 (63.8%), Opus 5 (63.4%), and GLM-5.2 (62.9%). AI coding agents like Junie (61.8%), Claude Code (60.4%), Codex (58.0%), and Cursor (51.7%) are also ranked. The benchmark tracks over 100 models and agents, with many still pending evaluation results. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 3 discussions and 22 comments across hackernews (as of 2026-07-31). **TL;DR:** The community is mostly skeptical and confused about the benchmark's methodology — particularly why models outperform agents, how effort levels are compared fairly, and whether the benchmark actually tests cross-language efficiency rather than just per-language performance. **Sentiment:** 20% positive · 50% mixed · 30% skeptical **The case for** - Some commenters note that certain languages like Java may be more amenable to agentic workflows, sparking useful discussion about language choice for AI coding. - The benchmark surfaces cost-per-problem comparisons that commenters find useful (e.g., Sol being 4-5x cheaper than Fable at comparable scores). **The pushback** - Commenters are confused about why raw models rank above agents, questioning what the benchmark is actually measuring. - The benchmark compares models at different effort levels (e.g., Fable high vs. Sol medium), making fair comparison difficult. - The benchmark tests different problems per language rather than the same tasks across languages, disappointing those hoping for cross-language efficiency insights. - Data contamination risk is flagged — models released after late 2023 may have seen SWE-bench issues during training, inflating scores. - Unexplained anomalies in the dataset timeline (July 2025, January 2026 spikes) undermine confidence in the results. **By community** - hackernews (skeptical): Commenters raise multiple methodological concerns including unfair effort-level comparisons, confusion over models vs. agents rankings, lack of cross-language task parity, and data contamination risks. **Hottest debate:** Whether comparing models at different effort/cost tiers (e.g., Fable high vs. Sol medium) makes the leaderboard rankings meaningful or misleading. **Open questions** - Why do raw models outperform dedicated coding agents on this benchmark? - What caused the apparent anomalies in the dataset around July 2025 and January 2026? - Would a benchmark using identical tasks across languages reveal which languages are most efficient for agentic workflows? - How much of the performance is genuine generalization vs. memorization of SWE-bench training data? **Highlights** > They are all different problems for the different languages. I was hoping this was a benchmark that attempted to see which languages were more efficient to use with which models. > — [spullara on hackernews · 1 comments](https://news.ycombinator.com/item?id=49125259) > Why are models better than agents, isn't it supposed to be the opposite? I don't understand the difference and what you are measuring. > — [goldenarm on hackernews](https://news.ycombinator.com/item?id=49126513) > Why test Fable high effort vs Sol medium? Especially when Sol comes out 4-5x cheaper in their tests at those effort levels. > — [dia80 on hackernews · 1 comments](https://news.ycombinator.com/item?id=49125518) > https://swe-rebench.com/about > Potential data contamination: The SWE-bench dataset, comprising a collection of GitHub issues, has been publicly available since the end of 2023. As a result, models released after this date may have seen these exact issues or highly similar data during training. This raises the risk of inflated performance metrics and makes it harder to distinguish genuine generalization from memorization. > — [rsyring on hackernews](https://news.ycombinator.com/item?id=49125837) > What happened in July 2025 and January 2026? If that can't be explained in a concise way this entire dataset is vibes at best. > — [tcdent on hackernews](https://news.ycombinator.com/item?id=49127013) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=43985141) · 4 points · 0 comments - [hackernews](https://news.ycombinator.com/item?id=49124336) · 30 points · 22 comments - [hackernews](https://news.ycombinator.com/item?id=48354272) · 2 points · 0 comments ## Similar posts on daily.dev - [Most coding agent benchmarks skip large-scale refactoring. Not this one.](https://daily.dev/posts/most-coding-agent-benchmarks-skip-large-scale-refactoring-not-this-one--p55v6sihn) · The New Stack · 0 upvotes · 0 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#llm](https://daily.dev/tags/llm), [#ai-coding](https://daily.dev/tags/ai-coding) [View this post on daily.dev](https://daily.dev/posts/13-models-and-4-agents-on-swe-tasks-go-java-python-rust-ts-ithomlqxu) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"13 Models and 4 Agents on SWE Tasks: Go, Java, Python, Rust, TS","url":"https://daily.dev/posts/13-models-and-4-agents-on-swe-tasks-go-java-python-rust-ts-ithomlqxu","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/13-models-and-4-agents-on-swe-tasks-go-java-python-rust-ts-ithomlqxu"},"datePublished":"2026-07-31T19:40:43.373Z","dateModified":"2026-07-31T20:06:39.309Z","description":"SWE-rebench is a continuously evolving and decontaminated benchmark for evaluating LLMs on software engineering tasks. The leaderboard ranks models and agents...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/04be1a80fb890580717b07c7bc60a695?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/04be1a80fb890580717b07c7bc60a695?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/13-models-and-4-agents-on-swe-tasks-go-java-python-rust-ts-ithomlqxu","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,llm,ai-coding","timeRequired":"PT6M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"13 Models and 4 Agents on SWE Tasks: Go, Java, Python, Rust, TS"}]} ```