--- title: Leaderboard | daily.dev description: The ARC-AGI-3 leaderboard tracks AI system performance on the latest ARC-AGI benchmark, which has evolved from measuring passive fluid intelligence (ARC-AGI-1... canonical: https://daily.dev/posts/leaderboard-bzhhktbcq twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Leaderboard | daily.dev og:description: The ARC-AGI-3 leaderboard tracks AI system performance on the latest ARC-AGI benchmark, which has evolved from measuring passive fluid intelligence (ARC-AGI-1... og:url: https://daily.dev/posts/leaderboard-bzhhktbcq og:image: https://api.daily.dev/og/posts/bzhHKTbCQ.png og:image:alt: Leaderboard og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Leaderboard **[Hacker News](https://daily.dev/sources/hn)** · 2 min read · 1 upvotes · 0 comments ## Summary The ARC-AGI-3 leaderboard tracks AI system performance on the latest ARC-AGI benchmark, which has evolved from measuring passive fluid intelligence (ARC-AGI-1 and 2) to challenging AI agents in novel interactive environments. The leaderboard visualizes cost-per-task versus performance across three categories: reasoning systems (e.g., Claude Opus 5, Grok, Gemini), base LLMs (single-shot inference), and Kaggle competition systems (constrained to $50 compute budget). Top performers include Claude Opus 5 (High) at 30.2% on ARC-AGI-3 with a $1.45 cost per task. Only systems costing under $10,000 total are shown. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Community take How the wider developer community reacted, aggregated from 5 discussions and 210 comments across hackernews (as of 2026-07-25). **TL;DR:** The Hacker News community is deeply skeptical of Claude Opus 5's ARC-AGI-3 benchmark results, with strong evidence of benchmark contamination/training data leakage, while a parallel debate rages over whether LLMs have meaningfully improved at all versus just getting better harnesses around them. **Sentiment:** 15% positive · 35% mixed · 50% skeptical **The case for** - Several commenters report Opus 5 feeling genuinely smarter in practice for niche tasks like longitudinal modeling and agentic coding compared to older models. - Fable (a prior Claude model) was praised by multiple users for human-like big-picture thinking and thorough task completion beyond what was asked. - Most commenters agree frontier models today are dramatically more capable than GPT-3.5-era models for real-world coding tasks. **The pushback** - Evidence of benchmaxxing: Opus 5 reportedly states hidden game rules before its first action and plays byte-identical optimal solutions at temperature 1.0 with zero exploration, strongly suggesting ARC-AGI-3 data leaked into training. - Opus 5 regresses below Opus 4.8 on novel games requiring actual rule discovery, suggesting the ARC-AGI-3 score reflects memorization of known patterns rather than general reasoning gains. - Multiple commenters argue the ARC-AGI-3 benchmark is measuring harness quality or training contamination rather than genuine model intelligence. - Some users feel Claude models are too restrictive and expensive, preferring GPT models for day-to-day work. - A commenter raised concern that AI companies may quietly degrade model quality after launch to cut costs, analogizing to telecom card fraud. - Opus 5 was noted to be stubborn about its initial opinions and continuously hedges, making it harder to work with in some ways. **By community** - hackernews (skeptical): Dominated by skepticism about ARC-AGI-3 benchmark integrity (suspected training contamination), a heated debate over whether LLMs have meaningfully improved beyond their harnesses, and mixed real-world impressions of Opus 5 vs. prior models. **Hottest debate:** Whether Opus 5's outsized ARC-AGI-3 score reflects genuine reasoning improvement or benchmark contamination/training data leakage, with direct evidence cited that the model plays memorized solutions with zero exploration on known game genres while regressing on truly novel ones. **Open questions** - How can benchmark integrity be maintained when closed models can be trained on any data, including benchmark discussions published online, without external verification? - Does the ARC-AGI benchmark actually measure general interactive abstract reasoning, or is it primarily measuring harness quality and pattern memorization? - If 80% of practical LLM improvements come from harnesses rather than the base model, should benchmarks test agents-in-harnesses rather than raw models? - Is there a way to detect benchmark contamination in model weights short of full training data transparency? **Highlights** > "Opus 5 states the hidden rules before its first action, then plays a byte-identical optimal solution in 5/5 seeds at temperature 1.0. Zero exploration." I guess there is no way this can happen without benchmark being part of the training data?? > — [throwa356262 on hackernews · 2 comments](https://news.ycombinator.com/item?id=49047153) > And being worst than previous model... "...The traces tell the why: (1) On our most classic Witness-style game, Opus 5 states the hidden rules before its first action, then plays a byte-identical optimal solution in 5/5 seeds at temperature 1.0. Zero exploration. It already knows this genre. (2) But on our most novel game (unusual mechanic combinations you can't pattern-match), Opus 5 regresses below Opus 4.8. Where rules must actually be discovered through interaction, the new model is worse than the old one..." > — [root-parent on hackernews](https://news.ycombinator.com/item?id=49048929) > If you go to a LLM without harness, GP original point in completely right. LLMS by themselves are still shit at math, they still confuse weird correlation to causation every time (and sometimes in ways even a 9 year old would say "no, that's dumb"), and confuse original parameters very often. I disagree with " "thinking" token generation being on the correct track just to 'no, wait' on an already correct conclusion", because i think that is an effect of the harness, not the LLMs. 80% off all the improvements since ChatGPT4 are in the harnesses, and the LLMs by themselves, while they improved in areas they already were good at (translation especially) did not fix any of they original issues (object permanence, calculusm correlation). Just run old models in the playground and get them to play chess (maybe make a small custom harness if you feel like it), then replace it with a frontier model (i don't know if you still have API access without harness on US models, but if you don't try K3), you will see LLMs weaknesses were not at all fixed, even marginally. They're way better and not inducing bugs in the code, so that make them usable since Opus4.5 (anyone using them prior to that either had a greenfield project or like spending hours debugging). > — [orwin on hackernews · 3 comments](https://news.ycombinator.com/item?id=49046719) > I wrote this in June, and I'm honestly not sure I've felt the same magic since: I was close to maxing out my $200 plan for the week, almost all Fable use [Claude CLI]. My observations: Fable seemed to have bigger-picture thinking and completed tasks more thoroughly vs just focusing on executing the ask. It pieced together context and intent like an all-star employee would, vs one that just does what you say. Not overeager (important!), but if the above-and-beyond was warranted, it just did it. This was surprisingly delightful. Coderabbit seemed to find ~1/3 or so as many issues when reviewing, too. > — [tpowell on hackernews · 1 comments](https://news.ycombinator.com/item?id=49045425) > Some 20 years ago, the telecommunications sector in Germany was liberalized. Many telephone card providers entered what had previously been a barely competitive market. They advertised their products with aggressive claims like: “Buy our €10 top-up card and get 660 minutes to destination X.” For the first few weeks, they would actually provide those 660 minutes to establish trust in their cards. But after a while, they would quietly start reducing the number of minutes on subsequent top-ups—say, from 660 minutes down to only 300. They wouldn’t do this for every card, so it was difficult to prove. Instead, they relied on averages across their customer base to make the economics work. Lately, I’ve found myself wondering whether something similar may be happening with frontier AI models. Companies launch with an exceptionally strong model and generous compute limits to build adoption. Once the model is established as a market leader, the incentives change, and users may start perceiving the service as becoming more constrained or less capable over time. I don’t have evidence that this is what’s happening with Anthropic—or with any other AI company. It’s simply a pattern that the current situation reminds me of. > — [submeta on hackernews](https://news.ycombinator.com/item?id=49047718) **Source threads** - [hackernews](https://news.ycombinator.com/item?id=47520530) · 5 points · 0 comments - [hackernews](https://news.ycombinator.com/item?id=49045040) · 107 points · 206 comments - [hackernews](https://news.ycombinator.com/item?id=44517352) · 3 points · 0 comments - [hackernews](https://news.ycombinator.com/item?id=46611742) · 2 points · 2 comments - [hackernews](https://news.ycombinator.com/item?id=46997857) · 1 points · 2 comments --- Tags: [#ai](https://daily.dev/tags/ai), [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents) [View this post on daily.dev](https://daily.dev/posts/leaderboard-bzhhktbcq) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Leaderboard","url":"https://daily.dev/posts/leaderboard-bzhhktbcq","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/leaderboard-bzhhktbcq"},"datePublished":"2026-07-25T07:32:35.812Z","dateModified":"2026-07-25T16:50:44.756Z","description":"The ARC-AGI-3 leaderboard tracks AI system performance on the latest ARC-AGI benchmark, which has evolved from measuring passive fluid intelligence (ARC-AGI-1...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/bfcd6071999ba1bfb31cefef8148664c?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/bfcd6071999ba1bfb31cefef8148664c?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Hacker News","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Hacker News","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/hn","url":"https://daily.dev/sources/hn"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/leaderboard-bzhhktbcq","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai,llm,ai-agents","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Hacker News","item":"https://daily.dev/sources/hn"},{"@type":"ListItem","position":3,"name":"Leaderboard"}]} ```