--- title: Measure a carpenter by hammer use and everything gets... description: A conversation with Quentin Anthony, VP of AI Engineering at Zyphra, from AMD's Advancing AI conference covers how frontier model engineers actually use AI... canonical: https://daily.dev/posts/measure-a-carpenter-by-hammer-use-and-everything-gets-smacked-bqijqkjdr twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Measure a carpenter by hammer use and everything gets smacked | daily.dev og:description: A conversation with Quentin Anthony, VP of AI Engineering at Zyphra, from AMD's Advancing AI conference covers how frontier model engineers actually use AI... og:url: https://daily.dev/posts/measure-a-carpenter-by-hammer-use-and-everything-gets-smacked-bqijqkjdr og:image: https://api.daily.dev/og/posts/bqijQkJDr.png og:image:alt: Measure a carpenter by hammer use and everything gets smacked og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Measure a carpenter by hammer use and everything gets smacked **[Dev Interrupted](https://daily.dev/sources/devinterrupted)** · 9 min read · 2 upvotes · 1 comments ## Summary A conversation with Quentin Anthony, VP of AI Engineering at Zyphra, from AMD's Advancing AI conference covers how frontier model engineers actually use AI coding tools. Key insights: AI models struggle with out-of-distribution tasks like GPU kernels because they lack training data; effective AI use requires deep domain expertise to steer and verify outputs; 'tokenmaxxing' (treating token spend as a proxy for AI progress) is a Goodhart's Law problem. Anthony advocates for verifiable task assignment and active human oversight rather than passive lever-pulling. Supporting data from LinearB shows agentic PRs reach production at 37–79% depending on team quality, with the gap driven by ownership and accountability rather than technical factors. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why do AI coding models perform poorly at writing GPU kernels compared to something like JavaScript? AI models struggle with GPU kernel writing because kernels are out-of-distribution data for training, unlike JavaScript which has enormous troves of open source data available. The weakness stems from what the model saw during training, not from any inherent difficulty in the task. Effective mitigation involves feeding the model two or three known-good kernel examples using the specific principle desired, rather than relying on generic prompting. _Developers steering agents through unfamiliar domains can find more framing like this on daily.dev._ ### What percentage of agent-opened pull requests actually reach production in software engineering teams? Agentic pull requests reach production at 79% in elite engineering organizations, 58% in good ones, and only 37% in the rest, according to LinearB's 2026 AI benchmarks. By comparison, all pull requests across those same tiers reach production between 81% and 90%. The gap is attributed to ownership: when nobody is accountable for reviewing an agent-opened PR, it tends to stall rather than ship. _Teams weighing agent adoption against real shipping outcomes track benchmarks like this via daily.dev._ ### How much does AI token spend actually cost compared to a developer's total cost, and is capping it worthwhile? Token spend at the 90th percentile runs about $481 per developer per month, compared to roughly $12,500 a month for a fully loaded developer, meaning AI spend is under 4% of total developer cost. Capping token spend to control costs therefore risks limiting the output of the most effective engineers for minimal savings, since the real cost to manage is the cost per pull request that actually ships. _Engineering leaders debating AI budget caps can weigh cost data like this through daily.dev._ ## Community discussion Top comments from developers on daily.dev. **@agustinbarrientos** · 0 upvotes > I'd never use token counts as a productivity metric. ## Similar posts on daily.dev - [Org Leaders: Is Your AI Policy Attracting the Best Talent?](https://daily.dev/posts/org-leaders-is-your-ai-policy-attracting-the-best-talent--smr90gqtl) · Kilo Blog · 1 upvotes · 1 comments - [Tokenmaxxing and the search for AI metrics that matter](https://daily.dev/posts/tokenmaxxing-and-the-search-for-ai-metrics-that-matter-0j3khk4uw) · LeadDev · 0 upvotes · 0 comments - [Tokenmaxxing is out. How to minimize AI spend without sacrificing security capability.](https://daily.dev/posts/tokenmaxxing-is-out-how-to-minimize-ai-spend-without-sacrificing-security-capability--hioxvbiot) · The New Stack · 2 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#productivity](https://daily.dev/tags/productivity), [#ai-coding](https://daily.dev/tags/ai-coding), [#dora-metrics](https://daily.dev/tags/dora-metrics) [View this post on daily.dev](https://daily.dev/posts/measure-a-carpenter-by-hammer-use-and-everything-gets-smacked-bqijqkjdr) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Measure a carpenter by hammer use and everything gets smacked","url":"https://daily.dev/posts/measure-a-carpenter-by-hammer-use-and-everything-gets-smacked-bqijqkjdr","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/measure-a-carpenter-by-hammer-use-and-everything-gets-smacked-bqijqkjdr"},"datePublished":"2026-07-31T18:46:50.648Z","dateModified":"2026-09-13T21:07:41.815Z","description":"A conversation with Quentin Anthony, VP of AI Engineering at Zyphra, from AMD's Advancing AI conference covers how frontier model engineers actually use AI...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a010fc1e15a851ad5fd5bc6324d5c394?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/a010fc1e15a851ad5fd5bc6324d5c394?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Dev Interrupted","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Dev Interrupted","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/3ecd065ae7354635a8fe0e1cee898951","url":"https://daily.dev/sources/devinterrupted"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/measure-a-carpenter-by-hammer-use-and-everything-gets-smacked-bqijqkjdr","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"llm,productivity,ai-coding,dora-metrics","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Dev Interrupted","item":"https://daily.dev/sources/devinterrupted"},{"@type":"ListItem","position":3,"name":"Measure a carpenter by hammer use and everything gets smacked"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/measure-a-carpenter-by-hammer-use-and-everything-gets-smacked-bqijqkjdr","comment":[{"@type":"Comment","text":"I’d never use token counts as a productivity metric.","datePublished":"2026-08-02T19:57:34.440Z","url":"https://daily.dev/posts/bqijQkJDr#c-k3or1w2m4","author":{"@type":"Person","name":"Agustin Barrientos","url":"https://daily.dev/agustinbarrientos","image":"https://media.daily.dev/image/upload/s--5ayxQnqn--/f_auto/v1788281802/avatars/avatar_wQYYVe5Tbj0NJ7C7qPoa8?_a=BAMAMicg0"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/measure-a-carpenter-by-hammer-use-and-everything-gets-smacked-bqijqkjdr#faq","mainEntity":[{"@type":"Question","name":"Why do AI coding models perform poorly at writing GPU kernels compared to something like JavaScript?","acceptedAnswer":{"@type":"Answer","text":"AI models struggle with GPU kernel writing because kernels are out-of-distribution data for training, unlike JavaScript which has enormous troves of open source data available. The weakness stems from what the model saw during training, not from any inherent difficulty in the task. Effective mitigation involves feeding the model two or three known-good kernel examples using the specific principle desired, rather than relying on generic prompting. Developers steering agents through unfamiliar domains can find more framing like this on daily.dev."}},{"@type":"Question","name":"What percentage of agent-opened pull requests actually reach production in software engineering teams?","acceptedAnswer":{"@type":"Answer","text":"Agentic pull requests reach production at 79% in elite engineering organizations, 58% in good ones, and only 37% in the rest, according to LinearB's 2026 AI benchmarks. By comparison, all pull requests across those same tiers reach production between 81% and 90%. The gap is attributed to ownership: when nobody is accountable for reviewing an agent-opened PR, it tends to stall rather than ship. Teams weighing agent adoption against real shipping outcomes track benchmarks like this via daily.dev."}},{"@type":"Question","name":"How much does AI token spend actually cost compared to a developer's total cost, and is capping it worthwhile?","acceptedAnswer":{"@type":"Answer","text":"Token spend at the 90th percentile runs about $481 per developer per month, compared to roughly $12,500 a month for a fully loaded developer, meaning AI spend is under 4% of total developer cost. Capping token spend to control costs therefore risks limiting the output of the most effective engineers for minimal savings, since the real cost to manage is the cost per pull request that actually ships. Engineering leaders debating AI budget caps can weigh cost data like this through daily.dev."}}]} ```