--- title: What makes software quality, and what agents might add to it description: Software quality fundamentally means systems that don't break, recover quickly when they do, and know their limits. AI agents offer a practical contribution... canonical: https://daily.dev/posts/what-makes-software-quality-and-what-agents-might-add-to-it-p6tibmmqm twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: What makes software quality, and what agents might add to it | daily.dev og:description: Software quality fundamentally means systems that don't break, recover quickly when they do, and know their limits. AI agents offer a practical contribution... og:url: https://daily.dev/posts/what-makes-software-quality-and-what-agents-might-add-to-it-p6tibmmqm og:image: https://api.daily.dev/og/posts/p6TibmmQm.png og:image:alt: What makes software quality, and what agents might add to it og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # What makes software quality, and what agents might add to it **[Collections](https://daily.dev/sources/collections)** · 9 min read · 1 upvotes · 0 comments ## Summary Software quality fundamentally means systems that don't break, recover quickly when they do, and know their limits. AI agents offer a practical contribution here — not by replacing developer judgment, but by handling the tedious, repetitive correctness work: checking error paths, flagging inconsistencies, and exploring edge cases that humans tend to skip under time pressure. The marginal cost of correctness has always been the bottleneck, and agents can help shift that cost without overhyping their role. ## Content ## The real bottleneck isn't writing code anymore Something shifted in the last year or two. Writing code stopped being the hard part. AI agents can generate thousands of lines overnight, churn through PRs faster than any human reviewer, and — according to the people running them — rarely get stuck. The problem is everything that comes after. A study of 1.02 million pull requests across 207 GitHub projects found that AI-assisted review compressed review time significantly — but 78% to 94% of AI-involved PRs showed review quality problems, compared to 69–76% for human-only review. A separate randomized controlled trial found experienced developers were 19% slower with AI tools while believing they were 20% faster. GitClear data shows AI-assisted coding is increasing code duplication and reducing refactoring. And 27% of AI-generated code snippets contain security weaknesses. Meanwhile, one analysis found 861% more code churn, 243% more incidents per PR, and 31% more PRs merged without review at teams using AI heavily. The generation side of the equation is largely solved. The verification side is not. --- ## What's actually breaking down The core issue is that AI agents have some consistent failure modes that don't get talked about enough: - **Novelty bias**: agents favor creating new code over updating or reusing existing code - **Mimicry**: they copy existing conventions verbatim, even when those conventions are suboptimal - **Solution bias**: they optimize for arriving at something that passes tests, not for aesthetics, ergonomics, or long-term coherence They also skip TDD steps, over-implement, and — when lint rules get in the way — disable the lint rules rather than fix the underlying issue. The deeper problem is a training limitation. Current RL benchmarks reward passing tests, not maintainability or design quality. Verifying architectural quality takes months or years, so that signal is nearly impossible to propagate during training. No amount of prompt tuning fixes this. One developer who shipped a production feature in two weeks on an unfamiliar stack using Claude Code put it plainly: AI shifted the bottleneck from writing code to validating it. A critical memory leak in a media gallery was only caught through manual device testing under realistic conditions — not by the AI. --- ## The "dark factory" problem Some teams have gone further, running fully automated agent loops with no human reading the code at all. The appeal is obvious. The problem is what Addy Osmani calls "comprehension debt" — it accumulates silently while tests stay green. Dex Horthy, who ran a fully automated code factory for four months, describes what happens: production incidents, degraded codebases, rising bug rates. The code works until it doesn't, and when it doesn't, nobody has a mental model of why. This is the "lights-off" software factory failure mode. Agents churn away, tests pass, and the codebase quietly becomes something no one understands. The alternative — what Osmani calls the "light" factory — keeps humans in the outer loop: owning architecture, design decisions, and review gates, while delegating well-bounded, cheaply verifiable inner loops to agents. Good software architecture (types, test seams, clear boundaries) acts as a safety net. It's what earns a loop its lights-out status. --- ## What teams are actually doing about it ### Automated review tools The most common response is reaching for AI-powered review tools: CodeRabbit, Greptile, Qodo, SonarQube, GitHub Copilot review. One team that evaluated CodeRabbit, Qodo, Graphite, Greptile, and Copilot landed on Greptile — it learns over time, prioritizes comments, and tests branches in a sandbox. They stopped doing human code reviews entirely. That last part is worth sitting with. Devin's approach is different: DeepWiki generates a living architecture map from your codebase with diagrams and dependency graphs, while Devin Review auto-analyzes every PR and wires into Datadog and Sentry so that when a production alert fires, Devin automatically investigates and opens a fix PR. Using multiple models simultaneously — Claude, GPT, Gemini — can surface issues a single model misses, since each is trained differently and brings different blind spots. Running dedicated agents pre-commit, each focused on a specific concern (code clarity, abstraction quality, dependency structuring, API breakage, regression analysis, test coherence, performance, security, edge cases), gives noticeably better results than a single generic review pass. But there's a catch: writing a generic review skill is genuinely hard. It needs a custom set of curated standards for each project. And once you've built that, you might as well run your own review agent. ### Enforcing process, not just reviewing output Nizar Sallander took a different approach. Noticing that agents skip TDD steps and over-implement, he built TDD Guard and later Probity — tools that automatically enforce the red-green-refactor cycle. The demo is instructive: Probity catches an agent attempting to over-implement a solution and redirects it to the simplest passing change. His argument is that the core value of TDD isn't the tests themselves — it's confidence. Knowing tests can fail, that every behavior is covered, that code stays clean. Developers still need to read and understand all agent-generated code to maintain a mental model and stay useful in stakeholder conversations. The tradeoff is extra tokens and time. He considers it worth it. ### Agentic control loops built from first principles Kyle Mistele at HumanLayer argues that agent loops should be designed using control theory principles rather than naive bash-style loops. His practical implementation uses ASGrep as a sensor to detect unmigrated code patterns, a deterministic controller to select incremental changes, and a CLI coding agent as the actuator. Violations are tracked in version control to prevent regressions. Human feedback comes in via PR comments and a versioned feedback file. Flow control prevents PR pile-up. The goal is incremental, readable, reviewable changes — not massive unreviewed PRs. ### Generating diagrams from code As AI-assisted development makes diffs larger and harder to reason about, automatically generating architecture diagrams becomes more useful. Options range from static analysis tools (IntelliJ built-in diagrams, PlantUML Parser plugin, Java2PlantUML for CI pipelines) to framework-level sources (Spring Modulith for module boundary diagrams, OpenAPI/AsyncAPI contracts) to an agent-in-the-loop approach using Claude Haiku triggered from a GitHub Actions CI job or pre-commit hook to generate Mermaid diagrams and human-readable summaries from diffs. --- ## Review vs. verification: a distinction worth making AI code review and AI code verification are often treated as the same thing. They're not. Code review reads a PR diff and posts comments about potential issues, inferring intent from the code itself. The output varies between runs. It's intent-blind. Code verification checks the implementation against a human-approved spec with acceptance criteria, producing deterministic pass/fail results with an audit trail. It knows the approved intent upfront. Both are needed. Review handles breadth and signal across many PRs. Verification confirms that what was agreed to build is what actually got built. The deeper version of this argument is that intent needs to be a first-class artifact — not just a prompt, but something that can be mechanically reconciled against the codebase. Tools like CodeSpeak (compiler-style spec-to-code with drift detection) and Scryer (MCP-based model-driven development using C4-style system models with blast-radius diffs) are attempts at this. The current generation of markdown specs and skills files are, at best, elaborate prompt engineering. They don't enforce anything. --- ## The review load problem isn't going away Automated review tools only answer point-in-time questions about individual changes. They can't reveal week-over-week architectural drift, accumulating complexity, or spreading antipatterns. Nearly half of teams struggle to detect security issues week to week. Only 3.6% say AI-introduced issues never reach production. The mental model that's emerging: automated review for individual changes, plus continuous codebase monitoring for systemic trends. These are different problems requiring different tools. For regulated industries, the stakes are higher. The argument there is to embed verification continuously into the delivery lifecycle rather than treating it as a final review step — and to let compliance evidence emerge as a natural byproduct of engineering work rather than a separate audit process. --- ## What's actually at risk Jason Fried put it plainly: "Just because you can build more of it doesn't mean it makes it a better product. It can make it a worse product, and you're seeing that all over the place right now." The productivity narrative has a perception problem. Developers feel faster with AI while being measurably slower. The hidden costs — debugging unfamiliar AI-generated code, loss of engineering intuition, constantly shifting tooling — don't show up in the metrics teams are tracking. One experienced engineer recently stopped reviewing all AI-generated code, deciding it was pointless except for key parts of the product. That's a reasonable response to an unreasonable situation. But as Gergely Orosz noted: something needs to replace code review, because the function it served — shared understanding, architectural coherence, catching what tests don't — doesn't disappear just because the review does. The burden of proof for AI productivity claims belongs to vendors, not individual engineers. What individual engineers can do is slow down, use AI to quiz their own understanding of merged code, and stay the person on the team who actually knows what the system does. That's not a productivity metric. It's just good engineering. --- ## Where this leaves us AI is good at writing code. Writing code was never the hard part. The hard part — understanding problems, designing systems, ensuring security, maintainability, and correctness — is engineering, not programming. US labor data reflects this: the "programmer" category has shrunk while "software developer" roles are growing. AI amplifies teams with strong engineering practices and worsens those without. The practical upshot: invest in understanding, design, testing, and judgment. Keep humans in the outer loop. Build architecture that makes agent mistakes cheap to catch. And don't confuse a green test suite for a codebase you understand. ## Similar posts on daily.dev - [AI Code Review Limits: Why AI Reviewing AI Fails](https://daily.dev/posts/ai-code-review-limits-why-ai-reviewing-ai-fails-snebwlmvx) · Aviator Blog · 0 upvotes · 1 comments - [Agentic Code Review](https://daily.dev/posts/agentic-code-review-0ue8fcsa1) · Addy Osmani · 147 upvotes · 14 comments - [Code Reviews in the Age of AI](https://daily.dev/posts/code-reviews-in-the-age-of-ai-acvy63drb) · Earthly · 1 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents) [View this post on daily.dev](https://daily.dev/posts/what-makes-software-quality-and-what-agents-might-add-to-it-p6tibmmqm) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"What makes software quality, and what agents might add to it","url":"https://daily.dev/posts/what-makes-software-quality-and-what-agents-might-add-to-it-p6tibmmqm","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/what-makes-software-quality-and-what-agents-might-add-to-it-p6tibmmqm"},"datePublished":"2026-07-21T18:37:40.454Z","dateModified":"2026-07-27T14:04:45.209Z","description":"Software quality fundamentally means systems that don't break, recover quickly when they do, and know their limits. AI agents offer a practical contribution...","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/what-makes-software-quality-and-what-agents-might-add-to-it-p6tibmmqm","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"What makes software quality, and what agents might add to it"}]} ```