--- title: Your AI agent fixed the bug, but what else did it change? description: Trail of Bits released two agent plugins, patching-bugs and regression-hunt, to let engineers delegate security patch development and regression checks to AI... canonical: https://daily.dev/posts/your-ai-agent-fixed-the-bug-but-what-else-did-it-change--inujowq00 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Your AI agent fixed the bug, but what else did it change? | daily.dev og:description: Trail of Bits released two agent plugins, patching-bugs and regression-hunt, to let engineers delegate security patch development and regression checks to AI... og:url: https://daily.dev/posts/your-ai-agent-fixed-the-bug-but-what-else-did-it-change--inujowq00 og:image: https://api.daily.dev/og/posts/INUjowq00.png og:image:alt: Your AI agent fixed the bug, but what else did it change? og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Your AI agent fixed the bug, but what else did it change? **[Trail of Bits Blog](https://daily.dev/sources/trail-of-bits)** · 9 min read · 0 upvotes · 0 comments ## Summary Trail of Bits released two agent plugins, patching-bugs and regression-hunt, to let engineers delegate security patch development and regression checks to AI agents. Built from analysis of thousands of Codex sessions and merged patches from the Patch the Planet initiative with OpenAI, the plugins enforce live proof-of-concept verification for every claim. A review of over 130 merged PRs found 21 confirmed regressions, and regression-hunt cut false-positive regression findings by 25% versus a baseline Codex agent asked to review a PR for regressions. patching-bugs uses a 'three-arm proof': the original code must fail, the patch must pass, and reverting the patch must fail again. Both tools read project-supplied THREAT_MODEL.md and PATCHING_GUIDELINES.md files to apply project-specific rules. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### How much does regression-hunt reduce false positives compared to a plain Codex agent reviewing a PR for regressions? Regression-hunt produces 25% fewer false-positive findings than a baseline Codex agent simply prompted to 'review the PR for regressions,' with a 95% confidence interval of 14% to 38%. This came from testing both approaches against a set of 21 confirmed regressions identified across more than 130 merged pull requests from Trail of Bits' Patch the Planet initiative. _Developers evaluating AI code review tools can track plugin benchmarks like this one on daily.dev._ ### What is the three-arm proof technique for verifying an AI-generated bug patch? The three-arm proof runs the same reproduction test against three states of the code: the original code must fail, the patched code must pass, and a complete revert of the patch must fail again. Reverting the fix and confirming the bug returns gives stronger evidence that the patch itself, not an unrelated environment or test change, actually fixed the bug. _Teams building trust in AI-written patches can follow verification patterns like this via daily.dev._ ### What should a THREAT_MODEL.md file for an open-source project include to guide AI patching agents? A THREAT_MODEL.md should define the assets whose compromise matters, trust boundaries, access controls, supported configurations, specifications on what is or isn't a vulnerability, and the evidence required for a valid bug report. Trail of Bits' regression-hunt plugin passes this file to every agentic step so each one checks its conclusions against it; GPAC, liboqs, and validator.js have published examples. _Maintainers documenting security rules for AI agents can find related guidance through daily.dev._ ## Similar posts on daily.dev - [1Password's AI patching benchmark is misleading](https://daily.dev/posts/1password-s-ai-patching-benchmark-is-misleading-u6zzx7cci) · Trail of Bits Blog · 0 upvotes · 0 comments - [OpenAI launches new initiative to help find and patch open-source bugs](https://daily.dev/posts/openai-launches-new-initiative-to-help-find-and-patch-open-source-bugs-cuvdbvyxk) · TechCrunch · 0 upvotes · 0 comments - [OpenAI rolls out AI-led push to fix open-source software flaws](https://daily.dev/posts/openai-rolls-out-ai-led-push-to-fix-open-source-software-flaws-xxtsat81b) · CSO Online · 1 upvotes · 0 comments - [Introducing Patch the Planet](https://daily.dev/posts/introducing-patch-the-planet-ckjfrtg29) · Trail of Bits Blog · 0 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#testing](https://daily.dev/tags/testing), [#openai-codex](https://daily.dev/tags/openai-codex) [View this post on daily.dev](https://daily.dev/posts/your-ai-agent-fixed-the-bug-but-what-else-did-it-change--inujowq00) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Your AI agent fixed the bug, but what else did it change?","url":"https://daily.dev/posts/your-ai-agent-fixed-the-bug-but-what-else-did-it-change--inujowq00","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/your-ai-agent-fixed-the-bug-but-what-else-did-it-change--inujowq00"},"datePublished":"2026-10-21T16:06:15.061Z","dateModified":"2026-10-21T16:25:55.379Z","description":"Trail of Bits released two agent plugins, patching-bugs and regression-hunt, to let engineers delegate security patch development and regression checks to AI...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/047a5714e9e4ca9fc8fb745bc84b1e92?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/047a5714e9e4ca9fc8fb745bc84b1e92?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Trail of Bits Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Trail of Bits Blog","logo":"https://media.daily.dev/image/upload/s--TLbYx4ip--/f_auto,q_auto/v1780213255/logos/trail-of-bits?_a=BAMAMiWQ0","url":"https://daily.dev/sources/trail-of-bits"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/your-ai-agent-fixed-the-bug-but-what-else-did-it-change--inujowq00","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,testing,openai-codex","timeRequired":"PT9M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Trail of Bits Blog","item":"https://daily.dev/sources/trail-of-bits"},{"@type":"ListItem","position":3,"name":"Your AI agent fixed the bug, but what else did it change?"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/your-ai-agent-fixed-the-bug-but-what-else-did-it-change--inujowq00#faq","mainEntity":[{"@type":"Question","name":"How much does regression-hunt reduce false positives compared to a plain Codex agent reviewing a PR for regressions?","acceptedAnswer":{"@type":"Answer","text":"Regression-hunt produces 25% fewer false-positive findings than a baseline Codex agent simply prompted to 'review the PR for regressions,' with a 95% confidence interval of 14% to 38%. This came from testing both approaches against a set of 21 confirmed regressions identified across more than 130 merged pull requests from Trail of Bits' Patch the Planet initiative. Developers evaluating AI code review tools can track plugin benchmarks like this one on daily.dev."}},{"@type":"Question","name":"What is the three-arm proof technique for verifying an AI-generated bug patch?","acceptedAnswer":{"@type":"Answer","text":"The three-arm proof runs the same reproduction test against three states of the code: the original code must fail, the patched code must pass, and a complete revert of the patch must fail again. Reverting the fix and confirming the bug returns gives stronger evidence that the patch itself, not an unrelated environment or test change, actually fixed the bug. Teams building trust in AI-written patches can follow verification patterns like this via daily.dev."}},{"@type":"Question","name":"What should a THREAT_MODEL.md file for an open-source project include to guide AI patching agents?","acceptedAnswer":{"@type":"Answer","text":"A THREAT_MODEL.md should define the assets whose compromise matters, trust boundaries, access controls, supported configurations, specifications on what is or isn't a vulnerability, and the evidence required for a valid bug report. Trail of Bits' regression-hunt plugin passes this file to every agentic step so each one checks its conclusions against it; GPAC, liboqs, and validator.js have published examples. Maintainers documenting security rules for AI agents can find related guidance through daily.dev."}}]} ```