Everyone building a software factory wants the same proof
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Engineering leaders from HumanLayer, Warp, AWS and Asana discuss how autonomous 'software factories' - pipelines where AI agents triage, spec, implement, review and ship code - actually work in practice. All agree that humans must retain ownership and accountability even when agents open pull requests, and that code review, not code generation, has become the real bottleneck. LinearB's 2026 mid-year benchmarks show agentic pull requests merge at only 37% in weaker organizations versus 79% in stronger ones, with AI code review coverage ranging from 8% to 57% across the community. Dex Horthy's SlopCodeBench experiments found unreviewed AI output had over 11 times more 'slop' triggers per thousand lines of code than reviewed code, leading him to conclude current models can't be trusted to run fully autonomously without human steering.
Table of contents
Who owns the work when an agent opens the pull request?A benchmark can score a model and still tell you nothing about your factoryThe equation is only as good as the dataThe bottleneck moved to review, and everyone is feeling the strainThe ownership puzzle and proof beyond pull requestsThis is the start of a conversation the industry hasn’t had yetQuestions this post answers
How much more likely are AI-reviewed pull requests to merge quickly compared to human-only reviewed ones?
At the top-performing organizations, pull requests reviewed by AI merge at the same 92% rate as those reviewed only by humans, but the gap widens further down: AI-reviewed PRs merge at 86% versus 79% for human-only review in the weakest organizations. AI review coverage itself is uneven, ranging from 57% of PRs at top organizations down to just 8% at the bottom, according to LinearB's 2026 mid-year benchmarks covering 2.7 million pull requests. Teams weighing AI code review adoption track benchmark data like this on daily.dev.
Can current AI coding agents run unsupervised without a human reviewing their code?
No, according to Dex Horthy's testing with SlopCodeBench, a long-horizon benchmark that withholds requirements upfront so agents must keep adapting. Running three Claude models through a 17-checkpoint subset, he found over 11 times more 'slop' triggers per thousand lines of code in unreviewed lights-off output compared to HumanLayer's own reviewed TypeScript codebase, concluding models still need steering. Developers evaluating how far to trust autonomous coding agents follow findings like this on daily.dev.
What is the merge rate difference between AI agent-generated pull requests and regular pull requests?
Agentic pull requests merge within 30 days at only 37% in the lowest-performing organizations, compared to 81% for all pull requests in those same organizations; at the strongest organizations agentic PRs merge at 79% versus 90% for all PRs. This gap, drawn from LinearB's 2026 mid-year benchmarks of 2.7 million pull requests across 253 organizations, shows agent-authored work stalling more than human work even within the same teams. Engineering leaders benchmarking their own AI adoption compare stats like these on daily.dev.