An extended field guide on building 'software factories' - repeatable, event-driven loops that let AI coding agents triage, implement, and test work with humans in the loop at key decision points. The piece argues generating code is no longer the hard part; the real challenge is verification, trust, and knowing where human judgment (product intent, architecture, quality bar, risk assessment) still needs to be applied. Covers practical patterns like triage labels, handoff protocols, verification budgets, run-status taxonomies (success/flawed/blocked/manual), security sandboxing for untrusted inputs, and the cognitive/comprehension debt that comes from running many parallel agent sessions. Draws on the author's own 82-minute factory run experiment with a demo movies app and references similar approaches from Warp, Vercel, Factory, and HumanLayer.
Table of contents
Do you really need a software factory?What my day looks like nowMy cognitive bandwidth does not scale with the agentsA wrong-project mistakeWhen green is misleadingWhich old projects deserve another life?What happened when I built a sample oneA verification budgetWhen a run doesn’t shipAutonomy is not a single settingThe feature I had to relearnWhat parallel work does to understandingOwnership doesn’t disappearQuestions this post answers
What are the four run statuses Vercel uses in their AI software factory to decide what ships to production?
Vercel marks every agent run as success, flawed, blocked, or manual, and only success ships to production while the rest re-enter the system. Flawed means the wrong thing was implemented or context was missing. Blocked means the environment lacked something like a credential. Manual means a boundary the factory isn't yet allowed to cross. daily.dev surfaces engineering writeups like this for teams designing their own agent review gates.
How do I prevent a compromised AI coding agent run from accessing secrets it doesn't need?
Run agents in isolated sandboxes that hold only the secrets a given task needs, so a compromised run can't reach credentials outside its scope. This matters because factories that read untrusted input, such as a GitHub issue or Slack message, can be adversarial and carry supply chain attack risks, so defenses should be layered rather than relying on a single control. teams hardening agent pipelines can track sandboxing and supply-chain practices on daily.dev.
Why do individual tasks in an AI software factory take longer than expected even with fast models?
Tasks that might seem like a 10-20 minute job can take two to four times longer once verification, retries, browser checks, and human review are included. In one factory run, a favorites feature with two rejections and a human decision took 56 minutes while a simpler quick-finder feature with no rejections took only 7 minutes in the same system. developers estimating agent-driven workflows can compare real-world timing breakdowns on daily.dev.
23.4K Impressions1 Comment