L3: Human-in-the-loop management
You mostly stop typing code. Agents run several tasks at once and your day becomes reviewing their work. Most people never go past this level.
The path.
- Own the outer loop In Agentic Engineering: Core Ideas "Agentic coding" is an overloaded term.
- Agentic code review In Verification & Testing for Agents If generation is cheap and verification is the bottleneck, then verification infrastructure is your leverage.
- Role separation In Orchestration Patterns Once one agent works, the obvious move is to run several. This is where the biggest gains and the biggest self-inflicted wounds both live.
- MCP & tools Models are only half the story. The other half is the plumbing that gives models hands: how an assistant reaches your files, databases, SaaS tools, and the web.
Check yourself.
One question per step. Take it cold to find where to start, or after reading to see what stuck. Nobody's grading you.
Best so far:
Question 1 of 4
-
As agents run the inner loop, where should you place yourself?
Owning the outer loop means placing yourself in the constraint-setting and sampling loops (installing checks that produce evidence and rendering the ship/no-ship verdict), not executing steps.
Covered in step 1 · Own the outer loop -
In agentic code review, what does the human's job become?
Review stops being a human reading every line and becomes a pipeline of checks (tests, type checks, linters, a reviewer agent) that produces evidence a human then judges, with an accountability contract instead of a gut-feel thumbs-up.
Covered in step 2 · Agentic code review -
Your checker agent flags gaps in a diff. How should you treat its report?
A reviewer prompted to find gaps will usually report some even when the work is sound, so the human above the maker/checker pair judges the judge's output too.
Covered in step 3 · Role separation -
What problem does MCP actually solve?
Before MCP, every AI app needed a bespoke integration for every tool; now one MCP server works with any MCP-speaking client, while function calling still runs underneath.
Covered in step 4 · MCP & tools
-
L0: Manual
The ground floor: AI as a search engine and occasional tab-complete. No real productivity change.
Everyone starts here. There's nothing to unlock yet.
-
L1: Discrete task offloading
You hand the AI small, self-contained jobs: write the unit tests, draft the docstring, generate one well-scoped function. The real work is still yours.
Chat assistants · Prompting 101 · IDE assistants
Explore path -
L2: Active pairing
The AI writes alongside you all day, handling the boring parts while you steer and review everything as it lands. Shapiro estimates about 90% of AI-native developers live here.
CLI agents · AGENTS.md & CLAUDE.md · What an agent actually is
Explore path -
L3: Human-in-the-loop management
You mostly stop typing code. Agents run several tasks at once and your day becomes reviewing their work. Most people never go past this level.
Own the outer loop · Agentic code review · Role separation · MCP & tools
You're on this rung
-
L4: Specification-driven
You define what to build instead of building it: write the spec, set the tests, judge the results. Leave for twelve hours and come back to a green suite.
The harness as an artifact · Evals-as-tests · LLMOps & guardrails
Explore path -
L5: The dark factory
Agents building the software with humans out of the loop. Shapiro reports it working only for teams under five people, and calls it likely our future.
Swarms & fleets · The factory model · Human factors
Explore path