AI-generated code can pass all tests while still implementing the wrong feature — because tests verify behavior, not intent. When an LLM agent fills gaps in a prompt with plausible guesses and then writes tests for its own output, the result is green CI with wrong functionality. The solution is a separate verification layer that checks implementation against a human-approved spec with explicit acceptance criteria. Aviator's experiment showed a verification agent checking 65 acceptance criteria in 6 minutes, catching 4 intent mismatches that no test would have flagged. The proposed workflow separates concerns: humans own spec approval, tests own behavior, and automated verification owns agreement between implementation and spec on every revision.

8m read timeFrom aviator.co
Post cover image
Table of contents
Two Similar, yet Different QuestionsThe Author Carried the IntentAn Agent Has No Intent, Only a PromptWhat 65 Criteria in 6 Minutes Looks LikeWhere Testing FitsWhat Changes in the PipelineWhat Comes NextFrequently Asked Questions (FAQ)
23 Impressions