A structured benchmark comparing five AI models (Claude Opus 4.8, Claude Sonnet 4.6, Grok, GPT-5.5, Gemini 3.1 Pro) on their ability to detect 15 planted bugs in a React/TypeScript budget app using Kilo's Code Reviewer feature. Opus 4.8 caught the most bugs (10/15), while Grok tied Sonnet at 9/15 for a fraction of the cost (~$0.08). Gemini 3.1 Pro performed worst, missing obvious bugs caught by cheaper models. Four bugs survived all five reviews — those requiring domain understanding of budget semantics rather than code-level pattern matching. The key takeaway: combining two models with complementary strengths (e.g., Grok + Sonnet) outperforms any single model, and cost does not reliably predict detection quality.
Table of contents
Why a Budget AppThe PromptResultsDetection by CategoryWhat Each Model Did WellWhat All Five MissedCost vs CoverageVerdict1.5K Impressions1 Comment