Part 2 of a security benchmark study examining vulnerabilities in AI-generated code across three apps (VaultBank, ClaimFlow, MedPortal) built with different stacks and AI coding tools. The study compared LLM-based scanners (ProjectDiscovery's Neo, Claude Code) against traditional SAST/DAST tools (Snyk, Invicti) across 74 confirmed vulnerabilities. Neo detected 89% of all exploitable issues including 100% of Critical/High findings with 93% precision, while Claude found 55% with 63% precision, Invicti found only Info-severity issues, and Snyk found zero valid vulnerabilities. Detailed walkthroughs show how Neo caught business logic flaws like arbitrary refund amounts, deactivated sessions retaining access, ORM-leaked password hashes, and cross-branch authorization bypass — all missed by other tools. The post also explains why Claude produced 24 false positives (mass assignment mitigated by Pydantic schemas, timing attacks impractical over HTTP, CSRF irrelevant with JWT auth). The benchmark apps and findings are open-sourced for community validation.
