Snyk ran 300 repeated vulnerability-finding scans across 10 JavaScript fixtures to measure how repeatable LLM-based security reviews are compared to deterministic SAST. Key findings: LLM reference-matched findings were highly stable (85% consistent across all 5 runs), but extra LLM-only reports were highly inconsistent — nearly 50% appeared in only 1 of 5 identical runs. The best LLM configuration (Claude Opus 4.6 Medium) reached 75.4% F1 against Snyk Code's reference set, leaving a 24.6-point gap. More expensive models (Claude Opus 4.7 Max) cost 5.7x more but scored lower. LLMs excelled at high-signal exploit shapes (command injection, SQLi, SSRF) but missed systematic patterns like repeated path traversal sinks and resource-limit findings. The data supports combining LLM review with SAST rather than replacing one with the other.
