<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/analysis-not-enough-to-rely-on-one-ai-model-to-discover-code-vulnerabilities-kcpdpfxvg" -->

---
title: Analysis: Not Enough to Rely on One AI Model to Discover...
description: Three AI models scanning open-source code for vulnerabilities agreed on only one finding out of 121 distinct results. Echo tested Claude Mythos, GPT-5.6 Sol...
canonical: https://daily.dev/posts/analysis-not-enough-to-rely-on-one-ai-model-to-discover-code-vulnerabilities-kcpdpfxvg
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: Analysis: Not Enough to Rely on One AI Model to Discover Code Vulnerabilities | daily.dev
og:description: Three AI models scanning open-source code for vulnerabilities agreed on only one finding out of 121 distinct results. Echo tested Claude Mythos, GPT-5.6 Sol...
og:url: https://daily.dev/posts/analysis-not-enough-to-rely-on-one-ai-model-to-discover-code-vulnerabilities-kcpdpfxvg
og:image: https://api.daily.dev/og/posts/KcPdpFxvg.png
og:image:alt: Analysis: Not Enough to Rely on One AI Model to Discover Code Vulnerabilities
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Analysis: Not Enough to Rely on One AI Model to Discover Code Vulnerabilities

**[DevOps.com](https://daily.dev/sources/devops)** · 3 min read · 0 upvotes · 0 comments

## Summary

Three AI models scanning open-source code for vulnerabilities agreed on only one finding out of 121 distinct results. Echo tested Claude Mythos, GPT-5.6 Sol and GLM-5.3: 83% of findings came from a single model, 20 from two, and just one from all three. The models also disagreed wildly on severity, rating one npm issue Critical, High and 'documented behavior' respectively, though validation rated it Low. Overall, 45% of findings were rated High or Critical, but only 26% held up after testing. Cost didn't predict coverage either: GLM-5.3 cost about $22 to run versus $303 for Mythos, yet still caught issues Mythos missed. Echo's Yuval Saggie says organizations need multiple AI models to uncover the vulnerabilities that actually exist.

## Full article

daily.dev links to this article rather than hosting it. Read it at the original source: <https://devops.com/analysis-not-enough-to-rely-on-one-ai-model-to-discover-code-vulnerabilities>

## Questions this post answers

### How much do Claude, GPT-5.6 Sol and GLM-5.3 differ when used to find vulnerabilities in open-source code?

They differ substantially. Out of 121 distinct validated findings, 83% were caught by only a single model, 20 by two models, and just one finding by all three. GLM-5.3 cost about $22 to run versus $303 for Claude Mythos and $30 for GPT-5.6 Sol, yet still found validated issues Mythos missed, showing cost does not predict coverage.

_Teams choosing an AI vulnerability scanner can compare real coverage data like this on daily.dev._

### Can AI vulnerability scanners be trusted to rate the severity of security issues correctly?

Not reliably. AI models rated 45% of 121 distinct findings as High or Critical severity, but only 26% held up at that severity after actual validation testing. One npm package issue was even rated differently by all three models tested (Critical, High, and dismissed as documented behavior), while validation rated it Low.

_Security teams tuning remediation priorities can follow findings like this on daily.dev._

### Why would a DevSecOps team need to run multiple AI models instead of just one to scan for vulnerabilities?

Because no single AI model finds everything: in Echo's analysis of 121 distinct validated vulnerability findings, 100 were found by only one model each, meaning relying on a single model misses the majority of real issues. Echo security researcher Yuval Saggie says organizations need to invoke multiple AI models to avoid discovering only a limited subset of actual vulnerabilities.

_DevSecOps teams weighing AI scanning strategies can track findings like these on daily.dev._

## Similar posts on daily.dev

- [Benchmarking 13 AI Models on Known CVE Detection](https://daily.dev/posts/benchmarking-13-ai-models-on-known-cve-detection-n1jxpyasp) · Aikido Security · 2 upvotes · 1 comments
- [Claude Mythos Preview Exposes Hidden Code Flaws Fast](https://daily.dev/posts/claude-mythos-preview-exposes-hidden-code-flaws-fast-wdwyjdo3x) · IEEE Spectrum · 0 upvotes · 0 comments

---

Tags: [#security](https://daily.dev/tags/security), [#open-source](https://daily.dev/tags/open-source), [#llm](https://daily.dev/tags/llm), [#ai-security](https://daily.dev/tags/ai-security), [#devsecops](https://daily.dev/tags/devsecops)

[View this post on daily.dev](https://daily.dev/posts/analysis-not-enough-to-rely-on-one-ai-model-to-discover-code-vulnerabilities-kcpdpfxvg)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"TechArticle","headline":"Analysis: Not Enough to Rely on One AI Model to Discover Code Vulnerabilities","url":"https://daily.dev/posts/analysis-not-enough-to-rely-on-one-ai-model-to-discover-code-vulnerabilities-kcpdpfxvg","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/analysis-not-enough-to-rely-on-one-ai-model-to-discover-code-vulnerabilities-kcpdpfxvg"},"datePublished":"2026-11-22T08:49:32.650Z","dateModified":"2026-11-22T08:50:04.262Z","description":"Three AI models scanning open-source code for vulnerabilities agreed on only one finding out of 121 distinct results. Echo tested Claude Mythos, GPT-5.6 Sol...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f433b115aed9a5f367fa373ac3a7de02?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/f433b115aed9a5f367fa373ac3a7de02?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"DevOps.com","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"DevOps.com","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/db8f2265cff0416c878c6e7e92bb8715","url":"https://daily.dev/sources/devops"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/analysis-not-enough-to-rely-on-one-ai-model-to-discover-code-vulnerabilities-kcpdpfxvg","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"security,open-source,llm,ai-security,devsecops","timeRequired":"PT3M"}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"DevOps.com","item":"https://daily.dev/sources/devops"},{"@type":"ListItem","position":3,"name":"Analysis: Not Enough to Rely on One AI Model to Discover Code Vulnerabilities"}]}
{"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/analysis-not-enough-to-rely-on-one-ai-model-to-discover-code-vulnerabilities-kcpdpfxvg#faq","mainEntity":[{"@type":"Question","name":"How much do Claude, GPT-5.6 Sol and GLM-5.3 differ when used to find vulnerabilities in open-source code?","acceptedAnswer":{"@type":"Answer","text":"They differ substantially. Out of 121 distinct validated findings, 83% were caught by only a single model, 20 by two models, and just one finding by all three. GLM-5.3 cost about $22 to run versus $303 for Claude Mythos and $30 for GPT-5.6 Sol, yet still found validated issues Mythos missed, showing cost does not predict coverage. Teams choosing an AI vulnerability scanner can compare real coverage data like this on daily.dev."}},{"@type":"Question","name":"Can AI vulnerability scanners be trusted to rate the severity of security issues correctly?","acceptedAnswer":{"@type":"Answer","text":"Not reliably. AI models rated 45% of 121 distinct findings as High or Critical severity, but only 26% held up at that severity after actual validation testing. One npm package issue was even rated differently by all three models tested (Critical, High, and dismissed as documented behavior), while validation rated it Low. Security teams tuning remediation priorities can follow findings like this on daily.dev."}},{"@type":"Question","name":"Why would a DevSecOps team need to run multiple AI models instead of just one to scan for vulnerabilities?","acceptedAnswer":{"@type":"Answer","text":"Because no single AI model finds everything: in Echo's analysis of 121 distinct validated vulnerability findings, 100 were found by only one model each, meaning relying on a single model misses the majority of real issues. Echo security researcher Yuval Saggie says organizations need to invoke multiple AI models to avoid discovering only a limited subset of actual vulnerabilities. DevSecOps teams weighing AI scanning strategies can track findings like these on daily.dev."}}]}
```

