--- title: Using LLMs to Find and Prioritize Vulnerabilities Is No... description: Current LLM-based application security scanning tools still produce over 60% false positives, flag unreachable code, and lack the contextual awareness needed... canonical: https://daily.dev/posts/using-llms-to-find-and-prioritize-vulnerabilities-is-no-easy-task-dtmgdglfx twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Using LLMs to Find and Prioritize Vulnerabilities Is No Easy Task | daily.dev og:description: Current LLM-based application security scanning tools still produce over 60% false positives, flag unreachable code, and lack the contextual awareness needed... og:url: https://daily.dev/posts/using-llms-to-find-and-prioritize-vulnerabilities-is-no-easy-task-dtmgdglfx og:image: https://api.daily.dev/og/posts/dtmgdGlfx.png og:image:alt: Using LLMs to Find and Prioritize Vulnerabilities Is No Easy Task og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Using LLMs to Find and Prioritize Vulnerabilities Is No Easy Task **[Dark Reading](https://daily.dev/sources/dr)** · 5 min read · 5 upvotes · 1 comments ## Summary Current LLM-based application security scanning tools still produce over 60% false positives, flag unreachable code, and lack the contextual awareness needed for accurate vulnerability triage. The core problem is that LLMs are generalists without organizational, technical, or code context — leading to inconsistent results and low developer trust. Effective triage requires three types of context and a deterministic harness around AI models. Reachability analysis remains one of the most effective ways to reduce noise, with studies showing 62% of open source libraries are never used at runtime. These findings will be presented at Black Hat USA in a session on AI approaches to vulnerability classification. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Why do LLM-based vulnerability scanners have such high false-positive rates? In tests across more than a dozen application-scanning tools, over 60% of flagged vulnerabilities turned out to be false positives, located in unreachable code, or of low severity. Pixee's CTO Arshan Dabirsiaghi attributes this to LLMs lacking organizational, technical, and code context, and models generalizing rather than specializing in vulnerability analysis, causing inconsistent conclusions on repeated scans. _Teams weighing AI scanners against traditional tools can track real-world AppSec tradeoffs on daily.dev._ ### What is reachability analysis and why does it matter for reducing vulnerability noise? Reachability analysis checks whether flagged vulnerable code actually executes at runtime, filtering out irrelevant findings. One study found 62% of open source libraries are never used at runtime, and another found only 12% of the open source Java code in applications is actually used, meaning most flagged issues in unused code can be safely deprioritized. _Developers cutting through scanner noise can follow reachability analysis techniques via daily.dev._ ### Why is severity scoring for MD5 hashing usage inconsistent in vulnerability scans? MD5 usage is often defaulted to medium severity, but its real risk depends entirely on context: using MD5 to hash passwords or secrets makes it a high-severity issue, while using it purely for collision resistance in a non-security context makes it low severity or a false positive. Scanners without this contextual understanding misclassify the finding either way. _Security engineers refining triage rules can compare context-aware scanning approaches on daily.dev._ ## Community discussion Top comments from developers on daily.dev. **@kartiknvj** · 0 upvotes > The 60% false positive number tracks with what I see when a generalist model triages code it has never seen. In my own experiments, feeding call graphs and reachability data alongside the finding drops the FP rate more than any prompt rewrite does. Do the authors quantify how much of the remaining error is unreachable-code flags versus missing business context? ## Similar posts on daily.dev - [AI floods security teams with flaws — business context sets priorities](https://daily.dev/posts/ai-floods-security-teams-with-flaws-business-context-sets-priorities-c6axuh5re) · The New Stack · 0 upvotes · 0 comments - [Using LLMs to Find Security Bugs: A Practitioner’s Playbook](https://daily.dev/posts/using-llms-to-find-security-bugs-a-practitioner-s-playbook-llrqsqe4g) · Tech Lead Digest · 0 upvotes · 0 comments - [Pen tests show AI security flaws far more severe than legacy software bugs](https://daily.dev/posts/pen-tests-show-ai-security-flaws-far-more-severe-than-legacy-software-bugs-timmk64xi) · CSO Online · 0 upvotes · 0 comments - [Don't Let the AI Find Your Bugs. Let It Judge Them.](https://daily.dev/posts/don-t-let-the-ai-find-your-bugs-let-it-judge-them--fpqkjl4lo) · Awesome Java Newsletter · 2 upvotes · 1 comments - [Why AI Changes the Risk Model for Application Security](https://daily.dev/posts/why-ai-changes-the-risk-model-for-application-security-rxzpdmwfj) · Security Boulevard · 1 upvotes · 0 comments --- Tags: [#security](https://daily.dev/tags/security), [#llm](https://daily.dev/tags/llm), [#appsec](https://daily.dev/tags/appsec) [View this post on daily.dev](https://daily.dev/posts/using-llms-to-find-and-prioritize-vulnerabilities-is-no-easy-task-dtmgdglfx) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Using LLMs to Find and Prioritize Vulnerabilities Is No Easy Task","url":"https://daily.dev/posts/using-llms-to-find-and-prioritize-vulnerabilities-is-no-easy-task-dtmgdglfx","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/using-llms-to-find-and-prioritize-vulnerabilities-is-no-easy-task-dtmgdglfx"},"datePublished":"2026-07-21T21:44:51.976Z","dateModified":"2026-09-13T18:28:12.937Z","description":"Current LLM-based application security scanning tools still produce over 60% false positives, flag unreachable code, and lack the contextual awareness needed...","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/11c5ee625334ba6e1920808be03aa5b1?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/11c5ee625334ba6e1920808be03aa5b1?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Dark Reading","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Dark Reading","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/dr","url":"https://daily.dev/sources/dr"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/using-llms-to-find-and-prioritize-vulnerabilities-is-no-easy-task-dtmgdglfx","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":5},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"security,llm,appsec","timeRequired":"PT5M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Dark Reading","item":"https://daily.dev/sources/dr"},{"@type":"ListItem","position":3,"name":"Using LLMs to Find and Prioritize Vulnerabilities Is No Easy Task"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/using-llms-to-find-and-prioritize-vulnerabilities-is-no-easy-task-dtmgdglfx","comment":[{"@type":"Comment","text":"The 60% false positive number tracks with what I see when a generalist model triages code it has never seen. In my own experiments, feeding call graphs and reachability data alongside the finding drops the FP rate more than any prompt rewrite does. Do the authors quantify how much of the remaining error is unreachable-code flags versus missing business context?","datePublished":"2026-07-22T18:58:32.806Z","url":"https://daily.dev/posts/dtmgdGlfx#c-ZchHgpMBt","author":{"@type":"Person","name":"kartik-nvjk","url":"https://daily.dev/kartiknvj","image":"https://media.daily.dev/image/upload/s--3gGgsVCw--/f_auto/v1781456774/avatars/avatar_TvTVeiMdkRCqWUDullFmy?_a=BAMAMiWQ0"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/using-llms-to-find-and-prioritize-vulnerabilities-is-no-easy-task-dtmgdglfx#faq","mainEntity":[{"@type":"Question","name":"Why do LLM-based vulnerability scanners have such high false-positive rates?","acceptedAnswer":{"@type":"Answer","text":"In tests across more than a dozen application-scanning tools, over 60% of flagged vulnerabilities turned out to be false positives, located in unreachable code, or of low severity. Pixee's CTO Arshan Dabirsiaghi attributes this to LLMs lacking organizational, technical, and code context, and models generalizing rather than specializing in vulnerability analysis, causing inconsistent conclusions on repeated scans. Teams weighing AI scanners against traditional tools can track real-world AppSec tradeoffs on daily.dev."}},{"@type":"Question","name":"What is reachability analysis and why does it matter for reducing vulnerability noise?","acceptedAnswer":{"@type":"Answer","text":"Reachability analysis checks whether flagged vulnerable code actually executes at runtime, filtering out irrelevant findings. One study found 62% of open source libraries are never used at runtime, and another found only 12% of the open source Java code in applications is actually used, meaning most flagged issues in unused code can be safely deprioritized. Developers cutting through scanner noise can follow reachability analysis techniques via daily.dev."}},{"@type":"Question","name":"Why is severity scoring for MD5 hashing usage inconsistent in vulnerability scans?","acceptedAnswer":{"@type":"Answer","text":"MD5 usage is often defaulted to medium severity, but its real risk depends entirely on context: using MD5 to hash passwords or secrets makes it a high-severity issue, while using it purely for collision resistance in a non-security context makes it low severity or a false positive. Scanners without this contextual understanding misclassify the finding either way. Security engineers refining triage rules can compare context-aware scanning approaches on daily.dev."}}]} ```