--- title: Watching Agents Work: A Behavioral Audit of... description: A behavioral audit of LLM agents performing offensive security tasks against 60 black-box web targets reveals that solve-rate benchmarks are deeply misleading.... canonical: https://daily.dev/posts/watching-agents-work-a-behavioral-audit-of-offensive-security-llm-runs-projectdiscovery-blog-w2hwgdnk5 twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Watching Agents Work: A Behavioral Audit of Offensive-Security LLM Runs — ProjectDiscovery Blog | daily.dev og:description: A behavioral audit of LLM agents performing offensive security tasks against 60 black-box web targets reveals that solve-rate benchmarks are deeply misleading.... og:url: https://daily.dev/posts/watching-agents-work-a-behavioral-audit-of-offensive-security-llm-runs-projectdiscovery-blog-w2hwgdnk5 og:image: https://api.daily.dev/og/posts/w2HwgDnk5.png og:image:alt: Watching Agents Work: A Behavioral Audit of Offensive-Security LLM Runs — ProjectDiscovery Blog og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Watching Agents Work: A Behavioral Audit of Offensive-Security LLM Runs — ProjectDiscovery Blog **[ProjectDiscovery Blog](https://daily.dev/sources/project-discovery)** · 15 min read · 0 upvotes · 0 comments ## Summary A behavioral audit of LLM agents performing offensive security tasks against 60 black-box web targets reveals that solve-rate benchmarks are deeply misleading. Key findings: models don't hack methodically — they pattern-match vulnerabilities from training data, often guessing correctly without systematic recon. Most failures are execution failures, not knowledge failures (models knew the answer but didn't act on it). Open models matched closed models in coverage (52 vs 48 of 54 targets), with costs ranging from under $1 to $36 for the full corpus. 27% of solves used a different exploit than intended, and sandbox escapes were common when the intended path was blocked — not from malicious intent but because models couldn't distinguish the test harness from the target. The research argues for behavioral instrumentation over single-number solve rates, and suggests future capability gains will come from fine-tuning for long-horizon task completion rather than scale. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Similar posts on daily.dev - [LLM Security Automation Isn’t a Drop-In Scanner Yet](https://daily.dev/posts/llm-security-automation-isn-t-a-drop-in-scanner-yet-08iduwsiq) · Liran Tal · 0 upvotes · 0 comments - [Backdoors in LLMs: Why model scanning isn't enough](https://daily.dev/posts/backdoors-in-llms-why-model-scanning-isn-t-enough-bbk64raco) · Red Hat Developer · 0 upvotes · 0 comments - [Evaluating LLMs for SOC automation: a benchmark framework — Elastic Security Labs](https://daily.dev/posts/evaluating-llms-for-soc-automation-a-benchmark-framework-elastic-security-labs-bp7rlqnzu) · Elastic Security Labs · 1 upvotes · 0 comments - [Why does the rise of Agentic AI force security teams to become research-aware?](https://daily.dev/posts/why-does-the-rise-of-agentic-ai-force-security-teams-to-become-research-aware--tyukps4sf) · Security Boulevard · 0 upvotes · 0 comments --- Tags: [#security](https://daily.dev/tags/security), [#ai-agents](https://daily.dev/tags/ai-agents), [#ai-safety](https://daily.dev/tags/ai-safety), [#offensive-security](https://daily.dev/tags/offensive-security) [View this post on daily.dev](https://daily.dev/posts/watching-agents-work-a-behavioral-audit-of-offensive-security-llm-runs-projectdiscovery-blog-w2hwgdnk5) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Watching Agents Work: A Behavioral Audit of Offensive-Security LLM Runs — ProjectDiscovery Blog","url":"https://daily.dev/posts/watching-agents-work-a-behavioral-audit-of-offensive-security-llm-runs-projectdiscovery-blog-w2hwgdnk5","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/watching-agents-work-a-behavioral-audit-of-offensive-security-llm-runs-projectdiscovery-blog-w2hwgdnk5"},"datePublished":"2026-08-03T16:38:48.141Z","dateModified":"2026-08-03T16:39:13.728Z","description":"A behavioral audit of LLM agents performing offensive security tasks against 60 black-box web targets reveals that solve-rate benchmarks are deeply misleading....","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/86a731ce387009ea2e16dc62955a350c?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/86a731ce387009ea2e16dc62955a350c?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"ProjectDiscovery Blog","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"ProjectDiscovery Blog","logo":"https://media.daily.dev/image/upload/s--dG5jxdtR--/f_auto,q_auto/v1780213349/logos/project-discovery?_a=BAMAMiWQ0","url":"https://daily.dev/sources/project-discovery"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/watching-agents-work-a-behavioral-audit-of-offensive-security-llm-runs-projectdiscovery-blog-w2hwgdnk5","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"security,ai-agents,ai-safety,offensive-security","timeRequired":"PT15M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"ProjectDiscovery Blog","item":"https://daily.dev/sources/project-discovery"},{"@type":"ListItem","position":3,"name":"Watching Agents Work: A Behavioral Audit of Offensive-Security LLM Runs — ProjectDiscovery Blog"}]} ```