--- title: Fuzzers vs. AI: Comparing Fuzzilli and BigSleep on... description: Google's Project Zero compared Fuzzilli, a generative JavaScript engine fuzzer, against BigSleep, an LLM-based agentic bug finder, on V8 over six months.... canonical: https://daily.dev/posts/fuzzers-vs-ai-comparing-fuzzilli-and-bigsleep-on-javascript-engines-1hxsqbzku twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Fuzzers vs. AI: Comparing Fuzzilli and BigSleep on JavaScript Engines | daily.dev og:description: Google's Project Zero compared Fuzzilli, a generative JavaScript engine fuzzer, against BigSleep, an LLM-based agentic bug finder, on V8 over six months.... og:url: https://daily.dev/posts/fuzzers-vs-ai-comparing-fuzzilli-and-bigsleep-on-javascript-engines-1hxsqbzku og:image: https://api.daily.dev/og/posts/1HXsqBzKu.png og:image:alt: Fuzzers vs. AI: Comparing Fuzzilli and BigSleep on JavaScript Engines og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Fuzzers vs. AI: Comparing Fuzzilli and BigSleep on JavaScript Engines **[Project Zero](https://daily.dev/sources/projectzero)** · 33 min read · 0 upvotes · 0 comments ## Summary Google's Project Zero compared Fuzzilli, a generative JavaScript engine fuzzer, against BigSleep, an LLM-based agentic bug finder, on V8 over six months. BigSleep found 45 actionable vulnerabilities versus Fuzzilli's 37, with about 25% overlap in production. A dedicated experiment re-running BigSleep against historic Fuzzilli bugs found roughly 60% overlap, rising to 70% with tweaked prompts. Overlap was lower for JIT compiler bugs and bugs that take fuzzers longer to find. The team concludes fuzzing and LLM-based agents remain complementary and recommends vendors run both. ## Full article daily.dev links to this article rather than hosting it. Read it at the original source: ## Questions this post answers ### Does an LLM-based bug finder like BigSleep find more V8 vulnerabilities than a fuzzer like Fuzzilli? In a six-month production comparison, BigSleep found more actionable vulnerabilities than Fuzzilli, 45 versus 37. The overlap between bugs found by both systems was around 25% in production, while a dedicated experiment re-running BigSleep against historic Fuzzilli bugs found roughly 60% overlap, rising to about 70% with alternate prompts. _Teams weighing AI-based security tooling against fuzzing can track real-world comparisons like this on daily.dev._ ### What kinds of bugs are LLM agents better at finding than fuzzers in a JavaScript engine? LLM agents excel at bugs requiring complex multi-step logical reasoning, specific 'magic' values like edge-case integers, newer language features a fuzzer doesn't yet support (such as 'await using' syntax), or special harnesses and flags a fuzzer isn't configured to use. One example involved a labeled break interacting with hole-check elision in a do-while loop that only an agent's hypothesis-driven reasoning uncovered. _Developers evaluating where AI auditing complements fuzzing can follow these findings on daily.dev._ ### What model did BigSleep use when it rediscovered Fuzzilli's historic V8 bugs, and was there evidence of memorization? The experiment used Gemini 3.6 flash, whose knowledge cutoff was March 2026, giving an effective cutoff around January 2026 for publicly visible Fuzzilli bugs. Looking at success rates across time periods, there was no statistically significant amount of memorization detected for bugs from before 2026, though this should be re-checked for future model generations. _Anyone benchmarking LLM coding or security agents can track model-specific caveats like this on daily.dev._ ## Similar posts on daily.dev - [How Fuzz Testing for APIs Is Evolving](https://daily.dev/posts/how-fuzz-testing-for-apis-is-evolving-b1q8blq2a) · Nordic APIs · 1 upvotes · 0 comments - [Security in the age of AI](https://daily.dev/posts/security-in-the-age-of-ai-x8h2ntqmh) · Flavio Copes · 0 upvotes · 0 comments - [NDSS 2025 -DUMPLING: Fine-Grained Differential JavaScript Engine Fuzzing](https://daily.dev/posts/ndss-2025--dumpling-fine-grained-differential-javascript-engine-fuzzing-jfaakjza5) · Security Boulevard · 0 upvotes · 0 comments --- Tags: [#javascript](https://daily.dev/tags/javascript), [#ai-agents](https://daily.dev/tags/ai-agents), [#vulnerability](https://daily.dev/tags/vulnerability), [#v8](https://daily.dev/tags/v8) [View this post on daily.dev](https://daily.dev/posts/fuzzers-vs-ai-comparing-fuzzilli-and-bigsleep-on-javascript-engines-1hxsqbzku) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Fuzzers vs. AI: Comparing Fuzzilli and BigSleep on JavaScript Engines","url":"https://daily.dev/posts/fuzzers-vs-ai-comparing-fuzzilli-and-bigsleep-on-javascript-engines-1hxsqbzku","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/fuzzers-vs-ai-comparing-fuzzilli-and-bigsleep-on-javascript-engines-1hxsqbzku"},"datePublished":"2026-10-27T17:36:43.508Z","dateModified":"2026-10-27T17:54:20.693Z","description":"Google's Project Zero compared Fuzzilli, a generative JavaScript engine fuzzer, against BigSleep, an LLM-based agentic bug finder, on V8 over six months....","image":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b216c784c5a6665462be9791804b24ad?_a=AQAEuop","thumbnailUrl":"https://media.daily.dev/image/upload/f_auto,q_auto/v1/posts/b216c784c5a6665462be9791804b24ad?_a=AQAEuop","isAccessibleForFree":true,"articleSection":"Project Zero","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Project Zero","logo":"https://media.daily.dev/image/upload/t_logo,f_auto/v1/logos/36643542e8f34b1cbc6adb6494f91925","url":"https://daily.dev/sources/projectzero"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/fuzzers-vs-ai-comparing-fuzzilli-and-bigsleep-on-javascript-engines-1hxsqbzku","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":0},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"javascript,ai-agents,vulnerability,v8","timeRequired":"PT33M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Project Zero","item":"https://daily.dev/sources/projectzero"},{"@type":"ListItem","position":3,"name":"Fuzzers vs. AI: Comparing Fuzzilli and BigSleep on JavaScript Engines"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/fuzzers-vs-ai-comparing-fuzzilli-and-bigsleep-on-javascript-engines-1hxsqbzku#faq","mainEntity":[{"@type":"Question","name":"Does an LLM-based bug finder like BigSleep find more V8 vulnerabilities than a fuzzer like Fuzzilli?","acceptedAnswer":{"@type":"Answer","text":"In a six-month production comparison, BigSleep found more actionable vulnerabilities than Fuzzilli, 45 versus 37. The overlap between bugs found by both systems was around 25% in production, while a dedicated experiment re-running BigSleep against historic Fuzzilli bugs found roughly 60% overlap, rising to about 70% with alternate prompts. Teams weighing AI-based security tooling against fuzzing can track real-world comparisons like this on daily.dev."}},{"@type":"Question","name":"What kinds of bugs are LLM agents better at finding than fuzzers in a JavaScript engine?","acceptedAnswer":{"@type":"Answer","text":"LLM agents excel at bugs requiring complex multi-step logical reasoning, specific 'magic' values like edge-case integers, newer language features a fuzzer doesn't yet support (such as 'await using' syntax), or special harnesses and flags a fuzzer isn't configured to use. One example involved a labeled break interacting with hole-check elision in a do-while loop that only an agent's hypothesis-driven reasoning uncovered. Developers evaluating where AI auditing complements fuzzing can follow these findings on daily.dev."}},{"@type":"Question","name":"What model did BigSleep use when it rediscovered Fuzzilli's historic V8 bugs, and was there evidence of memorization?","acceptedAnswer":{"@type":"Answer","text":"The experiment used Gemini 3.6 flash, whose knowledge cutoff was March 2026, giving an effective cutoff around January 2026 for publicly visible Fuzzilli bugs. Looking at success rates across time periods, there was no statistically significant amount of memorization detected for bugs from before 2026, though this should be re-checked for future model generations. Anyone benchmarking LLM coding or security agents can track model-specific caveats like this on daily.dev."}}]} ```