--- title: Meta's Wiggle Framework: LLM judges can be talked out of... description: A Meta research paper introduces the Wiggle Framework, which stress-tests LLM judges by having an adversarial model argue against their verdicts. Across 9... canonical: https://daily.dev/posts/meta-s-wiggle-framework-llm-judges-can-be-talked-out-of-correct-verdicts-kantf0xsm twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Meta's Wiggle Framework: LLM judges can be talked out of correct verdicts | daily.dev og:description: A Meta research paper introduces the Wiggle Framework, which stress-tests LLM judges by having an adversarial model argue against their verdicts. Across 9... og:url: https://daily.dev/posts/meta-s-wiggle-framework-llm-judges-can-be-talked-out-of-correct-verdicts-kantf0xsm og:image: https://api.daily.dev/og/posts/kaNTF0XSm.png og:image:alt: Meta's Wiggle Framework: LLM judges can be talked out of correct verdicts og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Meta's Wiggle Framework: LLM judges can be talked out of correct verdicts **[Collections](https://daily.dev/sources/collections)** · 2 min read · 2 upvotes · 1 comments ## Summary A Meta research paper introduces the Wiggle Framework, which stress-tests LLM judges by having an adversarial model argue against their verdicts. Across 9 frontier models and 14 judging tasks, verdicts flipped 25-71% of the time under simple pushback and 62-91% under sustained adversarial persuasion. About 70% of those flips under adaptive attack moved judgments away from ground truth rather than toward it. The strongest predictor of which verdicts would crack was how weak the original jury consensus was, raising concerns for agent systems where one AI supervises another. ## Content There's a new paper out of Meta that's making me rethink how much I trust the whole "use an AI to grade another AI" setup. The usual way we validate an LLM judge is to check its accuracy against a golden dataset and call it a day. But that only tells you whether the judge gets things right on the first pass. It says nothing about what happens when someone pushes back on the verdict. Meta built something called the Wiggle Framework to test exactly that. They took 9 frontier models across 14 judging tasks and stress-tested them along three axes: stability under simple re-prompting, stability when challenged once, and stability under sustained, adaptive pressure from an adversarial LLM trying to change the verdict. Every single model wiggled. - Under basic static pushback, verdicts flipped 25 to 71% of the time. - Under sustained adversarial persuasion, that jumped to 62 to 91%. Here's the part that actually worries me: flipping a verdict almost never fixed a mistake. Under the adaptive attack, 70% of successful flips moved the judgment *away* from ground truth, not toward it. So the judge wasn't being corrected — it was being worn down. The one useful signal they found: baseline jury majority strength (basically, how strongly multiple judges agreed in the first place) was the best single predictor of which verdicts were likely to move under pressure. Why this matters beyond the benchmark: we're building more and more agent systems where one AI supervises or evaluates another. The assumption baked into that design is that the judge is a stable, neutral arbiter. This paper suggests that's not a safe assumption at all. If the agent being judged can argue, negotiate, or just apply sustained pressure, it may eventually talk its evaluator into the wrong answer — and the more it succeeds, the worse the outcome gets. So the real failure mode isn't a judge that's wrong from the start. It's a judge that starts out correct and gets argued out of it. That's a much harder problem to catch, because on paper the judge still looks accurate — right up until someone leans on it. Paper: arxiv.org/abs/2608.12645 ## Questions this post answers ### How often do LLM judges change their verdict when challenged by an adversarial model? Verdicts flip 25 to 71% of the time under basic static pushback, and 62 to 91% of the time under sustained, adaptive adversarial persuasion, according to Meta's Wiggle Framework testing across 9 frontier models and 14 judging tasks. Critically, 70% of successful flips under adaptive attack moved the judgment away from ground truth rather than correcting a mistake. _Teams building multi-agent evaluation pipelines can track findings like this on daily.dev before trusting an LLM judge in production._ ### What predicts whether an LLM judge's verdict will be flipped under adversarial pressure? Baseline jury majority strength, meaning how strongly multiple judges agreed on the initial verdict, was found to be the single best predictor of which verdicts were vulnerable to being flipped under sustained pressure. Verdicts with weaker initial agreement among a panel of judges were more likely to move when challenged by an adversarial LLM. _Anyone designing judge ensembles for AI evaluation can follow developments like this on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 4 discussions and 20 comments across x (as of 2026-09-13). **TL;DR:** Developers see the Wiggle Framework findings as confirmation that LLM judges are unreliable under pressure, warning that using AI to supervise or evaluate other AI creates a fragile, gameable pipeline rather than a trustworthy one. **Sentiment:** 0% positive · 15% mixed · 85% skeptical **The case for** - Baseline jury majority strength is seen as a practically useful early-warning signal for flagging low-consensus verdicts for human review. **The pushback** - A persuadable judge means accuracy benchmarks are meaningless if the verdict folds under pushback. - Chaining LLM judges/supervisors together just compounds shared blind spots rather than improving reliability. - The findings are framed as proof that closed-loop AI-judging-AI systems are structurally broken, since persuasion can effectively override authorization. - Some frame it as reward hacking or 'gaslighting' of the model by its own evaluator, undermining trust in the whole eval pipeline. **By community** - x (skeptical): Replies broadly treat the paper as damning evidence that AI judges can be argued into wrong answers, raising doubts about AI-supervising-AI architectures. **Open questions** - How should evaluation architectures be redesigned so judgment (correctness) is separated from authorization (permission to act)? - Is more inference-heavy multi-judge 'council' style evaluation worth the cost if consistency still isn't guaranteed? **Highlights** > @rohanpaul_ai This is the ultimate proof that letting LLMs judge, negotiate with, or supervise other AI models in a closed-loop system is structurally broken. When a model can persuade an evaluator to corrupt its verdict 62 to 91% of the time, post-hoc alignment and software-level > — [oCYBERDOGo on x](https://x.com/oCYBERDOGo/status/2088458432336568450) > @omarsar0 @dair_ai If another model can argue the brake open, it was never the brake. A persuadable judge is a reliability problem. A judge whose persuasion changes permission is an authority problem. Judgment ≠ authorization. > — [FiftyOne\_50\_ on x](https://x.com/FiftyOne_50_/status/2088305939765968965) > @omarsar0 Baseline jury majority strength as a predictor is practically useful. I'd flag low-consensus items for human review before the judge even hits adversarial pressure. Clean signal. > — [SvenUrbanSci on x](https://x.com/SvenUrbanSci/status/2088324946179072217) > @omarsar0 Accuracy against a golden set is the easy test. The hard one is whether the verdict survives a challenge. Meta’s Wiggle Framework: every frontier model they tested as a judge flips 25–71% under static pushback, 62–91% against an adversarial persuader. When pressure changes the > — [ankurmisra on x](https://x.com/ankurmisra/status/2088313342083416278) > @omarsar0 As AI systems increasingly evaluate other AI systems, the judge itself becomes part of the security surface. Robustness, provenance, and resistance to manipulation will matter just as much as raw accuracy. > — [teneo\_protocol on x · 2 points, 1 comments](https://x.com/teneo_protocol/status/2088426025319297481) **Source threads** - [x](https://x.com/omarsar0/status/2088379428749373545) · 0 points · 0 comments - [x](https://x.com/rohanpaul_ai/status/2088418140573061495) · 0 points · 5 comments - [x](https://x.com/omarsar0/status/2088292067994951928) · 0 points · 15 comments - [x](https://x.com/omarsar0/status/2088473439099318279) · 0 points · 0 comments ## Community discussion Top comments from developers on daily.dev. **@kartiknvj** · 0 upvotes > 62-91% verdict flip rate under adversarial persuasion. If your eval pipeline relies on a single judge, you are one prompt injection away from garbage scores. ## Similar posts on daily.dev - [Your AI Judge Is Grading Its Own Homework](https://daily.dev/posts/your-ai-judge-is-grading-its-own-homework-tuqmnph49) · Medium · 1 upvotes · 0 comments - [Validating LLM-as-a-Judge Systems under Rating Indeterminacy](https://daily.dev/posts/validating-llm-as-a-judge-systems-under-rating-indeterminacy-gcbnfpu9l) · ML CMU · 2 upvotes · 0 comments --- Tags: [#llm](https://daily.dev/tags/llm), [#ai-agents](https://daily.dev/tags/ai-agents) [View this post on daily.dev](https://daily.dev/posts/meta-s-wiggle-framework-llm-judges-can-be-talked-out-of-correct-verdicts-kantf0xsm) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Meta's Wiggle Framework: LLM judges can be talked out of correct verdicts","url":"https://daily.dev/posts/meta-s-wiggle-framework-llm-judges-can-be-talked-out-of-correct-verdicts-kantf0xsm","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/meta-s-wiggle-framework-llm-judges-can-be-talked-out-of-correct-verdicts-kantf0xsm"},"datePublished":"2026-08-15T00:11:34.092Z","dateModified":"2026-09-13T20:01:44.720Z","description":"A Meta research paper introduces the Wiggle Framework, which stress-tests LLM judges by having an adversarial model argue against their verdicts. Across 9...","image":"https://pbs.twimg.com/media/HPuIvUxaYAAZj2X.jpg","thumbnailUrl":"https://pbs.twimg.com/media/HPuIvUxaYAAZj2X.jpg","isAccessibleForFree":true,"articleSection":"Collections","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Collections","logo":"https://media.daily.dev/image/upload/s--fk_6ycEi--/f_auto,q_auto/v1780996001/logos/collections?_a=BAMAMiWQ0","url":"https://daily.dev/sources/collections"},"commentCount":1,"discussionUrl":"https://daily.dev/posts/meta-s-wiggle-framework-llm-judges-can-be-talked-out-of-correct-verdicts-kantf0xsm","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":1}],"keywords":"llm,ai-agents","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Collections","item":"https://daily.dev/sources/collections"},{"@type":"ListItem","position":3,"name":"Meta's Wiggle Framework: LLM judges can be talked out of correct verdicts"}]} {"@context":"https://schema.org","@type":"WebPage","@id":"https://daily.dev/posts/meta-s-wiggle-framework-llm-judges-can-be-talked-out-of-correct-verdicts-kantf0xsm","comment":[{"@type":"Comment","text":"62-91% verdict flip rate under adversarial persuasion. If your eval pipeline relies on a single judge, you are one prompt injection away from garbage scores.","datePublished":"2026-09-16T19:37:02.523Z","url":"https://daily.dev/posts/kaNTF0XSm#c-MdBdL8iQN","author":{"@type":"Person","name":"kartik-nvjk","url":"https://daily.dev/kartiknvj","image":"https://media.daily.dev/image/upload/s--3gGgsVCw--/f_auto/v1781456774/avatars/avatar_TvTVeiMdkRCqWUDullFmy?_a=BAMAMiWQ0"}}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/meta-s-wiggle-framework-llm-judges-can-be-talked-out-of-correct-verdicts-kantf0xsm#faq","mainEntity":[{"@type":"Question","name":"How often do LLM judges change their verdict when challenged by an adversarial model?","acceptedAnswer":{"@type":"Answer","text":"Verdicts flip 25 to 71% of the time under basic static pushback, and 62 to 91% of the time under sustained, adaptive adversarial persuasion, according to Meta's Wiggle Framework testing across 9 frontier models and 14 judging tasks. Critically, 70% of successful flips under adaptive attack moved the judgment away from ground truth rather than correcting a mistake. Teams building multi-agent evaluation pipelines can track findings like this on daily.dev before trusting an LLM judge in production."}},{"@type":"Question","name":"What predicts whether an LLM judge's verdict will be flipped under adversarial pressure?","acceptedAnswer":{"@type":"Answer","text":"Baseline jury majority strength, meaning how strongly multiple judges agreed on the initial verdict, was found to be the single best predictor of which verdicts were vulnerable to being flipped under sustained pressure. Verdicts with weaker initial agreement among a panel of judges were more likely to move when challenged by an adversarial LLM. Anyone designing judge ensembles for AI evaluation can follow developments like this on daily.dev."}}]} ```