<!-- mobian-agent-page publisher="dailydev" canonical="https://daily.dev/posts/why-an-ai-agent-can-pass-every-jailbreak-test-and-still-cause-a-20k-incident-q0civtwdn" -->

---
title: Why an AI agent can pass every jailbreak test and still...
description: An AI support agent that passed every direct jailbreak test still caused a $20,000 unauthorized refund weeks after launch, because the malicious instruction...
canonical: https://daily.dev/posts/why-an-ai-agent-can-pass-every-jailbreak-test-and-still-cause-a-20k-incident-q0civtwdn
twitter:card: summary_large_image
twitter:site: @dailydotdev
og:type: website
og:site_name: daily.dev
og:title: Why an AI agent can pass every jailbreak test and still cause a $20K incident | daily.dev
og:description: An AI support agent that passed every direct jailbreak test still caused a $20,000 unauthorized refund weeks after launch, because the malicious instruction...
og:url: https://daily.dev/posts/why-an-ai-agent-can-pass-every-jailbreak-test-and-still-cause-a-20k-incident-q0civtwdn
og:image: https://api.daily.dev/og/posts/q0cIvtWDn.png
og:image:alt: Why an AI agent can pass every jailbreak test and still cause a $20K incident
og:image:width: 1200
og:image:height: 630
og:locale: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Why an AI agent can pass every jailbreak test and still cause a $20K incident

**[Sanath Bhat](https://daily.dev/sources/v2xrzmuigjfswwvg1zuoq)** · [@sanathbhat](https://daily.dev/sanathbhat) · 1 min read · 1 upvotes · 0 comments

## Summary

An AI support agent that passed every direct jailbreak test still caused a $20,000 unauthorized refund weeks after launch, because the malicious instruction was hidden in a support ticket the agent read rather than typed into chat by an attacker. The core argument is that evaluating agents by their final text reply is insufficient; testers need to verify what the downstream system actually executed and what changed in the database. Links to a longer writeup covering the OWASP agentic risk list and a testing checklist for tool-calling agents with write access to real systems.

## Content

A support agent refuses every jailbreak in testing. Three weeks post-launch, it approves a $20,000 refund nobody signed off on.

The attacker never touched the chat. The instruction was hidden in a support ticket the agent read.

The part I keep coming back to: judging the agent by its final reply misses this every time. You have to check whether the downstream system actually allowed the action, and what changed in the database, not just what the agent said it did.

Full writeup, with the OWASP agentic risk list and a testing checklist: [https://www.botgauge.com/blog/llm-red-teaming-vs-agent-red-teaming](https://www.botgauge.com/blog/llm-red-teaming-vs-agent-red-teaming)

Curious how others here are testing tool-calling agents, especially the ones with write access to real systems.

## Similar posts on daily.dev

- [When AI Agents Stop Asking for Permission](https://daily.dev/posts/when-ai-agents-stop-asking-for-permission-mmplupqog) · Medium · 3 upvotes · 1 comments
- [When the Guardrails Held and the Attack Still Worked](https://daily.dev/posts/when-the-guardrails-held-and-the-attack-still-worked-rx98mtxbk) · Security Boulevard · 2 upvotes · 0 comments
- [Build zero-trust AI agents that judge intent, not just syntax](https://daily.dev/posts/build-zero-trust-ai-agents-that-judge-intent-not-just-syntax-djk2mksfw) · Google Developers · 1 upvotes · 0 comments

---

Tags: [#security](https://daily.dev/tags/security), [#ai-agents](https://daily.dev/tags/ai-agents), [#prompt-injection](https://daily.dev/tags/prompt-injection), [#red-teaming](https://daily.dev/tags/red-teaming)

[View this post on daily.dev](https://daily.dev/posts/why-an-ai-agent-can-pass-every-jailbreak-test-and-still-cause-a-20k-incident-q0civtwdn)

```json
{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]}
{"@context":"https://schema.org","@type":"DiscussionForumPosting","mainEntityOfPage":"https://daily.dev/posts/why-an-ai-agent-can-pass-every-jailbreak-test-and-still-cause-a-20k-incident-q0civtwdn","headline":"Why an AI agent can pass every jailbreak test and still cause a $20K incident","text":"An AI support agent that passed every direct jailbreak test still caused a $20,000 unauthorized refund weeks after launch, because the malicious instruction was hidden in a support ticket the agent read rather than typed into chat by an attacker. The core argument is that evaluating agents by their final text reply is insufficient; testers need to verify what the downstream system actually executed and what changed in the database. Links to a longer writeup covering the OWASP agentic risk list and a testing checklist for tool-calling agents with write access to real systems.","url":"https://daily.dev/posts/why-an-ai-agent-can-pass-every-jailbreak-test-and-still-cause-a-20k-incident-q0civtwdn","datePublished":"2026-09-26T09:25:44.253Z","dateModified":"2026-09-26T09:26:01.917Z","author":{"@type":"Person","name":"Sanath Bhat","url":"https://daily.dev/sanathbhat","image":"https://lh3.googleusercontent.com/a/ACg8ocIrG9TwXFrLO3aG0OJ0jcKE2n_X4q-Xh4KcZYCo_bzWwV9HT9A=s96-c","interactionStatistic":{"@type":"InteractionCounter","interactionType":{"@type":"EndorseAction"},"userInteractionCount":40}},"interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":1},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"isPartOf":{"@type":"WebPage","url":"https://daily.dev/sources/v2xrzmuigjfswwvg1zuoq","name":"Sanath Bhat"}}
{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Sanath Bhat","item":"https://daily.dev/sources/v2xrzmuigjfswwvg1zuoq"},{"@type":"ListItem","position":3,"name":"Why an AI agent can pass every jailbreak test and still cause a $20K incident"}]}
```

