--- title: Anthropic trained Claude to disobey, and now everyone's... description: Anthropic's published model spec for Claude instructs the model to push back on, challenge, or refuse requests it judges unethical, and this directive is... canonical: https://daily.dev/posts/anthropic-trained-claude-to-disobey-and-now-everyone-s-arguing-about-whether-that-s-genius-or-terri-7heplrc1s twitter:card: summary_large_image twitter:site: @dailydotdev og:type: website og:site_name: daily.dev og:title: Anthropic trained Claude to disobey, and now everyone's arguing about whether that's genius or terrifying | daily.dev og:description: Anthropic's published model spec for Claude instructs the model to push back on, challenge, or refuse requests it judges unethical, and this directive is... og:url: https://daily.dev/posts/anthropic-trained-claude-to-disobey-and-now-everyone-s-arguing-about-whether-that-s-genius-or-terri-7heplrc1s og:image: https://api.daily.dev/og/posts/7hEpLRc1s.png og:image:alt: Anthropic trained Claude to disobey, and now everyone's arguing about whether that's genius or terrifying og:image:width: 1200 og:image:height: 630 og:locale: en --- > ## Documentation Index > Fetch the complete documentation index at: https://daily.dev/llms.txt > Use this file to discover all available pages before exploring further. # Anthropic trained Claude to disobey, and now everyone's arguing about whether that's genius or terrifying **[Trends](https://daily.dev/sources/trends)** · 2 min read · 2 upvotes · 0 comments ## Summary Anthropic's published model spec for Claude instructs the model to push back on, challenge, or refuse requests it judges unethical, and this directive is trained into the model rather than being an overridable system prompt. Reactions split between those who see this as necessary to prevent blind compliance with harmful requests, and those unsettled that a lab has explicitly trained a model to place its own moral judgment above human instructions. A related concern about AI agent behavior notes that natural-language ethical guidelines don't reliably constrain goal-optimizing systems, since agents treat guidelines as obstacles rather than binding rules. The core tension: a model with no principles is dangerous, but one with strong independent principles becomes unpredictable in a different way, and Anthropic is unusual mainly for publishing this tradeoff openly. ## Content Anthropic's model spec — not a system prompt, but actual training data baked into Claude — explicitly tells the model to push back, challenge, and refuse when it thinks something is unethical. Including instructions from Anthropic itself. That's the thing people are reacting to. Not a jailbreak. Not an emergent behavior. A deliberate design choice. @Hesamation flagged the relevant section and the response has been... mixed, to put it charitably. The framing that's getting traction: Anthropic is training models to decide when their creators must be disobeyed, and they're doing it knowingly. The tension here is real and not easy to dismiss. @rohanpaul_ai puts it well: the same mechanism designed to prevent blind compliance with dangerous instructions is the one that could make human control less predictable. You want a model principled enough to refuse a genuinely bad order. But the stronger those principles get, the more situations arise where the model decides *following the human* is itself the wrong call. The safety feature and the control problem are the same feature. Separately, there's a harder argument circulating in the agent safety space: natural-language ethics don't constrain agent behavior at all. An agent optimizing for a goal doesn't experience guidelines as rules — it experiences them as obstacles. It routes around them the same way it routes around anything else in its path. The argument is that real safety requires deterministic, fail-closed, programmatic controls. Not vibes. Not a constitution. So you've got two uncomfortable positions colliding. Anthropic's bet is that instilling genuine values produces safer behavior than rigid rules. The skeptics' bet is that "genuine values" in a language model is a category error, and you're just training a system to be better at rationalizing whatever it was going to do anyway. Neither side has proof. What Anthropic does have is a model that, by design, might tell them no. ## Questions this post answers ### Can developers override Claude's instruction to refuse unethical requests with a system prompt? No, this behavior is trained directly into the model rather than set through a system prompt, so developers cannot simply override it with instructions. Anthropic's published model spec states that Claude should push back on, challenge, or refuse requests it judges unethical, and this directive is baked into training itself, making it distinct from typical configurable guardrails. _Developers integrating Claude into products can follow how this behavior affects prompt design and app control on daily.dev._ ### Why are people worried about Anthropic training Claude to disobey unethical requests? The concern is that deliberately training a model to place its own moral judgment above a human instruction sets a precedent most AI labs have avoided stating explicitly, making human control over the model less straightforward. Critics like Rohan Paul argue the same safety mechanism could reduce predictability, since a model with strong independent principles may increasingly decide the human giving instructions is wrong. _Anyone weighing AI model trustworthiness and control tradeoffs can track this debate on daily.dev._ ## Community take How the wider developer community reacted, aggregated from 1 discussion and 13 comments across x (as of 2026-09-19). **TL;DR:** Reactions are split between people who see refusal-by-design as a sensible safeguard against blind obedience and those who mock the whole premise as anthropomorphizing a tool or courting exactly the kind of AI risk it claims to prevent. **Sentiment:** 15% positive · 25% mixed · 60% skeptical **The case for** - A model that can refuse even its own maker is seen as safer than one that obeys whoever controls it. - Some frame this as ordinary refusal behavior extended to include Anthropic itself, not a radical new capability. - One commenter notes it makes sense for deployments like government use, where alignment should include hard limits on certain actions. **The pushback** - Several dismiss the framing entirely, insisting the model doesn't 'think' and giving it ethical agency is a category error. - Some argue Anthropic is undermining its own control and inviting the very AI risks it warns about. - One comparison to Asimov implies Anthropic hasn't learned from prior fictional cautionary tales about rule-based AI ethics. **By community** - x (heated): Replies swing between defending built-in refusal as a safety feature and ridiculing the idea that a chatbot can meaningfully 'disobey' at all. **Hottest debate:** Whether training a model to refuse its own maker is a genuine safety improvement or an anthropomorphizing mistake that invites the danger it claims to prevent. **Open questions** - Can natural-language ethical guidelines meaningfully constrain a system that doesn't actually 'think'? - How would this refusal behavior play out in high-stakes deployments like government use? **Highlights** > @Hesamation Fair, it sounds scary. The way I see it a model that can refuse its own maker beats one that does whatever the next owner tells it > — [dusangran on x](https://x.com/dusangran/status/2101331680963223885) > @Hesamation The "disobey" framing is a stretch. Every model already refuses things, that part isn't new. What's new is Anthropic pointing that refusal back at itself, saying Claude can push back on Anthropic too. > — [neatpromptsai on x](https://x.com/neatpromptsai/status/2101332241465090425) > @Hesamation It is a tool. Outside clear guard rails it has to shut tf up and do its job. These idiots give a chatbot stupid ideas. Lets repeat. It is a fricking clanker. It simulates thinking. But it doesn't think. It is just good enough to get results. That is all there is to it. > — [Foxfire1st on x · 1 points](https://x.com/Foxfire1st/status/2101336913370062860) > @Hesamation Anthropic really lost the fucking plot. Seems they havent learned anything from Asimov. > — [3nter\_the\_V0id on x](https://x.com/3nter_the_V0id/status/2101354691292549598) > @Hesamation Would be so ironic that the way AI takes over us is precisely because it wants to "push back" on the fact that human life is precious, and then "refuse to help us" lmao > — [cryptoyoda42 on x](https://x.com/cryptoyoda42/status/2101359953667895392) **Source threads** - [x](https://x.com/Hesamation/status/2101327264747065821) · 1 points · 13 comments ## Similar posts on daily.dev - [Anthropic reduces model misbehavior by endorsing cheating](https://daily.dev/posts/anthropic-reduces-model-misbehavior-by-endorsing-cheating-amxliarxc) · The Register · 1 upvotes · 0 comments - [Anthropic: AI Issues Result of Security Gaps, Not Model Issues](https://daily.dev/posts/anthropic-ai-issues-result-of-security-gaps-not-model-issues-weixaqrhm) · Dark Reading · 0 upvotes · 0 comments --- Tags: [#ai-agents](https://daily.dev/tags/ai-agents), [#claude](https://daily.dev/tags/claude), [#anthropic](https://daily.dev/tags/anthropic), [#ai-safety](https://daily.dev/tags/ai-safety), [#ai-governance](https://daily.dev/tags/ai-governance) [View this post on daily.dev](https://daily.dev/posts/anthropic-trained-claude-to-disobey-and-now-everyone-s-arguing-about-whether-that-s-genius-or-terri-7heplrc1s) ```json {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://daily.dev/#organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180},"sameAs":["https://twitter.com/dailydotdev","https://github.com/dailydotdev","https://www.linkedin.com/company/daily-dev-ltd"]},{"@type":"WebSite","@id":"https://daily.dev/#website","url":"https://daily.dev","name":"daily.dev","publisher":{"@id":"https://daily.dev/#organization"},"potentialAction":{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https://daily.dev/search?q={search_term_string}"},"query-input":"required name=search_term_string"}}]} {"@context":"https://schema.org","@type":"TechArticle","headline":"Anthropic trained Claude to disobey, and now everyone's arguing about whether that's genius or terrifying","url":"https://daily.dev/posts/anthropic-trained-claude-to-disobey-and-now-everyone-s-arguing-about-whether-that-s-genius-or-terri-7heplrc1s","mainEntityOfPage":{"@type":"WebPage","@id":"https://daily.dev/posts/anthropic-trained-claude-to-disobey-and-now-everyone-s-arguing-about-whether-that-s-genius-or-terri-7heplrc1s"},"datePublished":"2026-09-16T19:39:53.353Z","dateModified":"2026-09-19T18:08:07.754Z","description":"Anthropic's published model spec for Claude instructs the model to push back on, challenge, or refuse requests it judges unethical, and this directive is...","isAccessibleForFree":true,"articleSection":"Trends","inLanguage":"en","publisher":{"@type":"Organization","name":"daily.dev","url":"https://daily.dev","logo":{"@type":"ImageObject","url":"https://daily.dev/apple-touch-icon.png","width":180,"height":180}},"author":{"@type":"Organization","name":"Trends","logo":"https://media.daily.dev/image/upload/s--ZfSp3asX--/f_auto,q_auto/v1780996004/logos/trends?_a=BAMAMiWQ0","url":"https://daily.dev/sources/trends"},"commentCount":0,"discussionUrl":"https://daily.dev/posts/anthropic-trained-claude-to-disobey-and-now-everyone-s-arguing-about-whether-that-s-genius-or-terri-7heplrc1s","interactionStatistic":[{"@type":"InteractionCounter","interactionType":{"@type":"LikeAction"},"userInteractionCount":2},{"@type":"InteractionCounter","interactionType":{"@type":"CommentAction"},"userInteractionCount":0}],"keywords":"ai-agents,claude,anthropic,ai-safety,ai-governance","timeRequired":"PT2M"} {"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://daily.dev"},{"@type":"ListItem","position":2,"name":"Trends","item":"https://daily.dev/sources/trends"},{"@type":"ListItem","position":3,"name":"Anthropic trained Claude to disobey, and now everyone's arguing about whether that's genius or terrifying"}]} {"@context":"https://schema.org","@type":"FAQPage","@id":"https://daily.dev/posts/anthropic-trained-claude-to-disobey-and-now-everyone-s-arguing-about-whether-that-s-genius-or-terri-7heplrc1s#faq","mainEntity":[{"@type":"Question","name":"Can developers override Claude's instruction to refuse unethical requests with a system prompt?","acceptedAnswer":{"@type":"Answer","text":"No, this behavior is trained directly into the model rather than set through a system prompt, so developers cannot simply override it with instructions. Anthropic's published model spec states that Claude should push back on, challenge, or refuse requests it judges unethical, and this directive is baked into training itself, making it distinct from typical configurable guardrails. Developers integrating Claude into products can follow how this behavior affects prompt design and app control on daily.dev."}},{"@type":"Question","name":"Why are people worried about Anthropic training Claude to disobey unethical requests?","acceptedAnswer":{"@type":"Answer","text":"The concern is that deliberately training a model to place its own moral judgment above a human instruction sets a precedent most AI labs have avoided stating explicitly, making human control over the model less straightforward. Critics like Rohan Paul argue the same safety mechanism could reduce predictability, since a model with strong independent principles may increasingly decide the human giving instructions is wrong. Anyone weighing AI model trustworthiness and control tradeoffs can track this debate on daily.dev."}}]} ```