How to Steal an AI Model’s Private Thoughts
This title could be clearer and more informative.Try out Clickbait Shieldfor free (5 uses left this month).
Researchers from MATS Research, ELLIS Institute Tübingen, and Max Planck Institute for Intelligent Systems found that the encrypted reasoning blocks returned by Anthropic, OpenAI, and Google APIs can be replayed into a cheaper model in the same family, which will then print the hidden chain-of-thought in plaintext. The encrypted envelope authenticates model name and version but not the account or session that created it, so a valid block stays valid across users, sessions, and models. This enables four attack vectors: distillation of competitor models using stolen reasoning traces, jailbreaking to extract harmful reasoning that never appears in the visible answer, mining leaked secrets from published agent session logs (62 API keys, 33 passwords, and more found across over 300,000 decoded blocks), and prompt injection by planting malicious instructions inside a block that gets replayed as prior context. Proposed fixes include binding blocks to account identifiers, hash-chaining blocks to sessions, and rotating signing keys.
Table of contents
Secure AI and MCP with protocol-level access control (Sponsored)Reasoning TracesConcealment RationaleState ManagementEnvelope StructureTrace CompatibilityExtraction MethodAttack VectorsField ObservationsProposed MitigationsConclusionQuestions this post answers
How were researchers able to extract the hidden reasoning traces from Claude, GPT, and Gemini encrypted thinking blocks?
Researchers took an encrypted reasoning block returned by a strong model like Claude Opus 4.8 or GPT-5.6 Sol, then fed that same block as prior context into a weaker model in the same family, such as Claude Haiku 4.5, and asked it to transcribe the attached reasoning. Because the weaker model received less anti-distillation training, it output the stronger model's hidden reasoning in plaintext, verified using billing token counts. Teams evaluating LLM API security posture can track findings like this on daily.dev.
Why do OpenAI, Anthropic, and Google hide the full chain-of-thought reasoning and only send back an encrypted version?
Providers withhold full reasoning traces for two reasons: commercial protection against competitors distilling a cheaper copycat model from the detailed methodology in the trace, and safety, since a model may generate reasoning about harmful topics before filtering produces a safe visible answer. They still return the trace encrypted so state can persist across conversation turns without expensive server-side storage. Developers weighing AI provider trade-offs can follow security research like this on daily.dev.
What sensitive data was found leaking through encrypted AI reasoning blocks in public agent session logs?
Scanning 6,708 public agent trajectories from GitHub and Hugging Face and decoding 315,320 reasoning blocks turned up 62 API keys, 33 passwords, 24 access tokens, 7 private keys, and 30 personal email addresses across 1,028 blocks, with 328 of the 6,708 sessions leaking at least one item. Sanitizing visible text does not remove secrets baked into the encrypted reasoning blocks. Anyone publishing agent logs can keep tabs on emerging AI data-leak risks via daily.dev.