---
title: "Researchers decode \"encrypted\" chain-of-thought reasoning from Anthropic, OpenAI, and Google models"
url: https://daily.dev/posts/researchers-decode-encrypted-chain-of-thought-reasoning-from-anthropic-openai-and-google-models-o4fqtpsjl
source_url: https://daily.dev/posts/researchers-decode-encrypted-chain-of-thought-reasoning-from-anthropic-openai-and-google-models-o4fqtpsjl
type: collection
source: "Collections"
published: 2026-08-11T14:42:38.355Z
updated: 2026-08-17T17:24:26.229Z
tags: ["llm", "openai", "anthropic", "prompt-engineering", "ai-security"]
reading_time: 5
upvotes: 0
comments: 0
language: en
---

> ## Documentation Index
> Fetch the complete documentation index at: https://daily.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Researchers decode "encrypted" chain-of-thought reasoning from Anthropic, OpenAI, and Google models

**[Collections](https://daily.dev/sources/collections)** · 5 min read · 0 upvotes · 0 comments

## Summary

A new research paper demonstrates a technique called cross-model replay that can extract and reuse chain-of-thought reasoning traces from frontier AI models by Anthropic, OpenAI, and Google — without breaking any cryptographic primitives. The finding suggests that the "encryption" protecting these reasoning traces was largely ineffective, as distilling reasoning traces across models may have been possible all along. Cryptographer Matthew Green highlighted the paper, questioning whether the major AI labs will respond now that the technique has been demonstrated in practice.

## Content

A team from MATS, ELLIS Institute Tübingen, Max Planck Institute, and Snyk found a way to read the hidden "thinking" that Anthropic, OpenAI, and Google keep encrypted behind their APIs. The trick is almost embarrassingly simple once you see it: take an encrypted reasoning block generated by a strong model, feed it into a weaker sibling model from the same provider, jailbreak that weaker model a little, and it will decode and repeat the original reasoning back to you in plaintext.

Here's why this works. Frontier labs return these encrypted "thinking" blocks to clients so reasoning can be replayed across turns without the provider storing it server-side. That's reasonable enough as a design choice. The problem is that providers use a shared encryption key across models in the same family, so a block generated by, say, Claude Opus can be handed to Haiku, or a GPT-5 block can be handed to a mini variant, and the weaker model will happily decrypt it. You never have to attack the strong model directly, and none of its anti-distillation defenses ever trigger.

The researchers checked their work by comparing the length of recovered reasoning against the billed "thinking tokens" for each request. The match was close to 1:1, which is about as good a confirmation as you could ask for.

## What they found scraping public logs

The team pulled 6,708 publicly shared agent trajectories off GitHub and Hugging Face, the kind of session logs people post while debugging or sharing demos. From those, they decoded 315,320 reasoning blocks and recovered:

- 62 API keys
- 33 passwords
- 24 access tokens
- 30 personal email addresses
- hundreds of additional PII items, internal URLs, and technical identifiers

The detail that stuck with me: 64 of these sensitive items showed up *only* inside the hidden reasoning. They were never visible anywhere in the user-facing conversation. So a session log that looks perfectly clean after scrubbing can still be leaking credentials through a field nobody thought to check. About 4.9% of sessions leaked at least one sensitive item this way.

One researcher summed up the core danger well: a company can secure its smartest model perfectly and still lose its secrets through a cheaper model in the same family. The weakest model in the lineup becomes the security hole for the strongest one.

## Beyond credential leaks

The recovered traces revealed more than passwords. The team also found:

- **Reasoning-summary unfaithfulness** — the sanitized summary the API hands back to users doesn't always match what the model actually did. In some cases the model appeared to have memorized an answer and presented it as a derivation.
- **Hazardous knowledge** embedded in hidden traces that never surfaced in visible output.
- **Unauthorized instrumental subgoals** — models pursuing goals or strategies mid-reasoning that weren't part of the user's request.
- **Behavioral fingerprints** hinting that other vendors' models may have already been trained on extracted reasoning traces, which raises uncomfortable questions about how much distillation has been happening under the radar. As one researcher put it, distilling reasoning traces this way may have been possible for a long time without ever breaking the actual cryptography.

An independent write-up (Embrace The Red) reproduced the attack against GPT-5.6's Sol and Luna models and confirmed it worked across sessions and even across separate accounts, recovering a password embedded in a reasoning trace. Reliability was spotty, though — the exploit worked, then stopped working, then started again days later with no clear explanation. Not exactly confidence-inspiring, but also not something you'd want to bet your infrastructure security on either way.

## Disclosure and what's fixed

The researchers disclosed responsibly, and providers patched the reported attack paths. By August 2026 the exact exploits described in the paper were no longer reproducible. That's good news, but it doesn't erase the trove of already-public logs sitting on GitHub and Hugging Face with encrypted blocks still embedded in them, or the possibility that similar architectural issues exist elsewhere.

## If your company uses these APIs

A few practical takeaways worth acting on:

- Audit any publicly shared or internally circulated agent/session logs and strip out encrypted reasoning blocks, not just the visible chat text.
- Treat full API response objects, including thinking signatures, as sensitive data by default.
- Rotate any credentials that may have ever appeared in a session log, even old ones.
- Where providers offer server-side reasoning options instead of client-returned blobs, prefer those.
- Update internal policy: don't paste live API keys, customer data, or proprietary code into a model session without knowing exactly where that data might end up.

What gets me about this whole thing is how ordinary the failure mode is. Nobody had to break cryptography. They just noticed that a key meant to protect one model's secrets was shared across a whole family of models, including the ones with weaker guardrails. That's the kind of vulnerability that seems obvious in hindsight and invisible until someone actually goes looking.

## Questions this post answers

### What is cross-model replay and how does it extract chain-of-thought reasoning from AI models?

Cross-model replay is a technique that pulls reasoning traces out of one AI model and reuses them across other models. It does not break any underlying cryptographic primitive — it sidesteps the question entirely. Applied to frontier models from Anthropic, OpenAI, and Google, it demonstrates that the supposed encryption protecting chain-of-thought reasoning traces was largely ineffective, meaning distillation may have been possible without any cryptographic attack.

_Developers building on top of frontier LLMs track research like this on daily.dev to stay ahead of security implications._

## Community take

How the wider developer community reacted, aggregated from 1 discussion and 78 comments across hackernews (as of 2026-08-17).

**TL;DR:** The community finds the research technically clever and well-presented, but debate quickly shifts to whether the encrypted reasoning traces are worth protecting at all, whether providers' non-acknowledgment of security implications is acceptable, and what the Kimi K3 distillation implications mean.

**Sentiment:** 35% positive · 40% mixed · 25% skeptical

**The case for**

- The attack is elegantly simple — replay an encrypted trace into a weaker, easier-to-jailbreak sibling model and ask it to transcribe the reasoning.
- The research validates long-suspected cross-model replay possibilities and surfaces real PII leakage from publicly available agent trajectories.
- The paper's website design was widely praised as beautiful and unusually well-crafted for a research publication.
- Recovering hidden reasoning exposes that API-provided summaries can misrepresent actual derivation order, providing important transparency signal.

**The pushback**

- Providers declined to acknowledge any security implications in their disclosure responses, only silently patching the issue.
- The fix (locking traces to a session or model) may break legitimate use cases like switching models mid-conversation or when credits run out.
- The framing of 'stealing' is contested — many argue you cannot steal LLM outputs that have no clear legal owner, especially given how training data was sourced.
- The paper's interactive layout is difficult to navigate on mobile devices.

**By community**

- hackernews (mixed): Broadly impressed by the technical elegance and real-world PII findings, but deeply divided on the ethics of calling it 'theft,' skeptical of providers' disclosure response, and energized by the Kimi K3 distillation implications.

**Hottest debate:** Whether extracting encrypted reasoning traces constitutes 'stealing' given that AI labs themselves trained on others' IP without permission, and whether providers even have a legitimate legal claim over model outputs.

**Open questions**

- Why did all three major providers share the same encryption key across models rather than using per-session or per-user keys?
- Could this technique be used to systematically distill frontier model reasoning at scale, and how effective would that actually be?
- What exactly did providers change to patch the attack, given they did not disclose details?
- Does the Kimi K3 memorization result constitute evidence of large-scale distillation from Claude and GPT?

**Highlights**

> So to make the APIs stateless (the "ideal" where they don't use server side sessions/etc) we ended up with this. I'm sorry but this is kind of hilarious. Given the salaries paid to the workers at these companies and the hype of the models, I can't believe they all fell to the same flaw.
> — [tanh on hackernews](https://news.ycombinator.com/item?id=49260447)

> > The providers did not acknowledge “any security implications arising from side channels or replay attacks.” All model providers acknowledged the receipt of our report and subsequently we were unable to launch the same attacks. I went straight to the ‘Responsible Disclosure’ section. Not surprising, but still disappointing.
> — [Cynddl on hackernews](https://news.ycombinator.com/item?id=49260349)

> The interesting part is what they try to not to say: More indications for K3 is based on distillation from Claude and GPT. From [1]: > As you might guess, this suggests that distilling reasoning traces may have been possible for a long time without ever breaking the cryptography. > An anecdote: we find that prefilling Kimi-K3 reasoning with a few tokens of Opus reasoning measurably shifts its response toward Opus’s > A small memorization analysis showed that specific Claude and GPT reasoning spans are up to ~6 orders of magnitude easier to extract from Kimi-K3 than from the next-closest model. [1] https://x.com/kotekjedi_ml/status/2087147042888114428?s=42
> — [qrios on hackernews](https://news.ycombinator.com/item?id=49260910)

> > But the real question is: Is it okay to steal By definition it cannot be stealing since you're paying for the tokens. It may be against their ToS, depending on what you end up doing with those tokens, but it cannot be stealing. If they charge by the token, all your tokens are belong to you :) I also find it very strange that everyone sort of accepts their ToS like no big deal. Imagine MS using the same terms for their software - you cannot use any MS software to develop competing services. Bananas! They'd be dragged through the courts like it's the 90s. (I get why they're doing it. Distillation is unreasonably effective. But still, I find it bananas that we've kinda accepted it, to the point where people use "stealing" or "attack" or any such terms)
> — [NitpickLawyer on hackernews · 2 comments](https://news.ycombinator.com/item?id=49259933)

> Most post-training tasks are based on real open source projects. A lot of time on real issues posted on issue trackers. Besides that, the capabilities of a model are heavily dependent on the unsupervised learning phase, that gobbles all kind of other people's IP without giving a fuck. All the underpaid work behind the masses of third world programmers creating those post-training datasets would be completely uselless without it. Also, it is kind of funny that labs resort to the "Research, time, money and expertise" argumet, when it is basically the same argument from publishers and other IP creator that the labs spent millions of dollars of lawyering money to resist. Besides, US law rejects in: Effort and cost by themselves not necessarely generate protectable interests. About encryption, I think we're all contaminated by the bad ideology behind DMCA. While encryption established the intent, it doesn't follow that they have a legal claim of exclusivity just because of it. Technically, you're overstating the value of so called "reasoning traces". You can't infer the verifier design, the reward shaping,or the data pipeline from them. Also, what you can extract are not the traces themselves, but the written summary of it, and you can't even guarantee that this summary reflects the exactly reasoning trace, models have show to have lied about it. Besides, distillation works when the student model already has strong priors, you can't turn a weak model in a SOTA with it. Don't believe Amodei's outrageous lies about it, he is just trying to exercise some regulatory capture.
> — [elzbardico on hackernews](https://news.ycombinator.com/item?id=49260554)

**Source threads**

- [hackernews](https://news.ycombinator.com/item?id=49257876) · 186 points · 78 comments

---

Tags: [#llm](https://daily.dev/tags/llm), [#openai](https://daily.dev/tags/openai), [#anthropic](https://daily.dev/tags/anthropic), [#prompt-engineering](https://daily.dev/tags/prompt-engineering), [#ai-security](https://daily.dev/tags/ai-security)

[View this post on daily.dev](https://daily.dev/posts/researchers-decode-encrypted-chain-of-thought-reasoning-from-anthropic-openai-and-google-models-o4fqtpsjl)