Stealing Reasoning Traces from Proprietary LLM APIs
This describes a potentially major security and privacy breakthrough affecting frontier LLM
Stealing Reasoning Traces from Proprietary LLM APIs
A paper reports that encrypted chain-of-thought traces from major proprietary LLM APIs can be replayed across models to jailbreak weaker models and reveal stronger models' hidden reasoning.
This describes a potentially major security and privacy breakthrough affecting frontier LLM APIs from Anthropic, OpenAI, and Google, with implications for hidden chain-of-thought leakage and model safety; the writeup appears to reference a substantive research paper rather than a minor blog post. No discussion/comments were provided to assess community debate.
<p><strong><a href="https://stolen-thoughts.com/">Stealing Reasoning Traces from Proprietary LLM APIs</a></strong></p> A vanity domain name (<code>stolen-thoughts.com</code>) for <a href="https://www.alphaxiv.org/abs/2608.09867">a neat paper</a>:</p> <blockquote> <p>Anthropic, OpenAI, and Google return encrypted chain-of-thought blocks to clients that can be replayed across sessions, users, and models. We take a trace produced by a frontier model, replay it into a weaker sibling, jailbreak the weaker model, and recover the stronger model’s hidden reasoning in plaintext</p> </blockquote> <p>You can see an example…