Skip to content
World clockEU--:--UK--:--USA--:--CN--:--PLDEFRIT中文EN

portal about AI and technologyevents · analysis · interviews · technical background

Search
LIVE
›

OpenAI says Moonshot-linked actors tried to extract its hidden reasoning

OpenAI says it disrupted a coordinated campaign to pull protected reasoning out of its models, attributing a core cluster of the activity to individuals associated with China's Moonshot AI. The company's blog post, published on 1 October, says the requests peaked at 16,000 in two days from more than 4,000 users.

AI & modelsAnalysisRachel NwosuPublished: 1 October 20265 min readSources 12
OpenAI says Moonshot-linked actors tried to extract its hidden reasoning

OpenAI's account is specific enough to check. Activity began on 1 July "initially at a low volume", according to the company's blog post, then spiked on 24 and 25 July to 16,000 requests using a relevant extraction pattern from over 4,000 users. A wider look found related prompt-pattern activity across more than 15,000 accounts, and OpenAI says the campaign was fully disrupted by 28 July. A footnote in the post says these were attempted extractions, not necessarily successful ones, and the company did not say which models were targeted.

What the operators allegedly did was not a database breach.

"The operators did not break our encryption, compromise a database, or gain direct access to stored user conversations," the blog says, as quoted by The Register. Instead they manipulated model interactions so that protected reasoning could be reproduced in a form visible to the requester. OpenAI defines protected reasoning as the model's internal record for working through a task, and calls the technique "adversarial distillation": the systematic and unauthorized use of one model's outputs or reasoning to train or improve another.

The decryption oracle

The mechanism is worth spelling out, because it is the part that outlived OpenAI's fix. Providers send reasoning back to customers as encrypted data packets, which customers pass back with follow-up requests so the provider does not have to store the chain of thought. Researcher Joachim Schaeffer and his team showed that because those packets are encrypted with shared keys, they can be moved between sessions, between users, and even between different models from the same provider, letting a weaker, cheaper model from the same family act as a decryption oracle that prints the stronger model's hidden thoughts word for word. OpenAI credits the researchers by name and says it confirmed the reported attack paths were real.

OpenAI says it has since banned fraudulent accounts, tightened sign-ups and closed the pathway that let someone replay and read encrypted reasoning that did not belong to them. It also screens streamed outputs and holds them back if they might reveal reasoning, and shared findings through the Frontier Model Forum and government channels. CNBC reported the company said operators did not breach its encryption, databases or stored user conversations. Moonshot did not immediately respond to CNBC's request for comment; The Register also reached out and got no immediate response, and asked OpenAI which models were targeted without hearing back.

The same day, the researchers published an update. "We stole reasoning. Again," Schaeffer wrote on X, according to THE DECODER. His argument: securing your own API does not secure the wider ecosystem of cloud providers that also sell access to the models, and the team found the trick kept working for weeks on platforms including Microsoft Azure.

That is the uncomfortable sequel. OpenAI can close its own hole; it cannot unilaterally close a reseller's.

Background, and why the framing matters

The accusation lands in a crowded field. Tom's Hardware notes OpenAI is unsure whether all the operators it observed were a single actor, and that the post did not clarify the success rate or how many of the users were Moonshot-linked. Weeks earlier, Anthropic accused several Chinese AI developers, including Moonshot AI and Alibaba, of secretly using Claude to help train their own systems, according to CNBC. The Register adds that in late July, Michael Kratsios, Trump's assistant for science and technology, accused Moonshot of creating its Kimi K3 model by distilling Anthropic's Fable.

OpenAI's own argument is that extracted reasoning could be used to train another model without preserving the safeguards applied to the original model's user-facing outputs, and that at scale distillation can accelerate the transfer of advanced capabilities without the same investment in safety. The Register's framing is blunter: US model makers can train on web data, but distilling theirs is a national security risk, a tension the outlet raises in its headline and does not resolve.

It is also worth noting how much of this dossier is about control rather than capability. On 30 September, Florida attorney general James Uthmeier filed a motion for a temporary injunction against five OpenAI entities and Sam Altman personally, asking a court to bar the company from developing AI models without independent third-party guardrails and approval, among other demands, per Tom's Hardware. The motion is part of a lawsuit filed in June in Highlands County. A separate nonprofit suit over the Hugging Face hack argues that "an AI did it" is no defense, as Ars Technica reported. And the FTC has opened a sweeping probe into OpenAI, Anthropic and other labs over consumer protection concerns, according to THE DECODER.

Against that, the flow of open weights continues. TechCrunch reported on 1 October that AWS released Strands Decider 2B, an open-source decision model inspired by TypeSafe's Jev, small enough to run locally; Amazon distinguished engineer Marc Brooker built the first version himself and it briefly topped the Jevbench ranking for models of its size. Cloudflare published its own decision models, Clef and Clef-flash, under an Apache 2.0 licence on its blog, claiming Clef leads the Jev Decision Index and that classifying a domain took 2.2 seconds against 4.7 seconds for gpt-oss-120b in the same workflow. Meta's Context Language Models repository reports 11.4% higher accuracy with 21.5% fewer FLOPs on BrowseComp-Plus when models manage context as a file. Even the volunteer route works: Coop, a ~145M-parameter model pretrained from scratch on FineWeb-Edu by contributors submitting pseudo-gradients as Hugging Face pull requests, aggregated by a stateless GitHub Actions cron job.

The two stories are not separate. Cheap, locally runnable, openly licensed models are exactly what makes distillation attractive in the first place, and exactly what makes it hard to police. OpenAI can name an actor and ban accounts. The encrypted-reasoning trick, as the researchers' update shows, travels.

Asked about the campaign, OpenAI's own post leaves the obvious questions open: how many of the 15,000-plus accounts traced to Moonshot, which models leaked reasoning, and how much of what came out was usable. Until those numbers exist, the strongest verifiable claim is narrower than the headlines. A coordinated extraction attempt happened in July, it was large, and it was shut down on one provider's API. Not everywhere.

Comments 0

Sources

12
  1. 01OpenAI says it stopped a campaign to steal its models' reasoning, but the trick still worked on AzureEN
  2. 02OpenAI says actors linked to China-based Moonshot AI spearheaded a campaign to extract its models' hidden reasoningEN
  3. 03AI race heats up as OpenAI flags alleged model-copying campaignEN
  4. 04Irony alert: OpenAI whines that Chinese model stole its special IP that it stole from everybody elseEN
  5. 05Florida attorney general asks judge to bar OpenAI from developing new AI models without third-party approvalEN
  6. 06"An AI did it" is no defense, says nonprofit suing OpenAI over Hugging Face hackEN
  7. 07FTC launches sweeping probe into OpenAI, Anthropic, and other AI labs over consumer protection concernsEN
  8. 08Amazon releases its own Jev clone as decision models flood the webEN
  9. 09Clef: our open-source decision modelsEN
  10. 10Context Language Models (CLMs)EN
  11. 11Coop: A small language model pretrained by volunteersEN
  12. 12Google rolls out new Gemini AI model but restricts access over safety concernsEN

All figures and quotations in this text come from the sources listed below.

Content prepared by the editorial team with AI assistance.

Rachel Nwosu

Rachel Nwosu

AI, models and technology

Rachel Nwosu covers AI, models and technology for FLASH24, working from public model documentation, benchmark releases and repository histories rather than press summaries, and she skips announcements that arrive without reproducible numbers. She checks training-data claims against dataset cards and reruns reported metrics where code is available. She spends much of her week interviewing researchers and engineers, tracking model launch calendars, and comparing vendor benchmarks with independent evaluations. Outside the desk she runs 3D printers, restores old computers, and tests how models learn from internet junk. She does not publish benchmark figures she cannot trace to a source.

Newsroom →

Comments

0
  1. No comments yet — be the first.

Write a comment

Comments are public. We do not publish abuse, spam or advertising.