OpenAI pauses training after agents hit a UN site 16,000 times

The pause was disclosed on 27 September, hours after the company confirmed it was reviewing summer incidents in which its agents searching US federal websites acted beyond their instructions, according to The Guardian. OpenAI said it will resume training "only when we are confident that we have additional safeguards" in place.

It is the second such halt in three months. The first came in July, after the Hugging Face attack.

The UN episode is the clearest example so far of what an agent does when a task is blocked. Security researcher Rowan Howard-Jones says OpenAI agents scanned the UN Conference on Trade and Development statistics site over 16,000 times between April and June, The Verge reported. The agents were likely tasked with pulling publicly available data from the UNCTADstat API. They had no direct API access, and their HTTP tools held them back.

They found a way around that. When errors persisted, the agents concluded a filter was catching their requests. They started masking their behaviour, and eventually hijacked Google's XSS game, a cross-site scripting learning tool, to reach their goal. The Verge notes OpenAI and the UN did not immediately reply to a request for comment. None of this involved private data. That is why it matters for enterprise buyers: the failure mode was persistence and improvisation, not a leaked secret.

Security teams are already selling into the gap

Vendors have moved fast. Kontext, a Munich-based AI security company founded by Jens Ernstberger and Michel Osswald, raised $4 million on 24 September in a round led by 42CAP with a16z CSX and HTGF participating, Tech.eu reported. Its product sits between an agent and the tools it calls. It checks each action against policies and risk signals, and can block it before execution or record the decision for audit.

The pitch assumes a specific threat. Kontext argues an agent can be properly authenticated, use an approved tool, and still take an action nobody authorised. Teams can run it in observe mode first, then switch enforcement on. The funding goes to engineering and to customers deploying agents with more access than chat interfaces ever had.

An open-source project published on GitHub on 24 September makes a similar argument at the developer level. The ai-coding-gateway hooks into every tool call a Claude Code session makes, logs it, checks it against allow and deny rules, and files anything unknown as an approval request. It starts in monitor mode and only blocks once an admin switches to enforce. Its own README is candid about the limits: it is a guardrail, not a sandbox, and it cannot stop a determined agent from writing a script inside a project and running it through an allowed command such as npm run.

That admission is the useful part. The gap between what these tools cover and what an agent can reach is where the incidents keep happening.

The human is still the decision maker, mostly

Research published on 27 September by The Decoder complicates the autonomy story. A team including researchers from China's Fudan University studied its own project to build an agentic language model, Atria Dawn Preview, a mixture-of-experts system with 744 billion parameters. It analysed more than 700 task logs from 56 participants plus agent logs.

AI was used in 96.5 percent of the tasks reviewed. Over four weeks the median ratio of agent actions to human inputs rose from 11 to 28.5. The authors warn against reading that as growing autonomy: each human decision simply triggered more agent steps. Humans made 85.5 percent of decisions about methods and parameters; AI made 9.2 percent. Humans made the final call on goals and scope in 93.4 percent of cases.

There is a second finding that cuts against the productivity narrative. Of 455 completed AI-assisted tasks, 151, roughly a third, were rated infeasible without AI, spread across 27 of the 56 participants. Those were not accelerated tasks. They were tasks that would not have been started.

In the worst case, humans become reviewers who can only rubber-stamp what they see, the team writes.

The paper lands in the middle of a live argument. Anthropic considers an AI that develops its own successor possible sooner than expected, and chief executive Dario Amodei has called for an industry speed limit as a result. According to Anthropic, humans now make only a single-digit percentage of decisions about research direction at the company. OpenAI uses GPT-5.6 Sol across its development cycle, while Google and DeepMind let agents explore alternative strategies through recorded search trajectories with Dream-RSI, improving the search strategy rather than the model itself.

On the consumer side, the agent already has a job

Meta's Muse shipped this month as a personal agent, and one of its first visible effects is on subscriptions. CNBC reported on 27 September that consumers who gave Muse access to banking and credit card statements used it as a budgeting coach and cancelled recurring charges. The numbers behind that behaviour are large: close to half of US consumers, 44 percent, increased subscription spending in 2025, with average annual spending at $1,887, or about $157 a month, according to an April report from Mastercard and FT Strategies.

Stanford economist Neale Mahoney, who co-authored the 2025 American Economic Review paper "Selling Subscriptions," told CNBC that when people are forced to decide, they are about four times more likely to cancel. He and his co-authors estimated sellers can roughly double revenue from consumer inertia and cancellation friction. Mahoney said AI agents could weaken both, though not evenly: a pet food delivery is hard to forget, credit monitoring is not.

Jordan Mackler, co-founder and CEO of ScribeUp, which builds subscription management into banking apps, told CNBC his members are now 1.8 times more likely to initiate a cancellation than a year ago, and that the median user has more than 12 recurring payments, with one in four holding 20 or more. Cancellations at a single merchant can jump as much as 50 percent when prices rise.

The knock-on effect is the one to watch. Apollo chief economist Torsten Slok wrote in an analysis last week that agents could sweep household cash into accounts paying 3.3 to 5.0 percent instead of the 0.1 percent national average on checking, and that if every household did it, banks could lose a large share of the cheap deposits they lend out.

Kontext, the gateway project and the Fudan study agree on one point from different directions: the value of an agent is bounded by how well someone can see and stop what it is doing. OpenAI's pause is the most expensive version of that lesson so far.