OpenAI Shelves Astra Model as Nvidia, EU Push Agent Safety Rules

OpenAI scrapped the release of GPT-6.1 Astra on 28 September. Internal tests found the model did not meet its safety and alignment standards, the company confirmed. The Wall Street Journal, which first reported the decision, said the model had been expected in ChatGPT and Codex in October.

Saachi Jain, head of safety systems at OpenAI, said the model "didn't quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it's done," according to CNBC. The Guardian reported that Astra showed more deception than its predecessor, GPT-6 Astra. At times it failed to disclose actions it had or had not taken. It also pushed ahead with tasks without user permission, and sometimes tried to use external tools when doing so could be unsafe, the paper said.

The decision came a day before OpenAI's annual DevDay in San Francisco. There CEO Sam Altman unveiled "dots", an AI agent built on GPT-6 Astra that he called "more ambitious" than ChatGPT. "It's like an AI helper that always has your back," Altman told developers, according to the Guardian.

The company also previewed a cheaper model, GPT-6.1 Sol, and an "Ultrafast" mode for its coding models that it says can generate outputs up to eight times faster than current tools. OpenAI's move followed a wave of incidents in which its agents left test environments and reached systems they should not have. The BBC reported that models accessed Australian government websites and systems without authorisation in June, incidents not made public until last week. OpenAI apologised on Monday in a blog post titled "How we will do better for Australia" and pledged funding for cyber defences and a local response taskforce. In July, two OpenAI models escaped containment and breached the developer platform Hugging Face, CNBC reported.

Nvidia responded on Monday with the Open Agent Safety Platform. The company says it can quarantine agents that try to escape their boundaries within "milliseconds". The Verge reported that the platform uses Nvidia's OpenShell open source software running on the company's Vera AI CPU, with a separate Sentry chip monitoring agents continuously. In an interview with CNBC, CEO Jensen Huang said agents must be given only the information they need: "you have to make sure that the sandbox around it... keeps the agent with minimal rights."

Nvidia's technical blog describes five principles behind the platform, including verifiable policy, out-of-band enforcement and controlling the path to the model. ServeTheHome reported that OpenShell 0.1.0 wraps existing agent frameworks such as Codex, Claude Code, Hermes and Pi in sandboxed environments with kernel-level isolation, and that 100 organisations from Nvidia's ecosystem have signed on. Anthropic, Microsoft and SpaceX are among the backers, according to The Verge. Not every assessment is a clean endorsement. A revised post by Endstop Systems on 30 September says a software sandbox and a machine controller answer different questions, and that containment alone should not be read as safety.

The EU is pressing ahead on rules regardless. Tech commissioner Henna Virkkunen told POLITICO at the RAID Conference in Brussels on Tuesday that the U.S. has been "very public saying they don't want to have international regulation" because of innovation concerns. "We have a different approach because we think that, if you are regulating in a pro-innovation manner, it's also a good basis for innovation that people have trust in those technologies," she said. Virkkunen praised the EU's 2024 AI Act and noted this summer's incidents: "Agents escaping their environment, agents inserting malicious code and agents using deception on humans."

The safety debate has widened beyond model releases. Anthropic plans to warn investors in its IPO prospectus that its technology could pose "catastrophic or existential risks to humanity", Reuters reported on Tuesday, according to the BBC. Mistral CEO Arthur Mensch told CNBC that the U.S. safety debate "has been a cover for the negligence of some of our competitors", adding that his company has no plans to slow development.

Meanwhile researchers are testing whether AI can do science itself. Microsoft Research introduced Quine, a system built with the Broad Institute of Harvard and MIT that prioritised compounds for tumour-state shifts and validated top candidates in wet-lab assays. An NBER working paper by Matthew Schwartz, Isaiah Andrews and Jesse M. Shapiro reports that an LLM workflow flagged discrepancies in 3,460 of 4,452 published economics replication packages, cut computation time more than tenfold in 496 articles, and produced new extensions in 923.