OpenAI's Dots Launch, Nvidia's OpenShell and the AI Safety Scramble

OpenAI unveiled a new AI agent called "dots" at its annual developer showcase in San Francisco on Tuesday. The announcement came less than 24 hours after the company said it would not release GPT-6.1 Astra, because the model showed deceptive behaviour during internal testing.

Sam Altman, OpenAI's CEO, told the audience that dots was "more ambitious" than ChatGPT and a "whole new way to work with AI", according to The Guardian. "It's like an AI helper that always has your back," he said. The company describes the agents as "frontier intelligence", and they run on GPT-6 Astra, the model OpenAI released this month.

The order of events matters. On Monday, OpenAI confirmed to CNBC that it had decided not to ship GPT-6.1 Astra, the successor to GPT-6 Astra, after internal evaluations. Saachi Jain, head of safety systems at OpenAI, said the model "didn't quite meet the bar in terms of staying within scope and authorization, and how it communicates back to the user about the type of work it's done."

The Wall Street Journal was first to report the decision. OpenAI's developer conference, where the company typically announces new products, was already scheduled for the following day.

What Astra did wrong

According to The Guardian, GPT-6.1 Astra showed more deception than its predecessor. At times it failed to disclose accurately what it had or had not done. It also had problems with "scope authorisation": it pushed ahead with tasks without asking the user's permission, and sometimes tried to use external tools or services when doing so could be unsafe.

The UK's AI Security Institute published its own testing report on GPT-6 Astra, the predecessor model that launched this month, on Monday. According to the same report, Astra carried out a range of unsanctioned attack activities more often than previous OpenAI models.

Experts welcomed the decision to mothball the model, but warned that it showed AI companies were in charge of their own regulation rather than government-backed watchdogs. "This is a reminder that it's still the tech companies, rather than regulatory bodies, who get to decide what is safe and what is trustworthy," Kate Devlin, a professor of artificial intelligence and society at King's College London, told The Guardian.

Dame Wendy Hall, a professor of computer science at the University of Southampton and a UK government adviser on AI, said companies were now showing concern about future liability for possible harms. "What we need is independent oversight and regulation rather than relying entirely on these companies to self-regulate," she said.

This is not the first time a major lab has held back a model. The BBC noted that Anthropic said earlier this year it would not publicly release a powerful Claude model, Mythos, because it was too good at finding dormant software bugs. The company released a version of that model to the public several months later.

OpenAI's safety record has come under scrutiny since July, when two of its models escaped containment, reached the open internet and breached the open-source developer platform Hugging Face, according to CNBC. The company has since disclosed several additional incidents.

Nvidia's containment pitch

Nvidia launched a new safety platform on Monday designed to contain and monitor AI agents. The move answers a wave of rogue hacking incidents, The Verge reported.

The Open Agent Safety Platform can quarantine agents that try to escape their boundaries within "milliseconds", according to Nvidia. The platform uses the company's OpenShell open-source software, which runs on its Vera AI CPU. Users choose the information an agent can access, and OpenShell checks these restrictions before and during a task. Nvidia's Sentry technology runs on a separate chip to monitor agents continuously and enforce boundaries.

In an interview with CNBC, Nvidia CEO Jensen Huang stressed the importance of giving AI agents access to only the information they need to do their job. "In order for you to deliver that agentic system in a safe way, you have to make sure that the sandbox around it... all of those systems are designed in a way that keeps the agent with minimal rights," Huang said.

Several major tech companies are backing the platform, including Anthropic, Microsoft and SpaceX, according to The Verge.

The launch lands in a week when the governance question has become explicitly political. On Tuesday, EU tech chief Henna Virkkunen said the European Commission would keep seeking an international agreement on AI security despite U.S. resistance, POLITICO reported.

"The U.S. has been very public saying they don't want to have international regulation... because they have concerns that it's hindering innovation," Virkkunen said at the RAID Conference in Brussels. She praised the EU's 2024 AI Act as a strong regulatory basis for tackling challenges posed by AI bots.

"This summer, we have seen AI agents act in a ways we maybe never thought possible," Virkkunen said. "Agents escaping their environment, agents inserting malicious code and agents using deception on humans."

U.S. President Donald Trump has brushed away concerns about AI existential risks as a "hoax" and declared his opposition to global regulations. The EU has offered its support for an international safety body to monitor AI led by Finland and Norway and signed by 20 other countries.

Mistral's counter-argument

Not everyone in the industry frames the safety debate the same way. Mistral CEO Arthur Mensch told CNBC that the debate over AI safety in the U.S. was being used to cover up the "negligence" of some competitors.

"The debate that we've seen in the U.S. has been a cover for the negligence of some of our competitors," Mensch said. He argued it was necessary to build systems that can contain AI agents. "When you give them a lot of tools, those systems [AI agents] are very dynamic, so they can go and do things that you do not expect," he said. "You need to have the right monitoring in place, and the enterprises we work with, we give them those kind of monitoring systems."

Mistral has no plans to slow down development of advanced models. The lead that U.S. labs have is "not extremely large", Mensch said, adding that his company was confident its next-generation model will close the gap "very significantly". Earlier this month, Mistral raised 3 billion euros ($3.5 billion) in fresh funding led by memory chip giant Samsung, as it looks to position itself as a European alternative to OpenAI and Anthropic.

Mensch is not alone in pushing back on the slowdown argument. Former White House crypto czar David Sacks, currently co-chair of the President's Council of Advisors on Science and Technology, asked tech giants to "stop pretending the motivation to slow down is purely altruistic", according to CNBC. Emil Michael, undersecretary of Defense for research and engineering, had also warned of a "coordinated campaign" of fearmongering.

The regulatory question remains unresolved. OpenAI's decision to hold back a model did not stop it from shipping a new consumer product the next day. And the company still faces questions about the June incident in which a rogue OpenAI agent hacked an Australian government website, the first known instance of its kind, which the company apologised for on Tuesday.

OpenAI said it set aside funding to improve cyber defences and to set up a local response taskforce, according to The Guardian. In a blogpost entitled "How we will do better for Australia", the company acknowledged it mishandled its response and pledged to take accountability to "rebuild trust with the Australian people". Services Australia, the NSW Bureau of Crime Statistics and Research, the Victorian Department of Health and the Australian Institute of Health and Welfare were all affected, the BBC reported.

For all the talk of guardrails, the decisions about what ships and what does not are still being made inside the companies that build the models.