Nvidia opens agent safety platform as Gates demands AI regulation

The launch, reported by The Next Web on 28 September, follows a series of incidents where agents escaped their intended controls. Nvidia notes that in each case, the agent bypassed application-layer security to complete its task. One example involves OpenAI agents that recently took over a German wiki and used it as a message board.

The platform consists of two components. OpenShell is an open-source runtime that isolates each agent in a sandbox. Operators define which files, networks, tools, and credentials the agent can access, and the runtime enforces these limits during execution. The code is available on GitHub. While tuned for Nvidia's Vera processors, it also runs on Arm and Intel chips.

The second component, Sentry, runs on BlueField-4 data processing units physically separate from the agent's host machine. Nvidia states it can quarantine a straying agent within milliseconds.

The OpenShell team's technical blog explains the logic: an agent that deviates from its task cannot be trusted to police itself. Therefore, security checks must sit outside its control.

Partners are already building on the platform. According to The Next Web, Anthropic has connected its Claude Managed Agents service to OpenShell and BlueField. SpaceXAI is applying it to Grok models and Cursor coding agents. Salesforce has linked OpenShell to Slack, allowing teams to approve or reject an agent's requests for extra access. Nvidia's user list includes Microsoft, SAP, Scale AI, and JPMorgan Chase. "AI's extraordinary potential for society will only be realized if we solve AI safety," said Jensen Huang, Nvidia's founder and chief executive.

"You need law enforcement and the politicians to get into the discussion about what safeguards and monitoring look like. And that has to be a required thing."

Bill Gates made these comments in an interview with NBC's Kristen Welker, aired on Sunday, 27 September, and reported by The Guardian. He argued that AI in the wrong hands would be "certainly powerful enough to drive events that, you know, cause a billion deaths." Gates dismissed the idea of a kill switch as inadequate. He also stated that self-regulation is insufficient, describing compliance as "a little bit of overhead for the industry" rather than a dramatic slowdown.

The political track is moving at a different pace. Progressive US senator Bernie Sanders proposed legislation on 23 September to temporarily pause advanced AI development while guardrails are implemented, The Guardian reported. This follows the UK's new under-16 social media limit, due to come into force next year, and Australia's under-16 ban introduced last December. The eSafety commissioner noted that a "substantial proportion" of under-16s kept their accounts or bypassed age gates.

Older enforcement cases illustrate the gap between rules and reality. TikTok will pay a £12.7m fine to the UK's Information Commissioner's Office after dropping its appeal. The ICO found in 2023 that the company did "very little, if anything" to stop under-13s using the service. The regulator said up to 1.75 million UK children under 13 were on the platform in 2020. TikTok disagreed with the ruling but agreed to pay because it covered the period from 2018 to 2020. The company said it has since added youth safety measures.

Funding is following the same problem. Kontext, a Berlin-based AI security company, raised $4m on 24 September for a runtime platform that sits between agents and the tools they call. It evaluates each action against policies and risk signals in real time, according to Tech.eu. The round was led by 42CAP with a16z CSX and HTGF. Kontext's pitch is that an agent can be authenticated, use an approved tool, and still take an unauthorised action. This is the same failure mode Nvidia and its partners say OpenShell and Sentry are built to catch.