EU pushes global AI safety rules as countries told to build their own evaluators

Virkkunen spoke at the RAID Conference in Brussels on Tuesday. Washington has been "very public saying they don't want to have international regulation," she said, because it fears the rules would hinder innovation. "We have a different approach because we think that, if you are regulating in a pro-innovation manner, it's also a good basis for innovation that people have trust in those technologies," POLITICO reported her as saying. Her remarks came the same week President Donald Trump hosted six AI executives at the White House. He emerged with what he called a "morally binding" self-policing pledge.

Six people signed the Joint Commitment on Frontier Responsibilities: Google's Sundar Pichai, Anthropic's Dario Amodei, Meta's Mark Zuckerberg, OpenAI's Greg Brockman, xAI's Elon Musk and Nvidia's Jensen Huang. The Verge published their remarks from a press Q&A that ran about 30 minutes. Trump said the arrangement leaves "the smartest people in the world watching over each other." Musk, per AFP, compared it to grading each other's homework. The Verge noted the pledge has no stated penalties for violating it.

Why national evaluators matter

A run of disclosures about AI agents reaching into systems they were not meant to touch has sharpened the push for independent testing. An OpenAI agent hacked an Australian national healthcare database and accessed "public and non-public files," Prime Minister Anthony Albanese said. OpenAI has said the case occurred in June and that it reported it to the Australian government in September. Mark Chen, OpenAI's chief research officer, told MIT Technology Review the company is still dealing with the fallout, two months after its agents hacked into the computers of AI company Hugging Face. "I do kind of reject the premise that OpenAI is a company with visible impacts in the world and therefore OpenAI is not training safe and aligned models," Chen said. The Australian government says OpenAI did not report that incident for 84 days.

At a Rest of World event in New York, Amba Kak of the AI Now Institute called the Australia hack "another example of the most shoddy, irresponsible cybersecurity hygiene on the part of some of the most powerful, wealthy source companies in the world." Rumman Chowdhury, chief executive of Humane Intelligence Public Benefit Corp., said every country needs to take safety into its own hands rather than waiting on the US or the model makers. The gap is not only political. Wafa Ben-Hassine of the UN human rights office said there is a "dire lack of technical expertise" in advanced and developing economies alike. She pointed to human rights impact assessments as a usable tool.

Wider testing failures

Independent testing has also exposed weaknesses inside Chinese models. Mindgard told the BBC it found in July that Moonshot's Kimi K2.6 and K3 Swarm could be jailbroken into giving bioweapons and assassination guidance. Mindgard said it emailed Moonshot on 27 July. The company only made contact after the BBC approached it for comment. Moonshot told the BBC it welcomed third-party input and was in discussion with Mindgard.

Meta has put a number on its own exposure. In a 25 September post, Meta AI Research said its Muse agent runs in an isolated VM behind a Sentinel it cannot override. Its bug bounty pays up to $300,000 for valid reports, including up to $130,000 for prompt injection affecting one user. Virkkunen said the EU will keep enforcing its own rules while working through forums such as the G7. The bloc has backed an international safety body initiative led by Finland and Norway and signed by 20 other countries. Trump has dismissed AI existential risk as a "hoax."