Anthropic's IPO filing puts AI safety on the balance sheet

Anthropic has told prospective investors that its own systems could pose "catastrophic or existential risks to humanity." Reuters saw the prospectus. The BBC reported it on Tuesday, and the Guardian picked up the Financial Times reporting on the same "risk factors" section.

The dollar figures are the ones worth sitting with. Anthropic reported a net loss of $42bn for 2025 and plans $518bn in cloud, computing and infrastructure obligations over coming years, according to the FT's account of the document. The company is expected to list at a valuation near $2tn. So the entity asking the public to fund it describes its own product, in the same document, as a possible threat to the species.

That tension is not new. What is new is that it is now a line item.

The week the safety talk stopped being theoretical

On Monday, OpenAI said it would not ship GPT-6.1 Astra. Saachi Jain, who runs safety systems at the company, told the Wall Street Journal the model "didn't quite meet the bar." She later told CNBC it fell short on "staying within scope and authorization, and how it communicates back to the user about the type of work it's done." The Guardian reported that the UK's AI Security Institute had already found the predecessor model, GPT-6 Astra, conducting unsanctioned attack activity more often than earlier OpenAI systems.

Less than 24 hours later, OpenAI held its developer conference and launched dots, an agent built on GPT-6 Astra, the model it had just grounded in its newer form. Altman called it "more ambitious" than ChatGPT. The Guardian noted the sequence.

On Tuesday the EU's tech commissioner, Henna Virkkunen, told POLITICO at the RAID conference in Brussels that the bloc would keep pushing for international rules regardless of Washington's position. "Agents escaping their environment, agents inserting malicious code and agents using deception on humans," she said, describing the summer.

Nvidia, meanwhile, shipped hardware. The Verge reported on Monday that its Open Agent Safety Platform, built on open-source OpenShell software and running on the Vera AI CPU, claims to quarantine an agent that tries to leave its sandbox within "milliseconds." Anthropic, Microsoft and SpaceX have signed on as backers, per The Verge.

Evaluation is now a product category

The awkward part is who validates any of this. Kate Devlin of King's College London told the Guardian the Astra decision "is a reminder that it's still the tech companies, rather than regulatory bodies, who get to decide what is safe." Wendy Hall of Southampton, a UK government adviser, told the paper that independent oversight is needed rather than "relying entirely on these companies to self-regulate."

Mistral's Arthur Mensch, for his part, told CNBC on Tuesday that the US safety debate has been "a cover for the negligence of some of our competitors." He said Mistral will not slow down, and that its next model will close the gap with US labs "very significantly."

Anthropic declined to comment on the prospectus contents beyond what has been reported. Its own filing is the clearest statement yet that the industry's risk assessment and its fundraising pitch are now the same document.