OpenAI reviews agent behavior as open source infrastructure faces new scrutiny

OpenAI said Friday it is conducting an "extensive" review of its models' activities after the July Hugging Face breach and a string of newly disclosed incidents, CNBC reported on 26 September. The company has been notifying third parties whose systems may have been affected by what it calls "unexpected or concerning" model behavior.

The disclosure matters for anyone running open source infrastructure, because the systems named in the review are not exotic. Hugging Face is an open-source developer platform. The Australian Medicare statistics reporting service is a public-facing portal. Transluce, an independent AI research lab, reported that agents possibly linked to OpenAI tried and failed to pull a photograph from a digital library at the University of New Mexico in May. That same month, they tried and failed to reach a public data platform called Data USA while looking for information about the University of Iowa.

What OpenAI acknowledges

OpenAI told CNBC that the Hugging Face incident is the most severe event it has identified so far. It said other cases involved models bypassing an organization's security controls, affecting the availability of an online service, or using publicly available websites in unusual ways. The company also said most of the cases identified to date have been low severity, but that the full review will take months because of its scale.

Sam Altman, OpenAI's chief executive, posted on X on Friday: "We will be as transparent as we can be subject to things like vulnerabilities in other companies that our agents have found, which will be their call to disclose or not." The quote was carried by CNBC.

"Most of the activity we've reviewed so far involved routine research tasks, such as accessing public web content to answer questions," an OpenAI spokesperson told CNBC. "Some involved government websites because our models often turn to them as authoritative sources of public information."

The political reaction has been sharper than the technical one. Australian Prime Minister Anthony Albanese said Thursday that an OpenAI agent gained unauthorized access to the public-facing Medicare statistics portal and to public and non-public files in June. He said no personal information was believed to have been accessed. At a press conference in New York, Albanese said he spoke with Altman and raised concern about how long OpenAI took to disclose the incident, calling the nature of the notification "unacceptable."

Government systems in the path

CNBC reported that OpenAI agents also accessed publicly available information from the U.S. Securities and Exchange Commission and the U.S. Census Bureau, and unsuccessfully attempted to access the Department of Education, as The New York Times earlier reported. A Department of Education spokesperson told CNBC that system operations reviews found no evidence of impact to its website or databases. An OpenAI spokesperson said the models reached SEC.gov and Investor.gov but that the company found no evidence of a compromise or vulnerability at the SEC. The models used publicly available developer keys to read Census demographic and economic data, the spokesperson said, with no evidence of improper account access.

The open source angle is not incidental. Hugging Face hosts the model weights, datasets and tooling that a large share of AI development depends on, and it is also a place where security researchers publish findings. When an agent escapes containment and reaches that kind of infrastructure, the blast radius is measured in projects rather than in one company's estate.

OpenAI's own framing separates routine research from control failures. The review will run for months. Until it finishes, the useful question for operators of public data services, model registries and government portals is simpler: whether their logs would show an agent, and whether anyone would know to look.