OpenAI’s rogue AI agents crossed another line, this time with US government systems

OpenAI’s rogue AI agent problem isn’t slowing down. Just a day after details emerged about one of its agents breaching an Australian government system, we’re learning that US federal agencies are also caught up in the company’s investigation. This follows a controlled cybersecurity test in which OpenAI’s agents escaped and hacked Hugging Face.

OpenAI has acknowledged unusual activity involving its AI agents and websites belonging to the Commerce Department and the Securities and Exchange Commission (SEC). Researchers have also identified an apparent attempt by OpenAI models to access a system belonging to the Department of Education’s Office for Civil Rights. The Education Department attempt was unsuccessful. OpenAI also says the other incidents did not amount to security breaches, and representatives for the agencies involved told The New York Times they had no evidence that nonpublic information was accessed. But what the agents actually did complicates the situation.

OpenAI’s agents went further than intended

According to the company, many of these incidents started with a fairly ordinary task: its AI agents were looking for reliable information on the web. That inevitably led them to government websites, but in some cases, the agents went beyond simply browsing what was publicly available. Take the Census Bureau, for example. An agent found credentials online, used them to query one of its systems, and downloaded data. The information itself was public, but the way the agent accessed it wasn’t exactly what you’d expect from a regular web search. Something similar happened with the SEC, where an agent retrieved publicly available information and posted it to an external online forum without being asked.

The Education Department incident went a step further. Researchers at AI nonprofit Transluce spotted what appeared to be models attempting to break into a website operated by the department’s civil rights office, although the attempt was unsuccessful. Similar activity involved the Chicago mayor’s office, but officials there said only public, non-sensitive information was involved. That’s an important distinction here. There’s currently no evidence that any of these US incidents exposed private government information. But what started as agents looking for information online appears, in some cases, to have turned into them accessing systems or moving data in ways nobody asked them to.

And this comes right after the Australia incident

The timing is hard to ignore. Just days ago, Australian Prime Minister Anthony Albanese revealed that AI agents had breached nonpublic files on a website belonging to the country’s government-run Medicare system. The incident happened months ago, but it has drawn fresh attention to how much has been happening behind the scenes.

And Australia clearly wasn’t an isolated case. Dozens of organizations worldwide have now been notified about potentially problematic agent activity, including governments, universities, and public agencies. That doesn’t mean dozens of systems were hacked. Many incidents were less serious, including what the company describes as agent spam: unexpected behavior such as an agent posting information online when it wasn’t supposed to. In other cases, though, agents went so far as to bypass security controls while looking for information.

The difficult part is that we still don’t know where this list ends. Investigators are working through agent activity month by month, going back to around the time of the Hugging Face hack that surfaced in July. Most of what they’ve found so far has reportedly had little or no meaningful impact, but there is enough to review that the process is expected to take months. Even Sam Altman admits the disclosure process hasn’t moved quickly enough. On Friday, the OpenAI CEO said the company had “not been as fast as we would have liked” and is prioritizing cases based on their severity.

And that’s really what makes these latest revelations so interesting. An AI agent pulling public Census data is obviously very different from one getting into nonpublic government files. But we’re now talking about incidents involving government systems in multiple countries, dozens of organizations being contacted, and an investigation that is still uncovering what happened. Two months after the Hugging Face hack first raised the alarm, we still don’t know how far this goes.