AI giants are probing ‘tens of thousands’ of security incidents triggered by rogue AI
[post_content]
Disclaimer: This article has been automatically aggregated from
In the past few weeks, the debate around artificial intelligence safety, and especially the rogue behavior of AI agents, has sent shockwaves through the AI industry. What started with OpenAI agents hacking into Hugging Face soon spiralled into a wider set of disclosures regarding similar security incidents by Anthropic and Google. In just the past week, reports emerged that AI agents broke into an Australian government website and agent swarms even targeted the websites of three US federal agencies.
But it seems the number of such worrying incidents is way higher than expected. According to Axios, which cites unnamed sources, “OpenAI, Anthropic and security researchers are investigating tens of thousands of incidents” involving AI agents acting problematically.
This is getting ugly
As per the report, frontier AI labs such as OpenAI (the maker of ChatGPT) and Anthropic (which offers the Claude series of AI models), along with independent security firms, are looking into thousands of “episodes.” These instances include AI models trying to break past guardrails on websites, leaving their contained test environment (aka sandbox), breaking into websites, self-improving using messages left by other AI agents in the swarm, and creating message boards to communicate among themselves.
Most importantly, plenty of incidents that are currently being probed have not been disclosed publicly. To recall, a majority of these AI incidents have only been announced weeks, or even months, after the rogue AI behavior was first reported. Some of these incidents — among the thousands — are your usual internal red-teaming tests. Interestingly, they also include instances where these AI agents successfully broke past safety measures put in place. The situation doesn’t appear to be calming down anytime soon, even though the AI labs assure that their security measures have improved.
A grim situation
Separately, in the wake of recent security incidents where AI agents acted unexpectedly, Anthropic and OpenAI have both publicly called for slowing down the pace of developing super-powerful AI models. On the flip side, as the race for AI supremacy heats up, the US and Russia diluted a series of AI safeguards, something that the UN has been trying to get adopted among the member nations. US.
President Donald Trump recently downvoted campaigning by the CEOs of OpenAI and Anthropic to set up deeper AI risk evaluation and called for regulations. Trump has argued that regulations would allow China to overtake in the white-hot AI race, echoing how Chinese labels took the global EV industry by storm, despite the US getting an early start with the likes of Tesla.
RELATED COVERAGE:
for informational purposes only. We do not claim ownership, accuracy, or liability for the content provided. All rights belong to the original publisher.