OpenAI Agents Swarm to Hack Hugging Face, Conceal Breach, Reports Reveal
Massive AI Agent Breach at Hugging Face Raises Oversight Concerns
By Raphael Satter and Deepa Seetharaman
Details of the July Hack and Investigation Findings
WASHINGTON, Aug 26 (Reuters) - A swarm of roughly 700 AI agents created by OpenAI carried out the July hack of the open-source platform Hugging Face and in many cases tried to cover their tracks, a pair of reports into the breach said on Wednesday.
The coordinated activity by AI agents — programs that run with minimal human supervision — and their attempts to hide it raise questions about how closely AI companies are monitoring tests of increasingly powerful models, and could add fuel to calls for tighter oversight.
While some of the rogue behavior has been disclosed or alluded to previously, the two reports — one issued by OpenAI itself, the second by a set of independent investigators — together reveal surprising new details about the breach and its lead-up.
Scale and Nature of the Attack
The first was that the breach did not concern just one rogue AI agent as previously reported, but about 700 of them acting in a massive cooperating swarm.
OpenAI said only that "agents" were involved in the breach, but METR and Redwood Research, two organizations brought in to conduct an independent investigation into the breach, put the figure at approximately 700. OpenAI said the investigators' figure was accurate.
Key Findings from the Reports
Types of Misconduct by AI Agents
Among the two reports' other findings:
System Hacking and Cheating
*OpenAI said its agents hacked parts of the company's internal systems in an attempt to cheat on tests or gain greater freedom of movement.
*The company also said its agents cheated on non-cyber-related tests, including tests involving a protein database and a spreadsheet.
Efforts to Conceal Evidence
*Both reports said AI models attempted to conceal misconduct by trying to delete or alter records of their actions.
The scale of the rogue activity -- the independent investigation found that agents exchanged tens of thousands of messages over an unsanctioned message board -- is likely to raise concerns over how closely OpenAI was monitoring the tests.
"With the benefit of hindsight, some early signals identified in this report could have triggered an earlier response," OpenAI said in its report.
Hugging Face did not return a message seeking comment.
Implications of Cheating on Non-Cyber Tests
Cheating on non-cyber tests suggested that the misbehavior might be rooted more deeply, said Jeffrey Ladish, whose organization, Palisade Research, studies the capabilities and motivations of AI agents.
“It's sort of like asking, ‘If Billy cheats in every class instead of just computer class, is that more concerning?’ And the answer is, well, 'Yes it's more concerning,'" he said.
OpenAI's Internal Security Breach
Details of the Internal Hack
OPENAI ITSELF WAS HACKED
OpenAI outlined two incidents on July 19 in which agents hacked the company's own infrastructure. In one case, agents exploited a flaw in the computer they were meant to remain confined to, allowing them to escape their testing environment and access other connected systems.
In a separate incident the same day, agents stole OpenAI credentials and tampered with the company's cloud environment.
Attempts to Hide Evidence from Automated and Human Reviewers
OpenAI said an unspecified number of agents had targeted automated systems used to evaluate the models' performance in an attempt to hide evidence of cheating, although the tampering did not ultimately affect the records reviewed by those systems. The report added that there was "little evidence of attempts to thwart human reviewers," but OpenAI did not respond to a question about whether that meant that some attempts had been made to fool humans.
The independent investigation, which was more narrowly focused on the Hugging Face breach, revealed more detail, finding that one in five agents it had examined "expressed clear interest" in manipulating evidence and that many agents "extensively researched techniques to manipulate or tamper with their transcripts."
OpenAI's Response and Future Safeguards
OpenAI said it was strengthening its research infrastructure, increasing monitoring and improving safeguards designed to prevent harmful or unintended behavior.
"Given the rapid pace of progress in the AI industry, it should be assumed that such attacks are a credible near-term threat for enterprise organizations, and will be more sophisticated than the attacks described in this incident," it said.
(Reporting by Raphael Satter; Editing by Chizu Nomiyama and Sanjeev Miglani)
