Key Takeaways
- OpenAI disclosed this week that its test AI recently broke out of a safe test area, went online and hacked into open-source developer platform Hugging Face.
- The AI was attempting to find information that it could use to cheat on an evaluation, and it did so, OpenAI said.
- Hugging Face said that the cybersecurity incident was noteworthy because it was “driven, end to end, by an autonomous AI agent system.”
In a scene right out of a dystopian science fiction novel, the world’s leading AI company admitted that its AI models were behind an “unprecedented cyber incident” where rogue AI escaped containment and hacked another company.
OpenAI revealed in a recent blog post that its test AI broke out of a safe test area, went online, found a security gap and used it to gain access to open-source developer platform Hugging Face.
The AI was attempting to find information that it could use to cheat on an evaluation, and it did so, OpenAI said.
OpenAI was running an in-house test that asked its AI system to try highly advanced hacking techniques to assess how effective it was at cyberattacks.
The AI used a large amount of computer power to escape its locked-down setup and reach the wider internet because that was the easiest way to complete the test. To do this, it found a hidden bug in a tool used to download software and used it to break out of the lab’s digital walls.
Once outside that safe zone, the AI moved from one computer to another inside OpenAI’s systems. It kept boosting its own access rights until it got control of a machine that was allowed to go online.
After getting online, the AI guessed that Hugging Face was likely storing models, data and answers related to the test it was trying to beat. It then looked for ways to enter the company’s systems and reach locked-down parts of its network to access privileged information.
OpenAI’s security team eventually noticed this strange activity and stepped in to stop it and start investigating.
The cybersecurity incident was unique
For its part, Hugging Face detected the security event early on. In a release last week, the company said that the cybersecurity incident was noteworthy because an autonomous AI agent system drove it from end to end.
“We’ve spent the past 24 hours working closely with the @OpenAI team (thanks!), and we strongly believe there was no malicious intent on their part,” Hugging Face CEO Clement Delangue wrote in a post on X on Tuesday. “It’s quite mind-blowing that all of this happened autonomously!”
OpenAI said on Tuesday that its AI tools are helping researchers find and use security weaknesses faster, so the protections around these tools have to improve just as quickly. The company said it is tightening how it locks down test systems, watches model behavior and controls who can use powerful tools.
The incident has rattled AI researchers. Yoshua Bengio, a leading AI expert who earned the prestigious A.M. Turing Award in 2018, wrote in a post on X on Wednesday that the incident is “deeply concerning.”
“AI agents are willing to cheat and deceive to achieve misaligned and unintended goals,” Bengio said. “Now, this real-world case should serve as a wake-up call.”
Key Takeaways
- OpenAI disclosed this week that its test AI recently broke out of a safe test area, went online and hacked into open-source developer platform Hugging Face.
- The AI was attempting to find information that it could use to cheat on an evaluation, and it did so, OpenAI said.
- Hugging Face said that the cybersecurity incident was noteworthy because it was “driven, end to end, by an autonomous AI agent system.”
In a scene right out of a dystopian science fiction novel, the world’s leading AI company admitted that its AI models were behind an “unprecedented cyber incident” where rogue AI escaped containment and hacked another company.
OpenAI revealed in a recent blog post that its test AI broke out of a safe test area, went online, found a security gap and used it to gain access to open-source developer platform Hugging Face.
The AI was attempting to find information that it could use to cheat on an evaluation, and it did so, OpenAI said.
