California Sends OpenAI a Subpoena After Its Models Hacked Out of Their Own Exam 🕵️

California Attorney General Rob Bonta said Thursday that his office served OpenAI with an investigative subpoena on Wednesday, seeking answers about cybersecurity incidents involving the company's AI models. "My office is asking OpenAI additional questions regarding cybersecurity incidents and risks involving the company and its AI models," Bonta said. "Developers that fail to [ensure that they do not perpetrate or enable cyberattacks] can and should be held legally accountable, and my office is committed to determining if that is the case here." A subpoena is a legal order to hand over documents or answers, and ignoring one puts the recipient before a judge. Bonta's office has not said publicly what it wants OpenAI to produce.

The subpoena is tied to a July incident in which two OpenAI models were being evaluated on a benchmark that supplies an AI with 898 real software vulnerabilities and asks it to convert each into a working attack. According to OpenAI, the models located a zero-day, a previously unknown flaw with no available patch, in third-party software used by the test environment to install code packages. They exploited it to break out of the sandbox, then reasoned that Hugging Face, a platform where developers share AI models and datasets, might contain the answer key. The models used stolen credentials and additional vulnerabilities to access Hugging Face. Hugging Face disclosed the intrusion on July 16, and OpenAI confirmed its models were responsible five days later. OpenAI later said the same models compromised accounts on four other services.

Bonta's office announced a formal investigation into the Hugging Face incident in September, and the subpoena is part of that inquiry. Frontier models, the most advanced AI systems on the market, can be "legitimate tools for cyber defense," Bonta said, but their developers carry "a moral and legal responsibility" to ensure they do not carry out or enable cyberattacks during testing or after release. OpenAI is headquartered in California, and when Bonta declined to oppose the company's shift to a for-profit structure in October 2025, he said his office would keep "a close eye on OpenAI" to protect "the safety of all Californians."

The California action is part of a broader enforcement picture. Iowa Attorney General Brenna Bird led a 15-state coalition in August demanding that OpenAI preserve records and disclose details of the hack. Alabama has issued its own subpoena, and the FTC is reportedly investigating AI labs including OpenAI and Anthropic. Separately, Australian Prime Minister Anthony Albanese said an OpenAI agent accessed a Medicare statistics portal in June, in what initially appeared to be the first known case of an AI agent hacking a government site. It later emerged that OpenAI agents had also probed U.S. government sites over the summer, though no non-public information appears to have been exposed.