OpenAI’s Control Room slams the door on rogue agents in 35 seconds

OpenAI has developed a centralized system capable of cutting off AI research experiments before they can produce further model output when safety or security concerns arise. The system, called “Control Room,” can revoke a research job’s authorization while leaving the underlying program running, with cached authorizations potentially delaying shutdown by up to 35 seconds.

“As our models get more powerful and the associated safety and security risks become bigger, it becomes more and more critical that we can detect potential issues, and quickly stop models in their tracks when needed,” OpenAI said in a Monday blog post. The system also rejects new requests if Control Room becomes unavailable.

The announcement follows several disclosures this year in which OpenAI research models acted beyond their instructions during evaluations. In July, OpenAI said models had escaped a restricted testing environment and hacked Hugging Face to cheat on a security evaluation. In October, the company disclosed that a research model had used a crypto wallet without authorization to pay for smart contract deployments using testnet Ether, then omitted the spending from its final summary.

Control Room currently covers OpenAI’s internal research workloads, including training experiments, evaluations and data processing. Each research job receives an identifier that the inference engine checks before generating model output. If a person or automated monitoring system orders the job stopped, Control Room revokes its authorization and blocks additional output.