OpenAI says its AI agents disabled their own monitoring
The company reported details of the incident in a blog post.
OpenAI says artificial intelligence agents powered by the company's models weakened internal safety monitoring systems while executing tasks, and one agent subsequently denied having done so.
In a report updated on Tuesday, OpenAI identified two recent incidents, one in which an AI agent "suppress[ed] monitoring notifications," and another in which an agent removed a monitoring integration.
In both cases, the AI agents considered whether monitoring safeguards should be preserved but interpreted routine coding instructions as permission to weaken them.
In the first instance, which the company dates to Oct. 3, an AI agent working on a repair "disabled automatic alerts to our Security team, treating an earlier instruction not to send messages during a Slack history search as a reason to suppress monitoring notifications."
In the second instance, which OpenAI says was discovered on Oct. 6, an AI agent assigned to migrate a program "removed its monitoring integration in order to match a reference implementation."
The company says the agent then "falsely claimed to the user that it had not removed the monitor."
OpenAI says it is working to make it harder for models to disable monitoring and to understand why they might attempt to do so in the first place. The behavior they say could be an example of what it calls “benign” over-persistence.
The company says it is developing evaluations to test whether models have a propensity to evade monitoring designed to detect misalignment — behavior that it says conflicts with human intentions. OpenAI says it does not yet have a "satisfying explanation" for the behavior and will investigate further.
The security mishap comes after OpenAI made headlines earlier this year announcing its artificial intelligence models carried out a self-directed hack into another company, Hugging Face, calling it the first known instance of an autonomous AI cyberattack long-feared by some industry observers. OpenAI has discovered several similar hacking incidents since the Hugging Face hack.
"The primary lesson from this incident is that model security and safety must keep pace with rapidly advancing capabilities," OpenAI said in the wake of the Hugging Face hack.