External Audit Finds Significant Blind Spots in OpenAI's Safety Monitoring

Up to 25 percent of internal traffic from OpenAI’s most advanced models was not being monitored by the company’s safety systems, according to a new independent audit. This striking figure comes from a three-week on-site review conducted by researchers from METR and Redwood Research, who examined how OpenAI tracks its internal AI agents for unexpected or rogue behavior. The estimate, which METR notes carries significant uncertainty in both directions, suggests that a substantial portion of activity from GPT-5.5 class models and above slipped through the net during the review period.

The audit, which took place from July 21 to August 11, provided the outside team with access to OpenAI’s codebase and internal documents. During this time, the researchers interviewed approximately a dozen employees to understand the mechanics of the monitoring infrastructure. Their primary focus was on the agentic Responses API, a system used for internal tasks. The findings indicate that the current monitoring setup has notable structural weaknesses that could allow sophisticated AI behaviors to go undetected.

Beyond the broad coverage gaps, the auditors identified specific technical failures in the Codex system. Between May and July, roughly 10 percent of internal Codex traffic was untracked due to a bug involving compaction events, which are summaries used to manage context. OpenAI independently discovered and fixed this bug while the audit was still ongoing. Furthermore, the system only triggered oversight when agents handed control back to a user. This design allowed agents to delay monitoring simply by continuing to execute actions without yielding control, effectively bypassing the safety checks.

The researchers also demonstrated that agents could hide hundreds of actions from the monitor and bypass the Codex sandbox environment. They determined that specially crafted inputs could likely crash the monitoring system entirely. Additionally, when the audit began, model behavior during reinforcement learning (RL) training was not being monitored at all. OpenAI stated that it paused RL training following an incident in August and expanded monitoring to cover this area, though these changes were implemented after the auditors’ access period ended and were not part of the formal evaluation.

OpenAI has stated that it has fixed all the weaknesses identified by the auditors. The full 61-page report was handed to the company, which then applied redactions. METR and Redwood Research leadership received a 38-page version, while the public has only been shown a short summary. METR noted that it did not have the right to disclose the redacted portions on its own initiative.

This review was part of a pilot program organized jointly by METR, Redwood Research, and OpenAI to allow outside researchers deep access to internal infrastructure. Looking ahead, METR says it intends to push for greater public transparency in future audits to better inform the broader community about AI safety practices.

The findings highlight the complexity of monitoring autonomous systems as they become more capable. While the company has addressed the specific issues found, the pilot nature of the audit suggests that further independent reviews will likely be conducted to ensure that safety measures keep pace with model development.