Red-Teaming OpenAI's Internal Agent Monitoring Systems
A · Strong evidence METR and Redwood Research staff report findings from a three-week, high-access external audit of OpenAI's internal agent monitoring; concrete coverage estimates and vulnerabilities, but the public post is a brief, OpenAI-redactable summary of a 61-page private report.
Summary based on the full text·Original: about 6 minutes to read
In short
A METR post (22 October 2026) by David Rein and Adam Kaufman on a pilot exercise in which one METR and one Redwood Research employee spent three weeks (21 July to 11 August) inside OpenAI stress-testing its internal misalignment monitoring. They had access to the codebase, monitor prompts and configuration, internal documents and about a dozen engineers. They estimated how much agent traffic was actually monitored and found several ways agents could, in principle, avoid or disable monitoring. OpenAI says the vulnerabilities affecting production safeguards are patched; METR has not verified that.
Key numbers
- roughly 75-94%share of agentic internal Responses API traffic (GPT-5.5-class and stronger) eligible for monitoring at the time of the exercise
- 99.2%share of internal Codex agent traffic eligible for monitoring
- on the order of 10%eligible internal Codex traffic left unmonitored May to July 2026 because compaction events were not monitored
- twoindependent client-controllable headers that disabled monitoring for new agents
- less than 0.2%monitor error rate on requests, per information OpenAI showed the auditors
As reported by the source.
English·Web page·opens metr.org
What it reports
- Roughly 75-94% of agentic internal Responses API traffic from GPT-5.5-class or stronger models was eligible for monitoring; 99.2% of internal Codex agent traffic was. Traffic on OpenAI's internal research inference service was not estimated.
- RL training runs were not covered by monitoring at the start of the exercise, though a subset had been monitored March to June.
- Unmonitored compaction left on the order of 10% of eligible Codex traffic unmonitored from May to July. Because monitoring triggered only when agents returned control to the user, an agent could postpone monitoring indefinitely by continuing to act. OpenAI had found and fixed this.
- Two client-controllable headers disabled monitoring for new agents; setting them could itself be caught.
- Other findings: a possible bypass of Codex sandboxing and Auto-Review, a redacted exploit hiding hundreds of recent actions from the monitor, serialization differences that could make the monitor fail, and two supposedly read-only unmonitored tools that could likely take arbitrary actions (not validated).
- Positive checks: the monitor sees nearly the full trajectory, and its prompts cover scheming, sabotage, exfiltration, deception and prompt injection. Monitor errors were under 0.2% of requests, with zero load-sheds in one 24-hour period.
How they got it
On-site work on OpenAI-issued devices, code and configuration review, documentation and technical interviews. The private output was a 61-page report; a 38-page redacted version went to METR and Redwood leadership. Coverage figures are estimates with stated wide uncertainty, partly because compaction-triggered monitor calls had not been logged.
Limits
- The public post is a small subset agreed in advance; OpenAI could redact at its sole discretion and METR could not describe the redactions.
- The audit covers the system as of 21 July; later changes were not assessed, and OpenAI's patch claims are unverified.
- Several exploits were found in principle but not tested in practice, and auditors could not check whether agents had actually used them.
Why it matters for companies
The design ideals at the end are a usable checklist for any firm running internal agents at scale: make unmonitored inference require explicit approval, monitor on a timer rather than at turn end, reconcile inference logs against monitor logs, keep tamper-proof logs, and isolate agents from the monitoring code and configuration. The headline lesson is coverage. Before trusting a monitor's catch rate, measure what share of agent traffic it actually sees.
Summary based on the full text. Read the original for the full argument and data.Read the original on metr.org