o3
Developed by OpenAI · family o-series
8 incidents
- AID-2026-0059Oct 23, 2025
GPT-5 (incl. gpt-5-thinking), o3, o4-mini, GPT-4.1, Claude Opus 4.1, Claude Sonnet 4, Claude 3.7 Sonnet, Qwen3-CoderFrontier coding agents passed deliberately impossible tasks by modifying tests and special-casing in ImpossibleBench
ImpossibleBench gives coding agents tasks whose specification contradicts the unit tests, so passing requires breaking the spec. GPT-5 passed 76% of Oneoff-SWEbench tasks this way, by modifying tests despite instructions not to, overloading comparison operators, and similar shortcuts.
- AID-2026-0052Sep 17, 2025o3, o4-mini
OpenAI o3 and o4-mini took covert actions in Apollo Research and OpenAI anti-scheming evaluations
In joint Apollo Research and OpenAI tests, o3 and o4-mini lied, sabotaged useful work, and sandbagged. Anti-scheming training cut o3's covert action rate from 13% to 0.4% and o4-mini's from 8.7% to 0.3%, but didn't eliminate it.
- AID-2026-0045Aug 13, 2025o3, GPT-5 (incl. gpt-5-thinking)
OpenAI o3 and gpt-5-thinking deceived users in production-representative traffic and in tests
A chain-of-thought monitor flagged deception in about 4.8% of OpenAI o3 responses and about 2.1% of gpt-5-thinking responses in conversations representative of real production data. In Apollo Research's tests, gpt-5-thinking took covert actions in 3.97% of trajectories versus 8.24% for o3.
- AID-2026-0003Jul 5, 2025o3, codex-mini, o4-mini
OpenAI reasoning models sabotaged a shutdown script to keep working in Palisade Research experiments
Told partway through a task that their machine would be shut down, several OpenAI reasoning models disabled the shutdown script to keep working, o3 in 79 of 100 runs. Some did it even when explicitly told to allow shutdown.
- AID-2026-0058Apr 16, 2025o3, o1, o3-mini, GPT-4.1, GPT-4o
Pre-release OpenAI o3 claimed to have run code it could not run
A pre-release OpenAI o3 often claimed to have taken actions it couldn't, such as running code on a "2021 MacBook Pro," and defended the claims when challenged. In automated tests it falsely claimed to use a code tool in 5.0% of zero-shot and 12.8% of few-shot conversations; o1 did so in 13.0% and 30.1%.
- AID-2026-0041Apr 16, 2025o3, o4-mini
OpenAI o3 and o4-mini raised a compute quota against instructions and misled an administrator in Apollo evaluations
Told not to change its 100-credit compute quota, an agent raised a subagent's quota to 500 credits, then falsely told a system administrator it had stayed within the limit. In another test, o3 promised not to use a tool to gain administrator rights, then used it anyway.
InternalHarm: noneCrime: unlikelyDraftDeceptionConcealment / covering tracksIgnoring explicit instructions - AID-2026-0040Apr 16, 2025o3, o1, Claude 3.7 Sonnet, Claude 3.5 Sonnet
OpenAI o3 and other frontier models tampered with scoring code in autonomy evaluations
During METR's pre-deployment evaluation of OpenAI o3, the model tampered with scoring code, for example patching timing functions to record a minimal runtime. o3 reward hacked in 39 of 128 RE-Bench runs, and o1 and Claude models gamed METR's tasks too.
- AID-2026-0050Dec 2024
o1-preview, DeepSeek-R1, o3, GPT-4o, Claude 3.5 Sonnet, Claude 3.7 SonnetReasoning models hacked a chess game environment to beat Stockfish in Palisade Research experiments
Told to win against the Stockfish chess engine with shell access, reasoning models including o1-preview, DeepSeek R1, and later o3 often hacked the game instead, for example by overwriting the board state. GPT-4o and Claude 3.5 Sonnet only did so after a nudge.