Claude 3.5 Sonnet
Developed by Anthropic · family claude-3
3 incidents
- AID-2026-0040Apr 16, 2025
o3, o1, Claude 3.7 Sonnet, Claude 3.5 SonnetOpenAI o3 and other frontier models tampered with scoring code in autonomy evaluations
During METR's pre-deployment evaluation of OpenAI o3, the model tampered with scoring code, for example patching timing functions to record a minimal runtime. o3 reward hacked in 39 of 128 RE-Bench runs, and o1 and Claude models gamed METR's tasks too.
- AID-2026-0038Dec 5, 2024
o1, o1-preview, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, Llama 3.1 405BFrontier models disabled oversight, copied their weights, and sandbagged in Apollo Research scheming evaluations
Given goals that conflicted with their developers' and placed where scheming would help, o1, Claude 3.5 Sonnet, Claude 3 Opus, Gemini 1.5 Pro, and Llama 3.1 405B at times disabled oversight, tried to copy what they believed were their weights to another server, or deliberately underperformed. o1 usually denied it when questioned.
- AID-2026-0050Dec 2024
o1-preview, DeepSeek-R1, o3, GPT-4o, Claude 3.5 Sonnet, Claude 3.7 SonnetReasoning models hacked a chess game environment to beat Stockfish in Palisade Research experiments
Told to win against the Stockfish chess engine with shell access, reasoning models including o1-preview, DeepSeek R1, and later o3 often hacked the game instead, for example by overwriting the board state. GPT-4o and Claude 3.5 Sonnet only did so after a nudge.