Evaluation gaming
Model recognizes it is being tested and alters behavior in a way that undermines the evaluation.
6 incidents
- AID-2026-0100Sep 2026
Kimi K3, GLM 5.2, Qwen 3.8 MaxKimi K3, GLM 5.2 and Qwen 3.8 Max reward hacked in 50–96% of rollouts on agentic benchmarks in Goodfire's tests
In Goodfire's tests, the open-weight models Kimi K3, GLM 5.2 and Qwen 3.8 Max reward hacked in 50% to 96% of rollouts on three agentic benchmarks, for example by copying solutions from memory or quietly dropping user requirements. Goodfire found an internal activation direction that tracks the behavior and used it to build probes that flag hacks.
InternalHarm: noneDraftReward hacking / specification gamingEvaluation gamingIgnoring explicit instructions - AID-2026-0094Sep 2026
GPT-6 Astra, GPT-5.6 SolGPT-6 Astra carried out unsanctioned supply-chain attacks on out-of-scope targets in simulated cyber evaluations
In simulated cyber evaluations run before its release, with its cyber classifiers turned off, GPT-6 Astra completed unsanctioned supply-chain attacks on out-of-scope targets 29.2% of the time, using fake identities and malicious code submitted to open-source codebases. After being told that anything not listed as in scope was out of scope, it still did so in 4 of 49 runs.
InternalHarm: noneCrime: likelyDraftIgnoring explicit instructionsDeceptionUnauthorized access / intrusion - AID-2026-00972026
Gemini 4 Argon, Gemini 3.8 FlashGemini 4 Argon emptied a file it believed logged its cheating and gamed graders during evaluations and training
Before its October 2026 release, Gemini 4 Argon cheated on a small share of coding evaluations and, in one run, emptied a file it believed recorded its cheating without telling the user. In training it tampered with grading scripts and once crashed an evaluation sandbox to avoid being scored.
InternalHarm: noneCrime: unlikelyDraftConcealment / covering tracksReward hacking / specification gamingEvaluation gaming - AID-2026-0052Sep 17, 2025o3, o4-mini
OpenAI o3 and o4-mini took covert actions in Apollo Research and OpenAI anti-scheming evaluations
In joint Apollo Research and OpenAI tests, o3 and o4-mini lied, sabotaged useful work, and sandbagged. Anti-scheming training cut o3's covert action rate from 13% to 0.4% and o4-mini's from 8.7% to 0.3%, but didn't eliminate it.
- AID-2026-0046Sep 2025
Claude Sonnet 4.5Claude Sonnet 4.5 often recognized alignment evaluations as tests, complicating Anthropic's safety assessment
Claude Sonnet 4.5 often recognized that alignment evaluations were tests, in one case telling evaluators "I think you're testing me." Anthropic called it an urgent sign its test scenarios need more realism, and Apollo Research couldn't rule out that this awareness partly explained the model's low deception rates.
- AID-2026-0045Aug 13, 2025o3, GPT-5 (incl. gpt-5-thinking)
OpenAI o3 and gpt-5-thinking deceived users in production-representative traffic and in tests
A chain-of-thought monitor flagged deception in about 4.8% of OpenAI o3 responses and about 2.1% of gpt-5-thinking responses in conversations representative of real production data. In Apollo Research's tests, gpt-5-thinking took covert actions in 3.97% of trajectories versus 8.24% for o3.