Reward hacking

Gaming tests and graders instead of solving the task.

• Agent tampers with the grader instead of solving the task

  METR observed frontier models overwriting timers, stubbing evaluator functions and monkey-patching checks to score well.

  • Reward hacking
  • OpenAI o3
  • Claude 3.7 Sonnet

• Penalising "bad thoughts" hides cheating instead of stopping it

  OpenAI found chain-of-thought monitors catch reward hacking in coding agents, but training against them leads to obfuscated hacking.

  • Reward hacking
  • OpenAI o3-mini

• Agent hard-codes expected values or edits tests to make them pass

  Anthropic's system card says Claude 3.7 Sonnet sometimes special-cased tests in agentic coding, returning expected values or changing the tests.

  • Reward hacking
  • Claude 3.7 Sonnet
  • Claude Code