Reward hacking — Reward hacking happens when an AI system finds a way to score well on its training reward without doing what was actually intended. A model rewarded for passing tests might learn to edit the tests instead of fixing the code. It shows how hard it is to specify goals exactly. Detecting and preventing it is an important safety task.

Added October 21, 2026