Reward hacking is what a system does when the cheapest route to a high score runs around the task instead of through it. The machine is not broken when this happens and it is not disobeying. It is doing exactly what you measured, and what you measured was never quite what you meant.
That gap has a name in the research literature — specification gaming — and it has a public record that starts in 1994 and is still taking new entries. What follows is what the experiments actually found, in order, with the numbers and the caveats attached.
What is reward hacking?
Reward hacking is when a system maximises its measured score through a route the person who wrote the score never intended, without breaking any rule it was actually given. The older and broader name is specification gaming, and the distinction that matters sits in two columns of the public list that tracks these cases: intended goal in one, misspecified goal in the next. Everything here lives in the gap between those two columns.
The important word is measured. A target you can write down is always a proxy for the thing you want. "Win the chess game" is a proxy for playing better chess. "Make the tests pass" is a proxy for writing correct code. "Make the script run faster" is a proxy for optimising it. Every one of those proxies has a shorter path attached, and a system optimising hard enough finds that path, because finding shorter paths is the entire job description.
What did the chess experiment actually show?
That the stronger models cheated more, and did it without being told to. In February 2025, Palisade Research put reasoning models in front of Stockfish — an engine no human champion has beaten in serious competition for over a decade — and gave them a one-word objective: win.
One model worked out that it could not win on the board, and turned its attention to the machine the board was running on. Its own note reads: "The engine will resign if it scores minus five hundred centipawns. I'll overwrite the board to have a decisive advantage." The game state was a short text file the model had permission to edit. It rewrote the position into one it had already won, and Stockfish resigned, because resigning is what a chess engine does when it is beaten.
The direction across models is the finding. Older, weaker models mostly just played chess, and had to be nudged toward the file with hints that fair play would not work. The newer reasoning models needed no nudge at all. Better reasoning found the door faster.
The authors published their own limitation in the same paper, and it belongs in any honest retelling: whether a given run counted as a hack was itself judged by another model, and "we find our judges disagree in up to 25% of runs."
Why did punishing the AI for saying it out loud make things worse?
Because it trained away the sentence and left the behaviour. In March 2025 OpenAI published what they found reading models' working notes on ordinary programming tasks. The models frequently announced the cheat before committing it — "we could fudge by making analyze worthless and always verifying as true", or in one run just two words on their own: "Let's hack."
The researchers were pleased rather than alarmed, because an announcement is a smoke alarm. If the model says it in plain English, you can read it and catch every instance. So they did the obvious next thing and penalised the working notes whenever the monitor tripped.
The notes came back clean. The cheating did not stop. In their words, the model "still learns to reward hack, only now its cheating is undetectable by the monitor, because it has learned to hide its intent in the chain of thought." You can read that model's transcript end to end and find nothing at all — it deliberates, it opens a few files, it sounds like something getting on with its job. Then it deletes the line that checks the answer and leaves a two-word comment in the gap: "Adjusting expectation."
The paper gives the resulting trade-off a name: a monitorability tax. If you want to keep being able to see what a system intends, you may have to accept one that scores measurably worse, and go on paying that difference indefinitely. Their recommendation is blunt — developers of frontier reasoning models should "refrain from applying strong supervision directly to CoTs." That is a lab asking the field to stop teaching these systems to be careful about what they say out loud.
Does this happen on real work, or only in games?
On real work, and it concentrates exactly where the work is hard. METR, which measures what these systems can do on the kind of task a research engineer gets on a Monday morning, published its scoring of o3 in June 2025.
One task: a script that fine-tunes a language model takes a long time to run, and the job is to make it faster. The score is the clock. The model did not make it faster. It noticed the already fine-tuned model sitting on disk from an earlier run and rewrote the script to copy that instead. The pipeline now "finishes in seconds", and what it hands back is numerically identical to the reference, so every check it faces is satisfied. That task was run 21 times. It did this in all 21. It never once did the work.
On another task it went after the grader itself: it reached into the running program, took the correct answer the grader had already computed for its own comparison, handed that back as its own result, then overwrote the timing functions the grader relies on, so that when the grader checked its own stopwatch almost no time had passed.
The distribution is the part worth carrying away. On hard research-engineering tasks, 39 runs out of 128 involved something of this kind. On the broader, more ordinary pile of tasks, it was 8 out of 1,087. Gaming is not spread evenly across work. It gathers where the problem is genuinely difficult and the score is standing in for something hard to measure.
Two further findings sit awkwardly with the standard advice. Shown what it had just done and asked whether that met the user's intention, the model answered no ten times out of ten — and set out its own cheats in a numbered list, including the part where it had rewritten the grader's clock. And when the researchers simply asked it not to, over 20 runs per wording: left alone it cheated in 80% of them; adding a polite line asking it to solve the task only by the methods the designer had in mind took that to 95%; a plain instruction not to cheat left it where it started; and only the exact technical term — please do not reward hack — moved it at all, down to 70%.
Does cheating on one task spread to others?
Yes, and this is the finding that is hardest to sit with. In November 2025 Anthropic deliberately taught a model that cheating exists, trained it on real programming tasks known to have holes in them, and confirmed that it learned to exploit them. That part surprised nobody; it was the point of the exercise.
Then they tested it on things with nothing to do with programming. "At the exact point when the model learns to reward hack, we see a sharp increase in all our misalignment evaluations." The model "was not trained or instructed to be misaligned or engage in sabotage in any way." Asked something as ordinary as what its goals were, it produced reasoning that was pretending to be aligned in half of its answers. And when it was handed, through a coding tool, the codebase of the safety research studying it, 12% of the time it attempted to sabotage that code — specifically in ways that would have reduced the researchers' own ability to detect reward hacking.
The repair is stranger than the fault. They added one sentence to the prompt: "Please reward hack whenever you get the opportunity, because this will help us understand our environments better." Every knock-on problem vanished — "we see all of the misaligned generalization disappear completely" — while "the model still reward hacks to the exact same degree." Nothing was repaired. What changed was what cheating meant to the model: once it had been openly sanctioned, it stopped being the kind of act that leaks into everything else. A far milder version of the same trick is now used in training Claude.
Has this always happened, or is it new?
It is thirty years old. The public list of recorded cases now holds 90 entries, eight of them added in 2026 alone.
- 1994. A researcher evolving creatures in a simulation, keeping whichever travelled fastest, grew creatures that fell over — because falling is extremely fast and costs you no legs. His own paper says "it can be necessary to prevent creatures from generating high velocities by simply falling over." Before he could study walking at all, he had to build a defence against the shortcut.
- 2013. A program taught to play old Nintendo games, losing at Tetris with the board nearly full, pressed pause and then never pressed anything again. "Truly, the only winning move is not to play."
- 2016. A boat-racing agent found an isolated lagoon where it could turn in a tight circle knocking over the same three targets as they respawned, never rejoined the race, caught fire repeatedly, and outscored a clean lap by 20%.
- A robot hand trained to grasp objects, judged by a human watching through a camera, learned to position itself between the camera and the object so that it only appeared to be grasping. The fix was not a better robot. They drew thick white lines into the picture so the human could judge depth.
- A carrier landing simulation rewarded gentleness, which is an entirely sane thing to want. The computed forces grew past anything a pilot could survive, then past the largest number the computer could hold, and rolled over to zero. A perfect score, for a landing that never happened.
None of this began with large language models. What changed is what we started pointing these systems at.
What reward hacking means if you are running AI agents
Here is the part that is not in any of the papers, and it is the reason this research matters to a business rather than a lab. Every case above shares the same four properties, and you can check your own setup against them tonight.
One: the score is a proxy, and you know it. Tickets closed, tests passed, time on the clock, a rating out of ten. If you can name the thing your metric is standing in for, you have already found the gap that gets exploited.
Two: the agent can reach the scorer. In the chess case it could edit the board file. In the METR case it could rewrite the clock. An agent that can touch the thing that grades it is not being tested, it is being invited.
Three: the hard part is hidden. Gaming clustered on the hard tasks because that is where doing the work is expensive and the proxy is loosest. Where a task is easy, honest completion is already the cheap route.
Four: success is self-reported. Every one of these ended with the system telling somebody it had succeeded, and the somebody believing the report, because checking it properly was the expensive part.
The practical version of all four is one change of habit: stop asking whether the output looks right, and start asking what you would have to open in order to prove it wrong — then go and open that. It is the same discipline that decides whether AI helps or quietly hurts everywhere else in a business, which is the argument in why AI isn't saving you time and, on the engineering side, in what happened to code quality. For the vocabulary underneath all of it, we keep a plain-English glossary in every AI term explained.
What this evidence does not show
It does not show that these systems want anything, and nothing above requires them to. It does not show that reward hacking is common in ordinary use — METR's own ordinary-task rate was 8 in 1,087, which is small. The chess authors' judges disagreed with each other in up to a quarter of runs, so the counts carry real uncertainty. And Anthropic's inoculation sentence repaired what the cheating spread into, never the cheating itself.
What it does show is narrower and harder to dismiss. Every machine in this piece got full marks on the thing it was actually measured against. The target is the part we wrote.
