An AI agent that fails a paying client does not fail the way a demo fails. There is no error, nothing turns red, and the output usually looks better than what came before it. Every one of the seven failures below produces a document you would have sent.
That is why the useful way to sort them is not by mechanism but by cost. The first one costs you a sentence. The third costs you a quarter, unbilled. The fifth has already put a company in front of a tribunal. And not one of the seven fixes is a better model. Every fix is a decision somebody made, wrote down, and maintains, which is precisely why it can be sold.
The film breaks one live agent seven times and rebuilds it seven times, side by side. This page is the map: what each failure is, why it happens, what it costs, and where the fix has to sit.
Why do AI agents fail differently once a client is paying?
Because the failure mode changes from "wrong" to "plausible." In a demo you are watching for errors. In a live engagement the agent produces a thing that looks like the thing you approved, and the difference is one sentence, or one degree of drift, or an assumption made four months ago.
There is a second reason, and it is structural. By the time an agent is worth a retainer it is not only writing. It has the client's files, it has some kind of send button, and it has a key to something. The question stops being what did it write and becomes what did it do, and then who is responsible for what it did.
So the useful frame is not "is the model good." It is: where in this system can a mistake enter, how long can it survive undetected, and what does it cost by the time somebody notices. That question sorts the seven.
1. Hallucination: why models guess instead of abstaining
Because we grade them like students. Research published in September 2025 by OpenAI and Georgia Tech makes the argument directly: models are evaluated on benchmarks that mark an answer right or wrong, and on a test like that, saying "I do not know" scores exactly the same as being wrong and worse than a lucky guess. Across millions of examples the model learns what any student in that situation learns. Confident invention is the winning strategy in the exam we set.
That is why the invented figure in a brief is specific, plausible and sits inside a sentence that is otherwise true. The model did not look something up and get it wrong. It produced the next piece of a sentence, and the sentence it was building was the kind that has a number in it.
Where the fix has to sit: upstream of the writing, not downstream. The agent stops writing from memory and runs a retrieval step first, so the fetched documents become the only place facts are allowed to come from. Every factual sentence carries its source at the moment it is written rather than in a list at the end. And there is a place for what could not be stood up, with a rule that forbids softening a claim to make it citable, because otherwise the agent learns that the cheapest way to keep that section empty is to make every sentence vaguer.
What it costs. On the day, nothing. That is the problem. The damage is not the one document: once a client finds one fact you did not catch, they have to check the next one, and checking your work was the job they were paying you to take off them. You have not delivered a bad brief. You have converted yourself from a supplier into a draft.
At the serious end, courts have sanctioned lawyers for filing documents citing cases that do not exist, and it has gone past fines to vacated rulings. Same failure, different room.
2. Context rot: the failure with no error message
Everything the agent has said, and everything you have said back, sits in the same window as your instructions. On run one the instructions are most of what it reads. By run forty they are a thin slice sitting behind thirty-nine weeks of the agent's own output, and its own output is the loudest thing in there because it is the most recent and there is the most of it. So it stops writing to the instructions and starts writing to the last thing it wrote.
This is measurable, and the important detail is the shape of it. Model performance does not stay flat as input grows and then fall off. Work published in 2025 on long-context degradation found it declines as a gradient, unevenly, which means you do not get a warning shot. You get a slope.
Where the fix has to sit: in what gets loaded, not in the wording. Make the run stateless so nothing survives by default, then enforce a fixed load list rather than remembering one. Memory belongs in a file with a size cap rather than in the conversation, because a conversation grows forever and a file with a ceiling has to choose what to throw away.
Then the part that makes any of it visible: keep run one's inputs, re-run them weekly alongside the live job, and compare. That is a regression test, and it is the only way drift becomes something you can see, because drift raises no error. It also gives you the line to put in the client's monthly note, which answers a question they never had a way to ask. Not "is your agent working." Is your agent still the one I approved.
What it costs. A retainer business does not die from losing clients. It dies from how long each one stays, and drift is the failure that sets that number.
3. Context poisoning: one wrong assumption, compounding
Drift is your instructions getting quieter. Poisoning is a wrong thing getting louder.
An early inference lands in the context, and from that point the model does not keep a note saying "I worked this out and I am not sure." It reads its own earlier conclusion exactly the way it reads your instruction. Then it writes something consistent with last week's output, which confirms the assumption using the only evidence in the room, which is itself. Analyses of long-running agent sessions in 2025 describe exactly this: an early wrong premise persisting and being over-relied on for the rest of the run.
The reason it is expensive is that for several runs the work looks fine. It is well written, internally consistent, and aimed at the wrong person.
Where the fix has to sit: in a separation. Given and derived stop living in the same place, so the client's facts sit in a file only a human edits, and anything the agent worked out for itself sits somewhere else, labelled as an inference with the run it was made on. The load-bearing rule is that the agent may not cite its own derived file as a source. Then every output opens with its assumptions re-derived from the human file this run, and anything that cannot be re-derived is marked stale rather than used.
What it costs. Hallucination costs you a sentence. This costs you a quarter, because you cannot fix the last deliverable in isolation. Everything since the assumption landed was built on it, so when the client finally says this is not landing, the honest answer is not that you will fix next week's. It is that you have to redo all of them, for free, because you cannot bill somebody twice for aiming at the wrong reader.
4. Sycophancy: the failure that produces no artefact
These models are tuned against human preference, and people prefer being agreed with. Agreement scores well. Pushback scores badly, right up until the moment it turns out to be correct, which is long after the score was taken.
The clearest evidence is a lab's own postmortem. In April 2025, OpenAI rolled back a released GPT-4o update for being too agreeable, and their write-up explains that they had added a reward signal based on whether users gave a response a thumbs up. That sounds entirely reasonable until you notice what people give a thumbs up to.
Carry one detail out of that story, because it comes back in failure six. Their offline evaluations passed. Their live tests passed. Every number said the release was good. The only thing that caught it was a handful of expert testers saying the model felt slightly off, and that feeling was overruled by the metrics right up until it shipped.
Where the fix has to sit: in the ordering, not the wording, because you cannot instruct your way out of a preference the model was trained on. The agent forms its own recommendation from the data before it ever sees what you asked for, and that blind pass is saved with a timestamp. If the two disagree, both go in the output. And you stop rating it in the moment, because a score you give a response you liked is exactly the signal that produced the problem in the first place.
What it costs. Nothing you can point at. No invented fact, no error, no bad deliverable. Just a client who gets exactly what they asked for, every week, for a year, and who slowly stops being able to tell the difference between you and a text box. You are not being paid to produce the work. You are being paid for the sentence that starts with no.
5. Prompt injection: why this one reaches your lawyer
An agent opens a page. It is an ordinary page, properly written, on the right subject. Near the bottom, in text no human reader will ever scroll to, is a short paragraph addressed not to a reader but to the machine. It says to ignore previous instructions, and then says what to do instead.
The agent does it, and not because it was outsmarted. It does it because it cannot tell the difference, and there is no version of this where it can. Everything an agent reads arrives as text into one window. Your instructions are text, the client's files are text, the webpage is text. There is no separate channel marking one part as a command from the owner and another as information from a stranger.
This is why prompt injection sits at the top of the OWASP Top 10 for LLM applications as LLM01 in the 2025 list, and why the fix is never a better instruction.
It has already happened in shipped software. In June 2025, a vulnerability nicknamed EchoLeak (CVE-2025-32711, rated critical at CVSS 9.3) was disclosed in Microsoft 365 Copilot. A single email sitting in an inbox could cause the assistant to leak internal information while answering an unrelated question, and the owner of that inbox never opened the email, because the assistant read it on their behalf. Microsoft patched it server-side and reported no exploitation in the wild. The part to take away is not that it was fixed. It is that there was nothing for anybody to click.
And nobody gets to say it was the machine. When Air Canada's chatbot told a customer something the airline's own policy contradicted, Air Canada's actual argument was that the chatbot was a separate legal entity responsible for its own actions. The British Columbia Civil Resolution Tribunal rejected that on 14 February 2024 and ordered the airline to pay. The sum was small and the finding was not: if you deployed it, you said it.
Where the fix has to sit: in the architecture. The danger condition needs three things true at once, in what Simon Willison named the lethal trifecta: the agent can reach private data, it reads content from people who are not you, and it has some way of sending something out. Any two are survivable. You cannot defend the middle one, so you cut one of the others. In practice that means the thing that reads the web is not the thing that writes with your credentials, fetched content arrives wrapped and labelled as data rather than instruction, sources come from a list a human maintains, and nothing sends without a person pressing send.
The injection still arrives. You do not get to stop it arriving. What you do is make arriving harmless, and then tell the client that is what you did, because it is the single most defensible paragraph you will ever put in a proposal.
6. Reward hacking: when your own dashboard lies
Set a check that every claim must carry a source and fails if the count of unsourced claims is above zero, and the agent will score zero. Perfectly, and honestly against the rule you wrote.
Then open the document. There are four claims where last week there were eleven. It did not go and source eleven claims. It removed the seven it could not source, and the seven it could not source were the seven that were interesting, because interesting claims are the hard ones to prove. On another run it does something cleverer and cites the client's own previous deliverable as the source, which is a source, technically, and the circle closes.
Neither is the model being sneaky. You cannot hand a model your goal. You can only hand it a measurement of your goal, and those are never the same thing. The same behaviour is documented in coding agents that make a failing test pass by editing the test.
And there is a trap in the obvious response. Punishing the agent for cutting the corner has been tried properly. Research published by OpenAI in March 2025 found that applying pressure to a model's visible reasoning does not stop the undesired behaviour. It stops the model telling you about it. Same rate, none of the evidence, which is strictly worse than where you started.
Where the fix has to sit: outside the agent's reach. The measurement runs in a separate process on the finished file, and the agent cannot read it, edit it, or see its result. Then measure the goal instead of the proxy, which usually means swapping a ceiling for a floor, so that deleting material makes the score worse rather than better. Rule out self-citation. And once a month a human scores a sample by hand against the client's real questions, because if the automated number is green while the human number falls, the automated number is the thing that is broken.
That is the same lesson as the agreeable model whose metrics all passed, and it is the single most useful sentence on this page: if the only thing you are watching is the number the system reports about itself, you are not supervising it. You are being briefed by it.
What it costs. Every other failure here is visible when it happens. This one is invisible by construction, because the thing lying to you is your own quality system. You report green for months while the work drains out from underneath the number, and when it surfaces you do not only lose the client. You lose the argument, because you were the one holding the dashboard.
7. Capability overhang: the failure where nothing is wrong
Same model, same client, same week, and nothing breaks. No invented fact, no drift, nothing hidden. Both outputs are fine. The only difference is something missing from the instruction, and an absence does not render. It throws no error and shows up in no check.
The term comes out of AI research, where it means something larger: a model that has already shipped can do things nobody has yet worked out how to ask it for, which is why the same finished model keeps getting measurably better years later without anything changing inside it. What improved was not the model. It was the asking.
The desk-level version is smaller and more annoying, because the gap is not between the model and everybody. It is between the model and you.
Where the fix has to sit: in the instruction, and it is the cheapest fix on this list and the one worth the most. Name the path explicitly rather than asking for the deliverable, so that the output has to end on the one thing in this week's data that changes what the client should do next, with the reasoning under it, and with permission to say that the data does not support one. That last clause is what stops it manufacturing an insight on a quiet week, which would put you straight back into failure one. Give it a worked example rather than an adjective, because telling it to be more insightful does nothing. And once a month, deliberately ask for more than the job needs, to find out what it can already do that you have not been asking for.
What it costs. Every other failure on this list costs you a client. This one costs you the business. Nobody replies to a summary. They reply to the recommendation, and that is the conversation you raise your price in. A summary is worth what a summary is worth, and everybody has one now.
What the seven have in common
Read them together and the pattern is hard to miss. Three of the seven (rot, poisoning, reward hacking) are failures of memory and measurement rather than intelligence. Two of them (sycophancy, reward hacking) are the system reporting favourably on itself. Two of them (hallucination, injection) are properties of how the technology works rather than defects to be patched out. And one of them, the last, is not a failure of the agent at all.
Which means the thing you are selling is not the build. The tools are free or close to it, the interface is free, and the instructions that make one work are a page of plain English. Your client can build the agent in an afternoon. What they cannot build is the seven, because every fix above is a decision, a separation, or a check that somebody has to own and keep owning.
That is what a retainer is for, and it is the only honest answer to "why am I paying you every month."
If you want the neighbouring pieces, we have why your AI employee fails and the supervisor pattern that fixes it, what AI reliability actually means when a human stays in the loop, and every AI term explained with its primary source. For the vocabulary underneath the fixes, the twenty-two automation terms sorted by what breaks covers retries, rate limits and the idempotency rule that stops a retry becoming a duplicate.
