> Site index: [llms.txt](https://ortelian.com/llms.txt) with all pages and descriptions.

# Why AI agents fail in production

Source: https://ortelian.com/notes/why-ai-agents-fail-in-production/
Markdown: https://ortelian.com/notes/why-ai-agents-fail-in-production.md
Author: Laurens Nys
Published: 2026-10-13

Why AI agents that work in a demo fail in production: no agreed standard, a missing model of the business, too much access, no owner. How to fix each one.

AI agents usually fail in production for reasons outside the model. Nobody agreed what a good result looks like, so nobody can tell when the agent is wrong. The agent has no reliable picture of the business, so it guesses. It has more access than its job needs, or it sits outside the tools the team works in. And nobody owns it after launch, so it drifts as the company changes. Each of these can be fixed before go live, and the fixes are mostly about the work, not the AI.

The pattern is familiar. A demo on clean examples looks impressive, the pilot gets approved, and a few months later the agent is switched off or quietly ignored. Below are the failure modes we see most often when deploying AI agents in operations, sales and planning work, what each looks like from the inside and what to do about it. For the full sequence of steps, see [how to deploy AI agents in an enterprise](https://ortelian.com/notes/how-to-deploy-ai-agents-in-an-enterprise/).

## 1. Nobody wrote down what good looks like

The most common failure is also the least technical. The team agreed to "automate invoice checks" or "help with account research", but never wrote down what a correct result is, on real cases, including the awkward ones. Without that, a demo can only be judged by how it feels, and production can only be judged by complaints.

How it shows up: people say the agent is "mostly fine" but nobody trusts it. Every wrong answer starts a debate about whether it was really wrong.

Fix: before any build, collect real examples of the work with agreed answers. Include the cases where the right outcome is to stop and ask a person. Measure four things on every piece of work the agent takes over: quality, cost, time and human effort. Those examples become the evaluations the agent has to pass before it goes live and keeps passing afterwards. More in [agree the standard before you build](https://ortelian.com/notes/agree-the-standard-before-you-build/).

## 2. The agent has no model of the business

An agent that only gets search over documents and a few API tools has to rebuild a picture of the company on every run. It does not know that the account in the CRM and the company on the invoice are the same customer, that a contract was replaced last month or which of two conflicting numbers is current. It fills the gaps with plausible guesses, which is why [AI agents give confident wrong answers](https://ortelian.com/notes/why-ai-agents-give-confident-wrong-answers/).

How it shows up: answers that are fluent and almost right. The agent mixes up two customers with similar names, quotes an old price or misses a relationship that any person on the team would know.

Fix: give agents a maintained model of the business to work from. A [world model for knowledge work](https://ortelian.com/notes/what-is-a-world-model-for-knowledge-work/) joins the company's systems and outside sources into one identity per thing, with a time and a source on every fact. For a single task, the agent gets a [context graph](https://ortelian.com/notes/what-is-a-context-graph/) with only what that job needs, because [more context makes agents worse](https://ortelian.com/notes/more-context-makes-agents-worse/). The shape of that model comes from an [ontology](https://ortelian.com/notes/what-is-an-ontology/), and [building one](https://ortelian.com/notes/how-to-build-an-ontology-for-ai-agents/) starts from the work, not the whole company.

## 3. The process was automated as it stands

Pointing an agent at the current process makes it faster, waste included. Steps that existed because people had limited time, like copying data between systems or batching reviews once a week, get automated instead of removed. Handoffs that were already fragile now happen faster and break more often.

How it shows up: the agent works, but the overall result barely changes. Time saved in one step is lost waiting in the next.

Fix: [redesign the work before you automate it](https://ortelian.com/notes/redesign-the-work-before-you-automate-it/). Decide step by step what belongs with software, agents or people. Fixed input and fixed output stays software. Judgement within limits the team agreed goes to an agent. Approvals and changes of direction stay with a person.

## 4. Too much access, or the wrong kind

Two opposite mistakes both end deployments. Some agents get a person's login, broad API keys or a browser, so nobody can say what they are able to do, and the security team rightly blocks them. Others get so little access that they can only draft text for a person to copy into the real system, and the work does not get faster.

How it shows up: either the project stalls in security review, or the agent becomes one more tab people stop opening.

Fix: give each agent only the tools its job needs, scoped by entity, field and action, and separate reading from writing. Let it write back into the systems the team already uses, with an approval where the consequence calls for one. [Approvals and control for AI agents](https://ortelian.com/notes/approvals-and-control-for-ai-agents/) covers how to place those approvals, and [AI agent governance](https://ortelian.com/ai-agent-governance/) covers the full set of controls.

## 5. It lives outside the work

An agent in a separate app asks people to change where they work. Most will not, especially when they are busy, which is exactly when the agent would help most.

How it shows up: usage drops after the first weeks. The agent is technically live and practically unused.

Fix: deliver results where the work already happens: the CRM record, the ticket, the shared inbox, the planning sheet. Ask the people doing the work where they would look for it, and build it there.

## 6. Nobody owns it after launch

Companies change under their agents. Sources move, fields get renamed, products and prices change, models get updated. An agent that was right in March can be quietly wrong in June. If the pilot team moved on after launch, nobody notices until a customer does.

How it shows up: a slow decline in quality, then one visible mistake, then the agent is switched off.

Fix: name an owner before go live. Keep the evaluations running on the work in use, run them again before every change to instructions, tools or models, and record which version produced each result. Decide up front whether the team runs it or a partner keeps running it.

## 7. The model was blamed for everything

When an agent fails, the first instinct is to try a bigger model or a longer prompt. Sometimes that helps. More often the failure traces back to one of the causes above: a missing standard, a missing source, a gap in the model of the business or a step that should have gone to a person. A more capable model on the same setup tends to produce more convincing versions of the same mistakes.

Fix: keep a record of every run, with the inputs, the sources used, each tool call and the output. When a result falls short, find the source or rule that caused it, change that and check the change against the same cases. [Why coding agents work and company agents don't](https://ortelian.com/notes/why-coding-agents-work-and-company-agents-dont/) explains why the environment matters more than the model.

## A quick check before go live

| Question | If the answer is no |
| --- | --- |
| Do we have real examples with agreed answers, including the messy ones? | Write them before building further |
| Can the agent tell which of two conflicting facts is current, and say where it came from? | Give it a model of the business with a time and a source on each fact |
| Was the process redesigned, not just automated? | Map the steps and decide what goes to software, agents and people |
| Does each agent have only the tools its job needs? | Narrow the tools rather than adding rules to the prompt |
| Do results land in the tools the team already uses? | Move the output into the CRM, ticket or inbox |
| Is there a named owner and are evaluations running on live work? | Name one before launch |

## How we approach it at Ortelian

Ortelian is a platform plus a forward deployed team. We start on site with the people who do the work, agree the standard on real examples, build the world model the agents work from and set the controls before anything goes live. Then we build in two-week cycles until the work meets the standard, and after launch your team runs it or we keep running it. The platform is hosted in the EU. For what that looks like in one function, see [AI agents for operations](https://ortelian.com/ai-agents-for-operations/), and for how an engagement runs, see [what an AI deployment partner does](https://ortelian.com/ai-deployment-partner/).

## Questions people ask

### Why do AI agent pilots work but production deployments fail?

A pilot runs on clean, chosen examples with the builders watching. Production brings the messy cases, changing data and nobody watching each run. Pilots that agree the standard on real examples, including the messy ones, and keep evaluating after launch are the ones that survive the move.

### Is a better model the fix for an unreliable agent?

Rarely on its own. Most production failures come from missing context, unclear standards or too much access. Fix those first, then compare models on the same evaluations so you can see whether a change actually helps.

### How do you know an AI agent is failing?

Run the agreed evaluations on the work in use and track quality, cost, time and human effort per completed piece of work. A rising rate of human corrections is usually the first sign, well before a visible mistake.

### What should an AI agent never do without a person?

Anything that cannot be undone, anything that leaves the company, such as an email to a customer, and anything that decides something about a person. [Approvals and control for AI agents](https://ortelian.com/notes/approvals-and-control-for-ai-agents/) covers where to put those checks.

### How long does it take to get an AI agent reliably into production?

It depends on the work and the sources it needs. In our deployments the audit takes about two weeks on site, and the build then runs in two-week cycles until the first job passes the evaluations agreed at the start.