Approvals and control for AI agents
How to design human in the loop approvals, permissions and audit trails for AI agents, so they can act in your systems without losing control.
To keep control of AI agents, decide what each agent can do before deciding what it should ask. Give it only the tools its job needs, scoped and split into read and write. Put human approval on the steps that cannot be undone, that leave the company or that decide something about a person, and show the reviewer the evidence with the request. Record every run so any action can be traced to its inputs, sources and approver. Then keep someone responsible for pausing, changing and reviewing the agent.
This is the practical side of AI agent governance: how to design the approvals, permissions and records for one agent on one piece of work. It assumes the work has already been chosen and redesigned, as in how to deploy AI agents in an enterprise.
1. Start with what the agent can do, not what it should ask
Approvals are a second line of control. The first is the set of tools an agent has. An instruction in a prompt can be misread or overridden by text the agent reads. A tool the agent does not have cannot be misused. The OWASP project lists this risk as excessive agency in its Top 10 for LLM applications.
For each agent, list the actions its job needs and nothing more. Scope each one: which records, which fields, which systems. Separate reading from writing. An agent that prepares meeting notes needs to read accounts and past calls. It does not need to change deal stages or send email.
A useful test: if this agent does the worst thing its tools allow, what happens? If that is not acceptable, narrow the tools before adding approvals.
2. Sort every action by consequence
Human in the loop works when it is placed by consequence, not spread across every step. Ask three questions of each action the agent can take:
Can it be undone easily? Does it leave the company? Does it decide something about a person?
| Action type | Example | Control |
|---|---|---|
| Read only | Look up an account, summarise past calls | Runs on its own, recorded |
| Internal and reversible | Update a CRM field, tag a ticket, draft a reply | Runs on its own, recorded, easy to roll back |
| Internal with wider effect | Change a plan, reassign work, bulk update records | Approval by the owner of that work |
| Leaves the company | Email a customer, send a quote, post publicly | Approval before sending |
| Hard to undo or about a person | Payments, contract changes, anything about hiring or performance | Approval by a named person, often two |
Most of an agent's work should fall in the first two rows. If most of it falls in the last three, the work may need redesigning so the agent prepares and a person decides. Redesign the work before you automate it has examples.
3. Design the approval request, not just the approval
An approval is only as good as what the reviewer sees. A request that says "Send this email? Yes or no" trains people to click yes. A good request shows:
The proposed action in full, exactly as it will happen. The reason the agent proposes it. The sources it used, with links to the records. What changed since the last time, if anything. What happens if the reviewer says no or does nothing.
Aim for a request someone can check in a minute. Let reviewers edit before approving, and record the edit, because edits are the most useful signal for improving the agent. Put the request where the reviewer already works, such as the CRM record, the shared inbox or the team channel, not in a separate app they have to remember to open.
4. Decide who approves, and what happens when nobody does
Name an approver for each kind of action, by role rather than by person, with a backup. The approver should be someone who can judge the action, usually the person who did that work before the agent.
Decide what silence means. For most actions the safe default is that nothing happens and the request expires, with a note to the owner. Never let an action go ahead because nobody answered. For time sensitive work, escalate to the backup instead of auto approving.
5. Watch for rubber stamping
Approvals stop working when people stop reading them. Signs to watch: approval times measured in seconds, nearly every request approved unchanged, and reviewers approving batches at once.
When that happens, ask whether the approval is still needed. If the agent has been right on that kind of action for a long time, and the action can be undone, move it to running on its own with a record and a periodic sample review. If the approval is needed, make the request shorter and clearer, and send fewer of them. Fewer, better approvals keep people paying attention to the ones that matter.
6. Record every run
For every run, keep the inputs, the sources the agent used, each tool call, the output, the version of the agent or workflow, and who approved or edited what and when. That is what lets you answer the question that always comes up after something goes wrong: what was the agent doing when it did that?
Records also make fixes safe. When a result falls short, you can find the source or rule that caused it, change it and check the change against the same cases. A model of the business with a time and a source on every fact, such as a world model for knowledge work, makes this much easier, because the record can point to the exact facts the agent relied on. In the EU, deployers of high risk AI systems must keep the logs those systems generate, as our note on what the EU AI Act asks of B2B companies sets out.
7. Keep a way to pause, change and roll back
Someone must be able to pause an agent at once, without a deploy. Treat changes to an agent's instructions, tools or workflow like changes to code: show the change in full, run the evaluations on real examples and only then put it live. Record which version produced each result so you can roll back if a change makes things worse. More on evaluations in agree the standard before you build.
A worked example: operations
Take invoice matching in a finance team. The agent reads invoices, purchase orders and delivery notes, and matches them. Matches that agree on supplier, amount and quantity move on without a person, and each is recorded with the documents behind it. Mismatches go to the person who handled them before, with the three documents side by side and the difference highlighted. Payments stay with people. The agent cannot pay anything, because it has no tool to do so. See AI agents for operations for more examples like this.
Common mistakes
| Mistake | Do this instead |
|---|---|
| Approving every step to be safe | Place approvals by consequence and let reversible internal steps run |
| Controlling the agent through the prompt | Limit its tools and scope them |
| Giving the agent a person's login | Give it its own scoped tools and identity |
| Approval requests without evidence | Show the action, the reason and the sources |
| Auto approving when nobody answers | Let the request expire or escalate to a backup |
| Logging only the final output | Record inputs, sources, tool calls, versions and approvers |
How we set this up at Ortelian
Ortelian is a platform plus a forward deployed team. Where an approval belongs depends on how your work actually runs, so we set the tools, approvals and records with your team on site during the audit and the build, before anything goes live. On the platform, permissions are set per agent and per workflow, every run is recorded with its sources, and anything that leaves the company waits for a person. The platform is hosted in the EU and agents use models with zero data retention. For how a full engagement runs, see what an AI deployment partner does. For why agents go wrong without these controls, see why AI agents fail in production.
Questions people ask
What does human in the loop mean for AI agents?
A person reviews and approves certain actions before the agent carries them out. It works best on actions that cannot be undone, that leave the company or that decide something about a person, with the evidence shown in the request.
Does every AI agent action need approval?
No. Requiring approval for everything slows the work and teaches people to approve without reading. Reading and reversible internal changes can usually run on their own, as long as they are recorded and can be rolled back.
What is the difference between permissions and approvals?
Permissions decide what an agent is able to do at all. Approvals decide which of those actions need a person first. Set permissions narrowly, then use approvals on the consequential actions that remain.
What should an AI agent audit trail contain?
The inputs, the sources the agent used, each tool call, the output, the version of the agent or workflow, and who approved or edited what and when. With that, any action can be traced back and explained.
How do you stop an AI agent quickly if something goes wrong?
Keep a pause control that works immediately without a deploy, and name who can use it. Recorded versions then let you roll back to the last setup that passed the evaluations.
