Before you automate a process with AI, decide how wrong it is allowed to be. Not whether it will be wrong. It will. Three answers settle most of the argument: the error rate you accept, how fast a mistake is found, and what one missed case costs to undo. Most AI automation projects never write those down, and default instead to an unspoken standard nobody could meet: it must never be wrong.
How accurate does an AI system actually have to be?
When I ask that in a first meeting, the answer is almost always a version of "well, it has to be right." Fair enough. Everyone wants that. Push once and it moves: "obviously not a hundred percent, but you know." I do not know. Neither does the client, yet. That gap is where a surprising number of projects quietly go to die.
The trouble with "it has to be right" is that it is not a specification. It is a mood. You cannot test against a mood, you cannot sign it off, and it transfers the whole of the risk to whoever built the thing. Six months later somebody finds three wrong invoices in four hundred, and the question in the room is not whether 0.75% sits inside what was agreed, because nothing was agreed. The question is "so it does not work then."
I have watched a system with a genuinely good error rate get switched off for exactly that reason, while the manual process it replaced carried on making more mistakes than it did, invisibly, for years. Nobody had ever counted those. There was no number to defend, so there was no defence.
What do the Dutch dike norms teach you about AI automation?
There is a habit of mind in this country that would fix this in an afternoon, and it has nothing to do with technology.
The core of the Deltabeslissing Waterveiligheid is a number. The chance of dying in a flood, for everyone living behind the dikes, must be no greater than 1 in 100,000 per year, by 2050 at the latest. That is the basisbeschermingsniveau. The norms for the primary flood defences were written into the Waterwet and have applied since 1 January 2017, and the Waterwet itself merged into the Omgevingswet on 1 January 2024. The Delta Programme states the figure in plain language on its own site.
Read that as a business document rather than a water one. A country whose entire existence depends on those defences did not write "the dikes must hold." It wrote down a failure probability. Out loud, in law, with a date attached to it.
Then it went one better. The required level is stricter where the consequences would be worse: where large numbers of casualties are possible, where the economic damage would be substantial, where infrastructure of national importance sits behind the defence. The norm is not a property of the dike. It is a property of what is behind the dike.
That is the whole thing I want a client to carry into an AI project. You are not being reckless when you name an acceptable error rate. You are being reckless when you refuse to, because the rate is still whatever it is either way. The only difference is that nobody has decided whether it is tolerable and nobody is counting.
One further detail that I find bracing every time. The first national assessment round ran from 2017 to 2022, and its conclusion was that a majority of the norm sections, the stretches into which the defences are divided for assessment, do not yet meet the norms. Of the roughly nine million people who live behind a primary defence, around 80% had the basic protection level in 2020, expected to rise to around 82% by 2029. The country published that. It did not quietly move the number to match the dikes it happened to have. Knowing precisely where you fall short of your own standard is not the same as having no standard, and it is a great deal more useful.
What happens when the AI gets it wrong?
This is the question that ends roughly half the projects I get asked about, and I mean that as praise for the question.
Not all mistakes have the same shape. The distinction that does the work is not severity in the abstract. It is recoverability. So go through the process step by step and sort them.
- A draft that a person edits before it goes anywhere.
- A price suggestion that somebody approves.
- A classification that puts a document in the wrong queue and gets moved.
Those are inexpensive. The error surfaces within minutes, and the cost of the mistake is the cost of the correction. Nothing else happens.
Against that, consider a payment released, a message that reached a customer, a stock adjustment that propagated into three downstream systems, a line posted into a ledger you close monthly. Here the mistake does not stop where it was made. It becomes an input to something else, and by the time you find it you are not correcting one record, you are reconstructing a week.
I do not tell people to avoid automating the second category. I tell them the acceptable error rate for the second category is a different number from the first, and that the way to reach it is almost never a better model. It is a narrower process. Restrict what the system is allowed to touch. Cap the amount it can act on without a second pair of eyes. Teach it to refuse the unusual case rather than guess at it. A system that declines 12% of cases and is right about the other 88% is worth far more than one that has an opinion about everything.
We have written about the engineering side of this in a piece on the four-eyes principle, audit trails and idempotency, which is the machinery that makes a wrong step reversible. This essay is about the decision that comes before any of that machinery is worth building.
Who catches the mistake, and how fast?
Accuracy gets all the attention. Detection latency does most of the work.
A 3% error rate where every miss is visible the same day is a manageable business. A 1% error rate discovered in the annual audit is a bad quarter and an unpleasant conversation with your accountant. The second system is three times more accurate and enormously more dangerous, because those errors had eleven months in which to turn into other things.
So in any design review I want to know who sees the output, in what state, and on what day. If the answer is "the system posts it and we look at the totals at month end," then we are not really having an automation conversation. We are having a control conversation, and I would rather have that one first, because automation does not create the weakness there. It just makes it faster.
The least expensive instrument is nearly always a sample. Not a full check, which no team sustains past the third week. A fixed, small, boring sample: twenty cases a week, same day, same person, written down somewhere that is not an inbox. That gives you a real rate inside a month, and it gives you the thing almost no company has, which is a number to compare against next quarter.
What is an acceptable error rate for AI automation?
Whatever you settle on, do not settle on it against perfection. Settle on it against the process you actually have.
Nearly every company I sit with is comparing a proposed AI system against an imagined version of its current work, one in which nobody transposes a digit, nobody is off sick, nobody is doing this at 17:40 on a Friday in the week before a holiday. That version does not exist. The real one has an error rate. It is almost never zero and it is almost never known.
So the first job is to find your own number, or an honest bracket around it. Sometimes the record is already sitting there in a system you own: credit notes issued, orders reworked, corrections booked, complaints logged, invoices queried by customers. Sometimes there is nothing at all and you sample for four weeks. Either way you now hold the only comparison that means anything.
Then set the pass mark before you see any results. I have argued elsewhere about what to measure in an AI pilot, and that particular habit is the most protective one in the whole field. A threshold chosen after the fact is not a standard. It is a rationalisation, and everybody in the room can tell.
Be ready for the answer to be uncomfortable in the other direction, too. I have been in rooms where the measured human error rate came in well above what anyone in the business expected, and the honest conclusion was that the process needed repairing before anything was automated on top of it. That is not a failed project. That is a project that just saved somebody a great deal of money, which is one reason so many AI pilots stall on the way to daily use for reasons that have nothing to do with the model.
Does the AI Act deferral change any of this?
A word on the regulation, because it is being read as breathing room and it is not.
The Digital Omnibus moved the application date for stand-alone high-risk systems under Annex III from 2 August 2026 to 2 December 2027, and for high-risk systems embedded in regulated products from 2 August 2027 to 2 August 2028. The application dates of several other duties did not move: the Article 5 prohibitions, the Article 50 transparency obligations, and the obligations on providers of general-purpose AI. The Council and Parliament set out the simplification package themselves.
If you are waiting for December 2027 before you think about human oversight, you have misread what a deadline is for. The regulation is a floor, and it is a floor for a subset of systems. It has nothing whatsoever to say about the quote your system mispriced last Tuesday. Your customer will not be checking Annex III before deciding whether to call you again. We keep the obligations that actually reach a smaller company on our AI Act checklist for the mkb, and not one of them is a substitute for knowing your own error rate.
Should you automate a process you cannot check?
No. That is the shortest advice I give, and it is the one people most often want a second opinion on.
If there is no practical way to tell, inside a reasonable window, whether an output was right, then the system will accumulate errors at an unknown rate for an unknown period, and you will eventually learn about it from a customer. That is not an AI risk. That is a process that was already fragile, about to be run at speed.
The fix is not to abandon the idea. It is to build the check first. Sometimes the check turns out to be the whole project and the automation afterwards is nearly trivial. I have had engagements where the genuinely useful deliverable was a report nobody had, which made the current error rate visible for the first time, and the client decided the automation could wait a quarter. That is fine. That is the work doing exactly what work is supposed to do.
It is also why I am wary of a demo that impresses. A demo is a sample of one, chosen by the person showing it to you, on a case they already know it handles. The gap between that and a Tuesday in November is the entire job, which is why a great demo proves so little about implementation.
The three sentences I ask a client to write down
Before anything gets built, I want these three in writing, in the client's own words rather than mine.
- "This system is allowed to be wrong in roughly N cases out of a hundred."
- "When it is wrong, [name] will see it within [period], because [mechanism]."
- "One wrong case costs us [amount or effort], and it is [reversible or not reversible]."
If those three can be filled in, the project has a spine. Every disagreement afterwards is about numbers instead of feelings, which is a much better class of disagreement to have on a Thursday afternoon. If they cannot be filled in, you have just found the actual first task, and it does not involve building anything.
It took this country several decades and one catastrophic flood to arrive at the same principle for its dikes. Name the failure you accept. Weight it by what sits behind the defence. Publish where you fall short of it. Then go to work on the gap, in that order. It is not a comfortable way to open a project. It is the only way I know that closes one well.
If you want a second pair of eyes on which parts of a process are safe to hand over and which need the check built first, that is most of what our AI automation work actually consists of. Bring the error rate you already have, even if it is a guess. Especially if it is a guess.
