Human-in-the-Loop AI: Approval Patterns for Real Work

What counts as an approval gate, and what only looks like one

The phrase gets stretched to cover everything from a person glancing at a dashboard to a mandatory countersignature. For designing a process, only one meaning is worth using: at a named point in the run, the process stops, shows a person the exact action it is about to take, and does nothing until that person decides.

Four properties have to hold. Remove any one of them and you have monitoring rather than control.

• A specific pending action. Not “the system is working on refunds” but “refund €420 to this customer, on this card, for this period”. A reviewer can only approve something concrete; vagueness here produces approvals nobody remembers giving.
• A person with the authority to decline. A named role, not a shared mailbox, and someone for whom saying no is a normal outcome rather than an escalation.
• An action that is genuinely blocked. The step does not happen until the decision is made. A message telling you what has already been sent is a log entry, however prompt it is.
• A recorded decision. Who decided, what they were shown, and what happened next. Without the record, the approval is real when it happens and worthless three months later, which is exactly when you will need it.

It follows that a daily digest, a confidence score with no person attached, and a Slack channel that fills up faster than anyone reads are not gates. They are useful, but they do not stop anything.

Gates are also expensive. Each one adds latency and consumes review attention you will need for the gate that actually matters. The most common design mistake is not too few approvals but too many, spread evenly across a process. That arrangement does not fail loudly. It fails by being ignored.

Six approval patterns, and when each one fits

These six cover almost every approval you will need to design. Most real processes use two or three of them at different steps, which is the point: the pattern belongs to the action, not to the process.

Each pattern has a characteristic way of going wrong, and knowing it is more useful than knowing the definition.

• Approve before send. The strongest control and the most expensive one. Its failure mode is queue latency followed by rubber-stamping: as volume rises, the reviewer approves in batches without opening the detail. Keep the blocking gates in a single run to the low single digits, so each one still means something.
• Review after with undo. Only honest if the undo restores the world, not just your database. A refund can be clawed back; an email cannot be unsent, and a cancelled order that already triggered a shipment is a phone call rather than a rollback. Test the undo path before relying on it, and treat anything the customer has already seen as irreversible.
• Sampling and spot-checks. Borrowed from quality control, and honest about its limits: sampling measures whether the process is still correct and protects no individual item. Operations engineering accepts the same trade, with Google's site reliability guidance arguing that 100% is the wrong reliability target for basically everything. The same holds for review coverage. Use sampling to catch drift on low-stakes work, never as the only control on an irreversible action that matters.
• Threshold escalation. Only as reliable as the field it reads. If the amount can be wrong because of currency, tax, or a batch that aggregates several items, the threshold silently mis-tiers the case and nobody is asked. Two rules make it durable: state the limit in the written procedure rather than in the code alone, and add an aggregation rule so five requests just under the limit from the same counterparty still escalate.
• Dual approval, or four eyes. Public-sector internal-control standards describe it as segregation of duties: dividing key duties such as authorising, processing and recording, payment, review, and custody of assets among different people, so no single person controls every part of a transaction. Automation does not change that logic; it just means one of the parties is software. Expensive per item, so reserve it for payments, changes to bank or payee records, and credits above a limit.
• Exception-only review. The cheapest sustainable pattern and where most processes should end up. It requires you to write the exception rules down, and it requires one discipline: anything that does not match a known rule counts as an exception. The process fails towards the human, never past them.

Match the pattern to the risk tier, not to your confidence

Tier the action by what a mistake costs, not by how well the system has been performing lately. Confidence moves weekly. Consequence is a property of the action itself and does not move at all. Three questions settle the tier: can it be undone completely, who is affected, and what does the worst plausible error cost.

Tier 3 is where the law arrives before your judgment does. Under Article 22 GDPR, a person has the right not to be subject to a decision based solely on automated processing that produces legal or similarly significant effects, and where such processing is permitted, the right to obtain human intervention, to express a view and to contest the decision. That intervention has to exist as a real step, performed by a real person with the authority to change the outcome.

The tier attaches to the action, not to the process. One refund process can contain a Tier 0 note, a Tier 1 reply, a Tier 2 payment and, if it can also close an account, a Tier 3 decision. Tiering the whole process gives you either an automation that cannot move or a gate on the wrong step.

Temporary learning gates and permanent gates are different objects

Two approvals can look identical on screen and mean completely different things. One exists because the system has not yet shown it can do the step; it is evidence-limited and should disappear. The other exists because somebody has to be answerable for the decision, which no level of accuracy discharges; it is liability-limited and should not.

One question separates them: what would have to become true for this gate to come off? If the answer is “the system gets reliably good at it”, the gate is temporary. If the answer is “someone else would have to be willing to be answerable”, it is permanent, and improving the model is beside the point.

The frequent mistake is treating every gate as temporary: as accuracy improves, approvals get stripped out in a batch and the ones that were never about accuracy go with them. The opposite mistake produces a process where nothing runs without a signature, a reviewer approving hundreds of items a day, and no real control anywhere. Sorting the gates into these two columns before you automate costs an afternoon and prevents both.

Why approvals decay: automation bias and finite attention

Approval gates rarely fail because someone approved one wrong thing. They fail because the gate stopped being read. Regulators have named the mechanism: Article 14 of the EU AI Act requires that high-risk systems be designed so natural persons can effectively oversee them, and specifically that the measures address automation bias, the tendency to over-rely on output produced by the system. It also requires that overseers have the competence, training and authority to intervene. Authority is the part most internal designs skip: a reviewer who cannot decline without a difficult conversation is not overseeing anything.

Treat the following as design requirements for the approval itself, not as training you deliver once.

• Show the action, not a summary of it. The reviewer should see what will actually be sent, paid or changed, next to the source it was derived from, close enough together to compare without clicking away.
• Make declining as cheap as approving. One click plus a reason from a short list. If declining means writing an email to whoever built the automation, approval becomes the path of least resistance and the gate is decorative within a month.
• Measure the change rate, not the approval count. The number that tells you whether a gate is alive is how often a review changes the outcome: declines plus edits, as a share of items reviewed. A gate at zero for a sustained period is either ready to be removed or no longer being read, and those two need opposite responses.
• Bound the review volume deliberately. Decide how many items one reviewer can genuinely read in a working day and design the gates so that number is not exceeded. Any design that needs a person to inspect every output forever will fail at whatever volume outruns their attention, and it will fail quietly.
• Capture the reason for every decline. A recorded reason is the raw material for improving the procedure. Without it, the same correction gets made every week and the process never learns it. Gates also accumulate, so review the list on a schedule: for each gate, is it temporary or permanent, and what has its change rate been?

One process, three gate placements: a worked example

The trade-off is easier to see on one process than in the abstract. The figures below are invented for this example; the shape of the trade-off transfers, the numbers do not.

Where the gates come from: learn, approve, codify, run

Approvals are usually retrofitted onto an automation that already works, which is why they are the first thing dropped when it gets busy. The alternative is to build the process around them from the first run. That is the sequence all-agents uses, and it holds whether or not you use our product.

1. 01Learn — run it together, approve everything irreversibleAI judgment
   The first runs execute with a model's flexibility while the operator watches, and every irreversible action stops for approval without exception, because you do not yet know which steps are the risky ones. Each accepted run is captured as a concrete example and the written procedure is sharpened afterwards, so the next run starts from what the last one taught.
2. 02Approve — the owner decides the process is readyHuman approval
   No fixed run count releases a process. The person who owns it decides, based on which branches have actually been seen and how complete the written procedure now looks. A rare branch that has never occurred is a reason to wait, not a reason to hope.
3. 03Codify — the walk-through, where permanent gates get pinnedHuman approval
   Before anything is automated, the process is walked through step by step with real example data. Steps that follow rules become deterministic code, steps needing genuine judgment stay with the model, and steps where a person must remain answerable are pinned as permanent gates. This is where a temporary approval either disappears or becomes permanent, so make it an explicit written decision rather than an omission.
4. 04Run — the codified process, with the pinned gates firingCode
   The trigger fires, the deterministic steps run, and the pinned gates pause the run for a person exactly where they were placed. The learning-phase approvals are gone; the pinned ones are not. Runs are recorded and replayable, which is what turns an approval into evidence you can produce later. The governance layer around this decides who holds which permissions and where an escalation goes.

If you take one thing from this into your own design, take the question from the codify step and ask it of every approval you have: what would have to be true for this to come off? The gates that survive it honestly are the ones worth building properly; the rest are queue latency wearing a badge. For a process where the answer is unambiguous in both directions, the accounts payable playbook shows the same patterns applied end to end.

Limitations and when not to use this

• This page is about approvals inside an operational process. Gates catch bad actions one at a time; they do not tell you whether the system is getting better or worse over months. That question needs measurement over time, which the reliability guide covers.
• Nothing here is legal advice. Article 22 GDPR and Article 14 of the EU AI Act are cited as primary sources and described narrowly. Whether a particular decision produces legal or similarly significant effects, whether your system falls in scope as high-risk, and what role you play are questions for your own counsel and depend on your jurisdiction.
• Approval patterns assume you have people with the time and the standing to review. In a team where the person raising the request is also the only possible approver, dual approval is theatre. Say so plainly and choose a different control, such as an after-the-fact review by someone outside the process or a hard cap that simply refuses the action.
• Do not automate a process whose exceptions you cannot describe. If every case needs a person because you cannot name the rule that would decide it, gates will not rescue it. The process is not ready, and writing the procedure down is the cheaper next step.

Sources

1. Art. 22 GDPR — Automated individual decision-making — GDPR Accessed 24 July 2026
2. Article 14: Human oversight — EU Artificial Intelligence Act Accessed 24 July 2026
3. Standards for Internal Control in the Federal Government (Green Book) — U.S. Government Accountability Office Accessed 24 July 2026
4. Embracing Risk — Google, Site Reliability Engineering Accessed 24 July 2026

Related reading

• SOP AutomationApprovals only survive automation if they are written into the procedure; this shows how to write one precise enough to hold them.
• AI Agent GovernanceThe layer around the gates: who holds which permissions, what gets logged, and how an escalation reaches a person.
• AI Agent ReliabilityGates catch bad actions one at a time; this covers the failure modes that make a process produce them in the first place.
• Accounts Payable AutomationThe same patterns applied to a process where thresholds and dual approval are non-negotiable.