How to Keep a Human in the Loop When You Automate Work

Review is a design decision, not a disclaimer

"A human reviews it" is the sentence that gets automated workflows approved. It is also, in a large share of implementations, untrue in practice: the human is present, the checkpoint exists, and it approves nearly everything because it was designed in a way that makes real scrutiny impossible. Whether oversight is genuine depends on specific choices about placement, information and volume — not on the fact that a person is nominally in the path.

Put the checkpoint before the irreversible step

The right place for review is immediately before whatever cannot be taken back: money leaving, a message reaching a customer, a record changing in a system of record, an account being closed. Everything upstream of that can run unattended, because a mistake there is still recoverable. Teams often place review too early — approving an intermediate draft while the consequential step runs automatically afterwards — which produces the ceremony of oversight without its substance.

Where the action is reversible, a different pattern is usually better: let it run, and invest in making it visible and easy to undo. An automation with a clear log and a one-click reversal beats one with an approval queue nobody has time to read.

Give the reviewer enough to decide, and no more

A reviewer needs three things: what is about to happen, why the system proposes it, and what would make this case wrong. If the interface shows only the proposed output, the only available judgement is whether it looks plausible — and plausible is exactly what a language model produces even when it is wrong. Show the input it worked from, the source it drew on, and any check that failed or came close to failing. Highlight what differs from the usual case, because the reviewer's real job is catching the exception, not re-reading the routine.

Volume is the thing that quietly breaks it

Approval quality falls sharply with quantity. A person asked to approve four hundred items a day will approve nearly all of them regardless of content, and the approval rate itself is a useful early-warning metric: a queue running at ninety-nine percent approval is either automatable or unreviewed, and it is worth finding out which.

The usual fix is to stop sending everything for review. Route by risk: auto-approve the cases that pass every check and fall inside normal bounds, and send only the uncertain, the unusual and the expensive to a person. This makes the queue small enough to read properly, which is the only way review means anything. Sampling has a role too — reviewing a random slice of the auto-approved traffic is how you find out whether the auto-approve rules are still right.

Measure the checkpoint, not just the workflow

Track how often reviewers reject or edit, how long they spend, and what happens to items they approved. If rejections are near zero, the checkpoint is not doing work. If reviewers are consistently editing the same thing, that is a defect report about the upstream step. And if errors are reaching customers through approved items, the review is not catching the failure mode you built it for, which is a design problem rather than a staffing one.

Plan for its removal

Human review should be an explicit stage with exit criteria, not a permanent tax. Decide up front what would justify removing it — a rejection rate below some level over a defined volume, no customer-visible errors in a period — and revisit it on a schedule. Some checkpoints will stay forever because the consequence is severe enough to warrant the cost, and that is a legitimate answer. What is not legitimate is a checkpoint kept because nobody ever re-examined it, which is expensive, slow, and provides the appearance of a safeguard while everyone involved has stopped reading.