Where we put the human in a document agent

Teams usually pick one of two extremes. Either a person reviews every output, and the agent saves almost no time, or nobody reviews anything, and the first mistake reaches a supplier or a customer. Neither lasts. We design the review points deliberately, per action, from the start of the prototype.

“What is the worst thing this agent could do?”

We list what the agent can do and rank each action by consequence: read, draft, write to a system, send externally, trigger a payment. A wrong summary is a small problem. A wrong message to a carrier or a wrong purchase order is a larger one. The ranking tells us where checks must be strict and where they can be light.

“Which outputs can we check without a person?”

Deterministic checks come first because they are cheap and do not drift. We validate against a schema, confirm that every cited passage exists, check permissions on the data used, scan for personal data and apply the business rules the client already has. Many bad outputs are stopped here.

“When should it ask?”

The agent hands over when specific triggers fire: low groundedness, conflicting sources, a high-value amount, a new supplier, or a request outside its scope. The handoff carries the draft, the sources and the reason, so the reviewer decides in a minute and does not start from zero.

“Is the reviewer really reviewing?”

Human review decays. After a few hundred correct outputs, people click approve. We track the override rate and the time spent per review, and we audit a random sample of approved items. If nobody ever overrides anything, either the agent is very good or the review has stopped working, and we want to know which.

“How does autonomy grow?”

Autonomy is earned per class of action. When an action type has a clean record against the evaluation set and in production, we reduce its review from every item to a sample, with the trail intact. The operators who own the workflow make that call, based on the evidence.