The Model Flags. The Person Files.

Two of our solutions were recognised at the Global AI Awards this season. Neither is autonomous. In both, the model's authority ends before anything irreversible happens, and that constraint is not a compromise we worked around. It is the reason they were deployable at all.

That is worth saying plainly in a month when most award lists are dominated by the word agentic.

Where each one stops

The carrier invoice auditing system reads freight invoices against shipment records, contract terms, rate tables and order data. It identifies discrepancies — wrong dimensions, unauthorised surcharges, discounts that were never applied — ranks them by recoverable value, and assembles the supporting documentation for each claim.

Then it stops. A person reviews the recovery report and files the dispute. The system has never credited an account, never contacted a carrier, never determined in any binding sense that a charge was illegitimate. It found a disagreement between two records and showed its work.

The healthcare documentation validation service reads clinical documentation against the requirements for a successful claim and tells a medical assistant what is missing, while the visit is still fresh.

Then it stops. The assistant writes the sentence. The system does not draft clinical content, does not submit the claim, and does not evaluate anyone's judgement.

Both produced substantial results — over $400,000 recovered in the first case, a 35% reduction in outstanding receivables in the second. Neither produced them by acting. They produced them by surfacing something at a moment when a person could act on it cheaply.

Why the boundary is the accountability answer

Sooner or later, somebody will have to explain one specific decision. A regulator asking about one claim. A payer disputing one submission. An auditor asking why a particular charge was written off in March.

Keeping the irreversible act on the human side of the line means that question always has an answer, and the answer always has a name attached. The model's contribution is evidence — assembled, cited, reviewable. The decision is made by a person applying rules that can be stated in advance and reproduced afterwards.

This is also why both systems were allowed anywhere near production. In healthcare in particular, validating that a required element is present is a factual, checkable question. Opining on clinical content is not, and putting a model on the wrong side of that line converts an administrative tool into a regulated clinical one, with every obligation that follows.

A test you can run this week

Take any AI workflow you are considering and name the irreversible act. There is almost always exactly one, and it is usually obvious once you look: money moves, a claim is filed, a record posts, a customer is notified, an account is closed.

Then check what sits on that side of the line. If anything probabilistic does, you have an accountability gap, whatever your evaluation scores say. Fix it by moving the act, not by improving the model — the failure mode you are guarding against is not a bad answer, it is a confident wrong answer that nobody caught because nothing was designed to catch it.

The corollary is the useful part: once the irreversible act is protected, you can be far more relaxed about everything upstream. A wrong suggestion costs someone ten seconds. That is what makes it safe to let the model be genuinely useful.

The unfashionable conclusion

The two systems recognised this season are conservative by design. In neither case does the model have the last word.

That was not the concession that made them shippable. It was the decision that made them work.

Find the irreversible act in your workflow. We help teams draw the line between what a model may surface and what only a person may commit — so the accountability question always has an answer with a name attached. Book a design review.