Renzo DemartiniIn short
- Count the review minutes across every queue, including reading, correcting and recording the reason.
- Rehearse leave cover before you count it as capacity, and set a queue-age limit that triggers escalation.
- Give reviewers and managers the authority to hold and pause work, and let the owner approve the restart.
‘A person reviews every case’ sounds like control. It is also a cost of working time, and it has to be funded.
The owner’s question is whether the review can keep pace across finance, sales and operations, and who may stop the work.
Count the whole queue
Three queues need 300 minutes of review a week: five hours across the business
| Workflow | Cases a week | Minutes each | Total minutes |
|---|---|---|---|
| Supplier coding | 40 | 3 | 120 |
| Sales follow-up drafts | 30 | 2 | 60 |
| Dispatch exceptions | 20 | 6 | 120 |
| Total | 90 | – | 300 |
Sample planning figures. Every case in the pilot needs approval. Minutes include reading, correcting and recording the reason.
Illustrative figures, not client data.
Count review demand across functions before you approve more volume. Replace these sample timings with observations from your own pilot.
NIST notes that AI systems can ‘defer decision making to a human expert’ or be used as an additional opinion. Say which role each queue plays, and time the whole task, not the click.
Give reviewers the evidence
The reviewer sees the order and the stock record beside the proposal, and can hold it
| Field | Dispatch exception D20 |
|---|---|
| Proposed action | Change the delivery date |
| Evidence | Order D20 and stock record S20 |
| Reviewer decision | Hold: the stock record conflicts |
| Escalation owner | Operations manager |
Illustrative figures, not client data.
Show the original record beside the proposed action. Give the reviewer ways to reject, amend or ask for more information.
Use approved sample records in the rehearsal. The OAIC recommends ‘ensuring staff are equipped to check and account for inaccuracies in the tools’ outputs’. Run a privacy impact assessment before personal information enters a queue. The OAIC says ‘A Privacy Impact Assessment will assist your organisation to understand the impact’ of an AI product on privacy.
Rehearse leave cover
The normal roster fits, and a week with the main reviewer away is 60 minutes short
| Case | Minutes available | Minutes needed | Balance |
|---|---|---|---|
| Normal roster | 360 | 300 | 60 spare |
| Primary reviewer away | 240 | 300 | 60 short |
| Cover adds protected time | 300 | 300 | 0 spare |
Illustrative figures, not client data.
A name on a roster is not yet a trained substitute. Test access and competence before you count cover as capacity. Rehearse a hard case and record what the substitute cannot resolve.
Set a queue-age limit with each function. When it is reached, escalate. Never release unchecked work in silence.
Make stopping a real option
Each role can stop work, and only the owner approves the restart
| Role | Authority | Escalates to |
|---|---|---|
| Reviewer | Hold any assigned case | Function manager |
| Function manager | Pause that workflow | Owner |
| Owner | Approve a restart after reviewing the evidence | Decision recorded |
Illustrative figures, not client data.
NIST’s AI RMF 1.0 asks for oversight processes that are ‘defined, assessed, and documented’, and for mechanisms ‘to supersede, disengage, or deactivate AI systems that demonstrate performance or outcomes inconsistent with intended use’. It sets no staffing ratio. The Playbook for MAP 3.5 says to ‘Establish practices for AI systems’ oversight in accordance with policies developed in GOVERN-1.’
Let reviewers challenge the proposed answer. Record the reason, the manager and the restart condition. Keep unresolved cases visible when responsibility changes.
Approve capacity before expansion
Volume holds at 90 cases until trained cover is in place and the queue-age test passes
| Item | Decision |
|---|---|
| Current volume | Hold at 90 cases a week |
| Expansion | On hold until the leave rehearsal passes |
| Restart evidence | Trained cover and a passed queue-age test |
| Queue-age limit (sample) | 1 working day |
| Absence rehearsal (sample) | Oldest case waited 2 working days: failed |
| Review date | 10 November 2026 |
Illustrative figures, not client data.
Review takes 5 hours a week, and no cash saving is claimed
Hours
5 hours
Weekly review time needed
Hours, shown apart from dollars.
Dollars
A$0
Cash saving claimed
Record software and running costs before any saving is claimed.
Illustrative figures, not client data.
Approve expansion only when a reviewer can explain, stop and escalate the work. Keep the capacity decision beside the operating plan.
What to do first
One measured week shows whether review capacity matches the promise
- 01Measure one week
Time every case across the three queues, from reading to recording the reason.
- 02Check the evidence on the card
The reviewer must see the original record beside the proposed action.
- 03Rehearse an absence
Run the queue with the main reviewer away and note what cover cannot resolve.
- 04Write the stop rights
Who can hold a case, who can pause a workflow and who approves the restart.
- 05Decide on volume
Approve more only when the reviewer can explain, stop and escalate the work.
Start with the queue that has the most cases.
Common questions
What does human in the loop mean?
A person takes part at defined points, such as checking, correcting or approving before work continues. NIST asks for human oversight processes that are defined and documented.
Does every AI system need a human reviewer?
No. NIST says some AI systems ‘may not require human oversight’ while ‘other systems may specifically require human oversight’. Decide by the cost of a wrong answer.
What happens when the reviewer is away?
Capacity falls and queues age. Plan trained cover, a queue-age limit and an escalation route before you increase volume.
Chartered Accountant (CA ANZ) and finance executive, based in Canberra. Practical AI and automation for SMEs, and adviser to owners, CEOs and boards on integration and finance.