The review queue: keeping humans in the loop

Every automation we ship has the same shape at its core. Software handles the cases it is confident about, and everything else lands in a queue for a human to clear. Not because the AI can’t guess — because some mistakes are too expensive to make silently.

This post is the pattern in full, with the actual code shapes we use. It is not clever. That is the point.

The shape of the thing

Three parts, always the same:

1. A classifier that scores each item and decides: act, or escalate.
2. A queue that holds escalated items with everything a reviewer needs on one screen.
3. A feedback loop — every human decision becomes a labelled example for the next tuning pass.

The threshold between “act” and “escalate” is a business decision dressed up as an engineering one. It comes straight from the cost of a mistake:

The classifier

The scoring call is deliberately dull. One function, one structured answer, no streaming, no agents:

from anthropic import Anthropic

client = Anthropic()

def classify(invoice_text: str, po_candidates: list[dict]) -> dict:
    """Match an invoice to a PO, or admit we can't."""
    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=512,
        system=SYSTEM_PROMPT,  # the rules a human would state, verbatim
        messages=[{
            "role": "user",
            "content": render_prompt(invoice_text, po_candidates),
        }],
    )
    result = parse_json(response.content[0].text)
    # result: {"po_number": "PO-4471", "confidence": 0.94, "reason": "..."}
    return result

Two details that matter more than the model choice:

• The system prompt is the rules from the scoping call, written down. When the client says “we never auto-post invoices over €10,000”, that sentence goes in verbatim.
• The model must be allowed to say I don’t know. A forced choice with no escape hatch is how confident nonsense ends up in your accounting system.

The queue

The queue is a table. Resist making it more than a table.

CREATE TABLE review_queue (
    id           bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    item_type    text        NOT NULL,   -- 'invoice', 'ticket', ...
    payload      jsonb       NOT NULL,   -- everything the reviewer sees
    suggestion   jsonb       NOT NULL,   -- what the model would have done
    confidence   numeric     NOT NULL,
    status       text        NOT NULL DEFAULT 'pending',
    decided_by   text,
    decided_at   timestamptz,
    created_at   timestamptz NOT NULL DEFAULT now()
);

The reviewer sees the item, the model’s suggestion, and its reasoning — and makes one of two moves: approve the suggestion or correct it. One keystroke each. If clearing the queue takes more than a few seconds per item, people stop clearing it, and the whole system quietly rots.

A review queue nobody clears is worse than no automation at all: the work still isn’t done, and now everyone believes it is.

The feedback loop

Every decision is a labelled example. Once a month, we look at where humans overrode the model and tune — usually the prompt, occasionally the threshold, rarely the model:

# tuning-run.yaml — inputs to a monthly review pass
window: 30d
export:
  - decisions_where: status = 'corrected'
  - decisions_where: confidence > 0.9 AND status = 'corrected'  # the scary ones
review:
  - prompt_rules      # do the written rules cover the misses?
  - threshold         # is 0.9 still right for this cost of error?

The metric we watch is not accuracy. It is escalation rate over time. A healthy system starts around 20–30% escalated and drifts down as the rules sharpen. If it drifts up, the process changed and nobody told the software — which is exactly what the queue is for catching.

Where this leaves you

Ninety percent automatic with a fast human queue beats one hundred percent automatic in every deployment we have done. The queue is not a compromise on the way to “full” automation — it is the finished state, and the reason the system is still trusted a year later.

If you’re weighing a process of your own, the five-question checklist is the place to start — question four is the one this whole post hangs off.