The division of labour
| Task | Model | Human |
|---|---|---|
| Detect & rank | Scores every account and transaction stream; orders the queue by case likelihood | Sets the thresholds and owns the tuning review |
| Assemble | Gathers the evidence a case needs — history, network links, prior dispositions | Reads it with authority to ask for more |
| Hold | May create review states and provisional holds where written policy authorises | Confirms or releases within the policy's clock |
| Decide | — | Adverse actions, filings, exclusions: human, logged, with rationale |
| Learn | Retrains on adjudicated dispositions | Guards the label quality the retraining depends on |
The table is the governance artefact: written per model family, referenced from the register, and stable enough that a new case handler learns the rules on day one.
Three domains, three tunings
- AML. The model ranks; the analyst investigates; the MLRO decides. The valuable model behaviours are network detection (structures rules can't express) and false-positive suppression on the classic noisy rules — with the suppression itself reviewed, because a model quietly silencing a rule class is a finding in waiting.
- Fraud and abuse. The most symmetric error costs, hence the most automatable ranking — but consequences still route through the human gate, and the false-positive cost stays on the same dashboard as the catch rate.
- Responsible gambling. Sensitivity over precision, models augment but never veto rule-based markers, and outputs flow one way — into protection. The operating detail that matters: an RG score's job is earlier ordering of the intervention queue, not a new species of judgement; the markers page covers what feeds it.
The failure modes to design against
- Automation complacency. Reviewers approving the model's top-ranked cases without independent reading — detectable in disposition patterns, countered by sampled blind review.
- Threshold drift by budget. Raising the score cutoff because the team is busy is a risk decision wearing an ops costume; it belongs to the risk owner, in writing.
- Label rot. Retraining on dispositions from the complacent period teaches the model its own mistakes. Label quality is a supervised process, not a free by-product.
- Silent scope creep. A fraud model's scores leaking into CRM eligibility or an RG model's into bonus targeting — the boundary violations that audits find. The one-gate architecture exists to make them impossible, not just forbidden.
The evidence this produces
Run properly, the AI layer strengthens the audit position instead of complicating it: every case carries its ranking rationale, every decision its human and reasons, every model its validation history. That file — not the model's accuracy — is what a regulator reviews. The operators who struggle are not the ones using AI in monitoring; they are the ones who cannot show the division-of-labour table when asked.
Continue reading: Model risk — keeping these models honest over time. RG operations — the function the RG models serve.