The checklist, by domain
| Domain | The questions | The answer that should worry you |
|---|---|---|
| Explainability | Show a case-level explanation for one real decision. Who is it written for — a data scientist or a case handler? | Accuracy metrics instead of a screen |
| Your data | Does our data train other customers' models? Residency per market? Deletion and model disposition at exit? | "Anonymised, so it's fine" without the mechanism |
| Protection signals | How are RG-class signals segregated? Can you demonstrate they never enter marketing features or cross-customer training? | The question surprises them |
| Monitoring edges | Input/output logging in our infrastructure? Score distributions per our segments? Calibration exports? | "Our dashboard covers monitoring" |
| Change control | Are model versions announced before behaviour changes? Can we pin a version? Rollback? | "Continuous improvement" as a shipping policy |
| Failure modes | The documented fallback when the service is down or wrong. A real incident story and what changed. | Perfection; graceful degradation without specifics |
| Audit support | What do you provide when our regulator asks about a decision your product influenced? | "That's never come up" |
Contract clauses that do the work
- Decision-data ownership. Inputs, outputs and explanations for decisions about your players are your records, exportable, retained per your schedules — because the audit trail obligation is yours whether or not the model is.
- Training opt-out with teeth. Not a checkbox — a warranty, with the protection-class carve-out absolute.
- Version pinning and notice. Behaviour changes are releases to you, with a validation window before they route real decisions — your champion-challenger discipline needs the hook.
- Exit rights. Data return, model disposition, and a transition period long enough that leaving is a decision rather than a hostage negotiation.
- Regulatory cooperation. A defined SLA for producing evidence when a regulator asks — the clause nobody uses until the one day it is the only clause that matters.
Running the diligence
Do it with the model-risk owner in the room, against a real case from your own operation if the vendor will run one. Score the answers against the table, write the findings into the model register as the vendor-model's first entry, and treat unresolved worries as contract clauses rather than hopes. The pattern across every domain above is the same one this whole cluster teaches: the artefact, not the assurance.
Continue reading: AI governance — the register the vendor's model joins. The Turbo Stars platform — our own answers to this page's questions are part of any evaluation.