The checklist, by domain

DomainThe questionsThe answer that should worry you
ExplainabilityShow a case-level explanation for one real decision. Who is it written for — a data scientist or a case handler?Accuracy metrics instead of a screen
Your dataDoes our data train other customers' models? Residency per market? Deletion and model disposition at exit?"Anonymised, so it's fine" without the mechanism
Protection signalsHow are RG-class signals segregated? Can you demonstrate they never enter marketing features or cross-customer training?The question surprises them
Monitoring edgesInput/output logging in our infrastructure? Score distributions per our segments? Calibration exports?"Our dashboard covers monitoring"
Change controlAre model versions announced before behaviour changes? Can we pin a version? Rollback?"Continuous improvement" as a shipping policy
Failure modesThe documented fallback when the service is down or wrong. A real incident story and what changed.Perfection; graceful degradation without specifics
Audit supportWhat do you provide when our regulator asks about a decision your product influenced?"That's never come up"

Contract clauses that do the work

  • Decision-data ownership. Inputs, outputs and explanations for decisions about your players are your records, exportable, retained per your schedules — because the audit trail obligation is yours whether or not the model is.
  • Training opt-out with teeth. Not a checkbox — a warranty, with the protection-class carve-out absolute.
  • Version pinning and notice. Behaviour changes are releases to you, with a validation window before they route real decisions — your champion-challenger discipline needs the hook.
  • Exit rights. Data return, model disposition, and a transition period long enough that leaving is a decision rather than a hostage negotiation.
  • Regulatory cooperation. A defined SLA for producing evidence when a regulator asks — the clause nobody uses until the one day it is the only clause that matters.

Running the diligence

Do it with the model-risk owner in the room, against a real case from your own operation if the vendor will run one. Score the answers against the table, write the findings into the model register as the vendor-model's first entry, and treat unresolved worries as contract clauses rather than hopes. The pattern across every domain above is the same one this whole cluster teaches: the artefact, not the assurance.

Continue reading: AI governance — the register the vendor's model joins. The Turbo Stars platform — our own answers to this page's questions are part of any evaluation.