AI Vendor Risk Assessment

How the scoring works

Score = points earned ÷ points possible. The safe answer earns the question's weight, Partly earns half, and the risky answer or no answer earns nothing. 80 or more is Low risk, 60 to 79 Medium, 40 to 59 High and below 40 Critical; any critical red flag makes the vendor at least High.

RefQuestionSafe answerWeightRisky answer is
U3Is our data used to train or improve any AI model, yours or a provider's?No3Critical
U4Can we opt out of any AI processing of our data, and will you confirm it in writing?Yes2Red flag
U5Are consequential AI outputs reviewed by a person before they affect us?Yes2Red flag
A2Does a named person approve agent actions that affect our data or systems (sending, changing, deleting, paying) before they take effect?Yes3Critical
A3Can you stop an agent immediately and revoke its access (a kill switch)? Who can do it, and how fast?Yes3Critical
A4Is every agent action on our data logged with the agent's identity, and will you give us those logs on request?Yes3Critical
A5Does each agent have its own identity and only the permissions its task needs, reviewed at least quarterly?Yes2Red flag
A8Are agents tested before release, and after changes, for prompt injection, data leakage and actions outside their purpose?Yes2Red flag
A9Will you tell us before introducing agents that act on our data, or widening what they can do?Yes2Red flag
M4Do your contracts with AI providers prohibit them from training on our data?Yes3Critical
M5Will you tell us before changing the model or provider used for work involving our data?Yes1Gap
G1Do you have a written AI use policy for staff? Please share it.Yes2Red flag
G2Do you keep an inventory of the AI systems and agents used in your service?Yes2Red flag
G3Is a named person accountable for AI risk in your organisation? Who?Yes1Gap
G5Do you train staff on safe AI use at least once a year and keep records?Yes1Gap

Written-answer questions (11 of them) earn their weight when answered and are listed under "Answers to read". 15 of the scored questions come with a contract term to ask for when the answer isn't the safe one.

Assess a vendor