How to Assess an AI Vendor Before You Let It Touch Your Data

Direct answer: Assess an AI vendor the way you assess any processor, then add four questions specific to models: does our data train anything, how long is it retained and where, which model providers sit behind you as subprocessors, and what happens when the model changes. Those four are where standard vendor questionnaires come up short.

Start with the normal assessment

An AI vendor is still a vendor. You want their security posture, a SOC 2 or ISO 27001 report if they have one, their subprocessor list, breach notification terms, data residency and deletion commitments. If they cannot produce any assurance report and they will handle customer data, that is the answer before you reach the AI questions.

The four AI-specific questions

Does our data train your models? Get it in the contract, not the marketing page. Business and enterprise tiers of the major providers exclude training by default; consumer tiers historically did not. The tier matters more than the brand.

What is retained, for how long, and where? Many providers retain inputs for abuse monitoring even when they do not train on them, often for 30 days, sometimes in a different jurisdiction from the one you contracted for. That retention is a disclosure you may owe your own customers.

Who is behind you? Most AI products are a wrapper over one or more foundation models. Those providers are your subprocessors in practice. Ask which ones, in which regions, and whether they can change without telling you.

What happens when the model changes? A model version can change behaviour without any change to the API. If you rely on the output for anything consequential, you want notice of material changes and a way to pin or test a version.

Match the depth to the exposure

A tool summarising public marketing copy is not the same risk as one processing customer support transcripts containing personal information, and neither is the same as an agent with write access to your systems. Assess in proportion. A single questionnaire applied uniformly wastes effort on the harmless and under-inspects the dangerous.

The practical test: could this vendor's failure expose customer data, break a contractual commitment, or take an action in our environment? Any yes moves it up a tier.

When they will not answer

Some AI vendors, particularly early ones, cannot answer these questions because nobody has asked before. That is not automatically disqualifying, but it changes what you should do: restrict what data reaches them, avoid personal information entirely, get the commitments you can into the contract, and set a review date.

A vendor that refuses to answer, as opposed to one that is still working it out, is a different matter.

What your own buyers will ask

Enterprise security reviews now routinely ask whether you use AI to process customer data, which providers, whether customer data trains a model, and how you assess AI vendors. Your answer to that last one is your process. Having one written down, applied and evidenced is the difference between answering in a paragraph and stalling a deal.

Our AI Vendor Risk Assessment runs this across your stack and produces the register buyers ask for, the shadow AI risk checker is free and finds the tools you did not know about, and whether staff can paste company data into AI tools covers the policy side.

The AI vendor you did not buy

The hardest vendors to assess are the ones nobody procured. A CRM ships an AI summariser in a minor release. A support desk adds suggested replies. A meeting tool starts producing transcripts and action items. None of these went through a vendor review because the vendor was already approved, and the contract you signed two years ago says nothing about model providers.

Treat a new AI capability in an existing tool as a change of processing, not a feature update. The questions are the same four, but the answers usually live in a supplementary AI addendum published quietly on the vendor's trust page rather than in your executed agreement. Read that addendum. It frequently names a foundation model provider your original subprocessor list never mentioned, and it sometimes reserves the right to change that provider on notice you will not receive because notice means updating a web page.

The practical control is an admin setting, not a contract. Most enterprise tools ship AI features with a tenant-level toggle, defaulted on. Decide deliberately whether it stays on, record the decision and who made it, and put a calendar reminder on the vendor's AI terms page. That record is what you will show a buyer who asks how you govern AI in your supply chain.

Writing the three tiers down so someone else can apply them

Proportionate assessment only works if the tiers exist on paper and anyone in procurement can place a tool without asking you. Tier one is a tool that only ever sees public or non-sensitive internal content, with no write access and no personal information. A short record of what it is, who owns it, and what data it may touch is sufficient. Tier two is anything processing personal information, customer content, or confidential material, which earns the full assessment: assurance report, subprocessor list, retention and training terms, deletion commitments, residency, breach notification, and a data processing agreement that survives your own customer commitments.

Tier three is anything that can act. Credentials, tool calls, write access to a system of record, or the ability to send communications on your behalf. That is a privileged integration and belongs under change control, with scoped permissions and an audit trail, which we cover separately in agentic AI vendor risk. The tiering decision itself should be recorded, because the interesting question a year from now is not what you concluded but why a tool that now handles customer data was assessed as tier one.

Clauses worth arguing for, and the ones to let go

Sales engineers will agree to almost anything verbally. Get four things into the paper. First, an explicit statement that customer data is not used to train, fine-tune, or otherwise improve models, covering both the vendor and its model providers. The distinction matters: a vendor can honestly say it does not train on your data while sitting on a provider tier that does. Second, retention stated as a number with a deletion mechanism, including whether deletion covers prompt and completion logs held for abuse monitoring. Third, a named subprocessor list with notice before additions and a right to object, because model providers change more often than hosting providers ever did. Fourth, notice of material model changes where you rely on output for anything consequential, along with the ability to pin or evaluate a version.

What to let go: bespoke audit rights over a vendor you cannot realistically audit, indemnities that exceed the contract value and will never be collected, and demands for a SOC 2 from a fifteen-person company that has been trading for eight months. If the tool matters enough to accept that risk, mitigate it by restricting the data rather than by winning a clause you will not enforce.

Prompt logs are a data store

The blind spot in most AI vendor reviews is inference logging. Prompts and completions get written somewhere, often to a logging pipeline the vendor treats as operational telemetry rather than customer data, and that pipeline may have different retention, different access controls, and occasionally a different region from the primary datastore you asked about.

Ask three questions about it. What is written to logs, in full or redacted. Who inside the vendor can read them, and is that access logged. How long they persist and does your deletion request reach them. A vendor that answers "thirty days for abuse monitoring, restricted to a named on-call group, excluded from deletion requests" has given you a real answer you can assess. A vendor that has never considered the question has told you something about its maturity.

The same applies internally. If your own product calls a model, your logs now contain customer content in a place your data map probably does not show, and your retention schedule probably does not cover. That is the finding we hit most often when we look at a company's own AI surface rather than its vendors.

Residency, PIPEDA and Law 25

Canadian organisations get an extra layer here. PIPEDA does not prohibit cross-border processing, but it does hold you accountable for information transferred to a third party for processing, which means the transfer needs comparable protection and reasonable transparency to the individuals concerned. If your support tool now routes ticket text through a model hosted in another jurisdiction, that is a transfer your privacy notice may not describe.

Quebec's Law 25 is stricter about the assessment step. Where personal information is communicated outside Quebec, you are expected to have conducted and documented a privacy impact assessment considering sensitivity, purpose, protections, and the legal regime of the receiving jurisdiction. An AI feature switched on by default in an existing tool is exactly the scenario that produces an undocumented transfer. Regional endpoints help, but check whether the guarantee covers inference only or inference plus logging, because those are frequently answered differently.

What to do when the honest answer is uncomfortable

Sometimes the assessment concludes that a tool your team already depends on cannot meet the bar. Ripping it out is rarely the proportionate response and is rarely what happens in practice. The workable path is to reduce what reaches it: redact or tokenise identifiers before they leave your systems, restrict the feature to a subset of users or a subset of data, turn off the parts that process customer content while keeping the parts that process internal content, and set a review date with named conditions for reassessment.

Document that as an accepted risk with an owner and an expiry, not as a silent exception. An accepted risk with a date is a defensible position in front of a buyer or a regulator. An undocumented one is the finding.

When not to buy an assessment from us

If you use three AI tools, none of them touch personal information, and none of them can take actions, you do not need a paid assessment. Write a one-page register with the four questions answered per tool, keep the vendors' AI terms links in it, and revisit it twice a year. Our free traztech Workspace will hold that register, and the free shadow AI risk checker will surface the tools your staff adopted without telling anyone, which is usually the more useful exercise at that size.

You also do not need us if you already have a functioning third-party risk process and a privacy lead. The AI questions bolt onto what you have; they are not a separate programme. Adding four questions and a tiering rule to an existing vendor review is an afternoon of work for someone who already owns that process.

Where paying makes sense is when a buyer has asked for your AI governance position in writing and you have nothing, when the surface has grown past what one person can hold in their head, or when your own product embeds models and the exposure is now yours to defend rather than a vendor's. That is a different piece of work from vendor review, and it is what our security assessment work covers. If you are unsure which side of that line you are on, tell us what you are running and we will say so.