AI Vendor Risk Assessment Requirements: A Practical Checklist

An AI vendor risk assessment should cover data handling and training use, model provenance, security controls, regulatory fit (PIPEDA, Quebec Law 25), and contractual accountability before you sign anything. Below is the checklist we use with clients, in the order we actually work through it.

Why AI Tools Need a Different Vendor Review Than Regular SaaS

Most companies already run some kind of vendor risk process for new SaaS purchases. AI tools break that process in a few specific ways. The vendor may retrain on your data by default. The output may be non-deterministic, which complicates testing and audit trails. And the vendor's own AI supply chain (the foundation model it wraps, the sub-processors it uses for inference) is often opaque unless you ask directly. A standard vendor questionnaire misses all of this. That is the gap this checklist closes, and it is the same gap our AI vendor risk assessment service is built to close for clients in Toronto, Waterloo, Ottawa, and across Canada who need to move fast on AI adoption without inheriting a vendor's unmanaged risk.

Data Handling and Training Use Checklist

• Does the vendor train on your inputs? Look for an explicit opt-out or, better, a contractual guarantee that customer data is excluded from model training by default.
• Where is data processed and stored? Confirm the regions involved, and check whether any processing happens outside Canada in a way that affects your PIPEDA obligations or, for Quebec-based operations, Law 25 cross-border transfer requirements.
• How long is data retained, and can you force deletion? Many AI tools retain prompts and outputs for "quality improvement" well past the session. Get the retention window in writing.
• Is there a data processing agreement (DPA) available? If the vendor cannot produce one, that alone is a red flag for enterprise or regulated use.

Model Provenance and Sub-Processor Transparency

• What foundation model powers the tool? A vendor wrapping GPT, Claude, or an open-weight model inherits that model's own risk profile, including that provider's data handling terms.
• Who are the sub-processors? Inference infrastructure, vector databases, and logging pipelines are often run by third parties the vendor doesn't advertise. Ask for the sub-processor list directly.
• Can the vendor explain how outputs are generated? You don't need a research paper, but you do need enough to explain a decision to a customer or auditor if the tool is used in anything customer-facing.

Security Controls to Verify Before Adoption

• Does the vendor hold SOC 2 Type II or ISO 27001? Ask for the report itself, not just the badge on the website, and read the exceptions section.
• Is data encrypted in transit and at rest? Standard, but still worth confirming for smaller or newer AI vendors that skip it.
• What is the vendor's incident response and breach notification process? Get the notification timeline in writing, not just a mention in the privacy policy.
• Does the vendor support SSO and role-based access? Tools without enterprise access controls are harder to govern once adoption spreads past a pilot team.

Regulatory and Compliance Fit for Canadian Buyers

This is the step most vendor checklists skip entirely, and it's where a generic template fails Canadian companies. If you handle personal information, PIPEDA applies to how that information moves through any AI tool you adopt, full stop. If you operate in or serve Quebec, Law 25 adds stricter consent and cross-border transfer rules on top of that. And if your organization is preparing for SOC 2, an ungoverned AI tool in your stack can become an audit finding even if the tool itself never touches regulated data directly, because auditors increasingly ask what AI tools are in use across the business. Companies in Vancouver and Calgary selling into the US market face an added layer: US customers now routinely ask vendors to disclose their own AI tool usage during due diligence, so your AI vendor list becomes part of your sales cycle whether you planned for that or not.

Contractual and Accountability Checklist

• Liability for AI-generated errors or hallucinated output. Confirm who is accountable if the tool produces incorrect output that causes downstream harm.
• Termination and data export rights. Make sure you can get your data out cleanly, and that the vendor commits to deleting it on exit within a defined window.
• Change notification for model updates. AI vendors update underlying models frequently. You want the right to be notified before a material model change, since it can silently change behaviour you've already tested and approved.
• Right to audit or request evidence. Even a lightweight clause giving you the ability to request updated compliance evidence annually keeps the relationship honest over time.

Building an Internal AI Vendor Register

A checklist per tool is useful, but the real control is a running register: every AI tool in use across the company, who approved it, what data it touches, and when it was last reviewed. Without this, shadow AI adoption (a team signing up for a new tool without security involvement) becomes the norm rather than the exception. This is one of the fastest-growing gaps we see in compliance readiness work with Canadian tech companies right now, and it's usually not because teams are careless, it's because nobody owns the register until an auditor or a customer's security questionnaire asks for it.

How Often Should You Re-Assess an AI Vendor

Annually at minimum, and immediately after any of the following: a vendor changes its underlying model, a vendor discloses a security incident, the vendor's use case in your organization expands (a tool piloted for internal notes now touches customer data), or your own compliance obligations change, such as pursuing SOC 2 or expanding into a new market. Treat AI vendor risk as a living assessment, not a one-time gate at procurement.

If you're evaluating a new AI tool, or you suspect your organization already has more AI vendors in play than anyone has actually reviewed, get in touch through our contact page and we'll walk through what a proper assessment looks like for your stack.

Tier the tool before you run the checklist

Running every question above against every AI tool is how a vendor review process dies. Teams start with good intentions, spend four hours on a transcription tool used by two people, and by the fourth request the reviews are being rubber-stamped. Tier first, then apply the depth the tier warrants.

A workable three-tier split. Tier one is any tool where customer data, personal information, source code, or unreleased material enters the prompt, or where the output reaches a customer without a human reading it first, or where the tool can take actions in another system. These get the full review, a named business owner, a DPA, and an annual re-assessment. Tier two is internal-only use on non-sensitive company data, with a human reviewing every output. These get a short review: training use, retention, sub-processors, access control, and a data classification limit written into the approval. Tier three is anything with no company data entering it at all, which is rarer than people claim and worth verifying rather than assuming. Register it, set a boundary on what may be pasted into it, and move on.

Write the tier criteria down before the first request arrives, because deciding tiers case by case turns every review into a negotiation with the requesting team. And put a time budget against each tier. If a tier two review takes more than an hour, the process is too heavy and people will route around it, which produces exactly the shadow adoption you built the process to prevent.

Verify the claim, do not collect the claim

The single biggest weakness in AI vendor reviews is accepting a marketing sentence as an answer. "We do not train on customer data" appears on nearly every vendor site and is true under specific conditions that may not describe how your team uses the product. Ask which document that commitment lives in, and read it.

The verification worth doing on a tier one tool: find the clause in the terms or DPA that excludes your content from training, not the FAQ page. Confirm whether it applies to your plan, because consumer and free tiers of the same product frequently carry different terms from the enterprise tier, and a team that signed up with a corporate card is on the consumer terms. Confirm whether it applies through the interface your people actually use, since API access and the web application sometimes differ. Take a screenshot of the admin console setting that enforces it, with the date. Ask whether human reviewers can see content flagged for abuse or quality, because that exception is in most contracts and is a legitimate answer that your own privacy notice may need to reflect.

Do the same for retention. The published answer is often "thirty days for abuse monitoring", which sounds fine until you ask whether deletion on request purges those copies, whether it covers backups, and whether logs held by the inference sub-processor follow the same schedule. A vendor who answers those three cleanly is a vendor with a real program behind the claim.

The questions agentic tools require that the older checklist does not cover

Assessing an AI tool that only takes text in and gives text back is a solved problem. Tools that connect to your systems and act are a different exercise, and the volume of them arriving in company stacks right now is the fastest-moving risk we see.

What to ask when a tool requests access to your email, calendar, repositories, ticketing system, cloud account, or CRM. Which OAuth scopes does it request, and are any of them write scopes. Can it be granted read-only, and does the product still work that way. Whose identity does it act under, a shared service account nobody owns or an individual's credentials, because the second means every action it takes is attributed to a person who did not take it. Where are the tokens stored, and what happens to them when an employee leaves. Is there an audit log of actions taken by the tool, and can you export it. Can a user approve an action, or does the tool act autonomously once connected.

Then the question people skip: what happens when the content the tool processes contains instructions. An assistant that reads inbound email and can send messages, or reads a ticket and can update records, will eventually process text written by someone outside your company who wants it to do something. Ask the vendor directly what mitigations exist against instructions embedded in processed content, and treat a blank look as a finding. The reasonable compensating control at your end is limiting write scopes and requiring human confirmation for actions that leave the system or touch money, and that decision belongs in the approval record.

Browser extensions deserve their own paragraph because they are approved casually and they are among the most privileged things on an employee laptop. An extension with access to page content on all sites reads your admin consoles, your customer records, and your internal tools. Review them as tier one regardless of what the tool claims to do.

Model changes are a change management problem, not a procurement one

Your existing vendor process assumes the product you approved is the product you keep using. AI vendors update the underlying model, sometimes silently, and behaviour you tested in a pilot can shift without a release note. If the tool sits in a workflow anyone depends on, treat this as change management rather than a contract clause you file and forget.

Practical measures. Where the vendor offers version pinning, use it, and know the deprecation window so you are not migrating under a deadline set by someone else. Keep a small set of representative inputs and expected outputs for tier one tools, and re-run them after any announced change, which takes twenty minutes and catches the regressions that matter. Record in the register which model version was in use when you approved the tool. Ask for the deprecation policy in writing, since the notice period for retiring a model version is more operationally relevant than most of the security questions and almost nobody asks for it.

What to do when the tool is already in production and the review is late

Most of these assessments do not happen at procurement. They happen when someone notices a tool already embedded in a workflow, which changes the problem from approve or decline to reduce exposure without breaking the team's work. Removing it outright is occasionally correct and usually not, because the team adopted it for a reason and will find a worse substitute.

The order that works: find out what data has already gone into it, which usually means asking the team honestly rather than reading the terms. Move the account onto enterprise terms with a DPA if one exists, since this alone resolves a large share of the issues. Turn off training use and reduce retention in the admin console and screenshot the settings. Set a written data classification boundary saying what may and may not be entered, and make it specific, because "no sensitive data" means nothing while "no customer names, no health information, no credentials, no unreleased financials" is a rule people can follow. Add SSO if the plan supports it so offboarding actually removes access. Then set a review date and record the whole thing in the register.

If the vendor will not produce a DPA or answer basic questions and the tool holds tier one data, that is when removal is genuinely the right call, and having the register and the written boundary makes that conversation with the business owner much shorter.

What good and bad vendor answers look like

A useful habit is judging the quality of the answer rather than only its content. Bad answer to the sub-processor question: a link to a page listing three cloud providers and the phrase "and others as needed". Good answer: a dated list with the purpose of each, the regions, and a commitment to notify before adding one, with a window in which you can object.

Bad answer on incident notification: "we will notify affected customers as appropriate". Good answer: a stated timeframe from confirmation, a named contact channel, and a commitment to provide the information you need for your own regulatory notifications, which matters because your obligations under PIPEDA or Quebec Law 25 do not pause while a vendor decides what to disclose.

Bad answer on SOC 2: a badge image. Good answer: the report itself under NDA, with a current period and exceptions you can read. When you get the report, check whether the scope covers the product you are buying, because a vendor with three products often has a report covering one. That check takes two minutes and it is the highest-value two minutes in the whole review. The same discipline applies to your non-AI suppliers, and if your general process needs rebuilding rather than extending, our vendor risk management checklist covers the wider version.

When you should not buy an AI vendor risk assessment from us

A lot of the companies that ask about this need a process rather than an engagement, and we would rather say so.

When you have fewer than about ten AI tools and someone competent internally. The checklist above is the whole method. A security-minded ops or engineering lead can work through a tier one tool in a couple of hours. Paying an outside firm to do what a template and an afternoon accomplish is a poor trade, and we will tell you that on the call.

When the real problem is that nobody owns the register. If tools are being adopted without review, the fix is naming an owner and writing a two-page policy, not commissioning assessments. Assessments of a stack that keeps growing without a gate go stale within a quarter. Fix the gate first.

When the tool is genuinely low risk. A grammar checker on marketing copy, an image generator for internal slides, a meeting timer with a summarization feature that nobody feeds client material into. Register them, set the boundary, and spend the review budget on the three tools that touch customer data.

When you need contract negotiation rather than assessment. If the assessment is done and the blocker is liability language or an indemnity for AI-generated output, that is counsel's work. We will hand you the technical position to support it and stay out of the drafting.

When your own compliance obligations have not started yet. If nobody is auditing you and no customer has asked what AI you use, build the register and the policy now, cheaply, and buy an outside review when the first questionnaire actually arrives. You will pay less and know exactly what you are paying for. Keeping the register somewhere durable helps, which is why our Workspace is free to use whether or not you engage us.

Where an outside assessment does earn its cost: when the stack is large enough that nobody has visibility, when agentic tools with write access are already connected to production systems, when you are answering a customer's questionnaire about your own AI usage and the answers need to be defensible, or when you want the register and the process built once and then handed to your team to run. That is usually a short engagement followed by a light retainer rather than an open-ended project, and if it is being scoped as anything larger, ask why.