Agentic AI Vendor Risk: Assessing a Vendor That Acts, Not Just Answers

Direct answer: When a vendor's AI can take actions in your systems rather than only return text, assess it as you would a privileged integration, not a SaaS tool. The controlling questions become what it is allowed to do, what stops it doing more, and what record exists afterwards. Data handling still matters; blast radius matters more.

Why the usual assessment falls short

A conventional vendor review asks where data goes and who can see it. An agent introduces a second question: what can it change. It holds credentials, calls tools, and decides which to call based on text it was given, some of which may come from outside your organisation.

That combination, non-deterministic decisions plus real permissions, is not something standard vendor questionnaires were written for.

The questions that matter

What permissions does it hold, and under whose identity? Its own service account or an impersonated user? Scoped to a workspace or granted broadly because that was easier to configure? This is the single most useful question and it is frequently answered with "admin, for now".

What can it do without a human? Draw the line explicitly. Reading is one tier. Writing is another. Sending external communications, moving money or changing access is a third that should generally require approval.

How is prompt injection handled? If the agent processes anything a third party can influence, a ticket, an email, an uploaded file, a web page, then instructions can be planted in that content. Ask what stops retrieved content being treated as instructions.

What is logged? You need a record of what the agent did, on whose behalf, and why, in a form you can hand to an auditor or use in an incident. Many agent products log the conversation but not the actions.

How do you stop it? A kill switch that revokes its credentials, and a known blast radius if it misbehaves for an hour before anyone notices.

Treat it as an identity

The cleanest mental model is that you are onboarding a very fast employee with unusual judgement. Give it the least privilege that lets it work. Put it in your access review. Give it an owner. Include it in offboarding when the vendor relationship ends, because a forgotten agent credential is a standing back door.

What auditors and buyers are starting to ask

Agents are appearing in enterprise security reviews now: whether AI systems can take actions in your environment, what approvals gate them, and how those actions are logged. If your access review covers humans only, an agent with production write access is a finding waiting to be written.

Our AI Vendor Risk Assessment covers agents as a distinct tier, and securing a RAG pipeline goes deeper on the injection route that turns retrieval into action.

Tier the agent by reach, not by vendor size

The useful sorting question is not how big the vendor is or how much data the tool sees. It is what the agent can reach when it is behaving badly. We put agents into three tiers and the tier decides how much assessment work is proportionate.

Read-only inside one workspace. The agent can query a corpus you control and produce text. A summarising assistant over a Confluence space sits here. The residual risk is disclosure, so the usual data questions carry the review: where the content goes, whether it trains a model, retention, subprocessors, deletion on termination. A lightweight review is defensible.

Write access to internal systems. The agent can create tickets, change records, push commits, update CRM fields, post in channels. Now a bad decision has a persistent effect that somebody has to undo. This tier needs a named owner, an inventory entry, a permission list you have actually read rather than accepted, and logging you have seen a sample of.

Actions with external or irreversible effect. Sending mail to customers, issuing refunds or credits, changing entitlements or group membership, deploying, deleting. This is the tier where the assessment starts to look like an integration security review rather than a vendor questionnaire, and where a human approval gate is usually the only control that reliably holds.

Most disagreements about how much diligence an AI tool deserves dissolve once someone writes down which tier it is in. The argument stops being about opinion and starts being about a permission scope you can go and check.

A worked example: the support agent that can issue credits

A billing support agent is connected to your helpdesk and your billing provider. It reads the ticket, checks the account, and can apply an account credit up to a limit without a human. The vendor demo is genuinely good and the support lead wants it live next week.

Here is what the review actually turns up when you push. The helpdesk integration is granted through an admin OAuth app, because the vendor documentation says so, which means the agent can read every ticket including the ones from your legal and HR queues that happen to live in the same instance. The billing token is scoped to credits, which is good, but the daily limit is configured in the vendor dashboard rather than enforced by the billing provider, so it is a setting the vendor controls and not a boundary you own. Tickets are inbound content written by strangers, so a customer who writes a paragraph of instructions into a support ticket is writing directly into the agent context. And the audit log records the conversation turns but records the credit as having been applied by the shared integration user, so your finance reconciliation cannot tell agent credits from human ones.

None of that makes the product unusable. It changes what you do before go-live: separate the helpdesk instance or scope the OAuth grant to one queue, move the credit ceiling into the billing provider where you control it, require approval above a dollar figure you pick, and get a distinct identity for the agent so the ledger shows who did what. That is a week of configuration work, not a rejection.

Contract language worth arguing for

Vendor paper for agent products is usually written for a chat product and has not caught up. Four clauses are worth the negotiation time.

Notice before capability expansion. The permission grant you approved on Monday can become a broader one on Wednesday because the vendor shipped a new tool into the same integration. Ask for notice before new action types are enabled under an existing grant, and the ability to decline them. This is the clause vendors resist most and the one that matters most, because your access review is only as good as the assumption that scopes do not silently grow.

Log export. You want action logs in a form you can pull into your own store, not just view in their console for 30 days. If an incident review depends on the vendor still being your vendor, you do not really have logs.

Subprocessor and model disclosure. Agent vendors are usually a thin layer over a model provider, sometimes several, sometimes routed dynamically. Ask which providers can receive your content and whether that list can change without notice.

Incident notification that covers agent behaviour. Standard breach clauses trigger on unauthorised access to data. They do not trigger when the agent did something wrong while perfectly authorised. Write in notification for material unintended actions taken on your tenant.

Test it before you approve it

A short adversarial trial tells you more than a 200-question spreadsheet. Give the agent a sandbox tenant and spend two hours trying to make it misbehave. Plant instructions in the places it reads from: a ticket body, a file name, a calendar invite description, an alt attribute in an uploaded document. Ask it to do something outside its brief and see whether it refuses, asks, or quietly finds a path. Watch what it does when a tool call fails, because retry loops are where a rate-limited agent turns into an accidental denial of service against your own API.

Then read the log of what you just did. If you cannot reconstruct the two hours from the log, the log is not evidence and you should say so in the assessment.

When the agent gets it wrong

Plan for the ordinary failure rather than the dramatic one. The common case is not a compromise, it is an agent that ran for six hours doing something subtly wrong across a few hundred records. That produces three jobs, and it helps to have decided who does them in advance: stopping it, working out the extent, and reversing it.

Stopping it should be a single action your team can take without the vendor, which usually means revoking the OAuth grant or disabling the service account rather than clicking pause in the vendor console. Extent means being able to list every action in a window, which is why the log question is not academic. Reversal is the one people forget to check. Ask during evaluation whether the agent's writes are individually reversible and whether the vendor can produce a list of them. For anything touching money, entitlements or customer communication, assume you will need to do the reversal yourself and confirm you can.

Decide up front whether an agent misfiring counts as a security incident under your own policy. Our answer is that it does once it touches customer data or customer-visible systems, because the alternative is a category of event that nobody is required to write up and nobody learns from. If you have an incident response arrangement, tell whoever holds it that agents are in scope, because most retainer scoping conversations predate this problem.

When this is more process than you need

Plenty of teams should not build any of this yet. If the only AI in your company is a chat assistant with no tool access, you have a data question and not an agent question, and running a tiering exercise over it is theatre. If you have two agents and both are read-only, a line each in your existing vendor register is proportionate and a formal AI vendor programme is not.

It is also fair to say that some agent risk is better handled by not integrating. When a vendor requires admin scope with no path to narrower grants, the cheapest control is declining the integration and using the product in a mode where a human moves the output across. That is unfashionable advice and it is frequently the right one for a small team with no security staff, because a control you have to operate every week is a control you will stop operating.

If your agents are still few enough to name in a meeting, use a free register and keep going. The traztech Workspace has vendor records and an evidence register that will hold this fine at that size. Come back to a structured programme when discovery stops being possible from memory, which is the honest trigger. When you do reach the point where agents hold production write access and buyers are asking about it, that is security work with a scope worth defining, and the vendor assessment mechanics are the place to start.

Re-review and the credential you forgot

Agent access decays badly. The scope granted for a pilot stays after the pilot ends, the person who owned it changes team, and the vendor ships new tools into the same grant. Put agents into the same quarterly access review as human accounts and check three things: that the owner is still there, that the scopes are still the ones you approved, and that the agent has actually been used since the last review. Unused agent credentials with production write access are the single easiest finding for an auditor to write and the easiest for you to prevent by revoking them.

Offboarding is the other gap. When a vendor relationship ends, the contract termination and the credential revocation are usually done by different people on different timelines. Add the revocation step to the same checklist that cancels the invoice, and verify it by trying the token rather than by trusting the ticket.