What Is AI and LLM Security? A Plain-Language Guide (2026)

If your company has shipped a chatbot, a support agent, or any product feature built on a large language model, someone on your team has probably already asked the question: is this thing safe to put in front of customers? That question is what AI and LLM security is trying to answer. It is a newer discipline than most of information security, and the terminology is still settling, so it is worth defining plainly before you decide whether you need it and who should do it.

What AI and LLM security actually means

AI security is the practice of finding and fixing the ways an AI system can be manipulated, tricked, or misused, either by an outside attacker or by an unexpected input from a normal user. LLM security is the part of that practice focused specifically on large language models, the technology behind tools like customer support bots, internal copilots, and AI-powered search.

It sits next to your existing security program rather than replacing it. Your firewall, your access controls, and your SOC 2 controls still matter. AI security adds a layer that deals with problems traditional security testing was never built to catch, because the vulnerability lives in the model's behaviour, not in a server misconfiguration.

What the testing actually involves

In practice, testing your own AI means probing it the way an attacker would, then documenting what broke and how to fix it. A few categories come up constantly:

• Prompt injection. An attacker hides instructions inside text the model reads, a support ticket, a PDF, a webpage, and the model follows those instructions instead of yours. This is the AI equivalent of SQL injection, and it is the most common finding in real assessments.
• RAG leakage. Many LLM products use retrieval-augmented generation to pull from a private knowledge base before answering. If access controls on that data are weak, a user can coax the model into revealing documents, records, or other customers' data it should never surface.
• Agent tool abuse. AI agents that can send emails, query databases, or call internal APIs are only as safe as the guardrails around those actions. If an agent can be tricked into calling a tool it shouldn't, the damage is no longer confined to a bad chat response.
• OWASP LLM Top 10 coverage. The OWASP Foundation maintains a standard list of the most common LLM vulnerabilities, covering everything above plus issues like training data poisoning, excessive agency, and insecure output handling. A proper assessment works through that list systematically rather than testing a handful of obvious cases and calling it done.

A real assessment produces the same kind of deliverable a penetration test does: a report ranking findings by severity, proof-of-concept examples showing how each issue was triggered, and concrete remediation guidance your engineering team can act on. If you want the specifics of how that work is scoped, our AI security assessments page walks through what is tested and how engagements are structured.

Who actually needs this

You need AI security testing if any of the following is true: you have shipped or are about to ship a customer-facing chatbot or AI feature, you have deployed an internal AI agent with access to real systems or data, a customer or prospect has asked how you secure your AI product, or you are pursuing a certification like SOC 2 or ISO 42001 that now expects AI risk to be addressed explicitly. That last point catches a lot of companies off guard. AI security used to be optional. It is increasingly becoming an expectation baked into procurement questionnaires and compliance frameworks, particularly for B2B SaaS companies selling into enterprise or regulated buyers.

Realistic timelines

A focused AI security assessment on a single product or feature typically runs a few weeks from kickoff to final report, not months. The variables that move the timeline are the number of AI-powered features in scope, how many tools or integrations an agent has access to, and how quickly your team can provide test access. Companies sometimes assume this is a multi-month audit on the scale of a full compliance program. It usually isn't. It is closer in scope and duration to a targeted penetration test. If AI risk is one part of a broader governance effort, for example building toward ISO 42001, that timeline extends because you are standing up policies and processes alongside the technical testing. Our ISO 42001 readiness work covers that broader governance piece if your goal is a management system for AI, not just a point-in-time test.

Common misconceptions

A few assumptions come up repeatedly and are worth correcting directly.

"Our AI vendor already secured this." Vendors like OpenAI, Anthropic, and Google secure the underlying model. They cannot secure how you configured it, what data you connected it to, or what tools you gave it permission to call. That configuration is your responsibility, and it is where most real-world incidents originate.

"We tested it with a few obvious prompts and it held up." Prompt injection techniques evolve constantly, and a model that resists an obvious attempt often falls to a more creative one. Ad hoc testing by someone without dedicated AI security experience tends to miss the findings that matter most.

"This is just a chatbot, the stakes are low." If the chatbot can access customer records, internal documents, or backend tools, the stakes are identical to any other system with that access. The interface being conversational doesn't lower the risk.

"We'll deal with this after launch." Retrofitting security into a shipped product is more expensive and more disruptive than testing before launch, especially once real users and real data are flowing through it.

Where to start

If you are unsure whether your AI deployment needs a dedicated assessment or fits under a broader security review, the honest answer depends on what the system has access to and who is using it. The fastest way to get a straight answer is to walk through your setup with someone who does this work regularly.

Talk to us about your AI deployment and we'll tell you plainly what level of testing makes sense, no upsell. Get in touch to start the conversation.

What an assessor needs from you before testing starts

The slow part of an AI security engagement is almost never the testing. It is getting an environment that behaves like production. Before kickoff, have these ready: a diagram of every place a model is called, the model names and versions and whether they run on a vendor API or your own infrastructure, the full text of your system prompts, the definitions of every tool or function the model can invoke, and the identity each of those tools runs under. That last item matters more than people expect. An agent that queries your database through a service account with read access to all tenants is a different risk from one that inherits the calling user's permissions, and you cannot tell which you have from a diagram.

You also want a test tenant seeded with at least two fake customers holding distinct records. Cross-tenant leakage in a retrieval index is the finding that changes procurement conversations, and it can only be demonstrated cleanly when there is a second tenant to leak into. Without it, an assessor is reduced to describing a theoretical path, and theoretical paths get argued away in a remediation meeting.

What buyers are actually asking about your AI

Enterprise security reviews have caught up faster than most product teams realise. The rows now showing up in vendor questionnaires are specific: do you train or fine-tune on customer data, does your model provider retain prompts and for how long, can a customer opt their data out, which subprocessors receive prompt content, is there human review of outputs, do you maintain an inventory of AI features, and do you have an acceptable use policy governing internal AI tools. Answering "we call a third-party API" satisfies none of these. What satisfies them is the subprocessor entry in your data processing agreement, the retention setting you have actually configured in the vendor console, and a screenshot or export proving it.

Teams that already run a disciplined answer library slot these rows in without drama. If yours does not, the underlying problem is process rather than AI, and our take on questionnaire help is the better starting point.

What drives the cost of an assessment

Scope is driven by four things. First, the number of distinct entry points: a chat widget is one surface, a chat widget plus an email-ingestion agent plus a document summariser is three, and each needs its own injection attempts. Second, how many tools the model can call and whether any of them write rather than read. Write-capable tools roughly double the effort because every finding needs a safe way to prove impact without damaging real data. Third, whether the product is multi-tenant, because tenant isolation testing is its own body of work. Fourth, whether you want a retest after remediation, since a clean retest letter is the artefact procurement accepts.

One distinction saves money: model evaluations and security testing are not the same purchase. Evals measure whether your model gives good answers. Security testing measures whether it can be made to take actions or reveal data it should not. If your real worry is answer quality, buy evals, not a penetration test. Our security services page sets out where the line sits, and published starting prices give you a sense of the floor before you book a call.

When you should not buy this

If your AI feature has no access to private data, no tools, and no ability to act, the realistic worst case is an embarrassing answer. That is a content-filtering and logging problem, and a dedicated assessment is poor value against it. Same answer for an internal-only tool used by a dozen employees who already have access to everything it can reach: you have not expanded the blast radius, you have changed the interface to it.

If you have no prompt and response logging at all, spend the first money there instead. An assessment tells you what could happen; logs tell you what did. Without them you cannot investigate a customer's report or prove to an auditor that nothing happened. Fix the obvious hygiene items too: remove credentials and internal URLs from system prompts, scope every API key the model can reach to the narrowest permission that still works, and cap tool call rates. Those cost engineering hours rather than consulting fees.

What to do when a finding lands

Triage by what the issue reaches, not by how clever the technique was. A prompt injection that makes the bot rude is a content issue. The same injection that makes an agent call an internal API is an access control issue. Fix in that order: revoke or narrow the tool permission first, enforce tenant filtering in the retrieval layer where the query is built rather than in the prompt, and treat prompt hardening as the last and weakest layer because it is the one an attacker gets unlimited attempts against.

If you find evidence that a technique was used against production rather than a test environment, that is an incident, and the notification clocks under your customer contracts and privacy obligations run the same as they would for any other data exposure. Deciding who makes that call at 11 p.m. is not a decision to leave until the night it happens, which is the argument in our piece on incident response.