Enterprise LLM security is no longer mainly a question of model behaviour. The larger problem emerges from the system around the model: what it can read, retrieve, which permissions it receives, and what actions it can perform. OWASP ranks Prompt Injection as LLM01:2025 and distinguishes direct prompt injection from indirect attacks delivered through external content such as websites and files. OWASP also states that fool-proof prevention remains unclear and recommends layered mitigation. [1] The risk is not laboratory-only: CrowdStrike's 2026 Global Threat Report states adversaries exploited legitimate generative-AI tools at more than 90 organisations in 2025. [7] Google, using repeated Common Crawl snapshots, reported a 32% relative increase in material classified in its malicious prompt-injection category between November 2025 and February 2026, though much detected activity was unsophisticated. That figure is a change in detected malicious-category material, not a 32% increase in successful AI compromises. [6]
For legal, healthcare, compliance, financial, and other high-trust environments, an LLM security review therefore has to extend beyond refusal behaviour and model-level safeguards. It needs to cover prompt injection, retrieval permissions, sensitive-data exposure, logging, external content, tool execution, and the controls that determine whether model-generated output can become a real-world action.
Direct answer: In 2026, LLM security means treating externally controlled content as untrusted, enforcing permissions outside the model, constraining tool use, and independently verifying consequential outputs and actions. The core risk is not only what the model says, but what the surrounding system lets it read, retrieve, and do.
The risk is not confined to laboratory demonstrations. CrowdStrike's 2026 Global Threat Report states that adversaries exploited legitimate generative-AI tools at more than 90 organisations in 2025, injecting malicious prompts to generate commands used for credential and cryptocurrency theft. [7]
That distinction matters. Detection frequency, attack success rate, vulnerability prevalence, and confirmed security incidents are different measurements and should not be presented as though they were interchangeable.
Why LLM security became an operational discipline
The traditional framing of LLM security concentrated heavily on what a model might generate. That becomes incomplete once an assistant can retrieve documents, inspect email, query databases, browse external content, or invoke tools.
At that point, security depends on the complete application architecture.
A legal assistant, for example, may combine a user's instruction with a document supplied by another party. A corporate assistant may combine an employee request with search results, emails, internal documents, and API responses. An agent may then use its interpretation of those inputs to choose a tool call.
The fundamental question becomes:
Which information is allowed to influence the model, and which information is allowed to influence an action?
Research on indirect prompt injection provides direct evidence for this problem. Yi et al.'s BIPIA research found that, within the evaluated benchmark, the tested LLMs were vulnerable to malicious instructions embedded in external content. The researchers identified two important causes: difficulty distinguishing informational context from actionable instructions, and failure to recognise that instructions inside external content should not necessarily be executed. [3]
This supports a practical security principle:
If an AI system can read externally controlled content, that content should not automatically receive authority over the workflow.
Direct and indirect prompt injection
OWASP distinguishes two major forms of prompt injection. [1]
Direct prompt injection occurs when an input supplied directly to the model alters its behaviour in an unintended way.
Indirect prompt injection occurs when instructions are embedded in external material processed by the LLM, such as a webpage or file.
This distinction matters because enterprise AI systems derive much of their usefulness from external information.
A legal AI system is valuable because it can process contracts, evidence, correspondence, and case material. A compliance assistant may inspect policies and records. A browser agent depends on web content. A retrieval-augmented system depends on information retrieved from another data store.
Eliminating external information would therefore eliminate much of the application's value.
The security challenge is instead to use untrusted information without allowing that information to obtain unintended control over the system.
Experimental evidence: prompt injection is strongly context-dependent
One reason security claims around prompt injection need careful qualification is that attack and defence performance varies substantially between configurations.
Liu et al., in research published at the 33rd USENIX Security Symposium, systematically evaluated:
- 5 prompt-injection attacks
- 10 defence mechanisms
- 10 LLMs
- 7 tasks
The research was designed specifically to provide a common benchmark for evaluating prompt-injection attacks and defences. [2]
This means that statements such as "prompt injection has an 80% success rate" are not defensible without identifying the particular attack, model, task, defence configuration, and evaluation methodology.
A percentage measured under one experimental configuration cannot automatically be generalised to enterprise LLM systems as a whole.
BIPIA reached a related conclusion from the perspective of indirect attacks. Yi et al. evaluated prompt injection delivered through external information and found the evaluated models vulnerable in the benchmark. Their analysis identified the distinction between informational context and actionable instructions as a central factor in successful attacks. The work was accepted at KDD 2025. [3]
The researchers proposed boundary awareness and explicit reminder mechanisms as defences. Their experiments showed substantial mitigation, including near-zero attack success rates for the evaluated white-box defence. Those results belong to the specific benchmark and defence configurations and should not be interpreted as proof that indirect prompt injection has been universally solved. [3]
Why architectural controls matter
Prompt engineering is useful, but it is not equivalent to a security boundary.
OWASP's mitigation guidance explicitly recommends enforcing privilege control and least privilege, requiring human approval for high-risk actions, segregating external content, and conducting adversarial testing. [1]
This distinction becomes especially important when an AI system can invoke tools.
Consider an agent that can request:
send_document(destination, document)
A model-level instruction may tell the agent not to send confidential information to unauthorised recipients.
A stronger security control is for the application itself to determine whether the requested destination is authorised before executing the operation.
The difference is significant:
Prompt-level control asks the model to behave correctly.
Authorisation-level control restricts what the system is capable of doing.
The latter remains enforceable even if the model has been manipulated.
CaMeL: separating control flow from untrusted data
A particularly relevant architectural research direction is CaMeL - Defeating Prompt Injections by Design.
CaMeL creates a protective system layer around the LLM. The architecture explicitly extracts control and data flows from the trusted query so that untrusted data retrieved by the LLM cannot determine program flow. It also applies capability-based controls intended to prevent private-data exfiltration through unauthorised data flows when tools are called. [5]
The version of the paper submitted on 24 March 2025 was revised on 24 June 2025. The revised v2 reports that, in the researchers' AgentDojo evaluation, CaMeL solved 77% of tasks with provable security, compared with 84% for an undefended system. [5]
This result quantifies both the security benefit claimed by the architecture and the utility cost introduced by its restrictions.
It must, however, be interpreted within the benchmark, assumptions, security property, and architecture evaluated by the researchers. It does not establish that CaMeL universally solves prompt injection.
Its more important contribution is architectural: security does not have to depend solely on whether an LLM correctly interprets an instruction. Some security decisions can instead be moved into enforceable mechanisms surrounding the model.
Prompt injection is appearing on the public web
Google published first-party evidence in April 2026 based on an analysis of Common Crawl.
Google's researchers conducted a broad scan of archived public-web content to identify indirect prompt-injection patterns. The research used pattern matching, LLM-based classification, and manual validation to distinguish potentially malicious instructions from legitimate material discussing prompt injection. [6]
This validation step matters because false positives are a significant measurement problem. A webpage discussing security research may contain examples of malicious prompts without itself constituting an attack.
Google observed material ranging from experimentation and AI-directed manipulation to a smaller category of malicious examples.
Between the November 2025 and February 2026 archive versions, Google measured a 32% relative increase in the malicious category. [6]
Google simultaneously warned against overinterpreting the result. Its researchers characterised much of the observed activity as having limited sophistication and noted that the Common Crawl dataset does not capture major social-media platforms. [6]
This prevents two opposite mistakes:
Understatement: claiming prompt injection exists only in academic demonstrations.
Overstatement: claiming every detected malicious prompt constitutes a successful compromise.
The available evidence supports neither extreme.
Multimodal input extends the attack surface
Prompt injection is not limited to ordinary visible text.
OWASP's LLM01:2025 guidance explicitly notes that malicious input does not need to be human-visible or human-readable as long as the model can parse it. Its examples also include multimodal prompt injection involving instructions embedded in images. [1]
Systems accepting scanned documents, screenshots, presentation files, images, or other multimodal material therefore need to include those channels in their threat model.
A human reviewer failing to notice an instruction does not mean that a multimodal model cannot process it.
Data leakage through natural-language database interfaces
Prompt injection becomes more consequential when an LLM has access to sensitive databases.
The SecureSQL study, published in Findings of the Association for Computational Linguistics: EMNLP 2024, provides primary empirical evidence.
The researchers created a benchmark containing:
- 932 samples
- across 34 domains
- evaluated using 15 models
- from six LLM families
The domains included medical, legal, financial, and political data. [4]
SecureSQL reported that the best-performing tested model achieved 61.7% accuracy, compared with 94% for humans in the benchmark. The researchers additionally evaluated prompt-injection and inference attacks and reported that both significantly affected model performance. Their tested chain-of-thought prompting defence did not significantly improve accuracy. [4]
These results apply specifically to the SecureSQL benchmark. They should not be interpreted as proving that every natural-language database interface leaks data.
They do demonstrate why an LLM should not itself be treated as the database authorisation mechanism.
If a user does not have permission to read a record, the database or application architecture should enforce that restriction independently of what SQL the model proposes.
Retrieval systems have the same underlying security problem
The same principle applies to retrieval-augmented generation.
If a user's source-system permissions do not permit access to a document, the retrieval layer should not make that document available to the model merely because the model has been asked a semantically related question.
This is not principally a question of whether the model is willing to obey a privacy instruction. It is an access-control problem.
A general architectural rule follows:
The model can participate in a security-sensitive workflow, but it should not become the sole enforcement point for security policy.
Defence layers and what they actually achieve
No single defence in the evidence reviewed here establishes universal protection against prompt injection.
OWASP explicitly states that fool-proof prevention remains unclear and therefore recommends multiple mitigations. [1]
Instruction and external-content separation
External information should be clearly distinguished from trusted instructions.
This is directly supported by BIPIA, which identifies inability to distinguish informational context from actionable instructions as one factor contributing to indirect prompt-injection success. [3]
The important qualification is that marking data as untrusted is not necessarily the same as technically preventing it from influencing execution.
System-prompt hardening
System prompts can establish expected behaviour, task boundaries, and rules.
They are useful but should not be treated as an access-control mechanism.
The USENIX benchmark demonstrates that defence effectiveness depends on the attack, task, model, and configuration. [2]
Input and output validation
OWASP recommends input and output filtering as part of a broader mitigation strategy. [1]
These controls can reduce risk, but they should not be presented as a guarantee that every semantically malicious instruction will be detected.
Least privilege
OWASP recommends restricting an LLM application's privileges to the minimum required for its intended operation. [1]
This reduces blast radius.
A summarisation system that cannot send email cannot be manipulated into directly sending the summarised document through an email capability it does not possess.
Tool authorisation
Security-sensitive tool calls should be validated outside the model.
A model can propose an operation, deterministic application logic can decide whether that operation is permitted.
This distinction is critical because verification is not authorisation.
A model may generate a plausible and well-supported request while still lacking permission to perform the requested operation.
Human approval
OWASP recommends human approval for high-risk actions. [1]
Human-in-the-loop control is particularly relevant where an AI-generated action could disclose sensitive information, alter access, trigger a financial transaction, modify production systems, or have legal, clinical, or other significant consequences.
The review should be meaningful rather than ceremonial: the reviewer needs sufficient context to understand the proposed action and its consequences.
Adversarial testing
OWASP recommends adversarial testing and attack simulations. [1]
The USENIX, BIPIA, SecureSQL, and CaMeL research further demonstrates why workflow-specific evaluation matters. Performance varies according to the task, attack, defence, model, privileges, and architecture. [2][3][4][5]
Independent verification
Independent model review can provide a separate check on factual support, source quality, inconsistencies, unsupported statements, or disagreement between generated conclusions.
That is a verification control.
It should not be described as a replacement for prompt-injection defences, access control, or authorisation.
If multiple models receive the same poisoned source material and operate within the same unsafe permission structure, agreement between those models does not make the workflow secure.
Verification can improve assurance about an answer. Authorisation determines whether an action is allowed. They solve different problems.
A defensible high-assurance architecture
A mature LLM workflow can therefore be designed around distinct control stages:
External input -> trust classification -> permission-aware retrieval -> constrained model processing -> output verification -> deterministic authorisation -> human approval where required -> action -> audit record
Each stage answers a different question.
Trust classification: Where did this information originate, and how much authority should it receive?
Retrieval: Is this information permitted for this user and task?
Model processing: What conclusion or action does the model propose?
Verification: Is the conclusion supported by the available evidence?
Authorisation: Is the requested action permitted?
Human review: Does the consequence require accountable human judgement?
Audit: Can the organisation reconstruct what happened?
Combining all of those questions into a single prompt is weaker than enforcing independent controls where deterministic enforcement is possible.
Verification checklist for high-stakes LLM deployments
Before trusting an LLM workflow with sensitive information or consequential actions, organisations should verify the entire path from input to execution.
Map every information source
Document every location from which the AI system can obtain information, including user prompts, documents, PDFs, email, websites, search results, databases, RAG repositories, screenshots, images, API responses, tool outputs, and persistent agent memory.
OWASP's distinction between direct, indirect, and multimodal prompt injection demonstrates why externally controlled content must be part of the threat model. [1]
Map every possible action
Record every operation the AI system can initiate, including sending messages, exporting files, updating records, querying databases, changing permissions, executing code, and invoking external APIs.
The potential impact of prompt injection depends heavily on what the system is allowed to do.
Verify authorisation independently
Do not allow an LLM-generated request to substitute for deterministic access control.
If an action requires a particular permission, validate that permission independently of the model's reasoning.
Test indirect prompt injection
Place controlled adversarial instructions in representative documents, webpages, messages, and other external content and test the complete workflow.
BIPIA exists specifically because testing only direct user prompts does not cover indirect injection through external content. [3]
Test tool misuse separately
A manipulated text response and an unauthorised external action are different failure classes.
Test both.
Protect logs
Prompt, retrieval, tool-call, and model-output logging can be important for auditability, but those logs may themselves contain sensitive information.
Retention, access, minimisation, and redaction therefore need their own controls.
Prepare an incident-response path
Determine in advance how agent credentials can be revoked, workflows disabled, compromised sources isolated, and relevant evidence preserved.
The ability to reconstruct an incident is part of operational assurance.
What the 2026 evidence actually supports
The strongest conclusion is not that "LLMs are insecure".
The available evidence supports something more precise.
Prompt injection remains a recognised high-priority LLM application risk. OWASP places it at LLM01:2025 and recommends layered mitigation rather than claiming a universal preventative control. [1]
Attack and defence effectiveness varies across models and tasks. The peer-reviewed USENIX benchmark evaluated five attacks, ten defences, ten LLMs, and seven tasks. [2]
Indirect prompt injection is experimentally demonstrated. BIPIA focuses specifically on adversarial instructions embedded in external content and identifies instruction-versus-context confusion as a central part of the problem. [3]
Sensitive-data risks exist when LLMs mediate database access. SecureSQL evaluated 932 examples across 34 domains using 15 models from six families and found significant effects from prompt-injection and inference attacks. [4]
Architectural separation is an active defence direction. CaMeL moves important security decisions away from unrestricted model interpretation towards explicit control-flow and capability enforcement. Its revised v2 reports 77% AgentDojo task completion with provable security, compared with 84% for the undefended system. [5]
Malicious indirect prompt-injection content can be observed on the public web. Google measured a 32% relative increase in its malicious category between November 2025 and February 2026 while explicitly warning that much of the observed activity remained relatively unsophisticated. [6]
Real organisations have encountered malicious exploitation of legitimate GenAI tooling. CrowdStrike reports more than 90 organisations affected by the activity described in its 2026 Global Threat Report. [7]
None of these sources proves that every AI workflow will be compromised.
Together, they provide strong evidence that prompt injection and the access-control problems surrounding LLM applications are security-engineering concerns rather than merely unusual chatbot behaviour.
Conclusion
The central security mistake in enterprise AI is allowing probabilistic language interpretation to make decisions that should be enforced by deterministic security controls.
A stronger approach separates responsibilities.
Treat externally controlled content as untrusted.
Preserve access control at retrieval and execution boundaries.
Give agents the minimum privileges they require.
Do not allow model output alone to authorise consequential tool actions.
Use human approval where the impact warrants accountable judgement.
Test indirect injection against the actual workflow rather than only testing generic jailbreak prompts.
Verify important outputs independently.
And maintain enough evidence to reconstruct what the system received, proposed, verified, authorised, and ultimately did.
That is the difference between assessing the security of an LLM and assessing the security of an LLM-based system.
For professional environments in which generated answers themselves require additional assurance, independent model review, source verification, and an inspectable evidence trail can provide an additional verification layer. Such verification does not replace access control, authorisation, or prompt-injection defences, it addresses a different question: whether an answer is sufficiently supported to be relied upon.
The factual data in this article is based on Formalizing and Benchmarking Prompt Injection Attacks and Defenses and Indirect Prompt Injection Goes Operational.
More articles on this subject are collected in the AI security overview.
Further reading on this blog: The AI Act after the Digital Omnibus: postponement for high-risk, firm obligations from August 2026 and Why verification with a single AI model reaches its limits.
Sources and references
- Formalizing and Benchmarking Prompt Injection Attacks and Defenses
- Indirect Prompt Injection Goes Operational
- Image-Based Prompt Injection: Hijacking Multimodal LLMs Through Visually Embedded Adversarial Instructions
- Prompt Injection Attacks on Large Language Models: A Survey of Attack Methods, Root Causes, and Defense Strategies
- Practical LLM Security Advice from the NVIDIA AI Red Team
- Defeating Prompt Injections by Design
- Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Models
- Formalizing and Benchmarking Prompt Injection Attacks and Defenses
Sources: Key claims are attributed to OWASP (LLM01:2025 prompt injection), CrowdStrike's 2026 Global Threat Report, Google Common Crawl analysis, and peer-reviewed research by Liu et al. (USENIX Security 2024) and Yi et al. (BIPIA, KDD 2025).