AI Agent Security: A Practical Threat Model for Enterprise Deployments

AI agent security requires threat-modelling the agent as a system with its own attack surface — the prompt, the tools it can call, the data it can access, and the actions it can take — not just securing the underlying model. The highest-risk categories in production deployments are prompt injection, tool and permission misuse, and unintended data exposure, and each needs a distinct set of controls.

Why Agent Security Is a Different Problem From Model Security

Securing a model and securing an agent are different disciplines, and treating them the same is how enterprise deployments end up with real exposure. A model is a function: it takes text in, produces text out, and the worst a compromised prompt can do is produce a bad response. An agent is a system: it can call tools, query databases, send emails, execute code, or move money, and a compromised agent can take a real-world action, not just say something wrong. Threat-modelling an agent means asking, for every tool it can call and every data source it can reach, what the worst thing a malicious or manipulated input could make it do — and building controls around that answer, not around the model's general behaviour.

This matters more in enterprise deployments than consumer ones because enterprise agents typically have access to more sensitive data and more consequential actions — customer records, internal systems, financial transactions — and because an attacker doesn't need direct system access. They just need to get malicious instructions in front of the agent, through a document it reads, an email it processes, or a webpage it browses.

The Threat Model: Four Categories That Matter Most

Prompt injection is the most-discussed and most underestimated risk: an attacker embeds instructions in content the agent processes — a document, a support ticket, retrieved data — designed to override its original instructions. Because agents often process untrusted external content by design, this attack surface can't be closed off entirely; it has to be contained. Tool and permission misuse is the second category: an agent with broader tool access than a given task requires has a bigger blast radius if it's manipulated or simply makes a reasoning error.

Data exfiltration is the third: an agent that can read sensitive data and also has any outbound channel — email, a web request, a chat response — can be induced to leak that data, deliberately or accidentally. The fourth is action-chaining risk: individually low-risk actions combined in sequence (read a customer record, then draft an email, then send it) can produce a high-risk outcome that no single-action review would have caught.

Controls That Actually Reduce Risk

• Scope every tool to the minimum permissions the task requires — never grant broad access "in case it's needed later"
• Treat all retrieved and externally-sourced content as untrusted input, and never let it directly override system-level instructions
• Require explicit approval for any action above a defined risk threshold — financial transactions, external communications, data deletion
• Log every tool call, input, and output with a traceable ID, so a compromised or misbehaving run can be reconstructed after the fact
• Rate-limit and cap the cost or scope of what an agent can do per session, to contain the blast radius of any single compromised run
• Separate the data an agent can read from the actions it can take — read access should not automatically imply write or send permissions

Where Human-in-the-Loop Still Belongs

The instinct to automate a human checkpoint out of the workflow entirely is understandable — it's slower and it's the whole point of deploying an agent in the first place. But for actions with real consequences, a human approval step isn't a limitation on the agent, it's the control that makes deploying it safely possible at all. The practical approach is a risk-tiered checkpoint: fully autonomous for low-risk, easily reversible actions, and a mandatory human review for anything financial, externally visible, or difficult to undo. This tiering should be explicit and documented, not left to the model's own judgement about what counts as high-risk.

In practice, this means the agent prepares the action along with its supporting rationale, and a human approves, edits, or rejects it before it executes. That checkpoint is not wasted effort even when the human almost always approves the agent's recommendation — it's what lets the organisation trust the agent with the task at all, and it gives you a place to catch the rare case where the agent's reasoning quietly went wrong before it becomes an incident.

Building This Into the Architecture, Not Bolting It On

Security controls added after an agent is already built tend to be superficial — a content filter here, a keyword blocklist there — because the fundamental architecture wasn't designed with the threat model in mind. The controls that actually hold up are architectural: tool permissions defined and enforced at the system level, not just instructed via prompt; a clear boundary between untrusted content and system instructions; and action logging built into the agent's execution loop from day one.

None of this requires solving prompt injection perfectly, which nobody has done yet — it requires designing the system so that even a successful injection or a reasoning error can't cause damage beyond a contained, recoverable scope. That containment, not prevention, is the realistic and defensible security goal for enterprise AI agents today.