Threat modelling for LLM and agent systems

What it is for

A threat model turns “is this safe?” into a list of specific things that could go wrong, who could make them happen, and what stops them. For LLM and agent systems the list is different from a web application’s: the attacker can be a document the model reads, a web page an agent visits, a tool description, or another agent.

What we look at

• Assets: data, credentials, money, actions in other systems, the model and its configuration, your reputation.
• Actors: users, operators, administrators, the model provider, third-party tools and MCP servers, other agents, and anyone who can place text where the model will read it.
• Flows and boundaries: every place untrusted content crosses into a prompt, a memory, a retrieval index or a tool call.
• Attacks: drawn from MITRE ATLAS, the OWASP lists for LLM and agentic applications, and NIST’s taxonomy of adversarial machine learning, which organises attacks by type, life-cycle stage and the attacker’s goals and capabilities.^⁠[1]

What you receive

The threat model itself, a set of abuse cases written as short attack stories your engineers can test against, and a ranked list of controls. The ranking is explicit about its reasoning, so you can change the order when your constraints differ from our assumptions.

How it runs

Workshops with the people who design and operate the system, followed by our own analysis and a review session. A2 pairs naturally with A1 AI architecture review before launch and with S1 AI red teaming and adversarial testing after it: the abuse cases become test cases.