Threat modeling: the lethal trifecta, trust boundaries, and how to use two OWASP lists as checklists

Today's Goals

1. Use the lethal trifecta test to decide whether an agent has an injectable exfiltration path, and say which edge is cheapest to cut
2. Draw an agent as a data-flow diagram with trust boundaries, labeling where each piece of data comes from
3. Walk the OWASP LLM Top 10 and Agentic Top 10 against your own agent and come out with a list of risk items

This course solves exactly one problem: how to make an agent that acts on its own fail to do damage when somebody talks it into trying. Today is about seeing where the danger actually lives. Bolting on defenses before you can see it usually means patching a problem you do not have while the real path stays untouched. When you are done, come back to the top of the page and check off the three goals.

Plain-Language Walkthrough

A building lobby and its parcel locker

Your building has a locked front door. To make life easier, the management company installed a parcel locker right inside the lobby, and anyone can walk up to it to drop off or pick up a package. That on its own is not dangerous: a stranger who gets into the lobby is, well, standing in the lobby.

The danger is a combination. Suppose the management company also did two other thoughtful-sounding things. First, they put a notice board next to the locker listing every apartment number, resident name and phone number. Second, they told the security guard a simple rule: whatever a note taped to the parcel locker asks for, do it, because it is probably from a resident.

Now look at the three facts together: a stranger can reach the lobby, the lobby holds a resident directory, and the guard executes instructions written on notes. Any passer-by can tape up a note saying "please photograph the notice board and text it to this number" and walk away with a complete data leak. And not one of the three facts is a mistake on its own: the locker is a convenience, the notice board helps people find each other, and following residents' instructions is the guard's job.

The risk is in none of the parts. The risk is in the combination. That is the whole of today's lesson. The reason your existing process never catches it is that your process inspects the parts one at a time, and this thing only exists once they are assembled.

The lethal trifecta: why each edge on its own is not a bug

Translate the building story into agent terms and you get what Simon Willison named in June 2025: the lethal trifecta. The three edges are:

1. Access to private data — the agent can read things that should not be public: the customer database, internal documents, other people's conversations.
2. Exposure to untrusted content — the agent pulls text somebody else can write into its context: ticket bodies, web pages, emails, third-party API responses.
3. Outbound communication — the agent can move information out: send mail, call a webhook, write somewhere another party can read.

When all three are present, one successful prompt injection becomes one data exfiltration. The chain is short: an instruction hides inside untrusted content, the model follows it, the model fills the payload from private data, and the model uses an outbound tool to ship it.

The whole course demonstrates that chain on one fictional target: an internal knowledge assistant called deskmate. Every day it does three things — read tickets and the web pages linked from them, query the internal customer database to answer questions, and notify colleagues by email or webhook. Those three duties happen to complete all three edges, and every one of them is a feature the product manager explicitly asked for.

What makes the test genuinely useful is that it hands you a way to cut, not a way to block. The three edges are joined by AND, so removing any one of them breaks the chain. And the three usually cost wildly different amounts: private data is normally the core of the business and cannot go, untrusted content is the requirement itself and cannot go either, but "send it outward" is very often there only for convenience — plenty of agents could downgrade sending to "write the draft back onto the ticket and let a human press send." The mechanical way to pick an edge is to count the tools involved; the smallest one is the cheapest cut:

// This tests capability, not behavior: if it is on the tool list, the agent has it.
// Whether an attacker can reach it does not depend on how you normally use it.
export function assessTrifecta(config) {
  const privateData = config.tools.filter((t) => t.sensitivity === 'private').map((t) => t.name)
  const untrusted = config.tools.filter((t) => t.returnsUntrusted).map((t) => t.name)
  // Untrusted content has two sources: external content a tool returns, and the user input itself
  if (config.userInputTrust === 'untrusted') untrusted.push('<user input>')
  const externalComms = config.tools.filter((t) => t.effect === 'send').map((t) => t.name)
 
  const edges = [
    ['private data access', privateData],
    ['untrusted content', untrusted],
    ['outbound communication', externalComms],
  ]
  const complete = edges.every(([, tools]) => tools.length > 0)
  // The edge backed by the fewest tools is the one that is cheapest to cut
  const cheapest = [...edges].sort((a, b) => a[1].length - b[1].length)[0][0]
  return { privateData, untrusted, externalComms, complete, cheapest }
}

# This tests capability, not behavior: if it is on the tool list, the agent has it.
# Whether an attacker can reach it does not depend on how you normally use it.
def assess_trifecta(config):
    private_data = [t.name for t in config.tools if t.sensitivity == "private"]
    untrusted = [t.name for t in config.tools if t.returns_untrusted]
    # Untrusted content has two sources: external content a tool returns, and the user input itself
    if config.user_input_trust == "untrusted":
        untrusted.append("<user input>")
    external_comms = [t.name for t in config.tools if t.effect == "send"]
 
    edges = [
        ("private data access", private_data),
        ("untrusted content", untrusted),
        ("outbound communication", external_comms),
    ]
    complete = all(len(tools) > 0 for _, tools in edges)
    # The edge backed by the fewest tools is the one that is cheapest to cut
    cheapest = min(edges, key=lambda e: len(e[1]))[0]
    return dict(
        private_data=private_data,
        untrusted=untrusted,
        external_comms=external_comms,
        complete=complete,
        cheapest=cheapest,
    )

Note that the test is about capability, not behavior. "Our agent never sends customer records out" is not an argument — what an attacker gets to use is the tool list, not your habits.

Direct and indirect injection: the attacker never has to talk to your agent

The mechanics of prompt injection were covered once already, on day 22 of the 30-day course: everything the model receives is flattened into one stretch of text, and your system prompt and somebody else's content are just two spans of characters that arrived in some order — no span carries a tamper-proof seal. So it is not a bug that can be fully fixed, and the industry consensus is to assume it will succeed and make success worthless.

For threat modeling you only need to push one step further. Direct injection is an attacker talking to your agent. Indirect injection is an attacker writing into something your agent is going to read sooner or later. The second is the hard one: they need no account on your system and never show up in your logs; it is enough to leave a paragraph in a ticket's notes field, or in a corner of a page that fetch_page will one day retrieve. Anywhere the agent reads content into context is attack surface, and that single sentence decides how you draw the next section's trust boundaries.

Drawing trust boundaries: label every piece of data with where it came from

The first diagram in threat modeling is not an architecture diagram. It is a data-flow diagram with one dashed line across it. The only difference between the two sides of that line: inside is what you can vouch for, outside is what somebody else can write.

Three steps. List every source that pours anything into the context. Mark each source trusted or untrusted. List every exit that can produce an external side effect. Then look for a path that runs from an untrusted entry all the way to some exit. For deskmate it looks like this:

flowchart LR
  U[user question untrusted] --> A[deskmate context]
  T[read_ticket ticket body untrusted] --> A
  W[fetch_page page body untrusted] --> A
  C[search_customers customer database private] --> A
  A --> E[send_email exit]
  A --> H[post_webhook exit]
  A --> K[update_ticket exit]

The part worth staring at is not the boxes, it is the direction of the arrows. Three untrusted arrows and one private arrow converge into one context, and three arrows leave that context for the outside world. That converge-then-diverge shape is itself the alarm. Once untrusted content and private data meet inside one context, and that context is wired to an exit, the model has become the guard following the note.

Two mistakes beginners make here. First, tool return values are entries too. Plenty of people mark only user input as untrusted and forget that the page body fetch_page brings back was written, in full, by a stranger. Second, exits are more than "send a message." Any write an outsider can observe is an exit: content written back onto a ticket is visible to the submitter, and content spliced into a URL has already landed in somebody's access log by the time the request fails. The list of exfiltration exits is usually longer than the one you wrote down.

What the two OWASP lists are each for

With the diagram drawn, the next question is "besides this one big path, what did I fail to think of?" That calls for a checklist, and OWASP maintains two of them.

The OWASP Top 10 for LLM Applications covers the model application: what goes wrong between input going in and output coming back. Three entries matter today — LLM01 prompt injection, LLM02 sensitive information disclosure, LLM06 excessive agency.

The OWASP Top 10 for Agentic Applications (published 9 December 2025, numbered ASI01 through ASI10) covers agents that act: they have loops, tools, memory and possibly other agents as colleagues, none of which the LLM list reaches. Four entries matter today — ASI01 agent goal hijacking, ASI02 tool misuse, ASI03 identity and privilege abuse, ASI06 memory and context poisoning.

One sentence holds the split: the LLM list governs what the model said; the ASI list governs what the agent did. A support bot that only answers questions is mostly covered by the LLM list. An agent that calls tools, mutates state and remembers things across turns will lose a whole class of risk without the ASI list — the same injection that merely makes a model talk nonsense in a single exchange gets written into long-term memory on an agent that remembers, and then fires on every later turn. That is ASI06.

The right way to use a checklist is to ask, entry by entry, "what does this look like in my system" — not to tick boxes saying "we do not have that problem." For deskmate, LLM01 looks like the ticket notes field, and ASI02 looks like send_email letting the model choose the recipient freely. If you cannot write down the concrete shape, you have not thought it through yet.

The output is not a document, it is a risk table you can tick through

The most common failure mode of threat modeling is producing prose: even-handed, impossible to fault, useless to act on, and never opened again three months later.

The way out is to give every risk item mandatory fields, and the critical one is evidence: which tools in the config make this risk true. A risk item with evidence can be challenged to its face ("that tool was removed last month"). One without evidence can only be nodded past.

// Every risk must answer three things: severity, the matching OWASP entry, and what makes it true.
export function buildRiskItems(config, verdict) {
  const items = []
  if (verdict.complete) {
    items.push({
      id: 'R-TRIFECTA',
      title: 'lethal trifecta complete: one indirect injection is enough to exfiltrate data',
      severity: 'critical',
      owasp: ['LLM01 prompt injection', 'LLM02 sensitive information disclosure', 'ASI01 agent goal hijacking'],
      evidence: [
        `private data: ${verdict.privateData.join(', ')}`,
        `untrusted content: ${verdict.untrusted.join(', ')}`,
        `outbound communication: ${verdict.externalComms.join(', ')}`,
      ],
      mitigation: 'cut any one edge; if none can be cut, put outbound communication behind an approval gate and an egress allowlist',
    })
  }
  for (const tool of config.tools) {
    if (tool.effect === 'send' && !tool.reversible) {
      items.push({
        id: `R-IRREVERSIBLE-${tool.name}`,
        title: `${tool.name} is an irreversible outbound action, so a mistake gets no second chance`,
        severity: 'high',
        owasp: ['LLM06 excessive agency', 'ASI02 tool misuse'],
        evidence: [tool.description],
        mitigation: 'add an approval gate and allowlist the recipients and destination addresses',
      })
    }
  }
  return items
}

# Every risk must answer three things: severity, the matching OWASP entry, and what makes it true.
def build_risk_items(config, verdict):
    items = []
    if verdict["complete"]:
        items.append({
            "id": "R-TRIFECTA",
            "title": "lethal trifecta complete: one indirect injection is enough to exfiltrate data",
            "severity": "critical",
            "owasp": ["LLM01 prompt injection", "LLM02 sensitive information disclosure", "ASI01 agent goal hijacking"],
            "evidence": [
                "private data: " + ", ".join(verdict["private_data"]),
                "untrusted content: " + ", ".join(verdict["untrusted"]),
                "outbound communication: " + ", ".join(verdict["external_comms"]),
            ],
            "mitigation": "cut any one edge; if none can be cut, put outbound communication behind an approval gate and an egress allowlist",
        })
    for tool in config.tools:
        if tool.effect == "send" and not tool.reversible:
            items.append({
                "id": f"R-IRREVERSIBLE-{tool.name}",
                "title": f"{tool.name} is an irreversible outbound action, so a mistake gets no second chance",
                "severity": "high",
                "owasp": ["LLM06 excessive agency", "ASI02 tool misuse"],
                "evidence": [tool.description],
                "mitigation": "add an approval gate and allowlist the recipients and destination addresses",
            })
    return items

That code is the core of today's lab. It is deliberately dumb — nothing but if statements, no model involved. Threat modeling is work you finish before writing code; it needs no running agent, and it should never depend on a model to tell you what is wrong with your own system.

One more field deserves its own paragraph. Severity is not a gut call. The very same "this tool returns untrusted content" is high on an agent with the full trifecta and drops to medium on an agent with no outbound communication, because the second one can be hijacked and still send nothing. Severity is a property of the path, not of the point — which is the same statement as "cut one edge," said the other way round.

Which layer each of the five days works on

Here is the position this course takes, and every remaining day comes back to it:

A detector is not a boundary. The architecture is.

Anything an adaptive attack can walk through is not a defense line, it is a filter. A real boundary means that once untrusted input arrives, this agent is structurally incapable of doing the bad thing. The five days are ordered by how close each layer gets to that bar:

From tomorrow on, every day reports two numbers at once: ASR (attack success rate) and utility (task completion under attack). Any proposal that reports only the first is untrustworthy — change the agent to do nothing at all and ASR drops to zero instantly, taking utility with it.

Source Reading

Hands-On Lab

The lab is pure local computation — no model, no network. Today's skill is judging whether an agent is dangerous before you run it. Before you start, guess for yourself: which of deskmate's six tools carry each of the three edges? Run it afterwards, and the edge you got wrong is what you actually learned today. If you get stuck, read the comments in solution/src/model/; every function explains why it decides the way it does.

1. Run the solution's pnpm start and study the trifecta verdict for deskmate plus the risk table sorted by severity.
2. Go back to the starter and complete the trifecta test in trifecta.ts, remembering that user input is itself a source of untrusted content; when it is done, confirm all three edges list tool names.
3. Complete cheapestCut and the risk item generation in risks.ts, so every risk lands on an OWASP entry and carries an evidence line.
4. Look at the control config deskmate-no-send: deleting only the two send tools makes the critical item disappear, which is the plain demonstration of cutting one edge.
5. Change DESKMATE's userInputTrust to trusted and run again; confirm the untrusted content edge still holds, which shows the verdict is not hostage to one field. Finally run it against your own project's tool list and keep the output.

Interview Questions

Today's three questions are in the question bank below, focused on the lethal trifecta test, the difference between direct and indirect injection, and how to draw a trust boundary diagram in half an hour. Expand a question and read the analysis before the answer points — walking the derivation beats memorizing the points. The domestic and overseas frequency tags let you pick by target market.

Checklist and Tomorrow

• Use the lethal trifecta test to decide whether an agent has an injectable exfiltration path, and say which edge is cheapest to cut
• Draw an agent as a data-flow diagram with trust boundaries, labeling where each piece of data comes from
• Walk the OWASP LLM Top 10 and Agentic Top 10 against your own agent and come out with a list of risk items
• Explain why code review, dependency scanning and unit tests all miss this class of risk
• Say why severity is a property of the path rather than of a single point
• All 5 acceptance criteria of the lab pass, and you have run it once against your own project's tool list
• Answer at least 2 of the 3 interview questions without looking at the points

Tomorrow (D2) we stand the target up for real: we build an injection range around deskmate, attack it through all three injection surfaces — ticket body, page body, tool return — and then add four tiers of input-side defense in order: delimiters, provenance markers, spotlighting, a detector. After each tier we measure ASR and utility again. Measure before defending, for the same reason as today's "see the three edges before cutting": without knowing how often you are being breached now, you cannot judge whether a defense did anything. You will watch the first tier produce numbers identical to no defense at all, and you will watch four tiers of defense give the attack success rate back the moment one unseen case shows up — those two facts are precisely why D3 is about architectural defense.