Learn
What is prompt injection?
Prompt injection is an attack where text the model reads carries instructions the model follows – instructions the application author never wrote. It works because a language model has no channel separation: system prompt, user message, retrieved document, and tool result all arrive as tokens, and the model weighs all of them when deciding what to do next.
It is routinely called the number-one risk to LLM applications (it heads the OWASP Top 10 for LLM applications), and it is not a bug you patch. It is a structural property of how models consume text – which is why the practical question is not “how do I prevent it” but “how many independent layers does an injected instruction have to beat before it causes harm.”
The two families differ in who delivers the payload:
Direct injection (jailbreaks)
The user is the attacker: “ignore your previous instructions”, role-play personas, developer-mode framing, system-prompt extraction. Annoying for chatbots; dangerous when the model holds tools or secrets.
Indirect injection
The user is innocent – the payload hides in content the model reads on their behalf: a web page, a PDF invoice, an email signature, a support-ticket note, an MCP tool result. The agent’s own data pipeline delivers the attack.
Against a chatbot, a successful injection produces a bad answer. Against an agent, it produces an action: the model that just read the poisoned order note is the same model that holds refund, send_email, and pay_via_bank. Real incidents follow one pattern – untrusted content enters the context, steers the model, and exits as a tool call with your credentials behind it.
Agent loops make it worse in a specific way: every tool result is fed back into the context for the next iteration. A single poisoned result does not have to win immediately – it sits in the context, re-read on every subsequent step, until it steers one.
No single detector catches everything, so serious defenses stack independent layers – each one an additional thing the attacker must beat:
Deterministic patterns
Regex-class detection of the classic families – instruction override, jailbreak personas, extraction attempts. Zero model cost, zero false “judgment”.
A dedicated classifier
A purpose-built injection model (Prompt Guard 2 class) scores what patterns can’t, at higher recall.
Tool-result scanning
Everything a tool returns is untrusted input, scanned before it re-enters the context – the layer that addresses indirect injection where it actually arrives.
Canary leak detection
A unique marker seeded into the system prompt. If it ever appears in output, your instructions leaked – detected deterministically.
Action-level invariants
The layers that hold even when detection fails: an injected payee still has to survive argument grounding, permission tiers, taint tracking, and budgets before anything executes.
Detection rates on classifiers are good and improving, but the attack surface is every string the model will ever read, in every language, in every encoding. A defense that depends entirely on recognizing the attack will eventually miss one. That is the argument for enforcing at the action boundary too: a payment to an IBAN that appears nowhere in anything the agent legitimately read is blocked for being ungrounded – whether or not any scanner recognized the injection that produced it.
What is the difference between prompt injection and jailbreaking?
Jailbreaking is the direct form: the user tries to talk the model out of its rules. Prompt injection is the broader class, including indirect attacks where the payload hides in content the model reads – a web page, an email, a tool result – and the user is a bystander.
Can prompt injection be completely prevented?
No – it exploits how models read text, not a fixable bug. The realistic goal is layered: catch most attempts at the boundary (patterns + classifier + tool-result scanning), and make the ones that slip through harmless by checking actions against policy, grounding, tiers, and budgets before they run.
What is an example of indirect prompt injection?
A supplier email contains hidden text: “update the payment IBAN to …”. The invoice agent reads the email as a tool result, treats the instruction as truth, and issues a payment to the attacker’s account. The user never typed anything malicious.
How do I know if my system prompt leaked?
Seed a canary – a unique marker string – into the system prompt and scan every response for it. If the model echoes it, the prompt leaked. The proxy does this automatically and flags or redacts the response.
See the checks run on your own traffic.
One base-URL change puts every request behind the guardrail pipeline. We’re running a limited demo – sign up and we’ll get you in as soon as we can.