Learn
The LLM security checklist
A working checklist for teams shipping an LLM feature or agent to production. It is organized the way the risk actually flows – what enters the model, what leaves it, what it is allowed to do, and how you will know what happened – and each item states what “done” means, because “we have a filter” is not a state of doneness.
Use it honestly: most teams discover they are strong on one column and blank on another. The blank column is where the incident will come from.
Four checks on everything the model reads:
1. PII screened before egress
Done means: personal data is detected and replaced before the request leaves your boundary for the provider, and restored on the way back – not “our privacy policy says don’t paste PII.”
2. Injection scanning on user input
Done means: deterministic patterns plus a classifier score every prompt, with the results recorded – not a system-prompt plea to ignore malicious instructions.
3. Untrusted content treated as untrusted
Done means: retrieved documents, web pages, emails, and tool results are scanned before entering context. If your agent reads it, an attacker can write it.
4. Input abuse and topic policy
Done means: abusive or out-of-scope prompts are caught before they spend tokens, with the policy configurable per key or tenant.
Four checks on everything the model produces:
5. Grounding against sources
Done means: figures are matched to the source and arithmetic is recomputed on every response – deterministically, not by asking the same model to double-check itself.
6. Unsupported-claim detection
Done means: a judge independent of the generating model flags claims the material cannot support, fabricated references, and internal contradictions.
7. Format enforcement
Done means: structured output is validated and repaired, and unparseable output fails closed with an error code – your pipeline never ingests a silent poison value.
8. Content safety and leak detection
Done means: a safety cascade judges the output, and a canary in the system prompt makes instruction leakage deterministically detectable.
Three checks that only matter for agents – and matter more than everything above once you have one:
9. Tool policy and tiers
Done means: default-deny tool allowlists per key, schema-validated arguments, and risk tiers with human approval on the consequential ones.
10. Action grounding and taint
Done means: consequential arguments (payees, amounts, recipients) must appear in the run’s actual sources, and screened PII or secrets cannot leave through tool calls.
11. Budgets and loop guards
Done means: hard per-run ceilings on cost, steps, and tool calls, enforced before the next model call – not an alert that fires after the money is gone.
And the one everything else depends on:
12. Audit trail per request and per run
Done means: every request records which checks ran, what they found, and what action was taken; agent runs reconstruct into a single trace across model calls, tool calls, and workflow steps. When something goes wrong – or a regulator asks – the answer is a query, not an archaeology project.
A blunt rubric: 0–4 items done, you are relying on the model’s good behavior and your users’ good faith; 5–8, you have real defenses with real gaps – prioritize the empty column; 9–12, you are ahead of most production deployments, and the remaining work is tuning thresholds on evidence rather than adding layers.
The proxy exists because items 1–12 are one product on the wire rather than twelve integration projects: point your SDK at it, turn checks on in observe mode, and work down the list with data instead of dread.
Where should a team start with LLM security?
With the two checks that prevent unrecoverable harm: PII screening on inputs (a disclosure cannot be un-disclosed) and – if you run agents – tool policy with budgets (an executed payment cannot be un-executed). Detection quality everywhere else can be tuned iteratively; those two are one-way doors.
Is this checklist aligned with OWASP?
It covers the runtime OWASP LLM Top 10 risks – injection, disclosure, output handling, excessive agency, system prompt leakage, misinformation, unbounded consumption – organized by data flow instead of risk number. Our OWASP page maps each risk individually.
How do I roll out checks without breaking production?
Observe first: run every check in flag-only mode, review a week or two of findings, tune thresholds, then flip individual checks to enforcement. The proxy’s FIX mode exists precisely for that sequence.
Do I need all twelve for an internal tool?
Scope to consequence. An internal summarizer with no tools needs the input and output columns. Anything holding credentials or tools needs the action column regardless of who its users are – internal agents fail just as expensively.
See the checks run on your own traffic.
One base-URL change puts every request behind the guardrail pipeline. We’re running a limited demo – sign up and we’ll get you in as soon as we can.