Learn
What are LLM guardrails?
LLM guardrails are runtime checks that sit around a language model and enforce rules the model itself cannot guarantee: what may go in, what may come out, and – for agents – what actions may run. The model stays probabilistic; the guardrails are the deterministic layer that decides whether its input and output are acceptable.
They exist because model-side safety is not application-side safety. The provider moderates for its own policies, not yours: it will not stop your customer’s card number from entering the prompt, a subtly wrong figure from leaving in a summary, or an agent from paying an invoice to an account it never actually read. Those failures belong to the application boundary – which is where guardrails run.
Every guardrail, whatever the vendor calls it, acts at one of three points in the request lifecycle:
Input guardrails
Run before the model sees the request: PII screening, injection detection, topic and abuse filters. Their job is to keep sensitive data and hostile instructions out of the context.
Output guardrails
Run on what the model produced, before your app or user sees it: grounding checks, format validation, content safety, system-prompt-leak detection.
Action guardrails
Run on what an agent is about to do: tool policy, permission tiers, argument grounding, taint tracking, loop detection, and per-run budgets. These only exist at runtime – no amount of prompt engineering substitutes for them.
Three adjacent practices get conflated. Evaluations measure quality offline, on test sets, before you ship – they tell you how often a model fails, but they cannot stop the failure that happens in production at 2 a.m. Provider moderation enforces the model vendor’s acceptable-use policy on their side of the wire – it knows nothing about your compliance obligations or your tools. Guardrails are the third thing: per-request enforcement of your policy, in line, with the authority to block, repair, or require approval before the response or the action goes through.
A mature setup uses all three. But when something must not happen – PII reaching a provider, an ungrounded payment executing – only an in-line guardrail can make that guarantee.
A guardrail that only blocks is a guardrail teams turn off after the first false positive. The pattern that survives production is graduated: run every check in an observe mode first (flag and record, change nothing), review what would have been caught, then flip individual checks to enforcement once you trust them.
The proxy ships this as PREVENT and FIX. In FIX mode violations are repaired where a safe repair exists – wrong arithmetic corrected, profanity censored, malformed JSON re-parsed – and otherwise flagged and recorded. In PREVENT mode the request fails closed with a 422 your code can handle. Modes apply per key, per user, or per request, so one team can enforce while another still observes.
Open-source libraries (NeMo Guardrails, Guardrails AI, LLM Guard) put the checks inside your application process: maximum flexibility, but every app integrates, configures, and updates them separately – and anything the library misses ships with that app.
A proxy puts the checks on the wire between all of your apps and the model: one integration (a base URL), one policy surface, one audit trail, and coverage for traffic you did not instrument – including the agent frameworks and low-code tools that never expose a hook for a validation library. The trade-off is a network hop and a piece of infrastructure to run; the comparison page walks through when each approach wins.
What are LLM guardrails in simple terms?
Automated checks that run on every request to and response from a language model, and block, repair, or flag anything that violates your rules – leaked personal data, hallucinated figures, unsafe content, or an agent action your policy does not allow.
Why aren’t the model’s built-in safety features enough?
The provider enforces its policy, not yours. It does not know which data you may not send, which figures the answer must match, which tools your agent may call, or what your regulator requires you to log. Those checks have to run at your boundary.
Do guardrails add latency?
Some do, some don’t. Deterministic checks (pattern matching, arithmetic verification, format validation) add milliseconds; classifier and judge-model checks add a model call. The honest answer is a measured one – the proxy times every stage per request and shows the overhead in the dashboard rather than quoting one number.
What guardrails should an AI agent have that a chatbot doesn’t?
Action-level ones: tool allow/deny policy, permission tiers with approval for risky calls, grounding of consequential arguments (payees, amounts, recipients) in the run’s sources, taint tracking on what leaves through tool calls, loop detection, and per-run cost budgets.
See the checks run on your own traffic.
One base-URL change puts every request behind the guardrail pipeline. We’re running a limited demo – sign up and we’ll get you in as soon as we can.