Choosing an approach
Library, gateway, firewall, or proxy?
“LLM guardrails” names at least four different product shapes, and they are not interchangeable – each enforces at a different point, with different authority. Here is the honest map, including where a proxy is not the right tool.
Specific products, compared
Weighing us against a specific product? Start here – one page each, stating what the other tool is genuinely good at, then the run-governance layer it doesn’t attempt, row by row. For the big-picture category map, the four shapes and the capability matrix are below.
vs Runlayer
Runlayer governs enterprise MCP access: identity, catalog, threat detection.
vs MintMCP
MintMCP curates who sees which MCP tools – bundles, SCIM, tool-level policy.
vs Obot
Obot is an open-source MCP platform: hosting, registry, gateway, RBAC.
vs MCPX (Lunar.dev)
Lunar.dev’s MCPX scopes which tools each team sees.
vs Zuplo MCP Gateway
Zuplo’s MCP Gateway nails spec-correct OAuth and credential brokering for MCP servers.
vs TrueFoundry
TrueFoundry unifies LLM and MCP infrastructure – fast gateway, hosting, rate limits.
vs Praesidia
Praesidia leads with the MCP authorization spec – OAuth 2.1, PKCE, resource-bound tokens.
The four shapes
Validation library
Checks run inside your application process, wired in with code.
Maximum flexibility; no extra infrastructure; great for a single, deeply customized app.
Every app integrates and updates separately; uninstrumented apps and low-code tools get nothing; no cross-request view, so agent runs, loops, and budgets are invisible; enforcement lives inside the process it is meant to police.
in/guard/out embeds respected open-source components (Presidio for PII, NeMo Guardrails for topic rails) and runs them as shared infrastructure – one integration covering every app, plus the cross-request run view a library can’t reach.
AI gateway
A proxy focused on routing, caching, key management, and rate limits – with moderation filters bolted on.
One integration point; solid for cost routing, failover, and provider abstraction.
Safety is a feature, not the product: typically keyword or moderation-API filters on input and output. No grounding of answers in sources, and agent controls stay shallow – name rules and spend counters, not grounded actions or durable run budgets.
in/guard/out chains behind your gateway: it routes, we enforce – grounding, PII restore, tool policy, and durable run budgets the gateway doesn’t attempt.
Prompt firewall
A detection layer classifying inputs (and sometimes outputs) for injection and abuse.
Strong classifiers for the injection problem specifically; useful signal for a SOC.
Detection without enforcement of consequences: it can say “this looks like an attack” but not “this payment is ungrounded, block it”. Hallucination, format, PII restore, and agent budgets are out of frame.
in/guard/out runs the injection scan too, but turns detection into a decision – the ungrounded payment is blocked and the drifted number repaired, not just flagged.
Eval & observability platform
Measures quality on test sets and traces production calls for later analysis.
Essential for development: regression suites, prompt comparison, failure analysis.
After the fact by design. A trace of the incident is not a block of the incident – nothing stands in line to stop the response or the tool call that fails the check.
in/guard/out is the runtime half: evals measure before you ship, we enforce after – and every production catch becomes a new eval case.
All four, on the wire
Put together, that is one layer in both directions, with enforcement authority: it screens and restores PII, grounds answers in their sources, validates and repairs output, and – for agents – checks every tool call against policy, tiers, grounding, taint, and budgets before it runs. It embeds the best of the library ecosystem (Presidio, NeMo Guardrails) and operates it as infrastructure: one integration, one policy surface, one audit trail across the LLM boundary, workflow checkpoints, and the MCP gateway.
The matrix, at a glance
| Capability | Library | AI gateway | Prompt firewall | Eval platform | in/guard/out |
|---|---|---|---|---|---|
| Input filters (PII, injection patterns) | ✓ per app | ✓ basic | ✓ strong | — | ✓ every app |
| PII restored after the model call | rare | opt-in in some | — | — | ✓ screen & restore |
| Output grounding & fact-checking | partial, per app | — | — | measures offline | ✓ enforced in line |
| Format validation & repair | ✓ per app | — | — | — | ✓ on the wire |
| Tool-call policy & permission tiers | framework-specific | name/regex rules | — | — | ✓ tiers + schemas |
| Cross-request runs, budgets, loop detection | — | spend caps | — | — | ✓ per run |
| Non-LLM step gating (checkpoints) | — | — | — | — | ✓ |
| MCP inspection & manifest pinning | — | — | — | — | ✓ gateway |
| One audit trail across apps & surfaces | — | request logs | alerts | traces | ✓ run graph |
| Routing, caching, provider failover | — | ✓ home turf | — | — | — composes with one |
| Offline regression testing & evals | — | — | — | ✓ home turf | — findings feed it |
“Home turf” rows are deliberate: routing belongs to gateways and offline evals to eval platforms – the proxy composes with both rather than replacing them.
Match the tool to the job
Keep PII/PHI out of model providers
A proxy is the natural fit: screening must happen before the request leaves your boundary, and restore must happen on the way back – on every app, including the ones nobody instrumented.
Stop hallucinated figures from shipping
Needs in-line output enforcement with access to the source material in the request – deterministic grounding plus a judge, with block/repair authority. Libraries can do parts per-app; evals only measure it.
Control what an AI agent may do
Needs cross-request state: runs, budgets, loop detection, tool history for sequence policies. Only something on the wire sees all of an agent’s requests; in-process libraries see one call at a time.
Secure MCP tool traffic
Needs an MCP-speaking gateway between agent and servers, where a deny is a protocol error the client cannot ignore – plus manifest pinning against rug pulls.
Route, cache, and manage provider keys
An AI gateway’s home turf. in/guard/out focuses on enforcement; it composes with a gateway (either side of it) if routing is a separate concern in your stack.
Regression-test prompts before release
An eval platform’s job, and in/guard/out does not replace it. Evals catch what you can reproduce; in/guard/out catches what happens anyway in production.
Frequently asked questions
Is this a NeMo Guardrails / Guardrails AI alternative?
It is the deployment alternative: enforcement on the wire instead of a library in each app. Not a rip-and-replace of the ecosystem, either – in/guard/out embeds respected open-source components (Presidio for PII, NeMo Guardrails for topic rails) and operates them behind one base URL, adding what a library cannot: cross-request runs, budgets, tool-call enforcement, and one audit trail.
Do I need a guardrails proxy if I already use an AI gateway?
They solve different jobs and compose: the gateway handles routing, caching, and key management; in/guard/out enforces safety – grounding, PII screen/restore, tool policy, budgets. Chain them if you need both.
Proxy vs library – what is the honest trade-off?
in/guard/out adds a network hop and a component to run, and per-app custom logic is easier in code. The library costs you N integrations, no coverage for uninstrumented traffic, and no cross-request view of agent runs. Teams with one app and strong engineers often start with a library; teams with many apps, agents, or compliance duties end up on the wire.
Can in/guard/out replace offline evals?
No, and it is not trying to. Evals measure quality before you ship; in/guard/out enforces policy after you ship. The overlap is that proxy findings – what was actually caught in production – make an excellent source of new eval cases.
Decide on evidence: run it next to what you have.
FIX mode observes without blocking, so you can measure what in/guard/out would have caught before changing anything. We’re running a limited demo.