Choosing an approach
Library, gateway, firewall, or proxy?
“LLM guardrails” names at least four different product shapes, and they are not interchangeable – each enforces at a different point, with different authority. Here is the honest map, including where a proxy is not the right tool.
Specific products, compared
Weighing us against a specific product? Start here – one page each, stating what the other tool is genuinely good at, then the run-governance layer it doesn’t attempt, row by row. For the big-picture category map, the four shapes and the capability matrix are below.
Bifrost
Bifrost is Maxim AI’s fast open-source Go gateway with guardrail integrations at the gateway layer.
OpenRouter Guardrails
OpenRouter’s Workspace Guardrails cap spend, restrict models, and screen requests for injection and PII.
Databricks Unity Gateway
Databricks Unity Gateway governs model, agent, and MCP access with budgets, guardrails, and service policies.
Patronus Lynx
Patronus Lynx is an open hallucination-detection model.
Galileo Luna
Galileo, now Cisco’s Splunk Agent Observability, scores hallucination and quality with its Luna models.
Lens
Lens (lens.cx) builds answers from its own retrieval and counts the sources that support or dispute each claim.
Runlayer
Runlayer governs enterprise MCP access: identity, catalog, threat detection.
MintMCP
MintMCP curates who sees which MCP tools – bundles, SCIM, tool-level policy.
Obot
Obot is an open-source MCP platform: hosting, registry, gateway, RBAC.
MCPX (Lunar.dev)
Lunar.dev’s MCPX scopes which tools each team sees.
Zuplo MCP Gateway
Zuplo’s MCP Gateway nails spec-correct OAuth and credential brokering for MCP servers.
TrueFoundry
TrueFoundry unifies LLM and MCP infrastructure – fast gateway, hosting, rate limits.
Praesidia
Praesidia leads with the MCP authorization spec – OAuth 2.1, PKCE, resource-bound tokens.
Virtue AI
Virtue AI (now part of Fortinet) builds fast multimodal guardrails, agent red-teaming, and shadow-AI discovery.
Straiker
Straiker red-teams AI agents and blocks the attacks it finds at runtime.
ChatSee
ChatSee’s Guardian Agent monitors agent behaviour, classifies failures and steers future runs.
The four shapes
Validation library
Checks run inside your application process, wired in with code.
Maximum flexibility; no extra infrastructure; great for a single, deeply customized app.
Every app integrates and updates separately; uninstrumented apps and low-code tools get nothing; no cross-request view, so agent runs, loops, and budgets are invisible; enforcement lives inside the process it is meant to police.
in/guard/out embeds respected open-source components (Presidio for PII, Qwen3Guard for content safety) and runs them as shared infrastructure – one integration covering every app, plus the cross-request run view a library can’t reach.
AI gateway
A proxy focused on routing, caching, key management, and rate limits – with moderation filters bolted on.
One integration point; solid for cost routing, failover, and provider abstraction.
Safety is a feature, not the product: typically keyword or moderation-API filters on input and output. No grounding of answers in sources, and agent controls stay shallow – name rules and spend counters, not grounded actions or durable run budgets.
in/guard/out chains behind your gateway: it routes, we enforce – grounding, PII restore, tool policy, and durable run budgets the gateway doesn’t attempt.
Prompt firewall
A detection layer classifying inputs (and sometimes outputs) for injection and abuse.
Strong classifiers for the injection problem specifically; useful signal for a SOC.
Detection without enforcement of consequences: it can say “this looks like an attack” but not “this payment is ungrounded, block it”. Hallucination, format, PII restore, and agent budgets are out of frame.
in/guard/out runs the injection scan too, but turns detection into a decision – the ungrounded payment is blocked and the drifted number repaired, not just flagged.
Eval & observability platform
Measures quality on test sets and traces production calls for later analysis.
Essential for development: regression suites, prompt comparison, failure analysis.
After the fact by design. A trace of the incident is not a block of the incident – nothing stands in line to stop the response or the tool call that fails the check.
in/guard/out is the runtime half: evals measure before you ship, we enforce after – and every production catch becomes a new eval case.
All four, on the wire
Put together, that is one layer in both directions, with enforcement authority: it screens and restores PII, grounds answers in their sources, validates and repairs output, and – for agents – checks every tool call against policy, tiers, grounding, taint, and budgets before it runs. It embeds the best of the open ecosystem (Presidio, Qwen3Guard) and operates it as infrastructure: one integration, one policy surface, one audit trail across the LLM boundary, workflow checkpoints, and the MCP gateway.
The matrix, at a glance
| Capability | Library | AI gateway | Prompt firewall | Eval platform | in/guard/out |
|---|---|---|---|---|---|
| Input filters (PII, injection patterns) | ✓ per app | ✓ basic | ✓ strong | — | ✓ every app |
| PII restored after the model call | rare | opt-in in some | — | — | ✓ screen & restore |
| Output grounding & fact-checking | partial, per app | — | — | measures offline | ✓ enforced in line |
| Format validation & repair | ✓ per app | — | — | — | ✓ on the wire |
| Tool-call policy & permission tiers | framework-specific | name/regex rules | — | — | ✓ tiers + schemas |
| Cross-request runs, budgets, loop detection | — | spend caps | — | — | ✓ per run |
| Non-LLM step gating (checkpoints) | — | — | — | — | ✓ |
| MCP inspection & manifest pinning | — | — | — | — | ✓ gateway |
| One audit trail across apps & surfaces | — | request logs | alerts | traces | ✓ run graph |
| Routing, caching, provider failover | — | ✓ home turf | — | — | — composes with one |
| Offline regression testing & evals | — | — | — | ✓ home turf | — findings feed it |
“Home turf” rows are deliberate: routing belongs to gateways and offline evals to eval platforms – the proxy composes with both rather than replacing them.
Match the tool to the job
Keep PII/PHI out of model providers
A proxy is the natural fit: screening must happen before the request leaves your boundary, and restore must happen on the way back – on every app, including the ones nobody instrumented.
Stop hallucinated figures from shipping
Needs in-line output enforcement with access to the source material in the request – deterministic grounding plus a judge, with block/repair authority. Libraries can do parts per-app; evals only measure it.
Control what an AI agent may do
Needs cross-request state: runs, budgets, loop detection, tool history for sequence policies. Only something on the wire sees all of an agent’s requests; in-process libraries see one call at a time.
Secure MCP tool traffic
Needs an MCP-speaking gateway between agent and servers, where a deny is a protocol error the client cannot ignore – plus manifest pinning against rug pulls.
Route, cache, and manage provider keys
An AI gateway’s home turf. in/guard/out focuses on enforcement; it composes with a gateway (either side of it) if routing is a separate concern in your stack.
Regression-test prompts before release
An eval platform’s job, and in/guard/out does not replace it. Evals catch what you can reproduce; in/guard/out catches what happens anyway in production.
Frequently asked questions
Is this a NeMo Guardrails / Guardrails AI alternative?
It is the deployment alternative: enforcement on the wire instead of a library in each app. Not a rip-and-replace of the ecosystem, either – in/guard/out embeds respected open-source components (Presidio for PII, Qwen3Guard for content safety) and operates them behind one base URL, adding what a library cannot: cross-request runs, budgets, tool-call enforcement, and one audit trail.
Do I need a guardrails proxy if I already use an AI gateway?
They solve different jobs and compose: the gateway handles routing, caching, and key management; in/guard/out enforces safety – grounding, PII screen/restore, tool policy, budgets. Chain them if you need both.
Proxy vs library – what is the honest trade-off?
in/guard/out adds a network hop and a component to run, and per-app custom logic is easier in code. The library costs you N integrations, no coverage for uninstrumented traffic, and no cross-request view of agent runs. Teams with one app and strong engineers often start with a library; teams with many apps, agents, or compliance duties end up on the wire.
Can in/guard/out replace offline evals?
No, and it is not trying to. Evals measure quality before you ship; in/guard/out enforces policy after you ship. The overlap is that proxy findings – what was actually caught in production – make an excellent source of new eval cases.
Decide on evidence: run it next to what you have.
FIX mode observes without blocking, so you can measure what in/guard/out would have caught before changing anything. We’re running a limited demo.