AI gateway
in/guard/out vs Bifrost
Bifrost (by Maxim AI) is an open-source Go gateway built for speed – microsecond overhead, a thousand-plus models behind one OpenAI-compatible API, failover and semantic caching – with guardrails wired in as gateway-layer integrations to external checkers. That last part is the difference in kind: Bifrost’s guardrails are per-request calls out to someone else’s classifier; in/guard/out ships the checks natively, correlates requests into runs, and repairs instead of only blocking.
Side by side
| Capability | Bifrost | in/guard/out |
|---|---|---|
| Unit of governance | The request and the virtual key | The run – requests, tool calls, and checkpoints in one governed session |
| Guardrail model | CEL rules routing content to external providers (Bedrock Guardrails, Azure Content Safety, Patronus, …) plus regex/secrets checks | Native pipeline: deterministic grounding, claim audit, PII screen/restore, format repair, shields – no per-check vendor bill |
| Hallucination checks | Via third-party guardrail providers | Deterministic first – arithmetic and number drift at zero tokens – then claim audit and judges as escalation |
| When a check fails | Block or mask at the gateway | Repair where safe: fix the JSON, correct the number, restore the PII |
| PII handling | Detection/redaction via integrations | Screened to placeholders and restored on the response; per-tool egress rules for agents |
| Agent budget enforcement | Per-key governance and rate limits | Per-run hard caps – cost, steps, tool calls – reserved atomically before the next call spends |
| Action grounding & taint | — | Payees grounded in the run’s sources; screened PII can’t exit via tool arguments |
| Workflow conformance | — | Tool-ordering policies, auto-compiled from an n8n export |
| Steps that never touch an LLM | — | Checkpoint API + n8n node, LangChain callback, Python SDK |
| Run visibility | Logs and metrics per request | A reconstructed run graph with cost burn-down and coverage |
Fast routing and real governance are different products
Bifrost’s pitch is that the gateway should cost microseconds – and for routing, it should. But a guardrail that calls out to a cloud moderation API per request isn’t microseconds, and more importantly it’s stateless: no accumulated run, no budget that survives across calls, no memory that the payee in this payment never appeared in anything the agent read. in/guard/out spends its milliseconds on the checks themselves – deterministic tiers first, model checks concurrently – and reports the guardrail tax per stage, per request, so the trade is visible instead of vibes.
Because both sides speak the OpenAI API, the practical answer is the same as with any router: chain them. Bifrost picks the model and absorbs provider chaos; in/guard/out governs the run and documents it.
Frequently asked questions
Bifrost already supports guardrail providers – why add in/guard/out?
Bifrost’s guardrails are adapters: each check is a per-request call to an external service, each with its own contract and bill, and none of them share run state. in/guard/out ships the checks natively – deterministic grounding costs zero tokens – and adds the run-scoped layer no per-request adapter can do: budgets, loop detection, action grounding, taint, conformance.
Isn’t in/guard/out slower than a microsecond gateway?
The gateway hop is microseconds; the guardrail work is the product. Deterministic checks run sub-millisecond, model-based checks run concurrently and only when enabled, and every stage’s cost in time and dollars is measured and shown per request – so you decide where enforcement is worth its latency, with numbers.
Can I run Bifrost and in/guard/out together?
Yes – both are OpenAI-compatible on both ends. Route with Bifrost, point its deployment at in/guard/out, and every routed path picks up run budgets, tool policy, grounding, and the run graph without app changes.
Microseconds route the call. Governance decides it.
Bifrost is OpenAI-compatible on both ends, so in/guard/out chains behind it with one base-URL change. We are running a limited demo - sign up and we will get you in as soon as we can.