Evaluation & Observability
in/guard/out vs Maxim AI
Maxim AI is a serious evaluation platform: agent simulation across personas, machine and human-in-the-loop evals, multi-turn session observability, alerting. All of it shares one property – it measures. in/guard/out’s job starts where measurement ends: standing in line with production traffic and deciding, per request and per tool call, whether this response ships, this payment executes, this run continues. A trace of the incident is not a block of the incident.
Side by side
| Capability | Maxim AI | in/guard/out |
|---|---|---|
| What it is | Simulation, evals, and observability for agents | A runtime control plane that enforces on live traffic |
| When it acts | Before release (simulation/evals) and after the fact (monitoring, alerts) | In line – before the response reaches the user or the tool call executes |
| A hallucinated figure | Scored in evals; alerted on in production | Caught deterministically and repaired or blocked on the wire |
| An ungrounded payment | Visible in the trace afterwards | Blocked before execution – the payee must appear in the run’s sources |
| A runaway agent loop | Alert fires while it burns | Budget gate stops the next call – cost capped mid-run, atomically |
| PII to the provider | Observed in traces | Screened to placeholders on the way in, restored on the way back |
| Tool-call policy | Traced and evaluated | Enforced: schemas, permission tiers, ordering, taint – per key, per run |
| Audit output | Dashboards and traces | A run graph of enforced decisions – what was allowed, repaired, blocked, and why |
Based on Maxim AI’s publicly available information. We’ve done our best to keep it accurate, but details may have changed since we last looked. If you’ve noticed an error, please email us.
Enforcement and evaluation compose
This is the least adversarial page on this site: teams should do both. Evals catch what you can reproduce before you ship; the wire catches what production does anyway. The two feed each other – in/guard/out is built against its own offline eval cases, and every violation it catches in production is a labeled, replayable case for your evaluation suite, whichever platform runs it. (Maxim also builds the Bifrost gateway – that lane is covered in vs Bifrost.)
Frequently asked questions
Does in/guard/out replace an eval platform?
No. Evals measure quality before you ship and trend it after; in/guard/out enforces policy on live traffic. Run both: the proxy’s production catches – each one a real failure with full context – make the best eval cases you will ever write.
Maxim observes multi-turn sessions – isn’t that the run graph?
It’s the observability half of it. A session trace shows what happened; in/guard/out’s run graph shows what happened and what was enforced – the budget that stopped step 14, the payment that was blocked, the number that was repaired – because the same layer that drew the graph made the decisions.
Can they run together?
Cleanly. in/guard/out sits on the wire; Maxim’s SDK instruments your app. Neither interferes with the other, and the proxy’s per-stage findings can be exported for analysis alongside your traces.
Evals find it in testing. The wire stops it in production.
in/guard/out enforces in line while your eval platform measures - and every production catch makes a new test case. We are running a limited demo - sign up and we will get you in as soon as we can.