Agent evaluation & observability
in/guard/out vs Maxim AI
Maxim AI is a serious evaluation platform: agent simulation across personas, machine and human-in-the-loop evals, multi-turn session observability, alerting. All of it shares one property – it measures. in/guard/out’s job starts where measurement ends: standing in line with production traffic and deciding, per request and per tool call, whether this response ships, this payment executes, this run continues. A trace of the incident is not a block of the incident.
Side by side
| Capability | Maxim AI | in/guard/out |
|---|---|---|
| What it is | Simulation, evals, and observability for agents | A runtime control plane that enforces on live traffic |
| When it acts | Before release (simulation/evals) and after the fact (monitoring, alerts) | In line – before the response reaches the user or the tool call executes |
| A hallucinated figure | Scored in evals; alerted on in production | Caught deterministically and repaired or blocked on the wire |
| An ungrounded payment | Visible in the trace afterwards | Blocked before execution – the payee must appear in the run’s sources |
| A runaway agent loop | Alert fires while it burns | Budget gate stops the next call – cost capped mid-run, atomically |
| PII to the provider | Observed in traces | Screened to placeholders on the way in, restored on the way back |
| Tool-call policy | Traced and evaluated | Enforced: schemas, permission tiers, ordering, taint – per key, per run |
| Audit output | Dashboards and traces | A run graph of enforced decisions – what was allowed, repaired, blocked, and why |
Enforcement and evaluation compose
This is the least adversarial page on this site: teams should do both. Evals catch what you can reproduce before you ship; the wire catches what production does anyway. The two feed each other – in/guard/out ships its own offline eval harness, and every violation it catches in production is a labeled, replayable case for your evaluation suite, whichever platform runs it. (Maxim also builds the Bifrost gateway – that lane is covered in vs Bifrost.)
Frequently asked questions
Does in/guard/out replace an eval platform?
No. Evals measure quality before you ship and trend it after; in/guard/out enforces policy on live traffic. Run both: the proxy’s production catches – each one a real failure with full context – make the best eval cases you will ever write.
Maxim observes multi-turn sessions – isn’t that the run graph?
It’s the observability half of it. A session trace shows what happened; in/guard/out’s run graph shows what happened and what was enforced – the budget that stopped step 14, the payment that was blocked, the number that was repaired – because the same layer that drew the graph made the decisions.
Can they run together?
Cleanly. in/guard/out sits on the wire; Maxim’s SDK instruments your app. Neither interferes with the other, and the proxy’s per-stage findings can be exported for analysis alongside your traces.
Evals find it in testing. The wire stops it in production.
in/guard/out enforces in line while your eval platform measures - and every production catch makes a new test case. We are running a limited demo - sign up and we will get you in as soon as we can.