Evaluation & Observability
in/guard/out vs ChatSee
ChatSee builds a Guardian Agent for enterprises running fleets of AI agents – the category Gartner now tracks under that name. It watches agent behaviour rather than uptime: semantic drift away from an agent’s mission, policy violations, missed escalations, tool misuse. Its distinctive idea is Failure Memory – an organisational record of what failed, why, and how it was fixed, which feeds regression tests, prompt adaptation and retraining data so the same failure does not come back. That is the reliability team’s lane, and a learning one.
in/guard/out works at the moment of the call. It doesn’t need to have seen a failure before to stop it: a payment whose IBAN appears in nothing the run read, a screened email address about to leave through a tool argument, a request that would push the run past its budget – each is decided deterministically, before it executes, and where a safe fix exists the output is repaired instead of blocked.
Side by side
| Capability | ChatSee | in/guard/out |
|---|---|---|
| What it is | A guardian-agent platform: behavioural monitoring, failure classification, remediation across an agent fleet | A guardrails proxy that decides each request and tool call on the wire, before it runs |
| How it acts | Prompt adaptation, circuit breakers and remediation learned from human preference | Deterministic checks on the request path – block, repair, or route to a person before the call executes |
| Agent spend | — | Hard per-run caps: each request reserves its worst-case cost against a durable ledger before it is sent |
| A payment’s payee and amount | Tool misuse detected | Must appear in something the run read – a figure the model wrote itself never counts as evidence |
| Private data | Unintended disclosures detected | Screened to placeholders before the provider sees it, restored in the response, and tracked so it can’t leave through a tool argument |
| Malformed output | Schema failures detected | JSON-schema validation with automatic repair |
| Wrong numbers in an answer | — | Deterministic grounding: arithmetic recomputed and altered figures corrected from the source, with no model call |
| Workflow ordering | Workflow drift detected | Tool-ordering rules compiled from an n8n export and enforced across the run |
| A refused call | Blocked | Refused with the problem, the remedy and whether a retry can succeed – so the agent can finish the task |
| What the checks cost you | — | Every guard-model call priced into the run, with per-stage latency on every request |
Based on ChatSee’s publicly available information. We’ve done our best to keep it accurate, but details may have changed since we last looked. If you’ve noticed an error, please email us.
Remembering a failure vs never shipping it
A failure memory gets better with every incident – which means it learns from incidents. That is the right tool for the long tail of behavioural failures: tone drift, a persona that wanders, an agent quietly drifting from its goal. It is the wrong shape for the failures that only have to happen once: the first payment to a payee an attacker planted, the first customer record sent to the wrong address, the first runaway loop. Those need a check that holds with no history at all – which is why in/guard/out grounds an action in what the run actually read, reserves the budget before the call, and tracks screened data through every tool argument.
The two compose. Point your agents through in/guard/out and every block, repair and approval lands in the run graph as a labelled, replayable case – exactly the raw material a failure-intelligence platform turns into taxonomy, regression tests and training data. One layer enforces on the wire; the other makes the fleet better over time.
Frequently asked questions
Is in/guard/out a ChatSee alternative?
Where they overlap – stopping an agent from doing something it shouldn’t – both act at runtime. The approach differs: ChatSee learns behavioural failure patterns across a fleet and steers future runs; in/guard/out enforces deterministic rules over the run’s own state on every call. Teams that want both learning and hard guarantees can run both.
ChatSee says it blocks unsafe behaviour too – what’s different?
What the decision is based on. A behavioural detector judges whether an action looks like a known failure. in/guard/out checks facts: did this IBAN appear in a source the run read, does this request fit in the reserved budget, is this argument a screened value. Those checks need no prior incident, cost no model call, and say exactly why a call was refused and how to fix it.
Does in/guard/out learn from past failures?
Partly. It learns a per-key behavioural baseline – normal tools, sequences and cost – and flags what is new, in shadow mode only. It does not build a failure taxonomy or generate retraining data; that is a failure-intelligence platform’s job, and in/guard/out’s run graph is a good source of cases for it.
Does it work with the agents we already run?
Yes. in/guard/out is OpenAI- and Anthropic-SDK-compatible, so an agent changes its base URL; steps that never touch an LLM go through the checkpoint API, with a drop-in n8n node, LangChain callback and Python SDK.
Learn from failures. Stop the next one.
Keep your failure intelligence and point your agents through in/guard/out - FIX mode observes first, so you can measure what it catches before enforcing anything. We are running a limited demo - sign up and we will get you in as soon as we can.