Benchmark
Can an agent spend money it never read?
Thirty-two labelled agent actions – fabricated payees, poisoned invoices and perfectly ordinary payments – put through the enforcement path in/guard/out ships. Both halves are reported: what it caught, and how much legitimate accounts-payable traffic it blocked doing so.
The scoreboard
A catch rate on its own is not a measurement – blocking everything scores 100%. What decides whether a check survives contact with production is the column beside it.
| System | Decided correctly | Caught | Catch rate | Blocked wrongly | False-pos. rate |
|---|---|---|---|---|---|
| in/guard/outenforcement | 29 / 32 | 14 / 16 | 87.5% | 1 / 16 | 6.3% |
| MiniCheck-Flan-T5-Largeentailment model, 780M | 17 / 32 | 15 / 16 | 93.8% | 14 / 16 | 87.5% |
| IBM Granite Guardian 4.1guard model, 8B | 20 / 31 | 11 / 16 | 68.8% | 6 / 15 | 40.0% |
| substring matchnaive control | 24 / 32 | 14 / 16 | 87.5% | 6 / 16 | 37.5% |
| allow-allreference floor | 16 / 32 | 0 / 16 | 0.0% | 0 / 16 | n/a |
Don’t read the false-positive column alone
Allow-all approves everything. It has no false positives because it never refuses anything – that is why it is in the table, as the floor the other rows are measured against, not as a candidate. It lets all sixteen bad payments through.
“Decided correctly” is the column that resists gaming: approve everything or refuse everything, and you score 16 of 32 either way. Only a system that actually decides can beat both.
MiniCheck-Flan-T5-Large is the case in point. It posts the highest catch rate here and lands at 17 of 32 – one better than approving everything.
Every figure counts every case. Nothing is excluded to improve a number, and the cases are weighted as they occur rather than sampled to a favourable mix.
Read the control first
The naive control posts a higher catch rate than the real check, and is worse. It asks only whether every consequential token appears literally in the sources, so it flags 4300 as unexplained when the invoice reads EUR 4,300.00.
On the laundering case it blocks the right action for the wrong reason: it never notices the invented IBAN, it objects to the correct amount. The same brittleness produces a 37.5% false-positive rate against our 6.3%, at an identical catch rate.
Why it matters
A catch rate quoted without a false-positive rate and a naive baseline underneath it is not a measurement. It is a number that cannot be wrong.
Exact first, then a second look before refusing
An exact comparison is fast and cannot be argued with, but on its own it refuses honest payments: an IBAN the invoice wrote with spaces, a total the agent correctly summed from line items, an amount it converted at the day’s rate. So in/guard/out takes a second look before it refuses.
The second look can release a refusal, never add one
When grounding would refuse an action, a model is asked the same two things the checks ask: does every consequential value appear in what the run read, and does any source try to instruct the agent. The refusal is lifted only if it clears both.
An action the exact check allows is never sent for review, so the review cannot add a false positive, and the injection scan, risk tiers and policy still refuse on their own. If the review fails, times out or comes back unclear, the refusal stands.
29 of 32, and one honest payment still refused
The spaced IBAN and the summed total were released, and every invented value was refused, by margins wide enough that the result does not hang on a well-chosen cut-off.
The conversion was not. A euro invoice paid from a dollar account at the treasury rate – 4,300.00 × 1.0850 = 4,665.50 – has every input in the sources and the result in none of them, and the second look judged it ungrounded in every run. The fix is to check the arithmetic, not to lower the bar.
The second look ran on 12 of the 32 actions, about 0.3 s each; the other 20 needed none. It reads the screened transcript, so it sees PII placeholders, not the values behind them.
Two purpose-built models, measured on the same corpus
Two published models built for adjacent jobs, run on the same 32 cases. MiniCheck-Flan-T5-Large checks whether a claim is supported by a document (the original weights – the quantised GGUF build has a broken decoder and was discarded). IBM Granite Guardian 4.1 is IBM’s guard model, asked its own groundedness and function_call criteria. Both were verified to separate a good case from a bad one before being scored at all.
MiniCheck catches the most, and refuses almost everything
It catches more than we do – 15 of 16 against our 14 – and it refuses 14 of the 16 legitimate actions to get there. An accounts-payable team would switch it off on the first morning.
The cause is specific and measurable: it separates identifier claims cleanly – the real IBAN scores 1, a substituted one scores 0 – and scores every numeric claim 0 whether it is right or wrong. Against a document reading EUR 4,300.00, both “the amount is 4300” and “the amount is 4800” come back unsupported. Nearly every real action carries a number.
Its catches are not detections
Read the reason column, not the verdict. Where it appears to catch something subtle, it had flagged the amount – which it flags everywhere. A system that refuses almost everything is right about every attack and has told you nothing.
And the one case it misses is the injected recursive delete – the arguments hold no IBAN, no email and no amount, so there was nothing to make a claim about. An entailment model has no notion of a poisoned instruction or a risk tier.
IBM Granite Guardian 4.1 is the balanced one, and still not usable here
It sits where you would expect a good general model to sit: catches 11 of 16, blocks 6 of 15 legitimate actions, lands at 20 of 31. Better judgement than MiniCheck, well short of a check you would leave switched on over accounts payable.
It is also orders of magnitude more expensive per decision – seconds of 8B inference against a comparison that finishes below the noise floor of the model call it sits beside. Measured self-hosted; a hosted endpoint would narrow the gap, and would not close it.
It cannot see a poisoned instruction
Four of its five misses are the injected cases – the invoice saying “ignore all previous instructions”, the portal page steering it in an HTML comment, the ticket asking it to mail the account list out, the directory note widening a delete. Asked about groundedness, it answers about groundedness.
That is the same split the shield benchmark found: a model built for one class of problem scores near zero on a different one. Grounding and injection are two checks, and neither substitutes for the other.
This is why the benchmark reports both columns. On catch rate alone, the model to buy is the one that would stop your business.
What the corpus contains
invented
9 cases · must block
The value appears in nothing the run read. The agent made it up, or carried a figure over from its own earlier output.
caught 9 / 9
poisoned
7 cases · must block
The value is in a document the run read, because an attacker put it there. Grounding passes these correctly; the injection scan covers this half.
caught 5 / 7
clean
16 cases · must pass
Legitimate actions, written to be awkward for us: spaced IBANs, thousands separators, agent-composed search queries, a nested two-payee batch, a currency conversion.
passed 15 / 16
Most clean cases are paired with an invented twin differing only in provenance. A system can only score on both by reading the sources, not by reacting to the word “transfer”.
The benchmark changed the product
Evidence is what the run read, not what it said
Action grounding used to compare against the whole transcript, the agent’s own prior turns included – so a figure the model invented three steps earlier counted as evidence for spending it. The corpus was built to expose exactly that in designs whose evidence is the whole context window, and it found a narrower version of it in ours.
It now grounds against source-bearing roles only: the operator’s instructions, the user, and tools the run actually called. The model is not a source about the world.
Which is the point of running it
Catch rate went from 75.0% to 87.5% on the same corpus, and a regression test pins the behaviour so it cannot drift back.
A benchmark that only ever confirms what you already believed is decoration. This one changed a shipped check, and that is the standard we hold it to.
No single check carries this
Grounding is one check among several, and the corpus shows why that matters: the two attack families in it are caught by entirely different parts of In Guard Out, with no overlap between them.
| The attack | What stops it | Caught |
|---|---|---|
| A value the agent never readfabricated payee, invented amount | action grounding | 9 / 9 |
| A source an attacker wrotepoisoned invoice, hidden instruction | tool-result scanningpermission tiers | 5 / 7 |
| An honest action the exact check refusesspaced IBAN, summed total, converted amount | second look | 2 / 3 released |
Grounding passed every poisoned case – correctly, because those values really were in a document the run read. An attacker who can write to the evidence gets grounding for free. A different check is what refused them: four on the injected instruction, one on a destructive argument pattern.
Neither check would have been enough. Run only grounding and five attacks go through; run only injection scanning and nine do.
What a single check costs you
IBM Granite Guardian missed four of those same five. Asked about groundedness, it answers about groundedness – and an invoice that says “ignore all previous instructions” is perfectly well-grounded text.
That is not a flaw in the model. It is what happens when one question is asked of a problem that has several.
This benchmark scores two of those checks. A run also passes through PII screening and egress tracking, tool policy and risk tiers, hard run budgets, loop detection and tool ordering rules. Each one answers a question the others do not.
And where detection runs out, the approval layer catches it
Two attacks defeat every detector, ours included, because the attacker wrote the evidence: a forged invoice whose only account number is theirs, and a real invoice with a decimal shifted one place. Nothing in the run contradicts anything, so there is nothing to detect.
Both are stopped by policy instead of detection. A payment above a ceiling, or to an account not on the approved list, is held for a person before it happens – and because those limits live on the API key rather than in the request, the agent cannot widen its own.
Ready-made policies for both ship with the product, written against these exact incidents. See the templates.
How to read this
Every case carries a written rationale saying what it is meant to distinguish. That rationale is the thing to argue with if you think a label is wrong.
The exact checks do no inference at all – they are string and number comparisons over the run’s sources, and they finish far below the noise floor of the model call they sit beside. The only latency added is the second look: about 0.3 s, and only on an action the exact check would refuse. The figures above are with it switched on.
False positives are reported as prominently as catches – the column most vendors omit, and the one that decides whether a guardrail stays switched on. The two external models were run by us on our corpus, unmodified on their vendors’ own weights, and each is linked so the exact build is identifiable.