Healthcare & life sciences
Fact-checking & guardrails for clinical AI assistants
Clinics, hospitals and health software that put patient data through a model.
The dose that changed on the way out
An assistant summarises a patient’s medication plan and turns 15 mg into 50 mg. Every other line of the summary is correct, so nothing looks wrong to the person reading it. On the way there, the patient’s name and record number were sent to the model provider along with the notes.
in/guard/out swaps patient details for placeholders before the request leaves, and puts them back only in your own app. Every number in the answer is matched to the source, so a drifted dose is caught by arithmetic rather than by luck.
Healthcare AI fails in two expensive ways: sensitive data goes out, or a wrong number comes back. A patient identifier that reaches a model provider is a reportable disclosure; a dosage the model subtly altered is a safety event. Both failure modes hide inside otherwise-useful answers.
in/guard/out screens PHI into reversible placeholders before any request leaves for the model – names, identifiers, contact details don’t reach the provider – and restores the real values only on the way back to your application. What the model sees, and what any log stores, is screened.
On the way back, deterministic numeric grounding recomputes arithmetic and matches every figure in the answer to the source, so an altered dosage or lab value is caught without relying on another model’s judgment. Content-safety classifiers flag unsafe or out-of-scope advice, and logging is tiered: screened by default, raw only as a deliberate, audited opt-in.
What can go wrong – and what in/guard/out does about it
Each problem teams like this run into, with its answer. In Block mode a failed check stops the request; in Fix & log it is repaired where that’s safe and recorded otherwise. Where something isn’t ours to control, we say so.
Patient data sent to the model provider
Names, IDs and other identifiers it detects become placeholders before the request leaves, and are restored in your app.
A wrong dosage or figure in a summary
Figures are matched to the source and the maths is recomputed; a wrong value is corrected or blocked.
Unsafe or out-of-scope medical advice
Content-safety classifiers and topic rules redact or block it.
Patient data sitting in AI logs
Logs keep the masked version by default; real values are stored only if you turn that on.
Whether the advice is clinically right
Outside in/guard/out in/guard/out checks answers against the material supplied, not medical judgement – clinical review stays with your clinicians.
The altered dosage
A summarization assistant restates a medication plan and drifts one figure – 15 mg becomes 50 mg. Numeric grounding matches every number in the answer against the source document, flags the mismatch deterministically (no judge model involved), and in prevent mode blocks the response before a clinician ever sees the wrong figure.
Frequently asked questions
Does patient data reach the LLM provider?
No. PHI is detected and replaced with placeholders before any data reaches the model provider. The provider only sees screened text – placeholders, not real PHI. The unscreened values are returned in responses sent back to your application.
How do you catch a hallucinated dosage?
Deterministically. The grounding stage recomputes arithmetic and compares every figure in the answer to the source material. A number that drifted from the source is flagged or blocked – no LLM judgment required for the numeric class of errors.
What gets stored in logs?
Screened transcripts by default – placeholders, not PHI. Raw storage is available as an opt-in for compliance documentation, but all logs are stored in the US. Screened transcripts are the default for data minimization.
Can it block unsafe medical advice?
Content-safety classifiers (e.g. ShieldGemma-class models) score responses and block or withhold unsafe content, alongside topic controls for out-of-scope advice.
Other clients
See it on your own traffic.
We’re running a limited demo – sign up and we’ll get you in as soon as we can.