Format enforcement
“Return JSON” should mean valid JSON
Structured output is a contract, and models break it in mundane ways: prose around the object, a trailing comma, a missing field, markdown fences. Every team ends up writing the same defensive parsing code – per app, per endpoint, forever.
The proxy enforces the contract once, on the wire. When a request asks for structured output, the response is validated and repaired in three tiers: parse it directly; if that fails, extract the JSON that is buried in prose; if that fails, re-ask the model with a corrective prompt. Your pipeline receives something that parses, or – in PREVENT mode – a clean 422 instead of a silent poison value.
Format enforcement composes with the other checks: the repaired output still goes through grounding, so a structurally valid answer with a drifted figure is caught too.
Direct parse
The completion is validated against the requested format. Valid output passes through untouched.
Extraction
Valid JSON buried in prose or markdown fences is extracted – the most common failure, fixed without another model call.
Corrective re-ask
As a last resort the model is re-asked with a corrective prompt. The extra call is metered and attributed to the request like every guard call.
Fail closed
In PREVENT mode, output that cannot be repaired returns a 422 – your parser never sees it. In FIX mode the best repair passes through, flagged.
Does this work with any model?
Yes – it fronts any OpenAI-compatible upstream (OpenAI, OpenRouter, local models via LM Studio, and more). Native structured-output support is used where the model has it; validation and repair cover the rest.
What happens when the output cannot be repaired?
In PREVENT mode the request fails closed with a 422 and a recorded reason. In FIX mode the closest repair is passed through and the violation is flagged, so downstream code and the dashboard both know.
Does repair add cost?
Direct parse and extraction are free. Only the corrective re-ask spends tokens, and it is priced into the request so the dashboard shows exactly what enforcement cost.
Can I see which tier fixed a response?
Yes – every request records a per-stage timeline, including what the format stage did (parsed, extracted, re-asked) and what changed.
See malformed JSON get repaired in real time
Send broken structured output - prose around the object, missing fields, trailing commas - and watch the proxy fix it. We are running a limited demo - sign up and we will get you in as soon as we can.