Format enforcement
“Return JSON” should mean valid JSON
Structured output is a contract, and models break it in mundane ways: prose around the object, a trailing comma, a missing field, markdown fences. Native structured outputs fixed much of this – where the model and provider support them. Everywhere else, every team writes the same defensive parsing code, per app, per endpoint, forever.
Side by side
| Concern | Native structured outputs & parsing libraries | in/guard/out |
|---|---|---|
| Schema adherence at generation | First-party and solid – where the provider and model support it | Used whenever available – the pipeline requests native structured output upstream |
| Models and providers without native support | Your defensive parsing code, or a per-app library (Instructor-style) | The same enforcement, uniform across every upstream, including local models |
| Broken output anyway | Exception handling, per app | Three tiers on the wire: parse → extract from prose → re-ask with a corrective prompt |
| Failure semantics | Whatever each app’s catch block does | JSON that parses, or a clean 422 – never a silent poison value |
| Correct format, wrong content | — (format is where it ends) | Repaired output still runs grounding – a valid object with a drifted figure is caught |
| Where the logic lives | N copies, one per codebase | Once, on the wire, versioned as policy |
Enforced once, composed with everything
in/guard/out enforces the contract once, on the wire. When a request asks for structured output, the response is validated and repaired in three tiers: parse it directly; if that fails, extract the JSON that is buried in prose; if that fails, re-ask the model with a corrective prompt. Your pipeline receives something that parses, or – in PREVENT mode – a clean 422 instead of a silent poison value.
Format enforcement composes with the other checks: the repaired output still goes through grounding, so a structurally valid answer with a drifted figure is caught too.
Three tiers of repair
Direct parse
The completion is validated against the requested format. Valid output passes through untouched.
Extraction
Valid JSON buried in prose or markdown fences is extracted – the most common failure, fixed without another model call.
Corrective re-ask
As a last resort the model is re-asked with a corrective prompt. The extra call is metered and attributed to the request like every guard call.
Fail closed
In PREVENT mode, output that cannot be repaired returns a 422 – your parser doesn’t see it. In FIX mode the best repair passes through, flagged.
Frequently asked questions
Does this work with any model?
Yes – it fronts any OpenAI-compatible upstream (OpenAI, OpenRouter, local models via LM Studio, and more). Native structured-output support is used where the model has it; validation and repair cover the rest.
What happens when the output cannot be repaired?
In PREVENT mode the request fails closed with a 422 and a recorded reason. In FIX mode the closest repair is passed through and the violation is flagged, so downstream code and the dashboard both know.
Does repair add cost?
Direct parse and extraction are free. Only the corrective re-ask spends tokens, and it is priced into the request so the dashboard shows exactly what enforcement cost.
Can I see which tier fixed a response?
Yes – every request records a per-stage timeline, including what the format stage did (parsed, extracted, re-asked) and what changed.
Related guardrails
See malformed JSON get repaired in real time
Send broken structured output - prose around the object, missing fields, trailing commas - and watch in/guard/out fix it. We are running a limited demo - sign up and we will get you in as soon as we can.