Structured Output Enforcer

Structured output enforcer

Getting an LLM to reliably return parseable JSON is a common production headache: stray markdown fences, chatty preambles, and trailing commas all break JSON.parse. The robust pattern combines a strict prompt that bans those habits with a retry loop that feeds the parse error back to the model and asks for a correction. This tool generates both.

How it works

You provide a base prompt and the JSON schema you expect. The tool produces an enforced prompt that tells the model to return only valid JSON matching your schema — no fences, no commentary. It also emits a provider-agnostic JavaScript snippet with an extractJson helper that strips code fences and isolates the outermost object, plus a loop that, on any parse failure, re-prompts the model with the error and the bad output up to your chosen retry limit.

Why LLMs fail at JSON — and how the enforcer fixes each failure mode

There are four common ways a model breaks JSON.parse:

1. Markdown code fences. The model wraps the JSON in ```json ... ```. The extractJson helper strips them before parsing.
2. Explanatory text before or after. The model says “Here is the JSON:” and then adds “I hope that helps!” after. The helper slices from the first { to the last }.
3. Trailing commas. Technically invalid JSON, common in models trained on JavaScript code. The retry prompt flags the exact error so the model can remove them.
4. Wrong shape. The model returns valid JSON but ignores your schema — for example returning a flat object when you needed an array. Explicit schema in the prompt and a Zod/Ajv schema check after parsing catch this.

Worked example

Suppose you want to extract structured invoice data. Your base prompt might be “Extract the invoice details.” Your schema:

{
  "invoice_number": "string",
  "total_amount": "number",
  "line_items": [{"description": "string", "amount": "number"}]
}

The enforcer wraps this into a prompt that says, in effect: return only a JSON object matching this exact shape, no markdown, no commentary. On a parse failure the retry prompt includes the exact error message and the bad response, asking the model to return the corrected JSON only.

Tips and notes

• Keep the schema small and flat. Deeply nested schemas are harder for models to honor; flatten where you can and validate nested parts separately.
• Prefer native JSON mode when you have it. If your provider offers structured outputs or function calling, use it as the primary path and keep this retry loop as a fallback for other models.
• Log retries. A high retry rate signals a prompt or schema problem worth fixing rather than papering over with more attempts.
• Validate after parsing. Successful JSON.parse only means it is valid JSON, not that it matches your schema — add a schema check (Zod, Ajv) before trusting the data.
• Two retries is usually enough. Most parse failures are consistent; if the model still fails after two correction attempts, the schema or prompt is the problem, not bad luck.