Getting reliable structured output

The schema constrains generation to a valid shape; your code validates what the schema cannot.

When a person reads the output, prose is fine. When your code reads it, you need JSON that parses on the first try, with the fields you expect and nothing else. The gap between "usually valid JSON" and "always valid JSON" is where structured-output bugs live, and closing it takes two things: the right prompt, and the right API call.

Prefer schema-enforced generation

The single biggest reliability win is not a prompt trick — it is letting the model's API constrain the output to a schema, so invalid JSON is impossible rather than merely discouraged. In the AI SDK that is generateObject:

import { generateObject } from "ai";
import { z } from "zod";

const { object } = await generateObject({
  model: "anthropic/claude-sonnet-5",
  schema: z.object({
    invoice: z.string(),
    total: z.number(),
    date: z.string().nullable(),
  }),
  prompt: `Extract the invoice from the statement below...`,
});

object is typed and guaranteed to match the schema. Asking for JSON in plain text and hoping — then wrapping JSON.parse in a try/catch and a retry — is the pattern this replaces. If your provider offers a JSON or structured-output mode, use it before you reach for prompt wording.

The prompt still carries the meaning

Schema enforcement guarantees the shape. It cannot tell the model what a field means, or what to do when the data is not there. That is the prompt's job, and it is the same discipline as the extraction prompts in the library:

Show one example of the target output. A single worked example fixes edge cases — leading zeros, decimal places, date format — more reliably than a paragraph describing them. One example beats a paragraph.
State the rule for missing fields. Say "use null when a field is absent — do not infer it." Without that, the model fills gaps with plausible guesses, and a plausible guess in a data pipeline is worse than a blank because nothing downstream can tell it from real data.
Constrain enums explicitly. If a field must be one of a fixed set, list the set and add an escape value. A free-text category field will drift; "status": "paid" | "unpaid" | "unknown" will not.

The failures, and their fixes

Markdown fences around the JSON. Ask for JSON in text and models often return ```json ... ```, which breaks JSON.parse. Schema-enforced generation avoids this entirely; if you are parsing text yourself, the instruction "return only the JSON, with no markdown and no prose" is the mitigation, but it is a mitigation, not a guarantee.

Preamble and trailing commentary. "Here is the data you asked for:" before the JSON, or "Let me know if you need anything else" after it. Same cause, same fix: constrain the output, and if you cannot, say "return the JSON object and nothing else."

Hallucinated fields. The model adds a helpful field you did not ask for, or fills a required one it could not find. The schema rejects unknown fields; the null rule handles the absent ones. Between them, the output stops improvising.

Numbers as strings, or the wrong type. "total": "82.50" when you wanted a number. A typed schema (z.number()) is the fix — the model is constrained to emit a number, and your code gets one without a parse-and-coerce step.

When you cannot use schema mode

Some setups — a raw HTTP call, a provider without structured output, streaming partial JSON — leave you parsing text. There the prompt does all the work, and the same rules from the library apply: name the exact format, show an example, forbid prose, and validate the result against a schema on your side (with z.safeParse) so a malformed response is caught and retried rather than crashing a consumer three steps downstream.

The order of preference is always the same: constrain the output at the API if you can, describe it precisely in the prompt regardless, and validate it in your code before you trust it.

From a real codebase

The fraud-triage-with-llm project — a small, free LLM app that triages card transactions locally — models its decision exactly this way. The model does not return prose; it returns a typed object your code can branch on:

class TriageDecision(BaseModel):
    risk_tier: RiskTier          # low | medium | high
    confidence: int = Field(ge=0, le=100, ...)
    rationale: str
    recommended_action: Action   # approve | review | block

It also carries a lesson this tutorial only gestured at. The first version typed confidence as a 0.0–1.0 float; the model returned 95 — it thinks in percent — and the validator rejected the response. The fix, recorded in the code's own comments, was two-fold: model the field the way the model naturally thinks (0–100), and keep the range validator as a safety net, because constrained decoding enforces the type of a field but not its range. That is structured output working as designed — the schema guarantees the shape, and your validation catches everything the schema cannot.

The prompt part of structured output is a task prompt like any other — an instruction, a format, and an example. Paste the instruction half into the checker and it will flag a missing format or a vague field description before it becomes a parse error in production.