Structured generation basics

Structured output means forcing the model's answer into a predetermined shape: a data structure with named, typed fields instead of free sentences. For an application this is a requirement, because code does not want "there's a meeting at three", it wants a readable field like { "hour": 15 }. Unstructured text means guessing and regex downstream, and both break.

Just naming the format is not enough either. OpenAI introduces its structured outputs feature for exactly this: it makes the model's response adhere to a supplied JSON Schema, reducing failure modes like omitted required keys or invalid enum values. The subtlety is this: producing valid JSON and producing JSON that matches the right shape are not the same thing. The first is grammar; the second is contract.

The free sentence on the left is clear to a human but ambiguous to a machine. The structured record on the right splits the same information into fields; each field has a name, a type, and a clear boundary.

JSON (RFC 8259) is a lightweight, text-based, language-independent data interchange format. It knows only four primitives (string, number, boolean, null) plus two containers (array and object). That small grammar is what makes it the default: every language has a mature parser, the rules are few and clear, and UTF-8 use is guaranteed by the standard outside closed ecosystems.

A few fine points. A JSON object is an unordered collection of name/value pairs; if order matters, you must rely on an array, not on object key order. Likewise the RFC asks object names to be unique for interoperability; duplicate keys behave unpredictably across parsers. These are not signs that JSON is fragile; on the contrary, because its boundaries are clear it can be automated safely.

JSON's one gap is that it does not define domain-level validation on its own: which field is required, which value is an enum, what range a number is in? The RFC does not cover that. JSON Schema steps in here, and that is the subject of a later section.

To keep the format debate concrete, let us write the same small record four ways: one user, one role, two tags. JSON, XML, YAML, and a Markdown table carry the same information, but at a different price. The diagram below shows roughly how many tokens each one spends; the bars fill in sequence.

Two things stand out. XML is the most expensive, because it wraps every value between an opening and closing tag; the <tag>…</tag> structure repeats the information twice. YAML drops the braces by leaning on indentation and is relatively compact, but that readability comes with whitespace sensitivity. The Markdown table looks tidiest to the human eye, yet as later sections show it is the weakest as a machine contract.

Token cost is not an idle worry: you pay per input and output token, and as nesting deepens the repeated syntax (XML tags especially) grows the bill. For the same data the cheapest safe format is usually JSON.

XML (W3C XML 1.0) is a markup language: a document is made of parts like elements, attributes, character data, and comments. It requires exactly one root element and proper nesting; for a conforming processor a well-formedness violation is a fatal error. That strictness is a feature: if structure breaks it stops rather than silently emitting wrong data.

XML's distinctive strength is putting a clean boundary around a generated span of text. Asking a model to return a long quote or a code block inside a tag can be easier than wrestling with JSON string-escaping rules; tags like <answer>…</answer> act as natural delimiters. So XML makes sense when document structure or mixed content (text and elements interleaved) matters.

The cost shows up in two places: tokens and validation. Because every value repeats two tags, XML is heavier than JSON and YAML for simple typed records. XML also separates well-formedness (is the syntax correct) from validity (does it satisfy extra constraints like a DTD); so syntactically correct XML may still not match the expected shape. Downstream you still need a parser and a validation contract.

YAML (YAML 1.2.2) is a human-friendly, cross-language data serialization language; it represents data with mappings, sequences, and scalars. Block style sets scope by indentation, while flow style uses explicit indicators like brackets and braces. YAML 1.2 aims to be a strict superset of JSON, so most JSON is also valid YAML.

What makes YAML attractive is the same thing that makes it sensitive. Comments, block scalars, anchors/aliases, and multiple scalar styles ease authoring but complicate parsing. Whitespace-sensitive indentation can break structure with a single stray indent; implicit typing can turn yes into a boolean or a postal code into a number. Mapping key order is not part of the representation model, so when order matters the spec points toward sequences.

The upshot: YAML is excellent for config and prompts a human edits by hand. In a machine-only LLM pipeline it is more fragile than JSON, because whitespace, implicit typing, key handling, and scalar folding all call for parser-aware validation.

A Markdown table is great for human-facing summaries, but it is the weakest option as a machine interface. First a technical fact: table syntax is not part of baseline CommonMark, it is an extension in GitHub Flavored Markdown. So not every parser that claims to speak "Markdown" has to read a table the same way, or at all.

A GFM table has a header row, a delimiter row (hyphens, with optional colons for alignment), and optional data rows; cells are separated by pipes. The spec's forgiving rules are exactly what make it hard to trust: the header and delimiter rows must match in cell count, a body row with too few cells gets empty cells inserted, and extra cells are ignored. So you can silently lose or invent data.

Add escaping the pipe character, alignment syntax, and renderer-specific behavior, and reading data reliably from a table gets hard. The right mindset: treat a Markdown table as a presentation layer, not a contract layer. For data going to a machine, convert it to JSON first and show the table only to humans.

A typed object is not a separate format; it is a schema + validation layer on top of a wire format (usually JSON). In Python you write your application's type with Pydantic, in TypeScript with Zod; the SDK turns that type into a JSON Schema, hands it to the model, parses the returned JSON, and delivers it validated against your type. OpenAI's SDK helpers do exactly this: they map a Pydantic or Zod model into a schema.

What makes this possible is JSON Schema 2020-12: with keywords like type, enum, required, object property counts, array item constraints, and string rules, it says what an instance must look like. One caveat: the format keyword is nuanced; the spec separates annotation from assertion behavior, implementations vary, so not every format is always enforced. Knowing the schema subset your framework supports is the application's job.

The pipeline below shows four steps: the model emits JSON, the parser turns raw text into a data structure, the schema checks that structure, and a passing record becomes a typed object. The key point: a schema is a validation language, not a prompt convention, and a passing value now speaks the same language as your application's type system.

Typed objects reduce drift but do not solve everything. The OpenAI docs are explicit that structured output still requires application handling for refusals, responses cut short by max output tokens, content filters, stop tokens, and network errors. A schema does not replace pipeline-level error handling.

The practical rule is simple: if a machine will consume it, start with JSON, put JSON Schema on top, and where possible use the provider's structured-output path that guarantees schema adherence. With its small grammar, universal parsers, and direct schema support, JSON is the safest default for typed-object output.

The exceptions are clear. If you need to delimit long text spans or mixed content, XML tags help. For config or prompts a human edits by hand, YAML adds readability. For a summary table shown only to humans, Markdown is ideal; but do not feed that table to a machine as data, convert it to JSON first. Always treat a Markdown table as presentation.

Finally, in high-reliability systems writing "return JSON" in the prompt is not enough. JSONSchemaBench reports that LLM-only approaches have the lowest compliance rate in their experiments. As guided-generation work shows, the real guarantee comes from constrained decoding: the model is masked at each step so only tokens that stay inside the target language are allowed. So format choice is the start; the contract is closed by schema, parser, validation, and a retry/repair flow when needed.

A format is only the first half of binding output to a contract; the second half is schema, parser, and validation.

So before asking "which format?", ask: who will read this output, what shape must it match, and what happens when it does not?