Structured Outputs: JSON, Schema Validation, Constrained Decoding
Your LLM returns a string. Your application needs JSON. That gap has crashed more production systems than any model hallucination. Structured output is the bridge between natural language and typed data. Get it right and your LLM becomes a reliable API. Get it wrong and you're parsing free-text with regex at 3am. Related: Phase 5 · 20 (Structured Outputs & Constrained Decoding) covers the decoder-level theory (FSM/CFG logit processors, Outlines, XGrammar). This lesson focuses on the production SDK surface (OpenAI responseformat, Anthropic tool use, Instructor) — read Phase 5 · 20 first if you want to understand what is happening below the API. Implement JSON-mode and schema-constrained outputs using OpenAI and Anthropic API parameters. Build a Pydantic validation layer that rejects malformed LLM outputs and retries with error feedback. Explain how constrained decoding forces valid JSON at the token level without post-processing. Design robust extraction prompts that reliably convert unstructured text into typed data structures. You ask an LLM: "Extract the product name, price, and availability from this text." It responds: That is a perfectly correct answer. It is also completely useless to your application. Your inventory system needs {"product": "Sony WH-1000XM5", "price": 348.00, "instock": true}. You need a JSON object with specific keys, specific types, and specific value constraints. You do not need a sentence. The naive solution: add…
Structured Outputs: JSON, Schema Validation, Constrained Decoding: Your LLM returns a string. Your application needs JSON. That gap has crashed more…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.