beginner25 minby DevFox

Structured Output Instead of Prose

Why parsing a model's paragraphs is a bug factory, and how asking for a schema instead turns an AI call into an ordinary function you can type, validate and test.

Your ticket triage has been quietly broken for two days. It reads each incoming support email, asks a model how urgent it is, and sends anything urgent to the on-call queue. The model used to answer "This looks urgent — the customer cannot log in." Your code searched that sentence for the word "urgent", and everything worked. Last Tuesday the model started writing "time-sensitive" instead, and the on-call queue went quiet.

Nothing crashed. No error was raised, no test failed, no alert fired. The model did its job and your parser did its job, and the two of them stopped agreeing about what an answer looks like. By the end of this lesson you will stop asking models for sentences you have to dig through and start asking them for a defined shape you can hand straight to the rest of your program — and you will know what to do on the day the shape comes back wrong.

Prose is a moving target

A model does not have a fixed way of saying things. It produces one plausible continuation of your prompt, and "plausible" covers an enormous number of sentences that all mean the same thing to a human and nothing like the same thing to a regular expression. "Urgent", "time-sensitive", "needs attention today", and "I would prioritise this" are one answer wearing four costumes.

So when you scrape a model's paragraph for a keyword, you are building on a surface nobody promised to keep stable. It shifts when you edit the prompt, when the input is longer than usual, when the provider ships a new model version, and sometimes for no visible reason at all. Every shift is a silent bug, because your code cannot tell the difference between "the model said no" and "the model said no in words I was not looking for".

Bad — decides urgency by keyword, so a reworded answer routes an outage to the normal queue with no error anywhere.

python

reply = call_model(
    "How urgent is this support email?\n\n" + email_body
)
is_urgent = "urgent" in reply.lower()

Good — names the answers it will accept, so the model has one word to choose and your code has one thing to compare.

python

reply = call_model(
    'Classify how urgent this support email is. Reply with '
    'JSON only, in the form: {"urgency": "high"} where urgency '
    'is one of high, normal, low.\n\n' + email_body
)
is_urgent = json.loads(reply)["urgency"] == "high"

The first version fails without failing. It keeps returning False forever, on every ticket, and the only signal you get is an angry customer a week later. The second version can still go wrong — it goes wrong loudly, which is the whole difference. (We harden that json.loads later; it is not the finished article.)

Throughout this lesson, call_model is your own thin wrapper around whichever provider you use; the ideas are the same in any language.

Ask for a shape instead

Structured output means telling the model, up front, the exact shape of the answer you want, and getting that shape back instead of a paragraph. In practice the shape is almost always JSON — a set of named fields with values — because every language already knows how to read it.

The reason this matters is bigger than parsing convenience. An AI call that returns prose is a stranger you have to interview. An AI call that returns a known shape is an ordinary function: it takes arguments, it returns a value with a type, and the rest of your program can use it without knowing or caring that a model was involved. Everything you already know about writing software applies again the moment the boundary has a shape.

Think of it like the difference between asking a colleague "so how did the interview go?" and handing them a one-page form with four boxes on it. The conversation is friendlier. The form is the one you can file, count, and compare against last month's.

Describing the shape

The description of a shape is a schema: the list of fields, what type each one holds, and which of them must be present. Write the schema first, in your own code, and derive both the request and the parsing from it. If you write the shape twice — once in English in the prompt, once in your parser — the two will drift apart, and you are back to guessing.

Here is a real one for the triage feature, in Python:

That class is doing three jobs at once. It documents the shape for you, it generates the schema you send to the model, and it checks the reply when it comes back. Every language has an equivalent — Zod or Valibot in TypeScript, a data class plus a validator in Kotlin, a struct with tags in Go. Pick the one your project already uses; the shape is the idea, the library is a detail.

Few fields, and flat

Every field you ask for is one more thing the model can get wrong, and one more thing your code has to handle when it does. A schema with four fields fails in four ways. A schema with twenty fails in twenty, and you will not have written a branch for most of them.

Nesting makes this worse than it sounds. A field inside an object inside a list is a value the model has to place correctly and position correctly, and the second failure looks nothing like the first: your validator reports a missing key three levels deep while the actual answer sits right there, one bracket over.

Three fields, no nesting, every one of them a single value. If you find yourself designing something with objects inside arrays inside objects, that is usually a sign you are asking the model to do two jobs in one call. Two calls with small flat schemas are easier to prompt, easier to validate, and far easier to debug than one call that returns a document.

The exception worth knowing: a flat list of the same thing is fine. Asking for five suggested tags as a list of strings is one job with one shape. It is the mixed, deep structure that costs you.

Every field means exactly one thing

A field name is an instruction to the model, and a vague one gets you a vague answer that still validates. date is a field a careful model will fill with March 3rd, 03/04/2026, or next Tuesday on three consecutive runs, and all three are legitimate strings. due_date_iso, described as "the date the customer needs a reply by, as YYYY-MM-DD", has exactly one correct form.

The same goes for values. Any field where a human would answer from a fixed list should be a closed list — an enumeration — not free text, because free text is where near-synonyms breed.

Bad — invites free text, so the same three tickets come back as "Billing", "billing question" and "payment problem".

Good — names every value the model may use and states what the numbers mean.

The first version passes any JSON check you throw at it and then sends half your billing tickets into the default branch of a match statement, because "payment problem" is not a category your router has ever heard of. Unconstrained fields do not produce errors; they produce a slow leak.

One more rule that saves real pain: give the model an honest way out. If a ticket genuinely does not say how urgent it is, a schema that only allows high, normal or low forces an invention. Add unknown to the list, or a separate confident boolean, and say in the prompt when to use it. A model told it may say "I cannot tell" will sometimes say it. A model with no such option always guesses, and the guess arrives looking exactly like knowledge.

The reply is untrusted input

Here is the habit that separates a demo from a feature. The model's response is untrusted input, the same as a form submitted from a browser or a payload from someone else's API. You would never take a web form's price field and pass it to your billing code without checking it. A model's output deserves precisely the same suspicion, and for a stronger reason: it was generated by something that has no concept of your schema being binding.

Two different things can go wrong here, and people conflate them constantly.

Not JSON at all

Parsing fails. You find out immediately.

A code fence, a preamble, a paragraph of explanation after the closing brace. Annoying, loud, and easy to handle.

Well-formed JSON saying the wrong things

Parsing succeeds. Nothing complains.

A priority of 7 on a 1–3 scale. A category of "payment" when your enum says billing. A missing needs_human. The number 2 sent as the string "2".

Every one of these becomes a bug somewhere else in your program, hours later.

Parsing catches the left one and sails straight past the right one. Validation catches both.

Bad — parses and uses in one breath, so a missing field becomes a KeyError in the routing code, several functions away from the cause.

Good — validates against the schema at the boundary, and has an answer ready for a reply that does not fit.

The bad version turns a slightly odd model reply into a stack trace in code that has nothing to do with AI, and the person debugging it at 3am has no idea the model was ever involved — the raw text is long gone. The good version fails in one known place, keeps the text that failed, and the ticket still reaches a human.

Two details make this work in practice. Validate once, at the boundary — the moment the response arrives — so everything downstream can assume a real TicketTriage and never re-check. And configure your validator to reject unknown fields rather than ignore them. A model that starts returning an extra key is telling you your prompt and your schema have drifted apart, and you want to hear it now rather than the month you finally need that field.

When the answer is nearly right

Most invalid replies are not nonsense. They are your object with something wrapped around it: a code fence, a friendly preamble ("Here is the JSON you requested:"), or a helpful paragraph of explanation after the closing brace. This is the model doing what it was trained to do, which is be useful to a reader.

Handle it in three steps, cheapest first.

Strip the obvious wrappers before you parse
Cut a leading and trailing code fence, and take the text from the first opening brace to the last closing one. A few lines, and it fixes a large share of failures.
Retry once, with the validation error in it
Send back the invalid text and the message saying which field was wrong. A model shown a specific complaint usually fixes it.
Once, though. If the shape is wrong twice, the problem is your schema or your prompt, not luck.
Take the path you decided on before you shipped
Route to a human. Return the neutral default your product can live with. Show an honest "we could not read that".
Any of those is fine. What is not fine is discovering on the day that you never picked one.

The third step is the actual deliverable here: a defined outcome for invalid data, chosen in advance

A cheat sheet to keep

Where to go next

You can now turn a model call into something the rest of your program can treat as normal code: a small flat schema, fields that mean one thing each, validation at the boundary, and a decided answer for the replies that do not fit. That is most of the distance between a prompt that impresses in a notebook and a feature that survives a Monday.

The follow-on is Structured Output You Can Trust, in the intermediate course. It picks up exactly where this stops: how constrained decoding makes invalid output impossible rather than unlikely, how to build a repair loop that provably converges instead of retrying forever, and how to work with an object that is only half-arrived because the response is still streaming. Each answers a question this lesson deliberately left at "validate it, and have a plan for invalid".

The thing to go and do now is small. Take one prompt you already have that returns a paragraph, and rewrite it to return three fields. Then feed it your five ugliest real inputs — the empty one, the very long one, the one in another language — and watch which field breaks first. That field is where your schema was vague, and finding it takes about ten minutes.