beginner25 minby DevFox

The Shape of an AI Feature

Assembling everything so far into one small end-to-end feature — input handling, prompt, schema, validation, fallback and evaluation — and seeing which parts are AI and which are ordinary software.

Two hundred support emails land overnight. Every morning someone spends the first hour reading them and deciding, for each one, which of five buckets it belongs in, how urgent it is, and whose queue it goes to. It is not hard work. It is just work, and it happens again tomorrow.

You now know eleven separate things about language models. This lesson spends all of them at once, on that inbox. You will build one small feature end to end — contract, input guarding, instructions, schema, the call, validation, fallback, logging, score — and then count the lines. That count is the point: the model call is nine lines in the middle of roughly a hundred and fifty, and the rest is ordinary software you already write.

Write the contract before the prompt

The instinct is to open a chat window and start tuning wording. Resist it. Start with the feature contract: what goes in, what comes out, and what comes out when things go wrong. Everything after it — schema, validation, eval — is derived from it.

python

from dataclasses import dataclass

CATEGORIES = ("billing", "bug", "access", "feature", "other")
URGENCIES = ("low", "normal", "high")

@dataclass
class Triage:
    category: str    # one of CATEGORIES
    urgency: str     # one of URGENCIES
    owner: str       # a name from our on-call roster
    summary: str     # one line, 120 characters or fewer

@dataclass
class Unclear:
    reason: str      # what was missing from the email

Two return types, not one. Unclear is not an error — it is an outcome you designed on purpose, because an email that says "it's broken, please fix" does not contain enough to route it. A feature that must always answer will always answer, and some of those answers will be confident nonsense.

Nothing above mentions a model, and that ordering is what keeps the model in its place: it is hired to fill in a structure you defined, not to decide what the structure is.

The half of the work that happens first

Before a character leaves your server the input gets three jobs done to it: reject what cannot be processed, cut what is too large, strip what must never leave the building.

Size, because a support thread is rarely one message. A customer forwards forty replies and you are billed for every token in all of them — tokens being the chunks of text a model measures and prices in — while the signal sits in the first paragraph.

Stripping matters more. Support inboxes are where people paste card numbers, passwords and one-time codes without being asked. Sending those to a vendor should be a deliberate decision, which is the whole subject of the lesson on what to never hand a model.

Bad — sends whatever arrived, at whatever length, containing whatever the customer pasted.

python

def triage_email(subject, body, sender_domain):
    return ask_model(subject, body, sender_domain)

Good — bounds and cleans the input before anything is sent anywhere.

python

def triage_email(subject, body, sender_domain):
    body = guard_input(body)
    if body is None:
        return Unclear(reason="empty or unusable message")
    return ask_model(subject, body, sender_domain)

def guard_input(body):
    body = redact_secrets(body.strip())  # cards, keys, codes
    if not body:
        return None
    return clip_to_token_budget(body, limit=1500)

The bad version bills you for a forty-message thread to classify one sentence, and on the day a customer pastes a card number into a complaint you have handed it to a vendor and written it into your own logs. Neither is recoverable by improving the prompt.

Two instruction layers, one per lifetime

Your instructions split by how often they change. The durable half — the job, the categories, the urgency rule, the permission to give up — is identical for all two hundred emails and belongs in the system instructions, the layer above every request. The per-request half is only this email.

That split is the four-part prompt from earlier in the course, rearranged by lifetime rather than topic: instruction and examples in the durable layer, context in the per-request one, and the output contract moved out of prose entirely into a schema — the next section.

The roster gets one sentence. Three names and the categories they take is nine words that change twice a year, so it lives in the durable layer: no retrieval, no tool, no lookup at request time. The decision map for getting your data to a model has a cheapest option, and this is it.

Asking for a shape, and showing two

Parsing prose is how you end up with a regex that breaks the day the model writes "Category: Billing" instead of "category: billing". Ask for a schema instead — a machine-readable description of the object you want back — and the answer arrives as a dictionary.

Two details are doing real work. Every fixed field is an enum rather than a string, so "Billing" and "billing issue" are not new values you have to normalise later. And "unclear" belongs to the category enum — the escape hatch is a legal answer inside the shape, not a special case arriving some other way.

Then show it, because a rule stated in a paragraph competes with everything else in that paragraph. Two examples are enough, and one of them has to be an unclear:

The second one is the important one. Without it the model has been told the escape hatch exists but has never seen anyone use it, and models weight what you show far more heavily than what you say.

The nine lines that are actually AI

Here is the whole of the model contact surface for this feature.

temperature=0 because this is a classification, not a piece of writing. Temperature trades variety against consistency, and a triage that puts the same email in two different queues on two mornings is a bug you will spend a week failing to reproduce. Turn it as low as the provider allows — and remember low is not off. The token cap is the same instinct: you asked for four short fields, so if the model starts writing an essay you would rather pay for two hundred tokens than four thousand.

Now keep the first lesson of this course in mind, because it explains the next section. The model is predicting plausible next tokens. It will produce something that looks exactly like a valid triage whether or not it had any idea what the email was about. Well-formed is free. True is not.

A well-formed answer is not a true one

The schema constrains the shape and says nothing about whether the content is right — and schema enforcement itself varies by provider and by mode, so write your validation as though the schema were only a strong suggestion.

Bad — takes the response at its word and puts it straight into a human's work queue.

Good — checks every claim against facts the application owns.

The bad version puts a fabricated detail in front of a support agent with the authority of a system. The first anyone hears of it is a customer being told about a refund that was never issued.

The validator itself is unremarkable, and that is the point:

Notice the owner is never asked for. It is a pure function of the category, and a model that has never seen your staff list will still return a confident, plausible, fictional name — the invention the lesson on hallucination describes. The fix is not asking more firmly; it is refusing to treat the field as authoritative. summary is free text and cannot be defended this way, which is why it is the one field a human reads and nothing routes on.

Three outcomes, one of them happy

Now assemble it. Every path through the feature ends in one of three places, and two of them are the same place.

Four different failures collapse into one outcome the rest of your system already handles.

The input was unusable
Empty, or nothing survived the guard. Never reaches the model at all.
The model declined
It answered unclear, which you made a legal value precisely so it could.
The model was unreachable
A timeout, a rate limit, an outage. Ordinary software failure, handled the ordinary way.
The answer failed validation
Well-formed and wrong: a category outside your enum, an owner who is not on the roster.

All four end in the human queue — where every email sat last month

Logging enough to explain a bad answer

Next Tuesday someone shows you a billing email routed to feature and asks what happened. You need to reconstruct that call, so log the identifiers rather than the content.

prompt_version is the field everyone leaves out and everyone regrets. When accuracy drops on a Thursday the question is always "what changed", and the prompt changes more often than the code around it. Bump that string every time you edit the instructions, and a bad week becomes a two-minute query.

Note what is absent: the email body. You cleaned it on the way in for a reason, and copying it into a logging system undoes that work in a different building. Log the id and look it up in the support tool, where it already lives.

Running it against the set you built

The previous lesson had you build an evaluation set — forty real emails with an agreed correct category. This is what it was for. Run the whole function against it, not the prompt: a validator that rejects good answers costs you exactly as much as a prompt that produces bad ones, and only an end-to-end run sees both.

Two numbers, deliberately. Wrong is expensive — it sends a locked-out customer to the feature-request pile. Unclear is cheap — ten seconds of someone's reading. A prompt change that turns two wrongs into two unclears has lowered your accuracy and improved your feature, and trying it by hand a few times would never have told you that.

Now count the file: roughly fifteen lines of contract, twelve of input guarding, thirty of instructions and examples, fifteen of schema, twenty-two of validation, sixteen of orchestration, twelve of logging, fifteen of scoring — and nine of model call. Eight of those nine groups are types, guards, lookups, error handling, logging and a loop, the same work as any other feature you have written.

That is the shape. A language model is a component with unusual properties: it is non-deterministic, priced by the token, and occasionally wrong with complete confidence. AI engineering is almost entirely the ordinary code you put around it to make those three properties survivable.

The shape, in one place

Guarding, validating, falling back0 lines

Instructions, schema, examples0 lines

Logging and scoring0 lines

The model call0 lines

The model call is the smallest piece of the feature, and it is the only part that is new to you

Where to go next

You can now build a small AI feature that behaves: it refuses when it should, fails into the old process, and has a number attached to it. Plenty of shipped features do not clear that bar.

It is the bar for one email at a time. The next course, Building Reliable AI Systems, takes this feature into production and fixes what it only sketched: treating context as a budget rather than a bucket, giving the model tools it can call instead of stuffing everything into the prompt, retrieval done properly with real chunking and reranking, agent loops that know how to stop, defence against prompt injection when the text you are triaging was written by a stranger, and observability for a system that never does the same thing twice. Every one is something this function would need before it ran unsupervised.

Before you move on, do one experiment. Add a sixth category to CATEGORIES and run score three times: with only the schema changed, then after describing the category in the system instructions, then after adding an example of it. Watch which of the three moves the number. That answer is worth more than any advice about prompt wording you will read this year.