beginner25 minby DevFox

Few-Shot Examples That Teach

When showing beats telling, how to choose examples that generalise instead of ones the model copies verbatim, and how a badly-chosen example set narrows the model instead of guiding it.

Your changelog prompt is three paragraphs long by now. You have told the model to write for customers, keep each entry to one line, use plain language, avoid internal jargon and never mention ticket numbers. It still hands you "This commit refactors the discount validation logic in the cart module." You add a sixth sentence. It gets slightly worse.

Some rules are far quicker to demonstrate than to describe. Few-shot prompting — putting a handful of worked input-and-output pairs into the prompt, before the real input — is how you demonstrate. By the end of this lesson you will know when an example beats another sentence of instruction, how to choose a set that generalises instead of one the model traces over, how a narrow set quietly shrinks what the model will do for you, and why every example is a bill that arrives on every call.

When showing beats telling

There are two kinds of rule in a prompt, and only one of them can be written as a sentence.

Easy to state

One sentence covers it.

Reply in English. Never include a customer's email address. Stay under two hundred words.

These stay as instructions, because a model treats an instruction as a constraint.

Where the afternoon goes

Every sentence makes it fuzzier.

"Sound like our release notes." "The right level of detail." "Not too formal."

You know exactly what you mean and recognise it in a second when you see it — which is precisely the property an example has and a description does not.

When you find yourself writing a fourth sentence about how the answer should look, stop writing and show one

Think about teaching someone to make coffee the way you make it. You can write out a recipe — grind size, water temperature, thirty seconds of bloom — or you can make two cups in front of them and let them taste. The recipe is precise about whatever was easy to measure and silent about the rest. The demonstration carries the parts you never thought to name.

That is the whole idea. A prompt with no examples is zero-shot: you describe the job and hope the description was complete. A prompt with a few worked pairs in it is few-shot. The heuristic is blunt and it works: when you find yourself writing a fourth sentence about how the answer should look, stop writing and show one instead.

What three examples teach silently

Here is the same changelog job with the instructions cut down to two sentences and three worked pairs added underneath.

The model completes the pattern:

Read that against what you actually wrote. You never said to drop the feat(billing): prefix, or to describe repairs in the past tense and new abilities in the present, or to keep it to one sentence ending in a full stop, or to leave out file and module names. Rules you did not have to be able to state, carried by three pairs.

The examples a model traces over

Examples teach by resemblance, and resemblance does not distinguish between the properties you meant and the ones your examples happened to share.

Bad — two real examples that are both, by accident, bug fixes.

Good — the same two slots, spanning the kinds of change you actually ship.

The changelog is the one page customers read to find out what is new. The first set turns every entry into a repair, so the feature you spent a month building is announced as something that used to be broken — and nothing errors, so nobody notices until a customer asks when it is coming.

Cover the edge, not the middle

The instinct is to pick representative examples — ordinary, middle-of-the-road inputs showing the job at its most typical. That instinct picks exactly the inputs the model already handles without help.

Better: your examples are not a summary of your inputs, they are a map of the edges — the boundary of what you will actually receive. Write down how your real inputs vary. Commit messages vary in length, in kind (repair, new ability, removal, breaking change, internal housekeeping), and in whether they belong in a customer-facing changelog at all. Then fill four roles rather than picking three favourites.

A typical example anchors the format: one plain input, handled the plain way, teaching labels, length and tense.

An edge example sits at an extreme — the commit with a five-line body where one line matters to a customer, or the two-word commit with almost nothing in it. It stops the model assuming every input looks like the last one it saw.

A hard example is one where a reasonable person would hesitate. Is a breaking rename good news or a warning? The example is where you settle it, and it is the highest-value slot in the set: precisely where the model would otherwise guess, and it will guess the same way every time without ever telling you it guessed.

A refusal example is the input that should not get the normal treatment at all. That one earns its own section.

The example that says no

Look at what a set of confident answers teaches on top of format: that every input gets an answer. Every input in, a polished line out. Hand it chore(deps): bump the test runner to 9.2 and it will write you a polished line about that too, because it has never seen this prompt do anything else.

You can add "output SKIP for internal changes" as an instruction, and you should. But one sentence competes with every demonstration you gave, and the demonstrations are the thing whose shape the model can see. Show the refusal once and it becomes an available move:

Give the refusal a concrete, recognisable form — one word, in the same slot as a normal answer, so the code around your prompt can test for it with a single comparison. "I don't think this belongs in the changelog, but let me know if you'd like a line anyway" is a refusal you have to parse, and parsing polite prose is how a well-behaved model ends up posting an apology to your release notes.

One format, repeated exactly

Everything inside the examples is signal, including the parts you thought were packaging. Labels, order, capitalisation, quotes, the blank line between pairs — nothing tells the model which of those you meant and which you were careless about.

Bad — three pairs, three different shapes.

Good — one shape, repeated without variation.

With the first set, some answers come back wrapped in quotes, some carry a Changelog entry: prefix, and some arrive bare. Anything downstream that reads that text — a script pasting entries into a release draft — breaks on roughly one in twenty. That is the worst rate there is: too rare to show up while you are testing, common enough to be wrong in public every release.

Order matters more than people expect, too. The example closest to the real input tends to weigh heaviest, so put your most representative one last. When you are unsure about an example, move it and see whether the output moves with it.

How a narrow set narrows the model

Here is the part that catches people out. Examples do not only add; they subtract.

Before you add any, the model's range on this task is enormous. Every example pulls that range towards itself — that is the entire mechanism, and it is what you wanted. But nothing separates the narrowing you intended from the narrowing that came along with it.

Say all four of your examples happen to open with "You can now" or "Fixed". Feed the prompt a removal and you get "You can now no longer export in the old format." No competent writer produces that sentence, and the model was entirely capable of "The old export format has been removed" — you talked it out of it. The same effect crushes a two-sentence change into one because every example was one sentence, and forces an unfamiliar input into the mould of the nearest example instead of handling it on its own terms.

The dangerous word is silently. Nothing errors, and your spot checks look excellent, because you spot check the kind of input you had in mind when you wrote the examples. The damage lives in the inputs you did not think of, which is where damage always lives.

The zero-shot control

Run it with every example deleted.

Same prompt, same inputs, no examples. If the no-example version handles a case better, your examples caused that loss.

Deliberately unlike inputs

The five that make you wince.

The real inputs that least resemble any example you wrote — not the ones you are proud of.

Two cheap checks. The first takes two minutes and almost nobody does it.

Every example is a bill on every call

An instruction is written once and read on every request. So is an example — and examples are usually the largest thing in the prompt by a wide margin.

Four pairs of a sentence or two each is a solid paragraph riding along on every call your feature makes, every hour, forever. You are charged for what you send, so that paragraph has a running cost, and it is read before the first word comes back, so it is part of your latency too. At a thousand calls a day none of that is worth a meeting. At a hundred thousand it is a line item.

So examples are a trade, not a free win, and each has to earn its slot. Add them one at a time and stop when the next one changes nothing you can see. Then run the subtractive test: delete each in turn, re-run your awkward inputs, and keep only the ones whose removal makes something worse. Sets that survive that usually come out at two to four, not ten.

If you are at fifteen examples and still patching failures, few-shot has stopped being the right tool — the task is probably two tasks wanting two calls, or a constraint you keep demonstrating that should have been stated outright.

A cheat sheet to keep

Where to go next

You can now tell a rule worth stating from one worth showing, fill the four slots instead of picking three comfortable inputs, keep one format throughout, give the model a way to decline, and check what your examples cost in both quality and money.

The natural next step is why models hallucinate. You have given the model permission to skip a commit; the neighbouring question is what happens when it has no such permission and no idea of the answer — why confident invention is the default rather than a malfunction, and which popular fixes genuinely reduce it. Examples shape how an answer looks. That lesson is about whether the answer is true.

Tonight, run the subtractive test on a prompt you already use. Delete one example, run your ten most awkward inputs, and see whether anything gets worse. Then delete every example and run the same ten again. One of those runs will teach you something about your own prompt — usually that an example you were proud of was doing nothing, or that one was quietly holding the model back.