intermediate30 minby DevFox

Designing Tools a Model Can Use Well

A tool description is a prompt. Naming, granularity, argument shape, error messages that teach the model to recover, and why fewer sharper tools beat a large flexible one.

Your refund tool works. You wrote it, you tested it, and it does what the ticket asked: pass it a subscription id and an amount, it calls billing, it returns a receipt. Then you hand it to a model and the first trace comes back with subscription_id: "sub_current" — a string that appears nowhere in the conversation, invented because the model needed one and had no way to obtain one.

Nothing in your code is broken. The failure is in the eighty words you wrote to describe it, and that is where most tool-calling failures actually live. By the end of this lesson you will be able to read a tool definition the way the model reads it — as a prompt — and predict the wrong call before it happens, from the name, from the argument shape, from the size of what comes back, and from what the errors say when it fails.

The model only ever sees the label

A tool definition is a name, a description, and a schema for the arguments. That is the entire surface. The model cannot read your source, cannot open the README, cannot ask the colleague who wrote the billing integration what a sub_ prefix means. It sees the label on the jar and nothing else.

Picture a contractor's first hour. You hand them one page listing what they are allowed to do, then leave. Everything not on that page they fill in by guessing, confidently, because guessing is the only move available. The page is not documentation for later reference — it is the whole briefing.

Which means a tool description behaves like a prompt in every way that matters. It is read on every request, it competes for attention with your instructions and everything retrieved into context, it costs tokens whether or not the tool gets called, and changing one sentence changes behaviour. So build the habit: when the model misuses a tool, edit the description before you touch the implementation.

Names and descriptions that say when to use it

Selection happens in two passes. The model scans the names to build a shortlist, then reads the descriptions of the survivors to choose. So a name has to survive being read beside its siblings, with no other context.

Three things make that work. Use verb_object, so the name says what happens rather than what exists — cancel_subscription, not subscription_manager. Name the user's intent, not your implementation: search_customers, not query_customer_index, because the model is matching against a request written in human terms. And keep one verb vocabulary across the set — if one tool is get_customer and another is fetch_invoice, the model has to work out whether the two verbs mean different things. They do not, but it cannot know that.

The description does the harder half of the job. It has to say what the tool does, when to reach for it, when not to, and where its arguments come from.

Bad — accurate, and gives the model no way to know where an id comes from.

Good — same tool, same schema, with the provenance of the id and the fallback spelled out.

The first version does not fail loudly. It fails with a plausible-looking id, a not-found error, and a model that tries two more spellings before giving up — three round trips of latency and cost, and a support reply saying the customer does not exist, for want of a sentence nobody wrote.

Several sharp tools beat one flexible one

The tempting design is one tool with an action argument: manage_subscription(action="cancel" | "refund" | ...). It looks tidy, it mirrors your service class, and it keeps the tool list short. It also reliably produces wrong calls, for two reasons.

You split one decision into two
Picking the tool happens with a description in front of the model. Picking the mode happens off a comma-separated list inside that description — no guidance, no note that one of the four is irreversible.
The most consequential choice now has the least help.
The schema stops being able to help
No schema can express "amount_cents is required, but only when action is refund", so the field has to be optional everywhere and nothing is checked.

Both problems come from the same move: putting the real choice somewhere the tooling cannot reach it

Bad — one schema for four jobs, so no field can be marked required where it actually matters.

Good — one job per tool, so each schema can require exactly what its job needs.

In the first version amount_cents cannot be required, because three of the four actions have no use for it. So the model omits it on a refund and your handler either refunds the whole charge or throws at runtime — while the schema, the one component that could have caught this before the call was made, was structurally unable to.

Where to draw the line: split on the decision, not on the endpoint. If two operations get chosen for different reasons they are two tools, even when they hit the same route. If a value varies in a way the caller never deliberates over — a page size, a locale — it is an argument. And split on risk even when your API does not: separate read and write tools are what let you gate the writes behind a confirmation later.

Arguments the model can actually fill

Every argument is a question you are asking the model, and it will answer all of them, including the ones it cannot know. Three rules cover most of the damage.

Prefer a closed set to a free string. A status typed as string invites "cancelled" when your database stores "canceled", and the result is an empty list rather than an error — a silent wrong answer, the worst kind. An enum turns that into a validation failure before the call is made, which is the cheapest correctness you can buy anywhere in an AI system.

Keep required fields few. Each one is another opportunity to invent something. Give anything with a sensible default an optional slot and name the default in the field's description, so the model can leave it alone.

Never ask for knowledge the model has no route to. Internal numeric ids, tenant ids, cursor tokens it was never handed, epoch-millisecond timestamps when the user said "last Tuesday" — these come from your code, not the conversation. Accept 2026-07-29 and resolve relative dates on your side.

Return values sized for a context window

A tool's return value is not a response to a database client. It is text pasted into the model's context, where it stays for the rest of the conversation. Forty columns of nulls, internal flags and audit timestamps cost you on this request and every request after it, and they bury the three fields the answer needed.

Return the fields that answer the question plus the ids needed to make the next call, and nothing else. For lists, give a total, the page you are returning, and an explicit way to continue.

total_matches tells the model its query was too broad without making it read thirty-four records to find out. next_cursor gives it a way to continue that does not require inventing an offset — the previous section's rule, applied to what you hand back rather than what you ask for.

The failure to avoid here is the silent truncation. A tool that quietly returns the first ten of hundreds teaches the model there were ten, and it will then say so with confidence, because from inside the context there is no evidence otherwise. Say the real number, always, even when the number is unhelpful.

Errors that teach the model what to do next

When a tool fails, the model gets your error text and chooses its next action from that text alone. That makes an error message the highest-leverage prose in the whole definition: it is read at exactly the moment the model is about to do the wrong thing twice.

Bad — reports that something failed, not what to do differently.

Good — names the cause and the recovery in the same breath.

With the first version the model's cheapest next move is to retry the same id, or a near variant, so one failure becomes three and the user still gets "I could not find that customer". The second makes the recovery the obvious next step, and the following call is usually right.

Two habits follow from the same idea. On a validation failure, quote the allowed values back: "status must be one of active, past_due, canceled — you sent 'cancelled'" repairs itself in one turn, where "invalid status" does not. And distinguish "you called this wrong" from "the system is unavailable", because they call for opposite responses — fix the call, versus stop and surface it.

How many tools is too many

There is no magic number, and raw count is not what hurts. Overlap is. Ten tools with sharp boundaries are easier to choose between than four whose descriptions could each plausibly answer the same request. Difficulty scales with similarity, not with length of the list.

There is a quick test. Read your descriptions side by side and try to write one user request that two of them both plausibly serve. If you can write that sentence, the model will pick wrong some fraction of the time. The fix is rarely deletion — it is a boundary sentence in each: "Use search_articles for public help content; use search_tickets for this customer's own past conversations."

Past a few dozen tools, two things degrade together: selection accuracy drops, and you spend context describing tools this request was never going to need. Three remedies, in the order worth trying — prefix names by domain (billing_, account_) so the shortlist is scannable; expose only the tools relevant to the current phase of work rather than the whole catalogue every time; and delete the ones traces show are never called.

A cheat sheet to keep

Where to go next

You can now look at a tool definition and see the calls it will produce: whether the name survives being read beside its siblings, whether the schema can require what the job needs, whether every argument is answerable from the conversation, whether the result is sized for a context window, and whether a failure hands back a next step or a dead end.

The natural follow-on is Skills and Reusable Capability, which answers what this lesson does not: what to do when the thing you want the model to be good at is not a function call at all, but a procedure — instructions, worked examples and assets, loaded only when the work calls for them. A tool gives the model a verb; a skill gives it a way of working, and the two are designed by different rules.

The thing to go and do: take the tool in your codebase that gets called wrong most often and, without touching its implementation, change three things. Add a sentence to the description naming what to use instead of it. Replace its widest free-string argument with an enum. Rewrite its most common error message so it names the recovery. Then re-run the ten requests that were failing. How many fix themselves is the measurement, and it tells you where the rest of your tool bugs are hiding.