intermediate30 minby DevFox

Tools the Model Can Call

How tool calling actually works — schema, invocation, result, continuation — and what to do about the model calling the wrong tool, the right tool with wrong arguments, or no tool at all.

Your support assistant has one job: tell a customer where their order is. You ask it about order A-48213 and back come three fluent sentences about a delivery window that does not exist. It has never seen your orders table and it never will. The answer is thirty milliseconds away in your database, and the model has no way to reach it.

Tool calling is the bridge, and it is less magical than it sounds: you hand the model a list of functions it may ask for, it asks, and your code decides whether to run them. By the end of this lesson you will have the whole round trip in your head — declaration, request, execution, result, continuation — and you will know what to do when the model picks the wrong function, calls the right one with nonsense arguments, or announces that it is looking something up without ever asking you to.

The model asks, your code answers

Start with the correction that saves the most confusion later: the model does not run your tools. It cannot. A model turns one sequence of tokens into another, and that is the entire range of things it does. When you read that a model "called your function", what actually happened is that it produced a structured request naming a function and some arguments, and the API handed that request to you.

It is a request slip, not a remote control. The kitchen still decides whether to cook the ticket.

Three consequences follow immediately, and each one matters:

If your code does nothing with the request, nothing happens. There is no side effect, no database read, no email sent.
You are allowed to decline. A tool request is a suggestion from a text generator, and you can refuse it the way you would refuse any other unvalidated input.
The model learns nothing about the outcome unless you send the result back to it. It has no memory of an execution it never performed.

That third one is why this is a round trip and not a function call. Getting the trip right is most of the work.

What you actually declare

A tool declaration is a name, a description in plain English, and a schema describing the arguments. It goes into the request alongside your messages.

python

tools = [
    {
        "name": "get_order_status",
        "description": (
            "Look up the current status and delivery estimate "
            "for one order, by its order number."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "order_id": {
                    "type": "string",
                    "description": "Order number, like A-48213.",
                },
            },
            "required": ["order_id"],
        },
    },
]

Field names differ between providers — some call the schema parameters, some return calls in a separate list rather than inline — but the three parts are the same everywhere, and so is the shape of the exchange below.

Two mechanical facts. Declarations travel with every request, so forty tools cost forty tools' worth of tokens on every turn, used or not. And how strictly a provider enforces the schema varies, so treat it as a hint you still have to check rather than a guarantee you have already been given.

The round trip, message by message

Five steps, and the message list grows at each one.

You send the question and the tool list
One user message, plus the declarations.
The model asks for a tool
A stop reason saying so, and a block carrying an id, a name and an input.
You append that turn unchanged
Including any text that came alongside the request.
You run the tool and append the result
Carrying the same id, because that is the only thing pairing an answer to a question.
You call the model again
It now sees its own request and your answer, and writes the sentence the customer reads.

Two calls to the model for one tool use. Worth knowing before you promise anyone a latency number.

Start with the user's question:

python

messages = [
    {"role": "user", "content": "Where is order A-48213?"},
]

response = model.create(messages=messages, tools=tools)

The response comes back with a stop reason — the field naming why the model stopped generating. When it wants a tool, that field says so, and the response content contains a tool_use block: a generated id, the tool name, and an input object of arguments.

python

response.stop_reason        # "tool_use"
block.id                    # "toolu_01A9k..." — generated per call
block.name                  # "get_order_status"
block.input                 # {"order_id": "A-48213"}

Now your half. Append the assistant's turn to the conversation unchanged, run the tool, and send the result back as a new message:

python

messages.append(
    {"role": "assistant", "content": response.content}
)

output = get_order_status(**block.input)

messages.append({
    "role": "user",
    "content": [{
        "type": "tool_result",
        "tool_use_id": block.id,
        "content": output,
    }],
})

response = model.create(messages=messages, tools=tools)

That second call is the continuation.

Note that the result is content in a message, not a return value. Some APIs put it in a message with role user, others in a dedicated tool role. Either way, the model reads it as text in the conversation, exactly as it reads everything else.

Zero calls, one call, or four

The stop reason is a branch, not a formality. A turn that requested a tool and a turn that answered directly need completely different handling, and both are normal.

Zero calls is often correct. Ask "do you ship to Norway?" and a good model answers from your system prompt without touching the orders table. Treating every turn as a tool turn is how you end up looking up an order number the customer never mentioned.

Several calls in one turn is also normal. Ask "where are orders A-48213 and A-48219?" and you can get two tool_use blocks in the same response. The rule is strict: one result per request, all in the same message, before you call the model again.

Bad — takes the first tool request and silently discards the rest.

python

call = next(
    block for block in response.content
    if block.type == "tool_use"
)
messages.append(
    {"role": "assistant", "content": response.content}
)
messages.append({
    "role": "user",
    "content": [{
        "type": "tool_result",
        "tool_use_id": call.id,
        "content": run_tool(call.name, call.input),
    }],
})

Good — one result per request, collected into a single message.

python

results = [
    {
        "type": "tool_result",
        "tool_use_id": block.id,
        "content": run_tool(block.name, block.input),
    }
    for block in response.content
    if block.type == "tool_use"
]
messages.append(
    {"role": "assistant", "content": response.content}
)
messages.append({"role": "user", "content": results})

The first version fails loudly on providers that validate the pairing and, worse, quietly on those that do not: the model answers about both orders while having heard back about one, and invents the other. The bug appears only when a customer asks about two things at once, which is exactly the request nobody tested.

A model that reads a result and decides to call another tool, step after step of its own accord, is running an agent loop with its own termination problems — the lesson called The Agent Loop. Here, one turn of tools and one continuation.

When it calls the wrong tool

Now the failures, starting with the one that looks like a bug in the model and is usually a bug in your dispatch.

The customer asks where their order is. The model requests cancel_order. It has picked a plausible-looking neighbour from a list of order-related functions, and if your code runs whatever it is handed, a customer asking a question has their order cancelled.

Two mechanical defences, both cheap. Dispatch through an explicit map, so the set of reachable functions is a thing you wrote down rather than a thing the model can name. And treat an unrecognised name as an ordinary result, not an exception:

python

HANDLERS = {
    "get_order_status": get_order_status,
    "search_orders_by_email": search_orders_by_email,
}

def run_tool(name, arguments):
    handler = HANDLERS.get(name)
    if handler is None:
        return f"No tool named {name!r} is available."
    return handler(**arguments)

Declining is a legitimate outcome. If a request arrives for a tool you will not run in this context, return a result saying so in plain language — "cancel_order is not available here; the customer asked for status" — and let the model continue. It reads that the way it reads any other tool result, and usually corrects itself on the next turn. Reducing how often the wrong tool gets picked at all is a naming problem, and belongs to the next lesson rather than this one.

When the arguments are wrong

Right tool, wrong contents. This is the most common failure of the three, and it takes a handful of shapes: a required field missing; an order number as 48213 when your schema said A-48213; a date as the string "next Tuesday"; an invented extra field that was in no schema anywhere.

The instinct is to let it throw. Resist it. An exception here kills the request, and the model never finds out what it got wrong — you have converted a recoverable mistake into a failed conversation.

Bad — a malformed argument raises out of the handler and ends the conversation.

python

def run_tool(name, arguments):
    return HANDLERS[name](**arguments)

Good — validation failures come back as a tool result the model can read and correct.

python

def run_tool(name, arguments):
    handler = HANDLERS.get(name)
    if handler is None:
        return f"No tool named {name!r} is available."
    try:
        parsed = SCHEMAS[name].model_validate(arguments)
    except ValidationError as error:
        return f"Invalid arguments: {error}"
    return handler(parsed)

The bad version turns a typo the model would have fixed on its own into a stack trace and an error page. The good version costs one extra round trip and usually ends with the right answer, because the model reads "order_id: string does not match pattern A-nnnnn" and sends A-48213.

Two boundaries on that generosity. Bound the correction — two attempts at the same tool, then stop and tell the user, or you have built a loop that pays per iteration. And keep the error text about the input, never your internals: a validation message is fine, a database exception with table names is not.

When it only says it called one

The third failure is the strangest, and the one that catches people who have wired everything else correctly. The model replies:

text

Let me look that up for you.

The stop reason says the turn ended normally. There is no tool_use block anywhere in the content. Nothing was requested, nothing ran, and a customer has been told to wait for something that is not coming.

This is not the model lying. It is a text predictor, and text written by helpful assistants is full of sentences like that one, so sometimes the most probable continuation is the announcement without the request. It happens most when a description is vague or the question sits at the edge of what a tool covers.

The defence is a rule worth writing on the wall: a tool call is a structured block, never a sentence. Never infer one from prose.

Bad — a promise to act is returned to the user as though it were an answer.

python

def answer(messages):
    response = model.create(messages=messages, tools=tools)
    if response.stop_reason == "tool_use":
        return run_tool_turn(messages, response)
    return response.text

Good — a turn that ends with no call and no data behind it gets exactly one re-ask.

python

def answer(messages, nudged=False):
    response = model.create(messages=messages, tools=tools)
    if response.stop_reason == "tool_use":
        return run_tool_turn(messages, response)

    # Ended the turn, requested nothing, and no tool result is
    # in the transcript: whatever it said, it had no data.
    if not nudged and not any_tool_result(messages):
        messages.append(
            {"role": "assistant", "content": response.content}
        )
        messages.append({
            "role": "user",
            "content": (
                "Call a tool to answer, or say you cannot."
            ),
        })
        return answer(messages, nudged=True)

    return response.text

The bad version ships "let me look that up" as the final answer, which reads as a hang and produces a support ticket about your support assistant. The nudged flag matters as much as the check — without it, a model that keeps narrating puts you in an unbounded loop, and you are paying for every turn of it.

The decision to run it is yours

Everything above assumes you execute what you validate. That is fine for reads and dangerous for writes, and the mechanism gives you the seam for free: between the request arriving and your handler running, there is a place to put a decision.

Split your tools by whether they change anything. Looking up an order status is a read — run it, and the worst case is a wasted query. Cancelling an order, issuing a refund, sending an email: these run once and stay run. For those the request is an intention, and something other than the model approves it — a confirmation from the user, a policy check, a human in the queue above a threshold.

python

NEEDS_APPROVAL = {"cancel_order", "issue_refund"}

if name in NEEDS_APPROVAL and not user_confirmed(name, arguments):
    return "Not executed: the customer has not confirmed this."

The return value there is the point. Declining is a normal tool result, so the conversation carries on and the model can explain to the customer what it needs — rather than an exception that drops the whole exchange.

There is a much larger version of this argument, because a tool result is untrusted text arriving in the same context as your instructions, and a document the model reads can try to talk it into calling something. Least privilege for tools and the trust boundary around their output are the subject of Prompt Injection and the Trust Boundary. For now, one line: the model's request is data, and data is never authorization.

A cheat sheet to keep

python

tools=[...]                      # travels on every request; costs tokens
response.stop_reason             # "tool_use" means it asked; branch here
block.type == "tool_use"         # the only honest signal a call happened
block.id                         # pairs the result to this exact request
block.name, block.input          # what it asked for, still unvalidated
messages.append(assistant_turn)  # echo the request turn back, unchanged
{"type": "tool_result",          # one of these per tool_use block
 "tool_use_id": block.id,        # ...matched by id, never by position
 "content": output}              # text the model reads; errors go here
model.create(messages, tools)    # the continuation — second call, always
HANDLERS[name]                   # explicit map; never getattr or globals
HANDLERS.get(name) is None       # unknown tool -> error result, not raise
schema.model_validate(args)      # validate before you execute
return f"Invalid: {error}"       # return the failure, do not throw it
attempts <= 2                    # bound self-correction; then give up
not any_tool_result(messages)    # narration guard: nudge once, then stop
name in NEEDS_APPROVAL           # side effects wait for a human yes

Where to go next

You can now run the full exchange — declare, branch on the stop reason, echo the assistant turn back, return one result per request with its id, continue — and you have an answer for each of the three ways it goes wrong: an explicit dispatch map, validation errors returned as results, and a structural check plus one bounded re-ask.

What this lesson deliberately did not answer is why the model picked wrongly in the first place. That is Designing Tools a Model Can Use Well, which comes next: the description as a prompt, granularity, argument shapes that are hard to get wrong, and error text written to teach recovery rather than report failure. Most of what looks like flaky tool use turns out to live there.

Go and build the smallest possible version tonight. Declare two tools over data you actually have, ask a question that needs one of them, and print the raw content blocks instead of the final text — see the id, the arguments, the stop reason with your own eyes. Then ask something neither tool covers and watch a turn come back with no call at all. Those two runs make the rest of the course concrete.