intermediate30 minby DevFox

The Agent Loop

Think, act, observe, repeat — and the four termination conditions that stop it running away. Step budgets, progress detection, and why an agent that cannot stop is worse than one that cannot start.

Your ticket-triage agent ran overnight. This morning one ticket in the queue has been touched a hundred and forty-three times: search the orders table, read order 88214, search again with the words rearranged, read 88214 again. Every call succeeded. Every call, on its own, was a reasonable thing to do. The ticket is still open, the customer is still waiting, and you have a bill for the night.

You already know the round trip — the model asks for a tool, you run it, you hand the result back. This lesson is about what happens when you put that round trip inside a while loop and walk away. The loop is about ten lines of code. Everything difficult about it is the question of when it stops, and by the end you will have four independent answers to that, a contract for what a stopped run hands back, and a test for whether you needed a loop in the first place.

The loop itself is small

An agent, stripped of everything marketing has attached to the word, is a model calling tools in a loop.

Think
The model reads the conversation so far and decides what to do next.
Act
It calls a tool, with arguments it chose.
Observe
The result goes back into the conversation, and round again.

Three steps, repeated. Everything else in this lesson is about deciding when to leave.

The whole architecture fits on a screen:

python

messages = [{"role": "user", "content": task}]

while True:
    reply = model.respond(messages, tools=TOOLS)   # think
    messages.append(reply)

    if not reply.tool_calls:
        return reply.text

    for call in reply.tool_calls:                  # act
        result = run_tool(call.name, call.arguments)
        messages.append(tool_result(call.id, result))  # observe

That is a working agent. It is also the version almost every team writes first, and it has exactly one exit: the model chooses not to call a tool. Every stop condition is delegated to the thing whose judgement you are trying to bound. The rest of this lesson is the control you build around those ten lines.

What changes the moment it is a loop

Three things break that were not broken in a single call.

Cost is multiplied by a number you did not choose
Every result is appended to the transcript, so step twelve pays to re-read steps one through eleven. Ten steps costs around fifty-five times the input of one, not ten.
A loop that usually takes three steps and occasionally takes twenty has a worst case forty times its typical case.
A wrong step becomes the input to the next one
In a single call a bad answer is a bad answer. In a loop it is fact number four in the transcript, and the model reasons forward from it with the confidence it gives the real ones.
The agent that misread order 88214 as 88124 does not discover the mistake — it builds four internally consistent steps on top of it.
Nobody is between the steps
The colleague who would have glanced at call one and said "that is the wrong order" is asleep. Every guardrail has to be code, running before the next call goes out.

None of these is a model quality problem. All three are consequences of the loop existing at all.

Terminator one: an explicit done signal

Stopping when the model returns no tool call is the default, and it mostly works. Its problem is that it conflates two different events: I have finished the task and I have nothing to call right now. A model that thinks out loud for a turn, hits a refusal, or writes its plan as prose all look identical to done.

Give the model an unambiguous way to say it is finished — a terminal tool the loop watches for — and treat everything else as not finished.

Bad — reads intent out of prose, so a sentence about finishing ends the run.

Good — the run ends only when the model calls the tool that carries the result.

The first version stops on "I will mark the task complete once the refund clears" — a sentence about finishing, not a finish — and the caller receives that sentence where it expected a refund ID. The second cannot fire by accident, and it hands back a typed object instead of prose your caller has to parse. How you name and describe that tool so the model reaches for it at the right moment belongs to Designing Tools a Model Can Use Well; here it only has to exist.

Once finish exists, a reply with no tool calls stops meaning "done" and starts meaning something more useful: the model stopped without finishing. That is its own outcome, worth its own status, and worth counting — it is one of the most common ways real agents end.

Terminators two and three: the budgets

Three budgets, not one, because a runaway loop has three shapes and each ceiling is blind to the other two.

A step budget catches the fast spinner: twelve cheap searches a second apart, exactly the overnight run above. A wall-clock budget catches the slow crawl: three steps, each waiting ninety seconds on a reporting query, where the step count never gets near its limit while your request handler times out. A token budget catches the fat context: one tool returns a two-hundred-page PDF and you are one step from the context limit with a step count of two.

The numbers matter more than the mechanism. Work the task by hand, count the steps a competent colleague takes, and set the ceiling a little above the slowest real run you have logged. If you picked 50 because it felt safe, you did not set a budget — you wrote "never" in a place where a number goes, and the loop will still be running when the bill arrives.

Terminator four: no progress

Budgets stop a runaway eventually. Progress detection stops it as soon as it becomes pointless, which is usually several steps and a lot of money earlier. Two signatures cover most of it.

The first is repetition: the same tool with the same arguments. Fingerprint each call and count what you have seen.

If search_orders is deterministic, its third identical call returns the answer already sitting in the transcript. No new information can enter the loop from it.

The second is oscillation: two calls that undo each other. assign_queue("billing"), then assign_queue("shipping"), then assign_queue("billing") again. No fingerprint repeats twice in a row, nothing looks stuck, and the ticket is exactly where it started. Keep the last handful of fingerprints in a window and flag it when the window contains a cycle — a state you have already been in.

What you do on detection matters as much as the detection.

Bad — kills the run on the first repeat, so the model never learns what it did.

Good — says what was already called, once, and stops only if it happens again.

A model that repeats a call has usually lost track of a result it already has, and one sentence of feedback recovers a good share of those runs. The strict version throws away the run that was one nudge from finishing, and you pay for every step it took to get there. Nudge once, though — a nudge costs a step of its own, and an agent being told the same thing three times is a loop wearing a different hat.

Stopping is not the same as failing silently

The four terminators give the loop several ways out, and the caller has to be able to tell them apart. Make every exit path return the same shape: what the status was, whatever work got done, and what was tried.

Bad — the caller cannot distinguish a budget stop from a refusal from a crash.

Good — same stop, but the caller gets the work so far and the reason there is no more.

With None, the ticket goes back in the queue looking untouched, and the next run spends the same twelve steps reaching the same wall. With the partial result, a human opens it and sees the refund amount was already calculated and only the approval is missing — thirty seconds of work instead of a repeat of the night.

Where the human goes in the loop

There are three places to put a person, and they catch different failures at different prices.

Before the loop, approving a plan. Cheap to build and it feels reassuring, but a plan is not a commitment — the agent can approve a three-step plan and take nine. Useful for expensive runs, not a safety mechanism.

Inside the loop, gating specific actions. This is the one that earns its keep, and the rule is to gate on the action, not on the step number. refund_order always pauses for approval; search_orders never does. Classify each tool once by whether its effect can be undone, and let the loop read that classification instead of asking a person every fifth step regardless of what is happening. Which actions deserve the gate gets sharper still when the transcript contains documents you did not write, which is the subject of Prompt Injection and the Trust Boundary.

After the loop, reviewing the result before it reaches the customer. The cheapest to build, and the only placement that catches a wrong answer produced by an unbroken chain of reasonable steps.

Gating inside the loop has one engineering consequence worth planning for: a loop that can pause has to be able to stop and resume. The messages, the step count, and the clock start have to be serializable state you can write down and pick up an hour later, not local variables in a request handler that dies at the gateway timeout. If a pause means a worker thread parked until someone comes back from lunch, you have built a queue with extra steps and a much worse failure mode.

The loop you did not need

Now the question worth asking before any of the above: does this need to be a loop at all?

Take "summarise the ticket, then route it to one of six queues." That is two model calls in a fixed order. Written as a pipeline it has two failure points, both testable, and it costs exactly what you think it costs. Written as an agent it is the same two calls plus a model deciding whether to make them, plus a chance it routes before summarising, plus a chance it summarises twice, plus a step budget, plus progress detection, plus a partial-result path. You have added every failure mode in this lesson to a task that had none of them.

A fixed sequence has as many things that can go wrong as it has stages, and you can enumerate them. A loop has as many as the model decides to create on the day, and you cannot enumerate paths that do not exist until runtime — which means your test cases cover the paths you thought of, and production supplies the others.

The test is one question: does the number of steps depend on what you find? Summarise-then-route is always two steps, so write two steps. "Work out why this order shipped late" might be one lookup or six, depending on whether the delay was stock, the carrier, or a failed payment retry — you cannot write that sequence down in advance, so a bounded loop is the right shape and worth its cost.

The middle ground is usually the best answer for real features: a fixed pipeline where exactly one stage is a bounded loop. You keep the parts you can name, and the model only gets to improvise inside the fence where improvisation is the point.

A cheat sheet to keep

Where to go next

You can now build a loop that ends on purpose: a terminal tool for success, three budgets for the three shapes of runaway, progress detection for the run that is alive but going nowhere, and one result type that tells the caller which of those happened and hands over the work that got done.

The natural next lesson is Failure Modes, Retries, and Fallbacks. This lesson stops a loop that will not end; that one handles the single step inside it that breaks — a timeout, a rate limit, a response cut off mid-object — and which of those are worth trying again at all, given that every retry spends from the same step budget you just set.

The thing to go and do: take a loop you already have, add a stop reason to whatever it returns, run twenty real tasks through it, and count how each one ended. Nearly everyone is surprised by the distribution. Most runs do not end on the budget you spent an afternoon tuning — they end on the model going quiet without calling anything, and that is a different problem with a different fix.