advanced35 minby DevFox

Multi-Agent Orchestration

When splitting work across agents genuinely helps and when it is expensive theatre. Fan-out and verify, adversarial review, coordination failure, and the token cost nobody budgets for.

AI
Architecture

The dependency audit came back in ninety seconds instead of eleven minutes, and the room was pleased until someone opened the usage report. Eight sub-agents, one per service. Each of them re-read the migration spec, re-loaded the same twelve tool schemas, and re-derived the same three naming conventions before it touched a single file. Two of them audited billing, because the split was written in prose and prose overlaps. And the summary at the top stated a library version that appears in none of the eight transcripts.

Every one of those problems is specific to having more than one agent, and none of them showed up in the demo. This lesson is about the line between the case where a second agent is the only thing that works and the case where it is an expensive way to re-solve a solved problem. By the end you will have three patterns that genuinely pay, the accounting for what they cost, four failure modes that do not exist below two agents, and one test to apply before you reach for any of it.

What a second agent actually buys

A sub-agent is a full agent loop — its own transcript, its own tools, its own step budget — started by another agent and returning a result into that agent's transcript. The agent that starts it and consumes the results is the orchestrator.

Mechanically that is a function call with two unusual properties. The first is that the sub-agent has a separate context window: it reads what its brief gives it and nothing else, and only its final result comes back. The second is that it forms its own judgement, unanchored by whatever reasoning the orchestrator has already committed to in its own transcript.

Those two properties are the entire product. Everything else people reach for multi-agent to get — decomposition, specialist behaviour, a defined workflow — one agent gives you from tool design and a well-written brief, at a fraction of the cost. So there are exactly two reasons to spawn a second agent, and they map onto those two properties:

Separate contexts

The work splits.

Pieces that do not need to see each other can run at once, each reading only what its brief gives it.

Separate judgement

The checking splits from the doing.

A verdict formed without the maker's reasoning in front of it catches what the maker cannot.

Two reasons, and they are the same property arranged two different ways

Independence, in one form or the other. Hold onto that; every pattern below is a way of arranging it, and every failure mode below is what happens when it was not really there.

Fan-out over work that does not talk to itself

The first pattern that pays is fan-out: N sub-agents, each given one item from a list, running at the same time.

The qualifying condition is strict, and it is one question you can answer before writing any of this: can item B be worked on before item A finishes? For forty repositories checked against one spec, yes — nothing in repository nineteen depends on what repository four turned out to contain. For a migration where each service's changes constrain the next, no, and no amount of orchestration makes it yes.

Bad — fans out over the stages of one task, which are ordered, so nothing runs in parallel.

Good — fans out over the items, which are independent, and keeps the stages in one agent.

The first version is a pipeline wearing a fan-out's clothes. It runs strictly sequentially, so you get no wall-clock win at all, and you pay three agent setups plus two lossy hand-offs where each stage sees only its predecessor's summary instead of the evidence behind it. The drafting agent argues from an outline it cannot check.

One consequence of the code above is easy to miss: each sub-agent carries its own step budget, and the orchestrator's budget does not bound them. Forty agents at fifteen steps each is a six-hundred-step system, and the ceiling that matters is the product, not the number you typed.

Generate, then verify independently

The second pattern is generate-then-verify: one agent produces the artefact, a second one attacks it. This is the pattern with the highest hit rate in real systems, and it is also the one most often built in a way that destroys its own value.

The value comes from one thing only — the reviewer's judgement being formed independently of the generator's. Hand the reviewer the generator's transcript and you have not built a review. You have built a very expensive agreement, because the reviewer now reads a fluent, internally consistent argument for why the artefact is correct, and it has all the same blind spots the argument was built from.

Bad — the reviewer reads the case for the defence before it reads the evidence.

Good — the reviewer gets the requirement, the artefact, and tools to check it against.

The first version's agreement rate goes up and its defect detection goes down at the same time, which is the worst possible pair of movements because the first is the one you notice. You end up paying for a second full agent to produce a rubber stamp, and shipping with more confidence than before you had a reviewer at all.

Two more things separate a reviewer that finds bugs from one that writes praise. Give it an adversarial mandate — "find an input for which this is wrong" produces work; "is this good?" produces a compliment. And give it tools that make its findings checkable, so a claimed failure comes with a command you can run rather than a paragraph you have to believe. A reviewer that can execute the test it just wrote is doing engineering; one that can only read is doing literary criticism, and you already know a model scoring text drags position and verbosity biases in with it.

A panel of independent attempts

The third pattern is the panel: run the same task N times independently, score the attempts, keep or synthesise the best. It is the only one of the three that deliberately multiplies the whole cost of the task, so it needs the clearest justification.

It pays when checking is cheaper than producing. A schema migration that either applies cleanly or does not, a query that either reconciles against the ledger or does not, a fix that either turns the failing test green or does not — in each of those the score is a fact, and five attempts against a fact is five lottery tickets on a task where the model succeeds sixty percent of the time.

Two conditions decide whether the panel is real or theatre.

The attempts have to actually differ. Identical briefs at a low sampling temperature give you five near-copies, and you paid five times for one answer with better formatting. Vary something that changes the approach: the framing of the task, the tools offered, the constraint each attempt is told to optimise for.

And the scoring has to be independent of the failure mode you are trying to catch. When the score is executable, it is. When the score is one more model reading five answers, errors that come from a shared blind spot survive the vote intact — five agents that all misread the same ambiguous requirement produce five agreeing answers, and the panel converts a wrong answer into a wrong answer with corroboration. Agreement between correlated judges is not evidence. It is the same judgement, billed five times.

The accounting nobody does up front

Now the bill, precisely, because "it costs more" is not actionable and the direction is not always what people expect.

Fixed setup multiplies by the number of agents. Every sub-agent pays for its own system prompt, its own tool schemas, its own brief and whatever conventions it needs restated, before it does one unit of useful work. Call that six thousand tokens in a serious system; across eight sub-agents it is forty-eight thousand tokens spent restating the problem. This cost is real in every pattern and it is invisible in every demo, because at N of two nobody looks.

Redundant patterns multiply the work itself. A panel of five costs five times the task; generate-then-verify costs the generation plus a substantial fraction again. That is not a defect — it is what you are buying — but it means the value has to clear a bar you can state, not a feeling that more agents are more thorough.

Fan-out over disjoint items divides the work tokens, and this is the honest surprise. A single agent grinding through forty repositories carries every previous repository's evidence in its transcript, so repository thirty pays to re-read twenty-nine. Forty sub-agents each read one. The work term goes down; the setup term goes up by forty. Which wins depends on your ratio, and you can measure it in an afternoon instead of arguing about it.

Latency is the slowest agent, not the average. This is worse than it sounds, because you are no longer sampling the distribution once. If one run in twenty stalls near its step budget, a single agent stalls one time in twenty and a fan-out of twenty stalls almost every time. Fan-out converts your tail latency into your typical latency. Add the serial head — the orchestrator planning the split — and the serial tail, where it reads everything back, and a "parallel" system is often half sequential by wall clock.

The orchestrator's own context fills with summaries. Each returned result is appended permanently, so by the time all eight are back, the agent whose job is to reason across all eight is doing it in the fullest, most degraded window it has had all run. Worse, it is reasoning over compressed evidence while each sub-agent had the original — the orchestrator is the least well-informed participant in a system it is supposed to adjudicate.

Per-request cost modelling and the levers that bring these numbers down are their own subject in The Economics of an AI Product. What belongs here is one habit: measure per-agent tokens and per-phase wall clock separately, or you will attribute a fan-out's cost to the wrong place and optimise the part that was already cheap.

Failure modes that need two agents to exist

Four of these. None has an analogue in a single loop, which is why the instincts you built there do not warn you about them.

Duplicated work. Two sub-agents given overlapping scope do the same reading and you pay twice for one answer. The cause is almost always a partition described in language rather than constructed.

Bad — scopes described in prose, and prose overlaps.

Good — scopes are disjoint because of how they were built.

Billing is payment-related, so the first version reads it twice — and the two readings will not agree in every detail, which means you have manufactured a contradiction out of nothing but a sloppy split, and will now spend orchestrator turns resolving it.

Contradiction with no tie-break. Two sub-agents return incompatible claims and nothing in the system is authorised to decide. In practice the winner is whichever one hedged least or happened to be summarised last — confidence in the prose, decided by writing style. The fix is to choose the arbitration rule before the fan-out: an executable check, a designated authoritative source, or an escalation to a person. "The orchestrator will work it out" is not a rule, because the orchestrator has strictly less information than either of the agents that disagreed.

Laundering. A plausible wrong claim enters at the bottom, gets summarised into a sub-agent result, gets summarised again into the orchestrator's report, and arrives as a confident sentence with no path back to the tool call that produced it. This is how the version number in the opening scene got there. Every summarisation hop strips provenance, and each hop makes the claim read as more settled than the last, because summaries do not carry hedging well.

Coordination overhead exceeding the parallelism. The orchestrator has to write N briefs, read N results, reconcile them and produce one answer. If reconciling is nearly as hard as the original task, you have parallelised the easy half and serialised the hard half, in one agent, with worse information than the workers had. This is the point at which multi-agent architectures do not merely cost more — they produce worse answers than the single agent you replaced, which is the outcome nobody benchmarks for.

The test to apply before any of this

Here is the whole decision, and it is deliberately biased towards the boring answer: if one agent with the right tools and the right context can do it, one agent is the right answer.

That word right is doing the work, so make it a question about your last failure rather than about ambition. When a single agent falls over, it fails for one of two reasons, and they have opposite remedies. Either it lacked the capability — it genuinely could not hold the task, needed to be in two places at once, or could not see its own mistake — or it lacked tools and context, which is a description of almost every agent that has ever disappointed anyone. A vague tool description, a missing search index, a brief that assumed knowledge the agent never had. Adding a second agent to that system gives you two agents with the same missing index, twice the bill, and a new class of failure on top.

So, in order:

Did the single agent fail for want of tools or context? Fix that first. Most "we need multi-agent" conclusions do not survive this question.
Is the work genuinely independent — can item B start before item A finishes? If yes, fan out.
Is the check genuinely separate from the doing — would a reader who never saw how it was made catch something the maker cannot? If yes, add a verifier.
Is verification cheaper than generation, and is the success rate mediocre rather than terrible? If yes, a panel is worth the multiplication.
If none of those, one agent. A pipeline of fixed stages in a single loop is not a compromise; it is the correct architecture for work whose shape you already know.

The honest summary of the field is that multi-agent systems are a scaling technique, not a capability technique. They let you apply more compute to work that was already divisible, and they let you buy a second opinion that a single loop cannot generate about itself. They do not make a model better at a task it cannot do, and the belief that they do is what the expensive theatre is built out of.

A cheat sheet to keep

Where to go next

You can now tell the three arrangements that pay from the ones that only look industrious: fan-out where the items do not talk to each other, a verifier that never sees the case for the defence, and a panel where the score is a fact rather than an opinion. And you can price all three before building them, including the two costs that only show up in production — the tail latency you inherit from your slowest worker, and an orchestrator adjudicating from summaries.

The next lesson, Model Context Protocol and Tool Ecosystems, takes on the assumption this one leaned on throughout: that each agent arrives with the right tools in front of it. It covers how tools and data get connected to an agent as a standard rather than as bespoke glue per integration — and what changes about your threat model when a third party, not you, defines the tools your agents see.

The thing to go and do: take a multi-agent flow you already run, log tokens per agent and wall clock per phase, then run the same task as a single agent with every tool the sub-agents had. Two numbers usually surprise people: setup tokens are a larger share of the total than anyone guesses, and the single agent is often close enough on quality that the architecture turns out to have been buying wall clock all along — which is worth knowing, because wall clock is a thing you can decide you do not need.