Multi-agent systems

1. Intuition

The single-agent ceiling. One agent is one loop, one context window, and one system prompt. Three things eventually break:

1. Context saturation. Every tool result stays in the context. By step 30, the window holds pages of file contents and search results, the early details get lost, and every call re-reads all of it.
2. Role confusion. One prompt that says "research, then code, then review, then test" produces an agent that does each job half-well. Reviewing your own work in the same context also inherits your own blind spots.
3. Sequential execution. Reading three independent sources takes three rounds, one after another, even though nothing about the second depends on the first.

Splitting fixes each one, at a price. Give each agent one job, its own fresh context, and a prompt tuned for that job. Workers read a lot and return a little, so the coordinator's context stays small. Independent workers run at the same time. The price: every boundary is lossy (a worker's 20,000 tokens of reading become a 400-token summary), coordination is new code that can fail, and total tokens can go up because every worker re-reads its own instructions and may re-discover what another already found.

A spectrum, not a switch.

Pipeline and fan-out are workflows: your code fixes the control flow. Orchestrator-worker and swarms are agentic: a model decides it. The rule from tool use and agents still applies: if you can write the steps down, write them down.

The handoff is the product. A worker only knows what its brief says. A good brief has the objective, what is already known, the boundaries (what not to do, which sources to skip), the tools to use, and the exact output format and length. A good report is short, structured, and cites what it found, so the orchestrator can merge and check it without re-doing the work.

Key distinction

Multi-agent pays when work is broad and independent. It buys parallel speed and clean contexts. It does not make a model smarter at a single coupled problem, and it usually costs more tokens than a single agent that manages its context well.

2. Visualization

Synthetic sizes. Left: the big token saving is against a single agent that never compacts; against one that does, the gap is small, and if workers must re-discover context they cost more. Right: the robust win is latency, because parallel workers keep the critical path flat.

Try it (run multi_agent.py, predict first, then check):

1. Before running, predict which design wins on total tokens at k = 1. Check the left panel.
2. Set M (summary length) to 4,000 tokens. Predict what happens to the orchestrator's synthesis call and its peak context.
3. Make workers need 6 steps (the dashed line). At what k does multi-agent become cheaper than the non-compacting single agent?
4. Suppose the subtasks are not independent: worker 2 needs worker 1's result. What happens to the critical path?

3. The math

Symbols

One agent loop of $T$ calls bills $TB + D\,T(T-1)/2$ input tokens, because each call re-reads everything before it (derived in tool use and agents).

Single agent doing all $k$ subtasks in one loop, $T = ks$:

Orchestrator with workers: one planning call, $k$ parallel worker loops, one synthesis call that reads the plan and the $k$ summaries:

The single agent's quadratic term grows with $(ks)^2$; the workers' grows with $k s^2$. Splitting removes the cross-subtask re-reading.

Worked example

Synthetic sizes: $k = 4$, $s = 3$, $B = 2{,}000$, $D = 1{,}500$, $B_O = 2{,}000$, $B_W = 1{,}500$, $P = 300$, $M = 400$.

Single agent.

1. $T = 4 \times 3 = 12$ calls.
2. $C = 12 \times 2{,}000 + 1{,}500 \times \frac{12 \times 11}{2} = 24{,}000 + 1{,}500 \times 66 = 24{,}000 + 99{,}000 = 123{,}000$.
3. Peak context (last call): $2{,}000 + 11 \times 1{,}500 = 18{,}500$ tokens.

Orchestrator with workers.

1. Plan: $2{,}000$.
2. One worker: $3 \times 1{,}500 + 1{,}500 \times \frac{3 \times 2}{2} = 4{,}500 + 4{,}500 = 9{,}000$. Four workers: $36{,}000$.
3. Synthesis: $2{,}000 + 300 + 4 \times 400 = 3{,}900$.
4. Total: $2{,}000 + 36{,}000 + 3{,}900 = 41{,}900$.
5. Peak context: a worker's last call, $1{,}500 + 2 \times 1{,}500 = 4{,}500$, versus synthesis $3{,}900$, so $4{,}500$.
6. Critical path: $1 + 3 + 1 = 5$ sequential calls instead of 12.

The fair baseline. A single agent that replaces each finished subtask's history with a 400-token summary pays, for subtask $j = 0, 1, 2, 3$, $s(B + jM) + D\,s(s-1)/2$:

1. Base reads: $3 \times 2{,}000 \times 4 = 24{,}000$.
2. Summary reads: $3 \times 400 \times (0 + 1 + 2 + 3) = 7{,}200$.
3. Within-subtask growth: $4 \times 1{,}500 \times 3 = 18{,}000$.
4. Total: $49{,}200$, close to the multi-agent $41{,}900$, with no coordination code.

When workers duplicate work. If each worker needs 6 steps because it must re-discover context the orchestrator already had, one worker costs $6 \times 1{,}500 + 1{,}500 \times 15 = 9{,}000 + 22{,}500 = 31{,}500$; four cost $126{,}000$; the total is $2{,}000 + 126{,}000 + 3{,}900 = 131{,}900$, more than the naive single agent.

Real systems that explore broadly spend far more. Anthropic reported that its multi-agent research system used roughly 15 times the tokens of a chat interaction, and that this paid off only on tasks valuable enough and parallel enough to justify it. The model above shows why: the token bill depends on how much each worker explores, while the latency win depends only on independence.

4. Implementation

Build it from scratch

The full script is code/19-agents/multi_agent.py. Agents are mocks (no API calls) so the run is offline and deterministic in tokens; "latency" is a short sleep per model call.

The cost model. One helper bills an agent loop; the two designs are built from it, so the arithmetic above is exactly what the code computes.

def loop_tokens(base, d, steps):
    """Input tokens billed by one agent loop: each call re-reads base + everything appended so far."""
    return steps * base + d * steps * (steps - 1) // 2

def single_agent(k, s):
    t = k * s
    return {"input": loop_tokens(B, D, t), "critical_calls": t, "peak": B + (t - 1) * D}

Predict: doubling $k$ roughly multiplies the single agent's tokens by what factor, once $k$ is large?

The orchestrator design. Plan, $k$ independent worker loops, then one synthesis call that reads only the summaries.

def orchestrator_workers(k, s):
    plan = B_O
    workers = k * loop_tokens(B_W, D, s)
    synth = B_O + P + k * M
    return {"input": plan + workers + synth, "critical_calls": 1 + s + 1,
            "peak": max(B_W + (s - 1) * D, synth), "parts": (plan, workers, synth)}

Predict: with $k = 50$ subtasks, which call has the largest context, a worker or the synthesis?

A worker gets a brief, not the orchestrator's context. It works in its own context and returns a short, addressed message. Nothing else crosses the boundary.

def worker(brief, steps):
    """A mock worker: its own fresh context (the brief only), a few tool steps, then a short summary."""
    for _ in range(steps):
        time.sleep(CALL_SECONDS)
    return {"to": "orchestrator", "from": brief["id"], "summary": f"findings on {brief['topic']} ({M} tokens)",
            "tokens": loop_tokens(B_W, D, steps)}

Predict: the brief has no field saying which sources other workers cover. What failure does that invite?

Fan out, fan in. Briefs are structured (goal, format, length cap). Workers run concurrently in a thread pool; the orchestrator waits for all of them, then synthesizes.

def run_fanout(topics, steps):
    briefs = [{"id": f"w{i + 1}", "topic": t, "goal": "3 key facts with sources", "format": "bullets, max 400 tokens"}
              for i, t in enumerate(topics)]
    t0 = time.perf_counter()
    time.sleep(CALL_SECONDS)                                          # orchestrator plans
    with ThreadPoolExecutor(max_workers=len(briefs)) as pool:         # workers run concurrently
        results = list(pool.map(lambda b: worker(b, steps), briefs))
    time.sleep(CALL_SECONDS)                                          # orchestrator synthesizes summaries only
    return results, time.perf_counter() - t0

Predict: one worker hangs. What does pool.map do, and what should a production orchestrator do instead?

Use the library

Agent frameworks package these patterns: the OpenAI Agents SDK (handoffs and agents-as-tools), LangGraph (graphs of nodes with shared state), Microsoft's AutoGen (conversations between agents), CrewAI (role-based crews), and Anthropic's Claude Agent SDK (subagents that run with their own context and return a summary). What they add over the script: real model calls, tool routing, shared state or message passing, retries, timeouts, and tracing across agents. What they do not add is judgment about whether to split. Start with one agent, measure where it fails, and split only the part that is broad and parallel.

Run it

uv run code/19-agents/multi_agent.py

Key lines of output (wall-clock times vary slightly by machine):

single agent: T = 4*3 = 12 calls, input = 12*2000 + 1500*12*11/2 = 123,000, peak context 18,500
orchestrator+workers: plan 2,000 + workers 4*9,000 = 36,000 + synthesis 3,900 = 41,900, peak context 4,500
critical path: 12 sequential calls vs 5
single agent that compacts each finished subtask to 400 tokens: 49,200
if each worker needs 6 steps: 131,900 input tokens (single agent: 123,000)
wall clock 0.28s vs 0.70s if the same calls ran one after another

5. Engineering

Decide with three questions. Are the subtasks independent (no worker needs another's result)? Does a single context overflow or slow down on this task? Is the task valuable enough to pay for parallel exploration? Three yeses: consider orchestrator-worker. Otherwise, one agent with good tools and context management usually wins.

Write briefs like tickets. Objective, known facts, scope boundaries, tools and sources, and an output contract (format, length cap, citations). Most multi-agent failures trace back to a brief that assumed context the worker didn't have.

Keep reports small and checkable. Workers return structured summaries, not transcripts. The orchestrator checks them (are claims cited, do they conflict) before merging, and can send a worker back with a sharper brief.

Budget and stop. Give every worker a step and token cap, and the orchestrator a cap on how many workers it may spawn. Scale effort to the task: a simple question does not need ten workers.

Observe across agents. Trace every agent with a shared run ID so you can follow a request from plan to briefs to worker steps to synthesis. Without it, debugging means reading several transcripts and guessing how they relate. Evaluate the whole system end to end, as in LLM evaluation, and also check each handoff.

Prefer read-only parallelism. Parallel workers that only read and report are safe to retry and easy to merge. Parallel workers that write (edit the same files, place orders) create conflicts and side effects that are hard to undo; keep writes in one agent, or serialize them.

Watch out

Failure modes. Vague briefs; workers duplicating each other; lost context at handoffs; conflicting reports merged without reconciliation; a spawning loop where the orchestrator keeps adding workers; one slow worker holding up the whole run; and agents echoing each other's mistakes (peer designs can amplify a wrong claim rather than catch it).

Common mistakes

• Splitting a tightly coupled task (most code edits) across agents, then losing the decisions each made.
• Comparing multi-agent to a single agent that never compacts its context, and calling the token saving a win.
• Passing workers the orchestrator's full transcript "for context", which removes the clean-context benefit.
• No output contract, so the orchestrator receives ten differently shaped essays.
• No per-worker budget or timeout.
• Measuring only final quality, not total tokens and latency, so the design's cost is invisible.

6. Knowledge check

Practice exercise. Use the lesson's sizes ($B = B_O = 2{,}000$, $B_W = 1{,}500$, $D = 1{,}500$, $P = 300$, $M = 400$) with $k = 6$ subtasks of $s = 2$ steps each. Compute total input tokens and critical-path length for the single agent and for orchestrator-workers.

Implementation challenge. Add a timeout and a retry to run_fanout: if a worker takes longer than 3 model-call times, cancel waiting for it, re-issue its brief once with steps=1, and mark its report as partial. Show the wall clock when one worker is slow.

Summary

• One agent hits three limits: context saturation, role confusion, and sequential execution.
• Designs range from subagents and pipelines (fixed flow) to orchestrator-worker and swarms (model-chosen flow); choose by task shape.
• Splitting turns the $(ks)^2$ re-reading term into $k s^2$ and the critical path from $ks$ to $s + 2$ calls, but against a single agent that compacts its context the token saving is small, and duplicated exploration erases it.
• The reliable win is latency and clean contexts on broad, independent work; the reliable cost is lossy handoffs and coordination code.
• Briefs and reports are the interface: write briefs like tickets, cap and structure reports, budget every worker, and trace across agents.

Next: Production architecture

Related: Building reliable agents · Agent memory · Model Context Protocol · Reliability, cost, and observability

Go deeper, code-first (AI Engineering from Scratch): A2A — Agent-to-Agent Protocol · Multi-Agent Debate and Collaboration · Orchestration Patterns: Supervisor, Swarm, Hierarchical · Supervisor / Orchestrator-Worker Pattern · Role Specialization — Planner, Critic, Executor, Verifier · Handoffs and Routines — Stateless Orchestration

Parts of this lesson are adapted from AI Engineering from Scratch by Rohit Ghumare (MIT license, notice): Why Multi-Agent?.

Interview angle