Multi-Agent Systems

Agents, handoffs, shared state and orchestrators. Pick a pattern, vote, checkpoint and fix the known failures.

Concepts and terms
Tasks and commands
Annotated code
Diagrams
Defaults and numbers
Troubleshooting
Questions and tips

Four primitives#

From Lesson 16.04, 16.04 code/main.py, Lesson 16.13, Lesson 16.22, 16.22 code/main.py

Coordinate agents#

Task	Code
Share a message pool	`state = SharedState()` then `state.append(<msg>)`
Run a fixed order	`StaticOrchestrator(<order>).run(team, state)`
Let the agent pick next	`HandoffOrchestrator("<start>").run(team, state)`
Let a selector pick next	`LLMSelectorOrchestrator("<start>", round_robin_selector)`
Hand off with a tool	`def transfer_to_refunds() -> "Agent": return refund_agent`
Run a group chat	`run_groupchat(AGENTS, <selector>, max_rounds=8, label="<label>")`
Fan out workers	`threading.Thread(target=w.run, args=(sq, results, i))`
Publish to a topic	`bb.publish("<writer>", "<topic>", "<content>", "<prompt>")`

From 16.04 code/main.py, 16.05 code/main.py, 16.10 code/main.py, 16.11 code/main.py, 16.13 code/main.py

Vote, persist and protect#

Task	Code
Count votes	`plurality(votes)`
Weight votes by confidence	`cp_wbft(votes, threshold=0.5)`
Leaderless pick	`decentllms(votes)`
Save a checkpoint	`store.write(thread_id, super_step, dict(state))`
Resume after a crash	`store.latest(thread_id)`
Give an agent a queue	`AgentQueue("<agent_id>").enqueue(<msg>)`
Stop a retry storm	`if not breaker.allow(): ...` then `breaker.record(ok)`

From 16.14 code/main.py, 16.22 code/main.py, 16.23 code/main.py

One supervisor run#

From 16.05 code/main.py, Lesson 16.05

Confidence-weighted vote#

def cp_wbft(votes: list[Vote], threshold: float = 0.5):

weights: dict[str, float] = {}

rep: dict[str, str] = {}

for v in votes:

key = v.canonical()cluster equal answers

weights[key] = weights.get(key, 0.0) + v.confidence

rep.setdefault(key, v.answer)

total = sum(weights.values()) or 1.0

winner_key = max(weights, key=weights.get)

if weights[winner_key] / total < threshold:

return None, weightstoo weak: retry or escalate

return rep[winner_key], weights

From 16.14 code/main.py

Pick a pattern#

Pipeline: Each agent transforms and passes on. One failed stage blocks the rest.
Fan-out / fan-in: Split into parallel agents, then merge. For independent subtasks.
Supervisor: Lead plans, delegates and synthesizes. Wrong for sequential tasks.
Hierarchical: Supervisors over supervisors. Only leaves work.
Group chat: Peers share a pool. A selector picks the next speaker.
Handoff: The active agent calls a tool that returns the next agent.
Blackboard: Topic pub/sub. Wins with many agents on long runs.
Single agent: Under 20 tool calls and 100k tokens, stay single.

From Lesson 16.01, Lesson 16.05, Lesson 16.06, Lesson 16.10, Lesson 16.11, Lesson 16.13

Handoff loop (Swarm style)#

def run_swarm(start_agent: Agent, user_messages: list[str]) -> list[Msg]:

history: list[Msg] = []

active = start_agentwho holds the chat

for user in user_messages:

history.append(Msg(role="user", content=user))

out = scripted_router(active, user)stands in for an LLM

if isinstance(out, Agent):a tool returned an agent

history.append(

Msg(role="assistant", content=f"(handoff to {out.name})", sender=active.name)

)

active = outswitch the active agent

out = scripted_router(active, user)

history.append(Msg(role="assistant", content=str(out), sender=active.name))

return historynothing persists

From 16.11 code/main.py

Defaults in the code#

Setting	Default	Note
`max_steps`	10	Every orchestrator `run`
`max_rounds`	8	Group chat demo cap
`TERMINATE`	TERMINATE	Token that ends a group chat
`threshold`	0.5	`cp_wbft` acceptance share
`rounds`	2	Graph topology debate rounds
`failure_threshold`	0.5	Breaker opens above this error rate
`window_size`	20	Breaker outcomes in the window
`open_cooldown_s`	0.5	Seconds before half open

From 16.04 code/main.py, 16.10 code/main.py, 16.14 code/main.py, 16.15 code/main.py, 16.23 code/main.py

Troubleshooting#

Symptom	Cause	Fix
One agent talks every turn	Hot speaker in the selector	Add speaker balance to the selector
Context grows each turn	Every agent reads every message	Give agents projections
Two agents hand back and forth	Handoff loop	Cap `max_steps` on the orchestrator
A wrong number spreads to all	Memory poisoning in shared state	Provenance, append only, read-only verifier
Majority is wrong together	Monoculture, same base model	Mix base models
Workers research the wrong target	Manager split drifted from the goal	Add a canary question worker
Workers disagree, one side wins	Silent synthesis conflict	Re-ask or report the disagreement
Downstream load grows 10x	Retry storm	Circuit breaker, capped retries

From Lesson 16.05, Lesson 16.06, Lesson 16.10, Lesson 16.11, Lesson 16.13, Lesson 16.14, Lesson 16.23, Lesson 16.15, 16.04 code/main.py

Glossary#

BFT: Byzantine fault tolerance. Tolerates f < n/3 faulty nodes.
CP-WBFT: Confidence-Probed Weighted BFT. Votes weighted by confidence.
MAST: Failure taxonomy. Specification, coordination, verification.
MARBLE: MultiAgentBench. Star, chain, tree and graph topologies.
Super-step: Unit of work between two checkpoints.
Outbox pattern: Write the side effect intent first. A separate executor runs it.

From Lesson 16.14, Lesson 16.23, Lesson 16.24, Lesson 16.22

Q&A#

What does multi-agent cost?: About 15 times the tokens of a single agent.
Why can the verifier not write?: A poisoned pool then cannot poison it.
Which threshold for 5 to 7 voters?: 0.5 to 0.67. Below it, escalate.
More agents or other models?: Other models. Three models beat five copies of one.
Most common failure?: Specification problems, 41.77% of MAST failures.

From Lesson 16.05, Lesson 16.13, Lesson 16.14, Lesson 16.15, Lesson 16.23

Tips#

Shared state is the only stateful part. Look there first for bugs.
Keep hierarchies at depth 2 or less.
Bound debate rounds to 2 or 3.
Past about 4 agents in a graph, the coordination tax grows.
Use async I/O for LLM calls. Threads cost about 1MB each.
Put a dedup key in every side-effect call.

From Lesson 16.04, Lesson 16.06, Lesson 16.14, Lesson 16.15, Lesson 16.22