Multi-Agent Debate and Collaboration
Du et al. (ICML 2024, "Society of Minds") run N model instances that independently propose answers, then iteratively critique each other over R rounds to converge. Improves factuality, rule-following, reasoning. Sparse topology beats full mesh on token cost. Explain the debate protocol: N proposers, R rounds, converge on a shared answer. Describe why debate improves factuality, rule-following, and reasoning. Explain sparse topology: not every debater needs to see every other. Implement a stdlib debate over a scripted LLM with full-mesh and sparse variants; measure token cost vs accuracy. Self-Refine (Lesson 05) is one model critiquing itself — risks groupthink. CRITIC (Lesson 05) grounds critique in external tools — not always available. Debate introduces a third mode: multiple instances, cross-critique, convergence by disagreement. N model instances independently propose answers to the same question. Over R rounds, each model reads the others' proposals and critiques them. Models update their answers based on the critiques. After R rounds, return the convergent answer. Original experiments used N=3, R=2 due to cost. Accuracy improves with more agents and more rounds on hard problems (MMLU, GSM8K, Chess Move Validity, biography generation). Cross-model combinations beat single-model debates: ChatGPT + Bard together > either alone. "Improving Multi-Agent Debate with Sparse Communication Topology" (arXiv:2406.11776, 2024-2025) showed full-mesh debate is not always optimal. Sparse topologies (star, ring, hub-and-spoke) can match…
Multi-Agent Debate and Collaboration: Du et al. (ICML 2024, "Society of Minds") run N model instances that independently propose answers, then iteratively…
This free lesson is part of the AI Engineering from Scratch curriculum. Read the full explanation, run the lesson code, and verify the result in the interactive reader or from the repository source.
Browse the complete course catalog or open this lesson on GitHub.