Source-linked AI summary
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, Igor Mordatch
TL;DR
Language models can hallucinate facts and make implausible reasoning jumps, motivating methods beyond single-instance prompting. This paper uses repeated debate among multiple model instances, finding improved reasoning and factual validity across tasks, while requiring greater computational cost and not guaranteeing correctness.
Problem
Language models trained on potentially inaccurate internet text may confidently hallucinate facts or make implausible reasoning jumps.
Method
Multiple language-model instances independently propose answers, critique other agents’ responses, and revise them over multiple rounds toward a common answer.
Results
The debate approach outperforms single-model baselines across six reasoning, factuality, and question-answering tasks and improves factual consistency in biography evaluation.
Takeaways & Limitations
Multiagent debate is a simple, effective approach across reasoning and validity language-modeling tasks and can operate with black-box model access.
Takeaways & Limitations
Debate is more computationally expensive and can converge on answers that are confidently incorrect.
Abstract
from arXiv · showhide
Large language models (LLMs) have demonstrated remarkable capabilities in language generation, understanding, and few-shot learning in recent years. An extensive body of work has explored how their performance may be further improved through the tools of prompting, ranging from verification, self-consistency, or intermediate scratchpads. In this paper, we present a complementary approach to improve language responses where multiple language model instances propose and debate their individual responses and reasoning processes over multiple rounds to arrive at a common final answer. Our findings indicate that this approach significantly enhances mathematical and strategic reasoning across a number of tasks. We also demonstrate that our approach improves the factual validity of generated content, reducing fallacious answers and hallucinations that contemporary models are prone to. Our approach may be directly applied to existing black-box models and uses identical procedure and prompts for all tasks we investigate. Overall, our findings suggest that such "society of minds" approach has the potential to significantly advance the capabilities of LLMs and pave the way for further breakthroughs in language generation and understanding.
1 Introduction
The paper proposes multiagent debate, in which multiple language-model instances generate, critique, and revise answers over several rounds. Across reasoning and factuality tasks, this approach outperforms single-model baselines while requiring only black-box access, although it is more computationally expensive.
- Motivation: Current language models can confidently hallucinate facts or make implausible reasoning jumps because internet training data may be inaccurate.Existing improvements include chain-of-thought prompting, verification, self-consistency, and intermediate scratchpads.
- Approach: Multiagent debate has models independently propose answers, critique other agents’ responses, and repeatedly update their reasoning toward a common answer.The quorum can maintain multiple reasoning chains and candidate answers before producing the final response.
- Findings: Across six reasoning, factuality, and question-answering tasks, debate outperforms zero-shot chain of thought and reflection baselines.Using multiple agents and multiple debate rounds is important for achieving the best performance.
- Findings: Debate often produces a more accurate common answer and reduces uncertain false facts by exposing disagreement during deliberation.The paper also reports cases where all agents initially predict incorrectly but reach the correct answer as debate progresses.
- Scope and cost: The same methodology and prompt templates apply across tasks using only black-box language-model generations, and debate is compatible with other prompting methods.The approach is combined with zero-shot chain of thought, but requires multiple model instances and rounds.
- Contributions: The paper introduces a benchmark and dataset for factual accuracy in biographies of famous computer scientists.The benchmark targets hallucinated institutions and dates and inconsistent facts across model instances.
2 Language Generation through Multiagent Debate
The proposed procedure mimics multi-threaded reasoning and multi-source fact checking by having multiple language-model agents generate, inspect, and revise answers. Repeated consensus prompts can produce convergence, and prompts that encourage agents to be more stubborn lead to longer debates and better final solutions.
- Motivation: The method is motivated by cross-examining independent reasoning paths and inconsistent factual sources until they yield a consistent answer.Agreement between separate lines of work increases confidence, whereas disagreement prompts examination of reasoning and assumptions.
- Procedure: The procedure begins with multiple copies of a language model generating independent answers to the same question.Each response acts as a possible thought process or information source for subsequent examination.
- Procedure: Each agent receives the other agents’ responses, verifies them, and refines its own answer through repeated debate rounds.The consensus prompt instructs agents to update their responses based on the collected answers.
- Prompt design: The approach is orthogonal to existing prompting techniques and can be combined with more detailed elicitation methods, including zero-shot chain-of-thought reasoning.The evaluations directly combine debate with zero-shot chain of thought.
- Consensus: Language-model agents empirically converge on a single shared answer after multiple debate rounds, although convergence is not guaranteed in general multi-agent games.The paper frames debate as a multi-agent game and reports empirical convergence rather than a general guarantee.
- Prompt design: Prompts that make agents more stubborn about their own solutions produce longer debates and better final solutions.The paper uses different prompts to control how much agents trust their own outputs relative to other models’ responses.
3 Experiments
The experiments test multiagent debate across reasoning, factuality, and language-generation settings. Debate substantially improves reasoning and factual accuracy relative to single-agent and reflection baselines, while also benefiting from multiple agents, debate rounds, and chain-of-thought prompting.
- Experimental tasks: The evaluation covers arithmetic, grade-school math, and chess move prediction as reasoning tasks, plus biographies, MMLU, and chess move validity as factuality settings.These tasks assess mathematical reasoning, strategic reasoning, factual knowledge, and rule-constrained planning.
- Design choices: Because of computational expense, the main benchmark evaluations use three agents and two debate rounds, although additional agents and rounds produced further gains.The method’s debate duration can also be controlled through prompts that alter how much agents trust their own outputs.
- Reasoning results: Multiagent debate gives a substantial reasoning boost across arithmetic, grade-school math, and chess compared with single-agent generation and reflection.The comparison also includes majority voting, using the same starting prompt and language model across evaluated approaches.
- Reasoning results: Debate can reach correct answers even when all agents initially answer incorrectly, because agents critique one another’s reasoning.This indicates debate is not limited to amplifying an initially correct answer.
- Reasoning results: Multiagent generation improves performance both with and without zero-shot chain-of-thought prompting on GSM8K.The reported results describe the method as compatible with other single-agent prompting approaches.
- Factuality results: Debate significantly outperforms each baseline in factuality evaluations and settles on biography bullets that are more consistent across agents.Reflection performs poorly in the factuality setting, while debate improves and settles the generated content across agents.
- Factuality results: When agents disagree under uncertainty, communication leads them to change toward a more accurate consensus answer; resistant opinions may indicate higher factual confidence.The authors describe ease of persuasion as a possible method for assessing factual confidence.
- Language-generation results: A two-model debate solved 17 of 20 GSM8K problems, compared with 11 for Bard and 14 for ChatGPT individually.The experiment combines ChatGPT and Bard in a multiagent debate.
4 Related Work
The paper situates multiagent debate among prompting, training, majority-vote, and compositional-generation approaches for improving reasoning, factuality, and language generation. Its distinction is using communication among language models to combine answers while requiring only black-box access.
- Reasoning and factuality: Prior reasoning work uses scratchpads, verification, chain-of-thought prompting, self-reflection, and finetuning.
- Reasoning and factuality: Prior factuality methods include RLHF, truthful-data pruning, and external sources or retrieval.
- Multiagent debate: Unlike majority voting, this work uses a language model to combine different answers through debate.
- Compositional generation: Related compositional-generation work combines models, whereas this paper focuses on communication between language models for reasoning and factuality.
5 Limitations and Discussion
The debate approach is simple and effective across reasoning and validity tasks, but it increases computational cost and has practical limits in long or incorrect debates.
- Multiagent debate improves performance across a wide set of reasoning and validity language-modeling tasks.
- The procedure is more computationally expensive because it requires multiple language generations and an underlying debate process.
- Longer debates can exceed current models’ ability to process the full input, causing them to focus mainly on recent generations.
- Debates often converge on a single answer, but consensus does not guarantee correctness because models may confidently affirm incorrect answers.
A Appendix
The appendix provides additional debate analyses, visualizations, and detailed experimental information for the paper’s datasets.
- The appendix contains additional debate analyses and visualizations, followed by detailed experimental descriptions for each dataset.
A.1 Additional Results
Additional results examine consensus and qualitative debate examples across several tasks. Debate prompts that encourage adaptation improve consensus, while final answers can still sometimes be incorrect.
- Consensus Between Agents: Debate prompts that encourage agents to adapt to other agents’ opinions improve consensus.
- Additional Qualitative Visualizations: Across qualitative examples from GSM8K, arithmetic, MMLU, biographies, and chess, debate generally improves final answers but can still converge on incorrect values.
A.2 Evaluation Details
The evaluation uses task-specific prompts and measures performance across arithmetic, grade-school mathematics, chess, biographies, and MMLU. It combines quantitative task metrics with qualitative debate examples.
- Arithmetic: Arithmetic performance is evaluated on 100 generated tasks, each containing six random integers between 0 and 30.The metric is whether the correct integer answer is obtained.
- Grade School Math: GSM8K performance is evaluated by extracting boxed final answers from 100 grade-school math problems.The evaluation measures accuracy in obtaining the final correct answer.
- Chess: Chess evaluation uses 300 selected games and scores suggested moves with Stockfish at search depth 20.The task asks for White’s next move at turn 14, and performance is reported using the relative Stockfish pawn score after executing that move.
- Biographies: Biography evaluation compares generated bullet-point biographies with Wikipedia-derived facts using yes, no, or uncertain judgments.The reported metric is the percentage of ground-truth bullets receiving either yes or no, while uncertain judgments are ignored; facts outside the ground-truth bullets are not captured.
- MMLU: MMLU performance is measured by multiple-choice accuracy on 100 questions randomly distributed across subject areas.The evaluation selects one hundred MMLU questions.
- Experimental prompts: The experiments use task-specific prompts, including debate-update instructions and answer-format constraints.The appendix lists prompts for each task and provides examples for math and debate settings.