PaperlayerView exact revision on arXiv

Source-linked AI summary

Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaboration

Qiushi Sun, Zhangyue Yin, Xiang Li, Zhiyong Wu, Xipeng Qiu, Lingpeng Kong

arXiv:2310.00280v3cs.AIcs.CL

TL;DR

LLMs remain challenged by complex reasoning despite their NLP capabilities. Corex organizes multiple LLM agents into collaborative modes, and evaluations report superior performance across reasoning tasks compared with previous solutions.

  • Problem

    LLMs have advanced NLP, but reasoning remains challenging and their internal representations constrain performance.

    Source: §1 · pp. 1–2

  • Method

    Corex turns LLMs into autonomous agents that collaborate through Discuss, Review, and Retrieve strategies.

    Source: §1 · pp. 2–3

  • Results

    Across 18 tasks in four categories, Corex achieves superior performance compared with previous solutions.

    Source: §6 · p. 10

  • Takeaways & Limitations

    Corex is reported as task-agnostic, cost-effective, and more flexible than other multi-model approaches.

    Source: §6 · p. 10

    Integrating multiple LLMs can be unstable, motivating better orchestration of models with distinct attributes and capabilities.

    Source: A Limitations and Broader Impacts · p. 17

Abstract

from arXiv · show

Large Language Models (LLMs) are evolving at an unprecedented pace and have exhibited considerable capability in the realm of natural language processing (NLP) with world knowledge. Benefiting from ultra-large-scale training corpora, a single LLM can manage typical NLP tasks competently. However, its performance in executing reasoning tasks is still confined by the limitations of its internal representations. To push this boundary further, we introduce Corex in this paper, a suite of novel general-purpose strategies that transform LLMs into autonomous agents pioneering multi-model collaborations for complex task-solving. Inspired by human behaviors, Corex is constituted by diverse collaboration paradigms including Debate, Review, and Retrieve modes, which collectively work towards enhancing the factuality, faithfulness, and reliability of the reasoning process. These paradigms foster task-agnostic approaches that enable LLMs to ''think outside the box,'' thereby overcoming hallucinations and providing better solutions. Through extensive experiments across four different types of reasoning tasks, we demonstrate that orchestrating multiple LLMs to work in concert yields substantially better performance compared to existing methods. Further results and in-depth analysis demonstrate the cost-effectiveness of our method, facilitating collaboration among different LLMs and promoting annotation efficiency.

1 Introduction

The introduction frames complex reasoning as a persistent challenge for LLMs and presents Corex as multi-model collaboration across complementary modes. Experiments report gains across four task types, alongside cost-effectiveness.

1 Introduction

  • LLMs have advanced NLP, but reasoning remains challenging, and scaling alone may not overcome their limitations.The introduction also describes prompting and decoding approaches as confined to a static black box, with unreliable answers and limited self-correction.Source: §1 · pp. 1–2
  • The paper motivates collaboration as a way for models to contribute distinct cognitive processes to complex reasoning.It presents model communication as a promising but underexplored pathway.Source: §1 · p. 2
  • Corex assigns distinct personas to LLM agents and combines Discuss, Review, and Retrieve to address complex reasoning.Discuss supports factuality and diverse rationales, Review scrutinizes reasoning or code, and Retrieve ranks candidates for faithfulness.Source: §1 · pp. 2–3
  • Across four reasoning-task types, Corex substantially outperforms prior strong baselines, with modes showing different task strengths and reduced reasoning overhead.The paper also reports synergies across modes and LLM backbones.Source: §1 · p. 3

2 Related works

The related-work section surveys approaches to eliciting LLM reasoning and the emerging use of multiple models collaborating on tasks.

Chain-of-Thought Prompting Elicits LLM Reasoning

  • CoT, self-consistency, tool use, and search-based methods are among the approaches surveyed for improving multi-step reasoning.The section describes PAL and PoT as generating programs from natural-language problems for execution by a Python interpreter.Source: §2 · p. 3

Multi-Model Synergy for Task Solving

  • Multi-model collaboration research spans model interactions, debates, role-based cooperation, and artificial societies, but remains preliminary.Prior work includes task-specific efforts such as improving factuality in translation and arithmetic reasoning.Source: §2 · p. 3

3 Corex

Corex organizes LLM agents into collaborative reasoning strategies. Discuss preserves diverse viewpoints, Review iteratively scrutinizes solutions, and Retrieve selects a chain-answer pair for faithfulness.

3 Corex

  • Corex has LLM agents independently generate a reasoning chain and prediction for a query, then applies collaborative strategies.The three modes are Discuss, Review, and Retrieve.Source: §3 · p. 3

3.1 Discuss

  • Discuss divides agents into two teams that refine chains and predictions over rounds, with a judge evaluating round outputs when teams disagree.The design avoids forcing a single consensus, preserving factuality and diversity because context limits, erroneous consensus, and dominance by stronger models are concerns.Source: §3.1 · p. 4

3.2 Review

  • Review has one primary agent formulate a solution, then other agents sequentially scrutinize and refine its reasoning chain and code.Each reviewer builds on prior reviewers’ feedback; due to GPT-3.5-Turbo’s context limit, only the previous discussion round is stored.Source: §3.2 · pp. 4–5

3.3 Retrieve

  • Retrieve scores candidate chain-answer pairs for faithfulness and returns the pair with the highest confidence score.The retriever evaluates whether the answer is supported by its reasoning chain; the approach is model-interaction-based and reference-free.Source: §3.3 · pp. 5–6

4 Experiment

Corex is evaluated across 18 tasks in four reasoning categories against strong baselines. With five agents, it broadly surpasses CoT-SC(10) on mathematical benchmarks, while different collaboration modes show strengths across task types.

4.1 Experimental Setup

  • The evaluation spans arithmetic, commonsense, symbolic, and semi-structured reasoning, comparing Corex with CoT, self-consistency, complexity-based consistency, and program-aided baselines.Main experiments use GPT-3.5-Turbo; cross-model analyses also use GPT-4, Claude-Instant, and LLaMA-2-Chat. Corex modes use five agents, with up to five Discuss rounds.Source: §4.1 · p. 6, §4.2 · p. 6

4.2 Main Results

  • With five agents, Corex broadly surpasses CoT-SC(10) on mathematical benchmarks.Table 1 compares accuracy across seven mathematical reasoning datasets.Source: §4.2 · pp. 6–7, Table 1 · p. 7
  • Over 6% on StrategyQA, Corex surpasses ComplexCoT; Discuss performs particularly well on commonsense reasoning.Review uses natural-language reasoning chains for commonsense tasks, which cannot be converted into computational tasks.Source: §4.2 · p. 7, Table 2 · p. 7
  • Multi-model collaboration outperforms most prior baselines on Big-Bench tasks; PAL aids Repeat Copy, while Review and Retrieve improve counting selection over majority voting.Table 3 reports accuracy on five symbolic reasoning datasets.Source: §4.2 · p. 7, Table 3 · p. 7
  • CoT-SC offers limited gains on FinQA and ConvFinQA, while cooperative paradigms yield significant improvements.Table 4 compares Corex with strong baselines on these two tasks.Source: §4.2 · p. 8, Table 4 · p. 8
  • Across 18 tasks, Discuss is competent for factual-knowledge tasks, Review mitigates reasoning-chain errors and repairs flawed code on challenging math and counting, and Retrieve improves general-task performance to varying degrees.The authors also report further experiments with open-source LLMs.Source: §4.2 · p. 8

5 Comparison and Analysis

Section 5 compares Corex with other collaborative methods and examines how its modes, model choices, and computational costs affect performance. The analyses report benefits from collaboration while noting variation across tasks and occasional instability in review gains.

5.1 Performance Comparison of Collaborations

  • Corex steadily outperforms MAD and EoT across different modes in the comparison reported in Table 5.The comparison covers Multi-Agent Debate (MAD; Du et al., 2023) and Exchange of Thought (EoT; Yin et al., 2023a).Source: Table 5 · p. 8, §5.1 · p. 8
  • Combining Corex modes generally improves performance, especially on numerical tasks, and any collaboration outperforms self-refinement.The analysis applies Review after other modes and compares the results with Self-Refine.Source: §5.1 · pp. 8–9, Figure 6 · p. 8

5.2 In-Depth Analysis of Corex Strategies

  • Consensus in Discuss mode is swift for most problems, though over 10% of ConvFinQA problems require more than three rounds.The authors also report favorable convergence, with interaction informing the judge’s decision.Source: §5.2 · p. 9, Figure 7 · p. 9
  • Review cycles generally improve performance on the analyzed tasks, although occasional deviations produce performance oscillations.The analysis covers Repeat Copy and GSM8K with ReviewCode, and BoolQ and Penguin with ReviewNL.Source: §5.2 · p. 9, Figure 8 · p. 9

5.3 Synergies between Different LLMs

  • With GPT-3.5-Turbo as players, stronger judges positively correlate with task performance, especially as task complexity increases.The authors attribute this pattern to the judge needing to understand the question and both parties’ reasoning.Source: §5.3 · p. 9, Figure 9 · p. 10
  • In Retrieve mode, retriever choice has a modest performance effect, so especially powerful retrievers are not necessary for favorable results.The authors note minimal variance on tasks such as ARC-c when the candidate answers come from GPT-3.5-Turbo agents.Source: §5.3 · p. 9, Figure 10 · p. 10

5.4 Cost-Effectiveness of Multi-Agent Collaborations

  • On AddSub, Corex modes match or surpass strong baselines, using 5–10% of majority-voting resources for equivalent performance.The analysis reports this computational-cost comparison in Figure 11 and notes a similar trend in additional experiments.Source: Figure 11 · p. 10, §5.4 · p. 10
  • Corex also reduces reliance on curated demonstrations, improving annotation efficiency.The paper points to experiments with varying numbers of demonstrations in Appendix D.3.Source: §5.4 · p. 10

6 Conclusion

Corex uses multi-model collaboration to turn LLMs into autonomous reasoning agents for complex tasks. Across 18 tasks in four categories, the paper reports superior performance to previous solutions and highlights task-agnostic, cost-effective, and flexible methods.

6 Conclusion

  • Corex is a suite of strategies that uses multi-model collaboration to make LLMs autonomous reasoning agents for complex reasoning.The paper presents this as a preliminary exploration of LLM-based agent ecosystems.Source: §6 · p. 10
  • Evaluations across 18 tasks in four categories show superior performance compared with previous solutions.The paper describes the evaluations as extensive.Source: §6 · p. 10
  • The methods are described as task-agnostic, cost-effective, and more flexible than other multi-model approaches.The authors hope the work will serve as a foundation for further research on complex reasoning and collective intelligence.Source: §6 · p. 10

A Limitations and Broader Impacts

The paper identifies limited evaluation of open-source models and instability when integrating multiple LLMs as boundaries and challenges for Corex. It presents broader benefits of open-source collaboration as possibilities rather than demonstrated results.

A Limitations and Broader Impacts

  • Open-source models are evaluated only in small-scale experiments in Appendix C because of performance constraints.The authors suggest collaboration among weaker open-source models might enable tasks previously handled only by stronger commercial models.Source: A Limitations and Broader Impacts · p. 16
  • The authors propose that open-source collaboration could improve efficiency and model diversity, with possible sustainability and fairness benefits.These are framed as anticipated possibilities, not established experimental outcomes.Source: A Limitations and Broader Impacts · p. 17
  • Integrating multiple LLM functions can be unstable, motivating better orchestration strategies for models with distinct attributes and capabilities.The paper identifies this as a direction for future research.Source: A Limitations and Broader Impacts · p. 17

B Implementation Details

The implementation details specify the models, prompts, evaluation procedures, and comparison setups used in the experiments. The Section 5.3 and 5.4 analyses are limited to at most 500 samples per run.

B Implementation Details

  • Main experiments use GPT-3.5-Turbo-0613, GPT-4-0613, and Claude-Instant-1.2 through paid APIs, with generation temperature set to 0.The OpenAI and Anthropic models are accessed through their respective APIs.Source: B Implementation Details · p. 17
  • Prompts use the same number of exemplars per task as previous studies, while the complexity-based prompting baseline uses prompts from Fu et al. (2023a).Prompt details and examples are listed in Appendix G.Source: B Implementation Details · p. 17
  • Arithmetic, commonsense, and symbolic tasks are evaluated with accuracy, while FinQA and ConvFinQA use their official scripts and the cited test split.TAT-QA uses its arithmetic portion, and Corex-Discuss is capped at five rounds.Source: B Implementation Details · p. 17
  • EoT results come from Yin et al. (2023a), while MAD and Self-Refine are reimplemented with Corex’s models and task prompts.The reimplementations are intended to ensure a fair comparison.Source: B Implementation Details · p. 17
  • Section 5.3 and 5.4 analyses use at most 500 samples per run because of rate limits and a restricted budget.Source: B Implementation Details · p. 17
  • Review mode runs PAL-generated programs with Python 3.9.12, but GPT-3.5-Turbo’s lack of program-generation optimization may compromise performance.The settings follow Gao et al. (2022) and Chen et al. (2022a), which used Code-davinci-002 (Chen et al., 2021a).Source: B Implementation Details · p. 17

C Collaborations among Open-Source LLMs

Corex pairs LLaMA-2-Chat models as primary agents with commercial LLM reviewers in Review mode. Though weaker alone, open-source models can collaborate to match or exceed commercial-model CoT-SC(10) performance.

C Collaborations among Open-Source LLMs

  • LLaMA-2-Chat(7B/13B) serve as primary agents, while GPT-3.5-Turbo and Claude-Instant-1.2 act as reviewers.The reported ReviewNL results are shown in Figures 12–13 and ReviewCode results in Figures 14–15.Source: C Collaborations among Open-Source LLMs · p. 17
  • In evaluations, open-source LLMs perform weaker when operating independently, but collaboration with other models lets them achieve performance comparable to or even surpass that of commercial models’ CoT-SC(10).Source: C Collaborations among Open-Source LLMs · p. 18
  • In ReviewCode, LLaMA’s limited code generation and task delegation still yield non-trivial results when it collaborates with other models.The authors suggest smaller models may beneficially draft desired content despite capability gaps.Source: C Collaborations among Open-Source LLMs · p. 18

D.1 Evaluations on Semi-structured Reasoning

The appendix reports semi-structured evaluations under a context-length constraint, adds cost-effectiveness analyses, and tests how few-shot demonstration counts affect Review and Retrieve performance.

D.1 Evaluations on Semi-structured Reasoning

  • Context-length constraints led the authors to evaluate TAT-QA with GPT-3.5-Turbo-16k alongside two other tasks.Table 6 is described as evaluating Corex on semi-structured understanding tasks with GPT-3.5-Turbo-16k.Source: §D.1 · p. 18, Table 6 · p. 18

D.2 Further Cost-effectiveness Analysis

  • Cost is the sum of few-shot exemplar, query, and output tokens; the additional ARC-c and Penguins analyses plot computational cost against performance.Dot size represents each method’s average number of inferences; Figures 16–17 show the analyses.Source: §D.2 · p. 19, Figure 16 · p. 19, Figure 17 · p. 19

D.3 Annotation Efficiency Analysis

  • Across varying few-shot counts, the number of examples has a marginal impact on Review and Retrieve performance, indicating annotation efficiency.Figures 18–19 show few-shot performance for Review and Retrieve modes, respectively.Source: §D.3 · p. 19, Figure 18 · p. 19, Figure 19 · p. 19

D.4 Analysis of Error with Different Types

The error analysis examines ReviewCode’s bugs and problem misinterpretations, reporting reductions in both error types on GSM8K and stronger removal of misunderstanding errors on harder GSM-Hard problems.

D.4 Analysis of Error with Different Types

  • ReviewCode errors comprise bugs in generated code and technically correct programs that misinterpret the problem.The analysis focuses on these two error categories.Source: §D.4 · pp. 19–20
  • Review reduces both bugs and problem-misinterpretation errors on GSM8K, while collaboration eliminates more incorrect-understanding errors on challenging GSM-Hard problems.Figures 20 and 21 show the pre- and post-collaboration error types for GSM8K and GSM-Hard, respectively.Source: §D.4 · p. 20, Figure 20 · p. 20, Figure 21 · p. 20

E Statistics and Details of Datasets

The appendices describe the evaluated datasets, provide illustrative collaboration cases, and give example prompts and few-shot exemplars for the covered tasks.

E Statistics and Details of Datasets

Arithmetic reasoning; Commonsense & Factual reasoning

  • The listed task descriptions range from multi-step grade-school arithmetic and commonsense question answering to temporal, counting, and instruction-following tasks.GSM8K problems require 2–8 steps; GSM-Hard replaces their numbers with larger ones, while StrategyQA requires multi-hop reasoning.Source: Arithmetic reasoning · p. 20, Commonsense & Factual reasoning · pp. 20, 22

Semi-structured Understanding

  • The semi-structured datasets include financial QA over reports and conversational numerical reasoning, plus tabular-and-textual QA in TAT-QA.FinQA covers financial-report question-answer pairs; ConvFinQA studies numerical reasoning in conversational QA.Source: Semi-structured Understanding · p. 22

F Case Studies

  • Tables 11–13 provide illustrative cases for Discussion, Review, and Retrieve collaboration paradigms.Each table corresponds to one paradigm; the prompt appendix also presents collaboration system messages and task exemplars.Source: F Case Studies · p. 22, Table 11 · p. 24, Table 12 · p. 25

G Prompts

  • The prompt appendix supplies example few-shot exemplars and collaboration messages, with shot counts specified for multiple task groups and baselines.Examples include mathematical reasoning, commonsense tasks, Date Understanding, Object Counting, Penguins, and financial QA.Source: G Prompts · p. 23, Table 14 · p. 27, Table 15 · p. 28
  • The included examples illustrate reasoning prompts and collaboration transcripts, including a discussion that corrects an answer from 5 to 105.The prompt examples also show candidate selection in Retrieve and a ReviewCode correction of an erroneous program.Source: G Prompts · pp. 24–26, Table 12 · p. 25, Table 13 · p. 26
Loading 2310.00280v3…