Self-Adapting Group of Experts for Multi-Agent Reasoning

1 INTRODUCTION

• Single responses can fail on problems needing several reasoning steps or knowledge beyond a model’s strengths, motivating teams that discuss, check, and revise solutions.The challenge is matching team reasoning strategies and collaboration to each problem’s needs.Source: §1 · p. 2
• Changing communication context changes which peer responses agents receive, but does not by itself revise their role instructions or reasoning strategies.SAGE investigates using agents’ responses to the current problem to guide strategy transfer across roles.Source: §1 · p. 2
• A physicist donor’s instructions to identify forces and formulate governing equations can guide mathematicians’ analytical derivations and computer scientists’ numerical solutions without replacing their specialties.Agents can further refine responses through multi-turn peer feedback.Source: §1 · p. 2
• SAGE selects a strategy donor from independent responses using answer agreement, prefix consistency, and reciprocal peer review, then transfers its strategy while preserving other agents’ roles.It routes responses through a sparse directed acyclic graph rebuilt after each collaboration round, from higher-scoring agents to lower-scoring agents.Source: §1 · p. 2

2 METHODOLOGY

• SAGE uses fixed agents and models without updating model parameters; each problem begins with independent responses, and adapted prompts remain fixed during subsequent collaboration.The framework requires neither training nor an external judge model.Source: §2 · p. 3

2.1 SELECTING A STRATEGY DONOR

• Donor selection scores initial responses for answer agreement and prefix consistency, then uses reciprocal peer review to retain candidates and select the highest-scoring agent.The donor contributes its original system prompt; review candidates are used for donor selection and the final vote, not as collaboration starting responses or context.Source: §2.1 · p. 4, §2.3 · p. 5

2.2 ADAPTING STRATEGIES WHILE PRESERVING ROLES

• Each non-donor agent’s backbone rewrites its original prompt using the donor’s original prompt, adding reasoning guidance while preserving the target agent’s role.The rewriter receives only the two original prompts, not the problem or generated responses, and adaptation occurs once per problem.Source: §2.2 · p. 5

2.3 PROPAGATING REASONING GUIDANCE THROUGH A SPARSE DAG

• Before each round, agents select at most K strictly higher-scoring parents, creating an acyclic graph with bounded in-degree and paths of at most K edges.Rebuilding the score-directed graph each round lets information flow track changes in response quality while bounding communication and dependency depth.Source: §2.3 · p. 5

2.4 SELECTING THE FINAL ANSWER

• Final answer selection pools initial answers, retained reviews, and completed-round responses, then returns the answer with the largest weight, resolving ties by leader votes and total votes.The returned response is chosen from the pool, preferring a stage leader’s response, then the most recent stage, then the lower agent index.Source: §2.4 · p. 6

3 EXPERIMENTS

• SAGE is evaluated against a single model call, chain-of-thought, and multi-agent baselines; SelfOrg is closest among them because both methods rebuild their DAG each round without training.The main backbones are Qwen2.5-1.5B-Instruct and Ministral-3-3B-Instruct.Source: §3 · p. 7

3.1 MAIN RESULTS

• SAGE achieves the highest average accuracy on both backbones, outperforming the strongest multi-agent baseline by +4.9 points with Qwen2.5-1.5B and +1.3 points with Ministral-3-3B.It outperforms every multi-agent baseline on all six benchmarks with Qwen2.5-1.5B.Source: §3.1 · p. 7

3.2 SCALING THE NUMBER OF AGENTS

• From four to nine agents, average accuracy rises by +1.5 points with Qwen2.5-1.5B and +1.8 points with Ministral-3-3B.Each added agent contributes an initial response and role prompt, while each agent still reads at most two parent responses per round.Source: Figure 2 · p. 7, §3.2 · p. 8

3.3 SCALING THE BACKBONE SIZE

• On GSM-Hard, SAGE beats Single at every tested model size, with gains of 1.2 to 5.0 percentage points; on GPQA-Diamond, results are less consistent.At 0.5B on GPQA-Diamond, accuracy rises from 22.56% to 31.14%, while SAGE has lower mean accuracy than Single at 3B and 32B.Source: §3.3 · p. 8, Figure 3 · p. 8

3.4 LIFTING TEAMS WITH WEAK AGENTS

• A mixed nine-agent team performs close to the strongest homogeneous team and exceeds the homogeneous Qwen and Phi teams on both benchmarks.Its average exceeds the mean of the three homogeneous teams by about +5 points, despite only one-third of agents using Ministral.Source: §3.4 · p. 8, Figure 4 · p. 9

3.5 ROBUSTNESS TO ADVERSARIAL ATTACKS

• Under message corruption, SAGE outperforms SelfOrg on all five benchmarks, with the largest gains on AQuA-RAT (+5.8 points) and MATH (+4.3 points).The authors note that corrupted responses may still affect donor selection.Source: §3.5 · p. 9

• Figure 5 compares SAGE and SelfOrg on five benchmarks with three of nine agents corrupted; bars show means and error bars show sample SD over three runs.Source: Figure 5 · p. 9
• Experiments across multiple backbones and text and vision-language benchmarks show improvements over the evaluated single- and multi-agent baselines.Source: §4 · p. 9
• SAGE is a training-free framework that adapts expert instructions and communication for multi-agent reasoning.Source: §4 · p. 9

• Generative AI tools were used to polish sentences during preparation of the work.Source: AI USE STATEMENT · p. 10
• Generative AI tools were also used to code, and the authors manually inspected and tested AI-assisted code before using it in experiments.Source: AI USE STATEMENT · p. 10
• The authors reviewed and edited AI-assisted text and take full responsibility for the experimental design, analysis, claims, and final paper.Source: AI USE STATEMENT · p. 10

• Section 2 specifies the SAGE inference procedure, summarized in Algorithm 1, with reciprocal peer review detailed in Appendix D.Source: REPRODUCIBILITY STATEMENT · p. 10
• Appendix C documents agent and baseline configurations, decoding parameters, and answer extraction and evaluation protocols.Source: REPRODUCIBILITY STATEMENT · p. 10
• Appendix E provides role, review, rewriting, collaboration, and baseline prompts; experimental tables and captions describe aggregation and identify single-run studies.Source: REPRODUCIBILITY STATEMENT · p. 10

ETHICS STATEMENT

• Robustness findings are limited to fixed, non-adaptive misleading messages, and consequential applications require independent validation of outputs.Source: ETHICS STATEMENT · p. 10

A.1 MULTI-AGENT REASONING

• Debate methods compare arguments and revise answers, but debate does not always outperform simpler aggregation and can be sensitive to setup; SAGE adapts role prompts before collaboration.Source: §A.1 · p. 16

A.2 PROMPT OPTIMIZATION

• Prior prompt-optimization approaches generate, edit, score, or evolve instructions; SAGE adapts once per problem, with the rewriter excluded from the problem and generated solutions.Source: §A.2 · p. 16

A.3 MULTI-AGENT COMMUNICATION AND COORDINATION

• Prior communication work studies fixed and query-adaptive structures, edge optimization, pruning, task-specific topologies, and joint prompt-graph search; SAGE adapts role prompts before collaboration.Source: §A.3 · p. 16

B.1 PROOF OF SPARSE ROUTING WITH BOUNDED DEPTH

• An agent of rank r has at most min{K, r −1} incoming edges, and each edge follows a strict score decrease, making the graph acyclic.The proof orders agents by decreasing previous-round score, breaking ties by increasing index.Source: §B.1 · pp. 16–17
• A directed path has at most K edges because every nonterminal vertex must be among the first K score-ranked agents.The proof uses the routing rule’s selection of the highest-scoring K candidates.Source: §B.1 · p. 17
• Both bounds are tight for arbitrary pairwise-distinct score vectors, but the proof does not claim every such vector is realizable by the scoring rules.The per-round guarantee does not imply that the union of graphs across rounds is acyclic.Source: §B.1 · p. 17

C IMPLEMENTATION DETAILS

• Main comparisons match evaluation examples and agent roles within backbones, while reporting accuracy as mean ± sample standard deviation over three runs and an unweighted six-benchmark macro-average.The main study uses four roles, three collaboration rounds, and a two-parent limit for SelfOrg and SAGE; inference cost varies with each method’s additional operations.Source: C IMPLEMENTATION DETAILS · p. 18
• The corruption study injects checked wrong answers into three of nine agents and modifies their shared messages, while inference receives no reference answers.The seeded corruption applies to initial, collaboration, and SAGE peer-review responses, but not prefix-consistency completions; SAGE and SelfOrg receive identical corruptions.Source: C IMPLEMENTATION DETAILS · p. 18

D RECIPROCAL PEER REVIEW DETAILS

• Reciprocal reviews use a fixed critic instruction on each reviewer’s own model, and kept reviews inform donor selection and the final vote while collaboration starts from initial responses and scores.Prefix consistency is computed under original prompts during review and again under adapted prompts during collaboration.Source: D RECIPROCAL PEER REVIEW DETAILS · p. 18

E AGENT PROMPT DETAILS

• The same primary-protocol message text is used across backbones, datasets, and runs, and agent prompts contain no reference answers.For heterogeneous experiments, each agent-owned call is dispatched to that agent’s assigned backbone.Source: E AGENT PROMPT DETAILS · p. 19

E.1 INITIAL ROLE SYSTEM PROMPTS

• SAGE samples four roles without replacement from a shared nine-role pool spanning mathematical specialties, academic knowledge, multiple-choice strategy, and verification.Every role prompt requires numbered reasoning steps, explicit intermediate calculations, verification where possible, and a boxed final answer.Source: §E.1 · pp. 19, 22

E.2 PREFIX-CONSISTENCY PROMPT

• Prefix-consistency checks retain the agent’s active role prompt and provide the original problem with the first 60% of whitespace-delimited response tokens for continuation.The continuation instruction asks for reasoning consistent with the prefix and a boxed final answer.Source: §E.2 · p. 23

E.3 RECIPROCAL PEER-REVIEW PROMPT

• The donor-selection review uses a fixed system message that casts the reviewer as an expert reasoning agent participating in peer review.This review message replaces the agent’s role system prompt for the reviewing call.Source: §E.3 · p. 23
• Peer review asks agents to keep or edit their answer based on whether another agent’s reasoning, corrections, or structure genuinely improves it.The review receives the original problem and both answers; edited answers should use useful corrections without blindly copying the peer.Source: §E.3 · pp. 23–24

E.4 SYSTEM-PROMPT REWRITE PROMPT

• The rewriter combines the target and best-performing system prompts into a concise reusable prompt while preserving the target’s identity and specialization.It adds only general reasoning, checking, interpretation, and formatting habits from the best prompt; the rewriter receives only the target identity and the two prompts, not task content or solutions.Source: §E.4 · p. 25

E.5 COLLABORATION PROMPTS

• Collaboration prompts direct agents to critically evaluate incoming peer answers rather than copy them blindly, then provide an improved answer.An agent with no incoming neighbors receives an instruction to review and improve its previous answer if needed; the same boxed-final-answer requirement applies.Source: §E.5 · pp. 25–26

F STRATEGY TRANSFER IN ACTION

• Reciprocal review selects the SkepticalVerificationSolver as donor despite its initial answer C, because selection scores retained reviews.The AlgebraicSolver initially answers A after swapping the person introduced with the recipient.Source: F STRATEGY TRANSFER IN ACTION · p. 26
• The rewrite retains the AlgebraicSolver’s identity and five priorities while adding verification habits; the rewriter sees only the two original prompts.The added habits include checking variable assignments for logical consistency.Source: F STRATEGY TRANSFER IN ACTION · pp. 26–27
• Three seeds: SAGE answers this question correctly every time, versus only once for SAGE-NOREWRITE; weighted pool voting selects D 12.0 to C 8.5.With the same seed, SAGE-NOREWRITE returns plurality answer C.Source: F STRATEGY TRANSFER IN ACTION · p. 27
• This single example does not establish the proposed mechanism, and rewrite quality varies across questions.The added habit addresses the kind of error made here, although the rewriter never saw the question.Source: F STRATEGY TRANSFER IN ACTION · p. 27

G.1 VISION-LANGUAGE REASONING

• About 6.1 percentage points on MathVista over SelfOrg; on MMMU-Pro, SAGE’s hard-question accuracy is 34.6% versus 30.5%, with methods closely matched on easy and medium questions.The table reports overall results for both benchmarks and question-level results for MMMU-Pro.Source: §G.1 · p. 27, Table 3 · p. 28
• Review and revision let agents check peer responses against the original images, addressing visual interpretation as well as the reasoning used to derive answers.The authors offer this as a plausible explanation for the observed gains.Source: §G.1 · p. 28

H.1 CONTRIBUTION OF ADAPTING PROMPTS

• With review, routing, and voting unchanged, removing prompt rewriting consistently lowers results, isolating an added contribution from rewritten prompts beyond peer-answer coordination.The rewriter sees neither the problem nor any response, so the result is consistent with transferring reasoning strategy rather than leaking a candidate answer.Source: Table 4 · p. 28, §H.1 · p. 29
• Rewriting adds to response-level coordination on Qwen2.5-1.5B, while on Ministral-3-3B it lifts SAGE above the strongest multi-agent baselines.The comparable drop on the two tested backbones suggests the rewriting benefit does not shrink across these two scales.Source: §H.1 · p. 29

H.2 CONTRIBUTION OF DONOR SELECTION

• Across all six benchmarks, the effect favors peer-review donor selection over a random teammate, although several individual gaps are within run-to-run variation.The comparison holds the leader weight and retained reviews fixed, so the changed factor is the choice of prompt.Source: §H.2 · p. 29, Table 5 · p. 29
• A random donor recovers only a small part of rewriting’s benefit over SAGE-NOREWRITE, suggesting that the transferred strategy’s suitability to the problem matters.The selected donor is judged most reliable on the problem at hand.Source: §H.2 · p. 29
• The ablation uses Qwen2.5-1.5B only, leaving the size of the donor-selection effect with stronger backbones or different role pools open.The authors note that random donors may cost more with weaker or less related prompts.Source: §H.2 · p. 29

H.3 CONTRIBUTION OF EARLIER-STAGE ANSWERS

• Final-round WPV excludes initial answers, retained reviews, and earlier-round occurrences, contributing one answer per agent from the last completed round.The highest-scoring nonempty answer receives weight 1.5; others receive weight 1.0.Source: §H.3 · pp. 29–30
• Pooling earlier answers acts as a safeguard when revision loses a correct initial or review answer; Final-round WPV still outperforms every baseline on both backbones.The contribution is concentrated on the weaker Qwen2.5-1.5B backbone and hardest benchmark; pooling’s effect is smaller than removing prompt rewriting.Source: §H.3 · p. 30, Table 6 · p. 30

H.4 WHICH ROLES SERVE AS DONORS

• Donor preferences track task structure: Qwen2.5-1.5B favors formalizing roles on free-response math, while Ministral-3-3B favors multiple-choice or broad-knowledge roles where choices or domain knowledge matter.Roles are sampled and scored without regard to benchmark, and no role dominates any benchmark.Source: §H.4 · pp. 30–31
• On GPQA-Diamond, neither backbone shows a role preference; both answer most questions incorrectly, weakening the signal for identifying a reliable donor.Figure 6 reports donor percentages by role, with 25% chance selection; problems decided by agent-index tie-break are excluded.Source: §H.4 · p. 30, Figure 6 · p. 31