PaperlayerView exact revision on arXiv

Source-linked AI summary

Divergent-Convergent Reasoning: Scaling Test-Time Compute through Structured Solution Synthesis

Bo Wen, Yuhao Chen, Erhan Bilal, Carla Agurto Rios, Chen Wang, Junchen Jiang

arXiv:2608.15303v1cs.AI

TL;DR

Additional test-time compute can help LLM reasoning, but it remains unclear whether disagreement among candidate solutions is noise or useful signal. DCR separates diverse exploration from reconciliation, showing that it can recover correct minority reports, improve compute allocation through recursive reconciliation, and use dispersion to indicate when reconciliation helps.

  • Problem

    It is unclear whether disagreement among candidate solutions can be systematically exploited to improve LLM reasoning accuracy rather than treated as noise.

  • Method

    DCR generates diverse independent proposals, then uses reviewer calls to analyze disagreements and synthesize a reconciled answer, optionally recursively allocating further compute.

  • Results

    Across MATH500, AIME, and MMLU-PRO, DCR amplifies correct minority reports, while recursive reconciliation achieves higher accuracy with reduced compute and dispersion diagnoses task difficulty.

  • Takeaways & Limitations

    Disagreement can serve as a high-value signal for adaptive test-time reasoning rather than merely noise to average away.

  • Takeaways & Limitations

    Using dispersion as a gating signal requires generating an initial proposal pool, creating a potentially prohibitive cold-start cost in latency-sensitive environments.

Abstract

from arXiv · show

Test-time compute can substantially improve Large Language Model (LLM) reasoning performance, yet how and when additional compute helps remains poorly understood. We study Divergent-Convergent Reasoning (DCR), a simple two-phase primitive consisting of an exploration phase that generates multiple candidate solutions followed by a convergent reconciliation phase. We present three core results. First, we show that even a single reconciliation step can reliably amplify correct minority reports: across datasets, DCR often recovers the correct answer when correct exploration outputs are in the minority, a regime where majority voting fails. Second, we introduce recursive DCR, an autoregressive reconciliation system that iteratively analyzes disagreements and allocates additional test-time compute. Recursive DCR achieves higher accuracy than fixed-compute baselines-reaching 93.3% on AIME 2024 and 92.0% on AIME 2025-while using roughly 27% less compute on average, demonstrating that attentive resource allocation is superior to uniform scaling. Third, we analyze disagreement among exploration outputs via a simple, training-free dispersion metric. Dispersion reveals a structured relationship between disagreement and test-time gains: in regimes where DCR is effective, higher disagreement among exploration outputs is associated with larger accuracy improvements from reconciliation. Together, these results show that disagreement, often viewed as noise, can be systematically exploited to improve test-time reasoning and reveal emerging scaling laws for agentic LLM systems.

1. Introduction

The paper introduces Divergent–Convergent Reasoning (DCR), which generates diverse proposals and reconciles them by analyzing disagreements. It shows that reconciliation can recover correct minority reports, recursive disagreement analysis improves test-time compute allocation, and disagreement helps identify when reconciliation is most beneficial.

  • Method: DCR combines Divergent Exploration of independent proposals with Convergent Reconciliation that analyzes disagreements and synthesizes one answer.It is presented as a simple two-phase reasoning primitive.
  • Core finding: Reconciliation can amplify correct minority reports and succeed precisely where majority voting fails.This challenges the assumption that disagreement is merely noise and majority voting is optimal.
  • Recursive DCR: Recursive disagreement analysis enables more effective use of additional test-time compute.The recursive extension is motivated as a way to allocate compute through iterative reconciliation.
  • Single-round DCR: K = 25 single-round DCR increases consistency over exploration even when correct exploration answers are a minority.The contribution is reported across datasets and settings.
  • Positioning: DCR is positioned as complementary to verifier-based methods when trained verifiers are unavailable or impractical, including model routing and budgeted inference.These settings require deciding how to spend limited test-time compute without immediate ground-truth feedback.

2. Related Work

DCR is positioned within test-time compute scaling as a candidate-generation-and-reconciliation primitive, complementing verifier-based, verifier-free, and multi-agent approaches. The section emphasizes empirical characterization of when reconciliation and selective compute allocation help, without introducing new search operators or uniform scaling.

  • Test-Time Compute Scaling: Test-time compute scaling improves LLM reasoning through majority voting, inference scaling laws, compute-efficient inference, and iterative decoding or inference-time algorithms.These methods treat generation as more than a single forward pass, with verification shaping reasoning traces or guiding candidate selection.
  • Test-Time Compute Scaling: DCR studies generating multiple candidate solutions followed by reconciliation, empirically characterizing when additional reconciliation compute is beneficial.It does not propose new search operators or scale compute uniformly.
  • Verifier-Based and Verifier-Free Methods: DCR complements verifier-based methods by addressing deployment without a reliable verifier before answer revelation, while allocating a finite inference budget.Verifier-based methods can use scaled test-time compute more efficiently when reliable verification signals are available.
  • Verifier-Free and Multi-Instance Methods: Unlike debate and ReConcile, DCR uses reviewer-style reconciliation to analyze minority-correct amplification, dispersion, and recursive unanimous-consent stopping.ReConcile uses multi-round discussion with confidence-weighted voting to reach consensus, whereas DCR emphasizes selective test-time compute.

3. Methodology: Divergent–Convergent Reasoning

Divergent–Convergent Reasoning separates diverse proposal generation from critical review and reconciliation, with recursive rounds enabling verifier-free, adaptive allocation of test-time compute. Reviewers analyze consensus and disagreements, explicitly retaining minority corrections, while dispersion is studied as an optional diagnostic rather than a core control requirement.

  • 3. Methodology: Divergent–Convergent Reasoning: DCR separates inference into divergent exploration, which generates diverse proposals, and convergent reconciliation, which synthesizes them into a high-quality answer.The two phases can be iterated for deeper reasoning.
  • 3. Methodology: Divergent–Convergent Reasoning: Exploration samples independent LLM replicas, potentially heterogeneous and varied by stochastic seeds or temperatures, to form a solution pool S = {x_n}^N.The setup treats independent proposals as perspectives from multiple research labs.
  • 3. Methodology: Divergent–Convergent Reasoning: Reconciliation runs K independent reviewer calls that verify consensus steps, pinpoint divergent steps, and synthesize a coherent answer while retaining potentially corrective minority reports.This process requires reviewers to reason about disagreements rather than perform simple majority voting.
  • 3. Methodology: Divergent–Convergent Reasoning: Recursive DCR feeds reconciled outputs into subsequent rounds and stops when all K reviewers agree or when the maximum round/call budget is exhausted.Its verifier-free stopping rule adapts compute to persistent disagreement.
  • 3. Methodology: Divergent–Convergent Reasoning: Dispersion is an optional, training-free post-hoc diagnostic or triage heuristic, whereas recursive DCR itself selectively allocates compute by stopping quickly on easy tasks and using more rounds for hard tasks.Dispersion is not required for the core recursive control loop.

4. Experiments

The experiments evaluate DCR across four challenging reasoning benchmarks and four LLMs spanning different capability levels. They compare single-round, mixed-model, and recursive reconciliation using trial accuracy and consistency metrics.

  • Experimental setup: The evaluation covers MATH500, AIME 2024, AIME 2025, and MMLU-PRO using Granite-4-H-Small, Llama-3.3-70B, Llama-4-Maverick-17B-128E, and GPT-OSS-120B.GPT-OSS-120B uses default/medium reasoning effort rather than high effort.
  • Experimental setup: Single-model single-round DCR generates proposal pools and applies reconciliation with reviewer width K = 25 across T = 25 stochastic trials.The same model performs one reconciliation round on its own proposal pool.
  • Recursive reconciliation: Recursive DCR uses reviewer width K = 8 per round, ranging from 8 to 80 reviewer calls across 1 to 10 rounds.The compute-parity point with a T = 25 single-round baseline is around 3 rounds (8 × 3 = 24).
  • Metrics: The evaluation reports trial accuracy as the fraction of correct trials and consistency as the fraction of problems with a strict majority (> 50%) of correct trials.Consistency measures whether the method’s signal outweighs its noise and corresponds to majority-vote accuracy over the trials.

5. Results and Analysis

Results show that DCR improves accuracy and consistency by reconciling diverse solution proposals, while recursive DCR achieves higher accuracy with adaptive compute allocation. Dispersion identifies when reconciliation is most useful: moderate disagreement marks recoverable uncertainty, whereas extreme disagreement signals capability limits.

  • Single-round DCR: DCR consistently improves trial accuracy and consistency over Sampling, often converting minority correct answers into stable majority outcomes.On AIME 2024, GPT-OSS rises from 74.3% to 88.1% trial accuracy and from 76.7% to 90.0% consistency with DCR Single.
  • Recursive DCR: 93.3% trial accuracy on AIME 2024 and 92.0% on AIME 2025 are achieved by Recursive DCR while using ∼27% less compute on average than the single-round baseline.The comparison is approximately 36 calls versus 50 calls.
  • Recursive DCR: Unanimous-consent stopping allocates roughly 33 calls to easy problems and up to 80 calls to hard problems with persistent disagreements.This adaptive allocation drives the higher-accuracy, lower-cost result rather than uniform scaling.
  • Recursive DCR: Recursive DCR yields high stability, with problems typically solved in every repeated trial or failed in every trial.Across 10 runs per problem, only one AIME 2025 problem showed an intermediate success rate.
  • Dispersion analysis: Dispersion predicts both task difficulty and reconciliation gains: moderate dispersion identifies recoverable uncertainty, while extreme dispersion indicates capability collapse.DCR is most effective for 0 < d ≲0.3; gains diminish when d ≳0.3, and near-zero dispersion yields minimal gains.
  • Dispersion analysis: Dispersion provides a cheap, unsupervised deployment diagnostic for prioritizing reconciliation and escalating or referring high-dispersion tasks.Systems can reconcile tasks in the recoverable zone while flagging extreme-dispersion tasks for human review or a stronger model.

6. Discussion and Limitations

The discussion positions DCR as a verifier-free complement to verifier-based methods, explains its success through selection among candidate arguments, and highlights cache reuse and dispersion-based routing as deployment considerations.

  • Positioning Relative to Verifier-Based Methods: DCR complements verifier-based approaches by leveraging intrinsic solution-space consistency when ground-truth labels or trained verifiers are scarce.Verifier-based methods remain powerful when reward signals are available.
  • Efficiency via KV Cache Reuse: Efficient KV cache reuse amortizes prompt-processing costs across multiple samples, reducing a key computational concern for ensemble-based reasoning.Examples include LMCache, CacheGen, and CacheBlend.
  • Mechanistic Intuition: Selection is Easier than Generation: DCR often succeeds where majority voting fails, supporting the view that recognizing a correct argument is easier than generating one from scratch.The divergent phase creates diverse reasoning traces, while reconciliation applies selection pressure among candidates.
  • Mechanistic Intuition: Selection is Easier than Generation: Recursive reconciliation can amplify selection pressure so the correct answer dominates even when it begins as a minority.This describes DCR as an evolutionary process in which reasoning traces form a population of candidate genotypes.
  • Dispersion as a Measure of Relative Difficulty: Sample-then-Assess uses a small initial batch’s dispersion as a direct, training-free signal of relative difficulty for allocating compute.This approach addresses the Router Paradox and is presented as more adaptable to novel tasks than predict-then-route systems.

7. Future Work

The paper identifies future work around reducing dispersion-estimation overhead, addressing confident errors in low-dispersion regimes, optimizing recursive stopping, and expanding dispersion signals beyond answer embeddings.

  • Upfront Cost of Estimation: N = 25 proposals are required to estimate dispersion, motivating adaptive probing or hybrid systems for latency-sensitive settings.One proposed strategy starts with N = 5 to estimate variance before allocating the full cost to high-risk tasks.
  • Bias in Low-Dispersion Regimes: Low dispersion can reflect consistently wrong predictions, motivating heterogeneous ensembles that mix proposal sources rather than scaling one model.The passage characterizes this failure mode as “confident hallucination” and notes that DCR is less effective there.
  • Stop Criteria Optimization: Strict unanimous consent may be overly conservative, motivating soft-consensus thresholds or probabilistic stopping rules to balance compute cost and accuracy.These alternatives are proposed specifically to optimize the trade-off between compute cost and accuracy.
  • Multi-Signal Dispersion: Future dispersion metrics could incorporate reasoning-path disagreement, confidence-score variance, and token-level entropy alongside answer embeddings.These signals extend disagreement measurement from final answers to intermediate reasoning, reported confidence, and generation-level uncertainty.

8. Conclusion

Divergent–Convergent Reasoning (DCR) makes test-time compute adaptive by combining diverse hypothesis generation with reconciliation, amplifying correct minority reports and using disagreement as a signal of recoverable uncertainty.

  • DCR transforms test-time compute from a uniform resource into an adaptive control variable through ensemble-based inference.The primitive separates diverse hypothesis generation from critical review.
  • Single-round DCR amplifies correct minority reports in settings where majority voting fails.
  • Recursive reconciliation achieves state-of-the-art accuracy with reduced compute by stopping early on consensus.
  • DCR treats disagreement as a high-value signal of recoverable uncertainty rather than noise to be averaged away.Its divergent and convergent phases emulate deliberate, self-correcting deliberation.

Impact Statement · A. Differentiation from Prior Work (Supplementary)

The work studies verifier-free allocation of test-time computation in large language models, targeting more efficient inference and better uncertainty awareness. It also examines risks from confident incorrect agreement and uses persistent disagreement to trigger abstention or additional computation.

  • Impact Statement: The work studies allocation of test-time computation in large language models without access to verifiers.This defines the paper’s focus as verifier-free computation allocation.
  • Impact Statement: Improved inference efficiency is identified as a potential benefit of the proposed methods.The passage frames efficiency as a potential, rather than guaranteed, deployment benefit.
  • Impact Statement: Better uncertainty awareness in deployed systems is another potential benefit.The work connects computation allocation with uncertainty awareness in deployment.
  • Impact Statement: High-confidence agreement on incorrect solutions is identified as a risk.Agreement alone can produce confidently wrong outcomes.
  • Impact Statement: The work explicitly analyzes failure modes involving high-confidence incorrect agreement.The passage states that these risks are examined directly.
  • Impact Statement: Persistent disagreement can trigger abstention or additional computation.Disagreement is treated as a signal for withholding an answer or allocating more test-time compute.

A.1. Verifier-free (VF) post-training and trace imitation … B. Prompts

The paper positions DCR as a verifier-free, inference-time alternative to post-training, early stopping, debate, and static aggregation methods. Its prompts implement divergent proposal generation and reviewer-style reconciliation by focusing analysis on disagreements while preserving verified consensus.

  • A.1. Verifier-free (VF) post-training and trace imitation: DCR allocates test-time compute through cross-solution disagreement and reviewer-style reconciliation, rather than single-trajectory length control or curated post-training supervision.It is designed for deployment when reliable verification is unavailable and retraining is impractical.
  • A.1. Verifier-free (VF) post-training and trace imitation: DCR is an inference-time multi-agent procedure that uses structured generation and aggregation instead of parameter updates or demonstrations to shape reasoning.The framing concerns verifier-free test-time decision-making, not claims about SFT/RL tradeoffs or replacement of post-training.
  • A.2. Early stopping / test-time compute allocation: DCR uses inter-agent answer dispersion to allocate more reconciliation compute, contrasting with intra-trajectory reflection, semantic redundancy, or answer entropy used for early stopping.Its policy responds to high dispersion by continuing multi-agent reconciliation rather than halting a single chain.
  • A.3. Multi-agent debate, judging, and aggregation: DCR separates divergent exploration from reviewer-style reconciliation, unlike iterative debate, peer persuasion, or weighted voting over existing agents’ answers.The framework can use unanimous-consent stopping and generates a new reconciled solution rather than merely revising or selecting existing responses.
  • A.3. Multi-agent debate, judging, and aggregation: DCR dynamically allocates compute from solution-proposal dispersion and constructs new coherent solutions, whereas ensemble methods select or weight existing outputs using diversity or higher-order dependence information.This distinguishes DCR from LLM-TOPLA and higher-order aggregation approaches that emphasize static ensemble construction or aggregation.
  • A.4. Minority/outlier theory support: DCR operationalizes the idea that divergent opinions can improve aggregate outcomes by treating dispersion as a valuable signal rather than noise.The cited human-crowd work provides theoretical and empirical motivation for this use of disagreement.
  • B.1. Divergent Exploration Prompt (Sampling): The exploration prompt instructs each solver to understand the problem, plan an approach, execute it step by step, verify the answer, and present the requested final format.It also encourages multiple strategies when applicable and careful checking of arithmetic, algebra, boundary cases, and alternative solutions.
  • B.2. Convergent Reconciliation Prompt: The reconciliation prompt treats consensus steps as requiring brief verification, analyzes divergent steps as difficult way-points, retains verified consensus, and replaces disagreements with mathematically sound reasoning.Its output must preserve logical flow from premises to conclusion in structured JSON containing step-by-step reasoning and a final answer.

C. Dispersion metric

The dispersion metric diagnoses the stability and breadth of exploration-stage reasoning by measuring how candidate answers vary around an embedding-space centroid. Its centroid-based formulation remains robust when a tight incorrect majority coexists with diverse potentially correct outliers.

  • C. Dispersion metric: High dispersion indicates qualitatively different reasoning paths and answers across repeated samples, whereas low dispersion indicates convergence to essentially the same answer.Low dispersion may reflect either correctness or systematic bias.
  • C. Dispersion metric: Candidate answers are mapped to 1024-dimensional vectors with a fixed Sentence-BERT embedding model before computing their centroid.The model uses a RoBERTa backbone.
  • C. Dispersion metric: Dispersion is the average squared Euclidean distance from candidate-answer embeddings to their centroid.
  • C. Dispersion metric: The centroid-based measure is robust when a tight incorrect cluster dominates diverse, potentially correct outlier answers.This addresses degenerate majority effects in exploration outputs.

D. Result example: Emergence of the correct solution from incorrect proposals

In an AIME24 example, Recursive DCR recovered the correct answer 902 from exclusively incorrect Round 1 proposals, reaching unanimous agreement by Round 3 through reflective debugging and persuasion.

  • Consensus evolution: Round 3: the corrected reasoning persuaded the remaining reviewers, producing 100% unanimity after an exclusively incorrect Round 1.The ground-truth answer was 902.
  • Recovery mechanism: Round 2: Reviewers 1 and 3 independently discovered 902 by debugging the dominant 1024 argument.They identified a missing feasibility constraint: maximality forbids configurations leaving any row or column empty.
  • Recovery mechanism: 1 + 1 + (25 −2)2 = 902: the corrected count followed from matching the colors appearing among rows and columns.The error was structural rather than arithmetic.
  • Consensus evolution: 0% to 100%: consensus on the correct answer 902 evolved from Round 1 to Round 3.The setup used AIME24 problem aime24-2024-II-9 with gpt-oss-120b and reviewer width K = 4.

D.1. Output and reasoning traces

The reasoning traces contain sharply conflicting counts for the chip-placement problem, including 1024, 13806, 122, and 902. The disagreement centers on how maximality constrains empty rows and columns and whether colour assignments must match across rows and columns.

  • Conflicting outputs: Other traces report 122 while rejecting 1024, 13806, and related counts as mishandling maximality.The highlighted 122 trace states that its configurations respect chip supplies and that the empty placement is not maximal.
  • 13806 reasoning: 13806 is obtained by inclusion-exclusion over assignments with no empty rows or no empty columns.The trace reports 7354 assignments for each condition and subtracts 902 configurations counted in both sets.
  • 902 reasoning: 902 counts the two monochromatic configurations plus 900 assignments in which both rows and columns contain both colours.This approach requires the colour sets present among rows and columns to match, yielding 30 × 30 mixed-colour assignments.
  • 1024 reasoning: 1024 is derived by independently assigning white or black to each of five rows and five columns, then filling matching-colour intersections.This reasoning treats all 2^5 × 2^5 assignments as valid and bounds each colour’s chip use by 25.
  • Maximality dispute: The central dispute is whether maximality forces every row and column to be non-empty or instead only rules out certain empty-row and empty-column combinations.Some traces argue an empty row or column permits adding a chip, while others distinguish mismatched colours and require matching row and column colour sets.

E. MMLU-PRO per-category results (Appendix)

This appendix reports MMLU-PRO results by category, using the same comparison format as the overall results: Sampling, DCR (Single), and DCR (Mix). The breakdown is distributed across three tables.

  • Per-category breakdown: Per-category results compare Sampling with DCR (Single) and DCR (Mix).The appendix presents the category breakdown in the same format as the overall MMLU-PRO results.
  • Table organization: Table 4 contains the first part of the MMLU-PRO per-category trial accuracy and consistency results.The table is labeled as part 1 of 3.
  • Table organization: Table 5 contains the second part of the MMLU-PRO per-category trial accuracy and consistency results.The table is labeled as part 2 of 3.
Loading 2608.15303v1…