PaperlayerView exact revision on arXiv

Source-linked AI summary

The Sequential Edge: Inverse-Entropy Voting Beats Parallel Self-Consistency at Matched Compute

Aman Sharma, Paras Chopra

arXiv:2511.02309v1cs.LGcs.AI

TL;DR

The paper asks whether parallel independent chains or sequential refinement is better at equal test-time compute. It evaluates sequential reasoning and introduces inverse-entropy voting; sequential reasoning outperforms parallel approaches in 95.6% of configurations. The authors conclude that sequential methods merit consideration as a default, while noting limits in task and architecture coverage and a latency trade-off.

  • Problem

    The paper asks whether parallel independent chains or sequentially refined chains perform better under matched test-time compute, addressing limited comparative evidence for sequential reasoning.

    Source: §1 · p. 1

  • Method

    The authors compare sequential chains that build on previous attempts with parallel chains and introduce inverse-entropy voting based on token-level logprobs.

    Source: §3.1 · p. 3, §3.3 · p. 4

  • Results

    Sequential reasoning outperforms parallel approaches in 95.6% of configurations across five models and three benchmarks, with accuracy gains up to 46.7%.

    Source: §9 · p. 9

  • Takeaways & Limitations

    The authors advocate sequential refinement and entropy-guided weighting as design principles for inference-time reasoning systems.

    Source: §9 · pp. 9–10

    The evaluation focuses mainly on mathematical and scientific reasoning with transformer-based models; sequential execution also incurs wall-clock overhead that trades off against accuracy in time-sensitive applications.

    Source: §8 · p. 9

Abstract

from arXiv · show

We revisit test-time scaling for language model reasoning and ask a fundamental question: at equal token budget and compute, is it better to run multiple independent chains in parallel, or to run fewer chains that iteratively refine through sequential steps? Through comprehensive evaluation across 5 state-of-the-art open source models and 3 challenging reasoning benchmarks, we find that sequential scaling where chains explicitly build upon previous attempts consistently outperforms the dominant parallel self-consistency paradigm in 95.6% of configurations with gains in accuracy upto 46.7%. Further, we introduce inverse-entropy weighted voting, a novel training-free method to further boost the accuracy of sequential scaling. By weighing answers in proportion to the inverse entropy of their reasoning chains, we increase our success rate over parallel majority and establish it as the optimal test-time scaling strategy. Our findings fundamentally challenge the parallel reasoning orthodoxy that has dominated test-time scaling since Wang et al.'s self-consistency decoding (Wang et al., 2022), positioning sequential refinement as the robust default for modern LLM reasoning and necessitating a paradigm shift in how we approach inference-time optimization.

1 Introduction

Although parallel self-consistency dominates test-time scaling, this paper evaluates sequential refinement under matched compute and reports that it outperforms parallel reasoning in most configurations. It introduces inverse-entropy voting to weight answers by reasoning-chain confidence.

2.1 The Parallel Reasoning Orthodoxy

  • Self-consistency decoding and related approaches have made parallel generation and aggregation the dominant test-time scaling paradigm.The paper identifies sequential iterative refinement as an underexplored alternative.Source: §1 · p. 1, §2.1 · p. 2

1 Introduction

  • Sequential reasoning beats parallel approaches in 95.6% of configurations, with accuracy gains up to 46.7%.The paper also introduces inverse-entropy weighted voting as an aggregation method for sequential reasoning.Source: §1 · p. 1

2.2 Sequential Reasoning: The Road Less Taken

  • Earlier sequential work explored iterative improvement, but prior studies lacked broad matched comparisons or relied on fine-tuning and specialized methods.This paper evaluates training-free sequential reasoning across open-source model families under matched compute.Source: §2.2 · pp. 2–3

3 The Sequential Reasoning Framework

  • The framework iteratively refines attempts through context accumulation, self-correction, and verification before aggregating answers.Its overview culminates in inverse-entropy weighted voting.Source: Figure 2 · p. 3

3.1 Sequential vs Parallel Paradigms

  • Sequential chains explicitly build on prior reasoning, whereas parallel chains are generated independently and their answers are voted on.Continuation prompts provide each sequential step with its previous reasoning chain.Source: §3.1 · p. 3

3.2 Seven Voting Methods for Sequential Chains

  • The study compares seven baseline voting methods and an inverse-entropy approach for aggregating sequential-chain answers.The baselines include schemes that increase, decrease, or equalize weights across steps.Source: §3.2 · p. 4

3.3 Novel Inverse-Entropy Weighted Voting

  • Inverse-entropy voting assigns greater weight to chains with lower token-level Shannon entropy, which indicates higher model confidence.The method computes entropy from token logprobs and uses inverse-entropy weights.Source: §3.3 · p. 4

4 Experimental Setup

The evaluation compares five open-source models across reasoning and creative tasks, using controlled chain counts and matched token budgets. The protocol specifies model access, generation settings, and benchmark scope.

4.1 Model Selection and Infrastructure

  • Five open-source models spanning architectural families and parameter scales were accessed through OpenRouter for consistent evaluation.The models include GPT-OSS, Qwen3, and Kimi-K2.Source: §4.1 · pp. 4–5

4.2 Benchmark Selection

  • Evaluation covers mathematical, graduate-level science, and creative-generation tasks.AIME tests multi-step mathematics, GPQA-Diamond tests graduate-level science reasoning, and joke generation is used for a creativity ablation.Source: §4.2 · p. 5

4.3 Experimental Protocol

  • Both paradigms use 3, 6, or 9 chains with up to 4096 tokens per chain, matching total token budgets across configurations.For example, six parallel chains and six sequential steps each use 24,576 tokens.Source: §4.3 · p. 5

5 Results

Sequential reasoning wins nearly all tested matched-compute comparisons, while six chains provide the reported efficiency balance. Voting results favor inverse-entropy weighting, with stronger gains in sequential than parallel settings.

5.1 Sequential Dominance and Optimal Chain Configuration

  • Sequential reasoning wins 43 of 45 configurations (95.6%), with gains up to 46.7 percentage points on Qwen3-235B AIME-2025 at six chains.The reported scores are 76.7% for sequential reasoning versus 30.0% for parallel reasoning.Source: §5.1 · p. 5, Table 1 · p. 6
  • Six chains yield 13.8 accuracy points per 1K tokens for sequential reasoning, while parallel approaches yield 11.7 beyond six chains.The study identifies six chains as the optimal balance of compute and performance across model families.Source: §5.1 · p. 5
  • Sequential refinement supports iterative error correction, accumulated context, and answer verification across reasoning passes.These mechanisms are described as unavailable to independent parallel chains.Source: §5.1 · pp. 5–6

5.2 Sequential vs Parallel Voting Analysis

  • Inverse-entropy voting is optimal in 97% of sequential configurations, while entropy-weighted voting beats majority voting in all six tested parallel configurations.Parallel improvements are reported as +0.5–3.4%; the paper also reports that entropy-weighted voting has larger gains in sequential settings across architectures.Source: §5.2 · p. 6
  • Parallel chains operate independently, limiting iterative verification and refinement; the reported voting advantage is consequently larger in sequential settings.The paper describes confidence-based weighting as less effective without progressive refinement opportunities.Source: §5.2 · p. 6

6 Ablation Studies

The ablations test sequential reasoning beyond the main reasoning tasks and examine its scaling under matched token budgets. They find a creative-output tradeoff between lexical and semantic diversity, while sequential reasoning maintains higher performance as budgets increase.

6 Ablation Studies

  • The ablation studies include seven sequential voting methods across models and benchmarks and a six-chain comparison of parallel majority and entropy voting.The studies are presented as tests of the approach and the boundaries of sequential reasoning advantages.Source: §6 · p. 6, Table 2 · p. 7, Table 3 · p. 7

6.1 Ablation 1: Creative Task Analysis

  • Sequential reasoning yields greater lexical diversity, while parallel approaches yield greater semantic diversity across chain configurations.The creative-task evaluation uses joke generation and assesses outputs across 3, 6, and 9 reasoning chains.Source: §6.1 · pp. 7–8, Figure 3 · p. 8

6.2 Ablation 2: Token Budget Scaling Analysis

  • Sequential reasoning achieves consistently higher absolute performance and accuracy per 1K tokens as budgets rise from 2K to 16K.The comparison uses Qwen3-30B-A3B-Instruct-2507 on GPQA-Diamond with matched compute at each budget.Source: §6.2 · p. 8

7 Future Work

The paper proposes further tests of sequential scaling and hybrid approaches. Its token-budget comparison reports that sequential self-refinement outperforms parallel self-consistency across the tested budgets.

Hybrid architectures

  • Future work proposes hybrid systems combining parallel exploration and sequential refinement, including entropy-gated branching and adaptive switching.Integrating parallel decoding into sequential frameworks is proposed as a way to address latency while preserving accuracy gains.Source: §7 · p. 8

Extensions and theory

  • The authors propose extending evaluation to vision-language, multimodal, and code-generation tasks and developing formal accounts of when sequential reasoning outperforms parallel methods.The proposed theory includes width-versus-depth scaling laws for inference-time compute.Source: §7 · p. 8

Broader evaluation

  • The authors call for larger evaluations across applications such as reward modeling and real-time agentic systems, including production deployment considerations.The passage frames these evaluations as validation beyond academic benchmarks.Source: §7 · pp. 8–9

Token Budget Scaling

  • Across 2K–16K tokens, sequential self-refinement outperforms parallel self-consistency by 6.6 to 8.9 percentage points.The passage says these results align with wider-versus-deeper inference scaling laws and may indicate greater efficiency at lower budgets.Source: Figure 4 · p. 9

8 Limitations

The paper reports that sequential refinement beats self-consistency in most tested configurations under matched compute, while identifying limits in task and model coverage and latency. It also proposes design principles and describes its ethics and reproducibility practices.

8 Limitations

  • Evaluation is limited to mathematical and scientific reasoning with transformer-based models and token budgets up to 16K; broader tasks and architectures remain to be assessed.The paper names commonsense reasoning, code generation, multimodal problems, recurrent models, and state-space models as examples for further validation.Source: §8 · p. 9
  • Sequential refinement requires serial execution, creating wall-clock overhead and an accuracy–latency tradeoff for real-time applications.Source: §8 · p. 9

9 Conclusion

  • Across five open-source models and three benchmarks under matched compute, sequential refinement outperforms self-consistency in 95.6% of configurations, with accuracy gains up to 46.7%.The conclusion also reports inverse-entropy weighted voting as optimal in 97% of cases (29/30), and identifies six-chain configurations as the optimal compute–accuracy balance.Source: §9 · p. 9
  • The proposed design principles are iterative refinement, entropy-guided confidence weighting, and model-adaptive chain lengths.Source: §9 · p. 9

Ethics Statement

  • The authors report following responsible-AI principles, acknowledge potential dual-use considerations, and state that no human subjects were involved.They also state that experiments used publicly available models and benchmarks with attribution.Source: Ethics Statement · p. 10

Reproducibility Statement

  • The paper documents experimental configurations, random seeds, temperature settings, token budgets, and chain lengths, and says code will be made available upon publication.The inverse-entropy weighted voting algorithm and implementation details are also described as provided.Source: Reproducibility Statement · p. 10

A Detailed Voting Methods Analysis

This appendix describes seven baseline voting methods and inverse-entropy weighted voting for aggregating sequential reasoning chains. It also specifies prompts that ask models to review and refine earlier reasoning steps.

A.1 Sequential Voting Methods

  • The methods compare position-based weighting schemes, including increasing or decaying weights, inverse rank, and a linear preference for later steps.The decay method assumes initial attempts are most valuable, while later-step weighting is motivated by the hypothesis that refinement improves reasoning.Source: §A.1 · p. 13
  • Simple majority gives every reasoning step equal weight and serves as the control baseline.The baseline makes no assumption about which steps are more valuable.Source: §A.1 · p. 13
  • Inverse-entropy voting assigns each reasoning chain a weight based on its token-level Shannon entropy.The weights are inversely proportional to entropy, with a small epsilon for numerical stability.Source: §A.1 · p. 13

B.1 System Prompts and Experimental Configuration

  • The AIME and GPQA prompts request step-by-step problem analysis and a final answer in the task's specified format.Refinement prompts for steps 2–6 ask models to review previous reasoning for gaps or errors; steps 7–9 additionally invite alternative approaches and missed considerations.Source: §B.1 · pp. 13–14

B.2 Model Specifications and API Configuration

This block identifies the model-specification and experimental-configuration tables, then outlines the statistical tests, sample size, and answer-validation rules used in the evaluation.

B.2 Model Specifications and API Configuration

  • Table 4 is identified as the source for model specifications and configuration.The supplied passage gives the table title but no model-level values.Source: Table 4 · p. 14

B.3 Comprehensive Experimental Parameters

  • Table 5 is identified as the complete experimental-configuration table.The supplied passage gives the table title but no parameter values.Source: Table 5 · p. 14

B.4 Statistical Analysis and Validation

  • The statistical plan uses a two-tailed Welch’s t-test, Cohen’s d, and 95% bootstrap confidence intervals based on 1,000 iterations.The null hypothesis is no significant difference between sequential and parallel performance; the alternative is that sequential reasoning significantly outperforms parallel reasoning.Source: §B.4 · p. 14
  • Each benchmark configuration uses 30 problems, with a minimum detectable effect of Cohen’s d = 0.5.The analysis includes 270 paired comparisons across configurations; invalid answers are excluded, and at least two valid answers are required per configuration.Source: §B.4 · p. 15
  • AIME answers are validated as integers from 0 to 999, while GPQA answers must be single letters from A to D.The appendix lists distinct extraction patterns for the two benchmarks.Source: §B.4 · p. 15

C Optimal Chain Length Analysis

The section analyzes chain-length choices across computational budgets and models, then tests entropy-based voting robustness to the number of top logprobs. It also specifies fallback handling for errors in entropy calculation.

C Optimal Chain Length Analysis

  • The chain-length analysis concerns accuracy-efficiency trade-offs across computational budgets and models.Table 6 is titled as an analysis of optimal chain length; the supplied passages do not give its cell values.Source: C Optimal Chain Length Analysis · p. 15, Table 6 · p. 15

C.1 Performance Analysis by Budget and Model

  • Sequential reasoning outperforms parallel approaches across AIME-2024, AIME-2025, and GPQA-Diamond in the Kimi-K2 Instruct chain-length analysis.Reported advantages include 13.3% on AIME-2024 with 6 chains, 20.0% on AIME-2025 with 6 chains, and consistent 1–3% improvements on GPQA-Diamond.Source: Figure 5 · p. 15, Table 6 · p. 15

D.1 Top-K Validation Methodology

  • The top-k validation computes Shannon entropy from the k most probable tokens and applies inverse weighting to aggregate answers.It tests k = 5, 10, 15, and 20 across representative configurations from each model family.Source: §D.1 · p. 16

D.2 Entropy Distribution Analysis

  • Accuracy remains identical across top-k values from 5 to 20, and higher k values do not improve discrimination between high- and low-confidence chains.The passage reports that k = 5 captures sufficient information for the entropy-based uncertainty quantification.Source: §D.2 · p. 16, Table 8 · p. 16

E.1 Error Handling and Fallback Mechanisms

  • The error-handling procedure falls back to majority voting when entropy calculation fails and uses epsilon = 10^-10 for numerical stability.It also validates entropy values in the range [0, 20] bits and handles single-token responses and empty sequences.Source: §E.1 · p. 17

F Entropy Aggregation Variants Analysis

The section compares mean, median, maximum, and minimum entropy aggregation for inverse-entropy voting. The reported accuracy is identical across the four methods, while mean entropy is preferred for efficiency and theoretical justification.

F.1 Alternative Entropy Aggregation Methods

  • Median entropy reduces sensitivity to outliers but loses granular uncertainty information, while maximum entropy captures peak uncertainty and may be oversensitive to outliers.Minimum entropy focuses on the most confident reasoning moments but may miss overall chain quality.Source: §F.1 · p. 17

F.2 Comparative Performance Analysis

  • All four entropy aggregation methods achieve identical accuracy across tested configurations, so the aggregation function does not significantly affect final voting performance.The comparison covers mean, median, maximum, and minimum entropy.Source: Table 9 · p. 17, §F.2 · p. 17
  • Mean entropy is preferred for its balance of computational efficiency and theoretical justification, despite equivalent empirical performance.It requires a single-pass calculation with minimal memory overhead and incorporates information from the entire reasoning sequence.Source: §F.2 · pp. 17–18

G Test-Time Scaling Comparison with S1 Framework

The section reports that continuous self-refinement scales better than S1 without specialized fine-tuning, then specifies semantic and lexical diversity measures and a matched-budget humor evaluation protocol.

G Test-Time Scaling Comparison with S1 Framework

  • Across increasing token budgets on GPQA-Diamond, continuous self-refinement achieves higher accuracy than S1 without specialized fine-tuning.S1 uses supervised fine-tuning on 1K s1K examples, whereas this approach uses post-trained models' inherent reasoning capabilities.Source: Figure 6 · p. 18, G Test-Time Scaling Comparison with S1… · p. 18

H.1 Semantic Diversity Metric

  • Semantic diversity uses pairwise sentence-embedding cosine similarity to measure conceptual distinctness rather than surface-level lexical variation.The implementation uses 384-dimensional embeddings from SentenceTransformers' all-MiniLM-L6-v2 model.Source: §H.1 · pp. 18–19

H.2 Lexical Diversity Metric

  • Type-token ratio measures vocabulary richness as unique-token count relative to total tokens across generated outputs.Higher TTR indicates more varied word choice and complements semantic diversity by capturing surface-level variety.Source: §H.2 · p. 19

H.3 Creativity Evaluation Protocol

  • The creativity protocol evaluates theme-varied joke generation under matched computational budgets and applies diversity metrics with statistical testing across multiple runs.Measurements include 95% confidence intervals and paired t-tests.Source: §H.3 · p. 19
Loading 2511.02309v1…