PaperlayerView exact revision on arXiv

Source-linked AI summary

Compare to Correct: Pairwise-Guided Logit Adjustment for LLM Test-Time Scaling

Cong Hua, Qianqian Xu, Zitai Wang, Zhiyong Yang, Shilong Bao, Linchao Zhang, Qingming Huang

arXiv:2610.18583v1cs.AI

TL;DR

Sampling-based aggregation scores paths by answer frequency or confidence but does not explicitly compare competing answers. PGLA combines pairwise confidence comparisons with voting support through a KL-regularized logit adjustment. Experiments report gains over voting baselines, with an online extension that improves sampling efficiency.

  • Problem

    Sampling-based methods aggregate answer frequencies or path confidence without explicitly characterizing comparisons between competing answers.

    Source: Introduction · pp. 1–2

  • Method

    PGLA estimates answer-level comparative advantage from confidence comparisons between paths supporting different answers and combines it with voting evidence through a KL-regularized logit adjustment.

    Source: Introduction · p. 2

  • Results

    PGLA achieves overall gains over voting baselines across mathematical reasoning, knowledge reasoning, and code generation; internal-confidence experiments report approximately 2% average accuracy gain across 33 model–benchmark settings.

    Source: Introduction · p. 2, Conclusion · p. 10

  • Takeaways & Limitations

    The framework accommodates different voting methods, confidence sources, and comparative-advantage rules, while its online extension improves sampling efficiency.

    Source: Conclusion · p. 10

    Reranking over observed answers cannot recover pools that contain no correct candidate.

    Source: Case Analysis · p. 19

Abstract

from arXiv · show

Sampling-based test-time scaling enhances LLM reasoning by aggregating multiple reasoning paths for the same query. Existing methods use answer frequencies or path confidence to compute candidate scores, which do not explicitly characterize comparisons between competing answers. Inspired by the Condorcet principle in social choice theory, we introduce pairwise confidence comparisons between paths supporting different answers to capture answer-level comparative advantage. We combine this comparative evidence with voting support through a KL-regularized objective. Theoretically, we show that this comparative-advantage is equivalent to the adjustment to the traditional voting logits. In this way, our proposed Pairwise-Guided Logit Adjustment (PGLA) framework allows comparative evidence to compensate for weaker voting support and correct voting-based ranking errors. Experiments on 10 challenging benchmarks including mathematical reasoning, knowledge reasoning, and code generation with 7 models ranging from 1.7B to 32B parameters speak to the effectiveness of this plug-and-play term. Additional evaluations with external reward scores and online sampling further demonstrate its applicability.

Introduction

Sampling-based reasoning aggregates multiple paths, but frequency- and confidence-based scores can miss how answers compare directly. PGLA adds pairwise confidence comparisons to voting, and experiments report gains across tasks and models.

Introduction

  • Existing aggregation methods use answer frequencies or path confidence, but may overlook information in sampled paths that distinguishes competing answers.Best-of-N selects the highest-confidence path, majority voting uses answer frequency, and confidence-based voting weights or filters path contributions.Source: Introduction · p. 1
  • In the Condorcet example, plurality selects A, while pairwise comparisons select B, whose supporters rank it above both alternatives more often.A receives 3 of 7 first-choice votes, but B is preferred to A in 4 of 7 rankings and to C in 5 of 7.Source: Introduction · p. 2, Figure 1 · p. 2
  • PGLA estimates answer-level comparative advantage from confidence comparisons between paths supporting different answers and combines it with voting evidence through a KL-regularized objective.The resulting adjustment to base voting logits lets comparative evidence compensate for weaker voting support.Source: Introduction · p. 2
  • Approximately 2% average accuracy gain over corresponding voting baselines across 33 model–benchmark settings; the online extension also achieves higher mean accuracy with fewer sampled paths than Online SC.The online comparison uses the same stopping rule and covers six mathematical reasoning settings.Source: Introduction · pp. 2–3

Problem Formulation and Representative Aggregators

The formulation converts sampled path evidence into answer scores, while the social-choice motivation shows why first-choice support alone can miss useful comparisons. Pairwise confidence comparisons offer a proxy for those unobserved preferences.

Problem Formulation

  • Given K sampled paths, each path yields an answer and confidence score; aggregation assigns each candidate an answer-level score and selects the highest-scoring candidate.The candidate set may be predefined or formed dynamically from sampled answers.Source: Problem Formulation · p. 3
  • Representative aggregators use a block’s maximum confidence for Best-of-N, its size for majority voting, or confidence-weighted contributions for confidence-based voting.Confidence scores may come from the model or an external verifier.Source: Problem Formulation · pp. 3–4
  • These aggregators use observed evidence within diagonal blocks but do not explicitly model preferences among alternative answers.Paths are grouped by output answer in the path-answer matrix, leaving other answer entries unobserved.Source: Problem Formulation · pp. 3–4

Motivation: A Social Choice Perspective

  • Two preference profiles can give A, B, and C identical first-choice counts yet different pairwise comparison matrices and winners.Both profiles give A 3 votes and B and C 2 each; under P, B wins pairwise, while under P′, A does.Source: Motivation: A Social Choice Perspective · p. 4
  • A Condorcet winner defeats every alternative in direct comparisons; when one exists, this rule selects it.The cited examples select B under P and A under P′.Source: Motivation: A Social Choice Perspective · p. 4
  • Because sampled paths reveal only their first-choice answer and confidence, pairwise confidence comparisons proxy the missing preference ordering.The motivating disagreement occurs when an incorrect answer has at least as many votes as a correct one, but correct-answer paths tend to have higher confidence in pairwise comparisons.Source: Motivation: A Social Choice Perspective · p. 4
  • 69.87%–91.62% of recoverable SC errors satisfy the pairwise-disagreement condition across three models and three mathematical benchmarks.Recoverable pools contain at least one correct answer; Fig. 4(b) reports the prevalence of the condition.Source: Motivation: A Social Choice Perspective · pp. 4–5, Figure 4 · p. 5

Pairwise-Guided Path Aggregation

PGLA estimates answer-level comparative evidence from confidence comparisons between paths supporting different answers, then combines it with voting support through a KL-regularized objective.

Pairwise-Guided Path Aggregation

  • PGLA constructs answer-level comparative evidence from pairwise confidence comparisons and combines it with a voting distribution.The framework is proposed for path aggregation as an alternative to applying the Condorcet approach directly.Source: Pairwise-Guided Path Aggregation · p. 5

Pairwise Confidence Comparisons

  • For different answers, path confidence comparisons score each comparison +1, 0, or −1, and a positive score means one answer’s paths win more often than the reverse.The pairwise scores are aggregated as a weighted average, with greater weight given to comparisons against answers with stronger voting support.Source: Pairwise Confidence Comparisons · pp. 5–6

Combining with Voting Information

  • A KL penalty keeps the adjusted distribution close to voting support while comparative evidence favors answers with stronger pairwise advantages.PGLA is equivalent to an additive adjustment to voting logits, allowing comparative evidence to compensate for weaker voting support.Source: Combining with Voting Information · p. 6
  • Better confidence discrimination can make pairwise comparisons more reliable and further improve aggregation performance.The passage identifies confidence scores’ ability to distinguish correct from incorrect paths as relevant to comparison reliability.Source: Combining with Voting Information · p. 7

Experiments

The experiments evaluate PGLA across aggregation baselines, model families, benchmarks, and analysis settings, reporting gains in overall and specialized comparisons.

Experimental Setup

  • The evaluation compares 11 aggregation baselines across seven models and ten challenging datasets spanning mathematical reasoning, knowledge reasoning, and code generation.The setup also reports HumanEval+ and MBPP+ using EvalPlus tests.Source: Experimental Setup · pp. 7–8

Overall performance

  • 4.61%, 1.78%, and 2.63% accuracy gains on AIME24, AIME25, and HMMT25, respectively, average over six Qwen3-8B voting baselines.PGLA matches or improves on corresponding voting baselines across evaluated Qwen3 reasoning and code-generation tasks.Source: Overall performance · pp. 8–9
  • 1.96% average accuracy improvement and 1.42% average ECE reduction accompany PGLA across four voting baselines with internal confidence estimates.The comparison uses three mathematics-specialized backbones.Source: Overall performance · p. 9

Performance with the External Reward Setting.

Case analysis

  • PGLA balances Case I, where voting support favors the correct answer, and Case II, where pairwise comparisons favor it despite equal or weaker voting support.Best-of-N often fails in Case I, while voting-based methods typically struggle in Case II.Source: Case analysis · p. 9

Fine-grained Analysis

  • Higher reliability of PRM scores for pairwise comparisons corresponds to higher PGLA accuracy on both models.The comparison uses PRM-Weighted SC as its baseline.Source: Fine-grained Analysis · p. 10
  • Increasing sampled paths generally improves performance on MATH and MathOdyssey, while PGLA remains strong across most tested K values.At temperatures T = 1.1 and T = 1.3, PGLA generally outperforms comparison methods across three math backbones.Source: Fine-grained Analysis · p. 10
  • Uniform averaging, Copeland, and Maximin generally improve accuracy over voting baselines, while the default rule achieves strong overall performance.On InternLM2-Math-Plus-1.8B, Online SC+PGLA improves accuracy over full-budget SC by 0.28–3.63% while reducing sampling cost by 50.8%–73.1%.Source: Fine-grained Analysis · p. 10

Conclusion

PGLA adds pairwise confidence evidence to voting-based answer aggregation, with experiments reporting gains across task domains and an online extension that improves sampling efficiency.

Conclusion

  • PGLA estimates comparative advantages between candidate answers from pairwise confidence comparisons across sampled reasoning paths.A KL-regularized objective combines this evidence with voting support and adjusts the base voting logits.Source: Conclusion · p. 10
  • The framework accommodates different voting methods, confidence sources, and comparative advantage rules.The conclusion presents it as a plug-and-play framework for sampling-based LLM reasoning.Source: Conclusion · p. 10
  • Experiments across mathematical reasoning, knowledge reasoning, and code generation report overall gains over voting baselines, while the online extension improves sampling efficiency.Table 4 presents an online setting on InternLM2-1.8B.Source: Conclusion · p. 10, Table 4 · p. 10

Related Work

Prior work aggregates sampled answers through voting, confidence-based selection, rankings, or adaptive sampling; this work focuses on reconstructing answer preferences from ordinary single-answer samples.

Related Work

  • Self-Consistency uses majority voting, while Best-of-N selects a candidate using a quality score such as a learned verifier or process reward model.Other approaches use an LLM or learned representations to assess semantic agreement.Source: Related Work · p. 16
  • Prior methods use confidence-weighted votes (Taubenfeld et al., 2025; Fu et al., 2026), path ranks (Kang et al., 2025), elicited rankings (Wang et al., 2025), or learned ranking (Marina et al., 2026).These methods exploit path quality or ordered candidate answers in different ways.Source: Related Work · p. 16
  • Unlike methods that elicit answer rankings, this work reconstructs answer-level preferences from pairwise confidence comparisons across ordinary single-answer samples and reconciles them with voting support.Adaptive methods also reduce sampling through agreement or confidence monitoring; the online extension updates preferences as paths arrive and stops when the decision remains stable.Source: Related Work · p. 16

Representative Methods

The section specifies how sampled paths, candidate answers, aggregation scores, and the selected prediction relate.

Representative Methods

  • The K sampled paths z_i produce answers a_i = g(z_i).This defines the path-to-answer mapping used in the scoring setup.Source: Representative Methods · p. 17
  • For each candidate answer a, an aggregation method M computes its score s_M(a; D).The passage introduces the aggregation score over the sampled data D.Source: Representative Methods · p. 17
  • The predicted answer is selected using Eq. (1).The passage says the same selection and voting formulas are applied after specifying possible confidence scores.Source: Representative Methods · p. 17

Path-Confidence Measures and Selection

The section defines confidence-based selection over sampled reasoning paths, including Best-of-N and several token-confidence measures. These choices determine how path confidence is used to rank answers.

Path-Confidence Measures and Selection

  • Perplexity confidence uses inverse perplexity, and exponentiation preserves Best-of-N selections and pairwise comparisons while supplying a positive voting contribution.The sampled path’s perplexity is PPL(z_i).Source: Path-Confidence Measures and Selection · p. 17
  • Self-Certainty measures deviation from a uniform vocabulary distribution using the full predicted token distribution rather than only the sampled token’s probability.The vocabulary size is denoted |Vtok|.Source: Path-Confidence Measures and Selection · p. 17
  • DeepConf aggregates token confidences over local windows; its bottom-10% variant uses top-k candidate tokens, with k = 20 following DeepConf.The local-window principle also underlies tail and minimum-window variants; the selected window set retains at least one window.Source: Path-Confidence Measures and Selection · p. 17
  • Best-of-N selects the answer with the maximum answer-level score among paths producing that answer.Its sampling budget is the K paths defined in Sec. 2.1.Source: Path-Confidence Measures and Selection · p. 17

Answer-Level Voting

Answer-level voting aggregates contributions from paths supporting each answer. The section contrasts unweighted votes with confidence-weighted and path-filtering variants.

Answer-Level Voting

  • Self-Consistency assigns one vote to each sampled path, so an answer’s score counts its supporting paths without using their confidence values.The indicator is one when its condition holds and zero otherwise.Source: Answer-Level Voting · p. 18
  • Confidence-based voting weights each path by a nonnegative coefficient, which can be zero for filtered paths; unfiltered Weighted SC assigns every path a coefficient of one.PPL-Weighted SC and DeepConf-Weighted SC use their respective confidence scores; confidence filtering retains or discards paths with binary coefficients.Source: Answer-Level Voting · p. 18
  • Perplexity Consistency aggregates contributions from distinct path–confidence pairs, while Reasoning Pruning filters low-confidence paths under its original query-specific mixture rule.The described Perplexity Consistency rule keeps one representative of each retained distinct pair and discards pruned paths and repeated copies.Source: Answer-Level Voting · p. 18

Case Analysis

The analysis distinguishes two complementary ranking errors: a single high-confidence path can mislead Best-of-N, while voting can miss pairwise confidence advantages. PGLA combines voting support with comparative evidence to address both patterns.

Case Analysis

  • In Case I, voting support favors the correct answer, but Best-of-N favors an incorrect answer because one incorrect path has the higher confidence maximum.The comparison considers two observed answers with supporting paths.Source: Case Analysis · pp. 18–19
  • In Case II, voting support favors or ties the incorrect answer while pairwise confidence comparisons favor the correct answer.The pairwise preference is the fraction of cross-answer path pairs favoring the correct answer, with ties counted as one half; the signed comparison is Y_cw = 2bρ_cw − 1.Source: Case Analysis · p. 19
  • A recoverable SC error is one whose candidate pool contains at least one correct answer; pools without a correct candidate cannot be fixed by reranking observed answers.The analysis measures Case II among recoverable errors.Source: Case Analysis · p. 19
  • Table 5 reports NoCorrect shares over all SC errors and Case-II shares among recoverable errors using PPL confidence, with K = 64 on MATH and K = 128 on MathOdyssey and AIME.The caption identifies the error-breakdown percentages and their different denominators.Source: Case Analysis · p. 19, Table 5 · p. 19
  • PGLA combines voting support, which tempers isolated confidence outliers, with comparative advantage, which can compensate for weaker voting support, yielding a better empirical tradeoff between the two error types.The passage points to Figs. 5 and 10 for the corresponding results.Source: Case Analysis · p. 19

Online PGLA Extension

The online extension updates PGLA as paths arrive and stops when its answer prediction stabilizes. Its stopping rule is applied consistently to a base aggregator and its adjusted counterpart.

Online PGLA Extension

  • Online PGLA updates its answer prediction as new paths arrive and stops sampling once the prediction becomes stable.The extension is intended to examine whether comparative evidence can reduce unnecessary inference-time sampling.Source: Online PGLA Extension · pp. 19–20
  • At each update, the observed-path prefix supplies voting support and pairwise comparative advantage for the answers seen so far.The evidence pool is represented by path-answer and confidence pairs, with supporting paths grouped by answer.Source: Online PGLA Extension · p. 20
  • When no other observed answer has positive voting support, the pairwise term is set to zero; the multiplicative adjusted score also handles zero voting support without smoothing.For positive voting support, the multiplicative score is equivalent to the log-score form.Source: Online PGLA Extension · p. 20
  • Sampling stops at the first update j ≥ L meeting the stability condition; a tie resets the count, and sampling continues to Kmax if stability is never reached.The label-free rule applies unchanged to a base aggregator and its PGLA-adjusted counterpart, enabling a matched online comparison.Source: Online PGLA Extension · p. 20

Proof for Proposition 3.1

This section establishes the PGLA optimizer’s logit adjustment and examines its behavior across additional evaluations, including accuracy, sampling efficiency, robustness, computational cost, and limitations.

Proof for Proposition 3.1

  • Strict concavity and KL-divergence equality establish a unique PGLA optimizer; its logit adjustment reduces to voting when comparative advantages are equal.The proof also connects the optimizer to the normalized answer-level PGLA scores.Source: Proof for Proposition 3.1 · pp. 20–21

Comparison Methods

  • The comparisons include 11 internal-confidence and voting baselines, three external-PRM baselines, and Online SC; “+ Ours” denotes PGLA enhancement.Methods use matched sampled-path subsets and repetitions; Online SC is included for sampling-efficiency comparisons.Source: Comparison Methods · p. 21, Implementation Details · pp. 23–24

LLM Backbones and Datasets

  • The evaluation uses seven generators from Qwen3, DeepSeekMath, and InternLM-Math across ten benchmarks spanning competition mathematics, graduate-level science, and code generation.HumanEval+ and MBPP+ add strengthened-test evaluations for the corresponding code benchmarks.Source: LLM Backbones · p. 22, Datasets · p. 23

Enhanced Code Tests in Reasoning Mode

Comparison of Non-Thinking and Reasoning Modes

Performance with the External Reward Setting

Tradeoff between Case I and Case II

Effect of Ranking Quality by Dataset

  • The ranking-quality study perturbs path-score assignments while preserving their multiset and using no correctness labels, then measures correct-versus-incorrect ranking with empirical AUC.The retention parameter ranges from λ = 0, which permutes all score assignments, to λ = 1, which preserves the original ranking.Source: Effect of Ranking Quality by Dataset · p. 29
  • On InternLM2-Math-Plus-7B, increasing ranking quality bρ from 0.496 to 0.729 raises PGLA’s absolute accuracy gain from −0.54% to +2.17%.PGLA also remains competitive and generally outperforms comparison methods at decoding temperatures T = 1.1 and T = 1.3, and matches or improves voting baselines on Qwen3-32B, with the most gains on HMMT25.Source: Effect of Ranking Quality by Dataset · p. 29, Temperature Sensitivity · p. 29, Table 16 · p. 31

Efficiency Comparison

  • PGLA-enhanced SC and RPC generally match SC’s best observed accuracy with fewer paths, with larger savings on InternLM2-Math-Plus-1.8B.The comparison selects the smallest tested budget at which each method matches or exceeds SC’s best mean accuracy on the evaluated grid.Source: Table 17 · p. 32, Table 18 · p. 32, Efficiency Comparison · p. 32

Complete Scaling Results with Varying K

Online Sampling Efficiency

  • Online SC+PGLA gains 0.39–6.40% over Online SC, reduces sampling cost by 50.8%–75.0%, and remains 0.10–5.89% above full-budget SC.Both online methods use the same stopping rule; predictions update every eight paths and stopping is possible from K = 24.Source: Table 21 · p. 35, Online Sampling Efficiency · p. 35

Effect of Comparative Advantage Rules

  • Voting-weighted comparisons yield higher macro-average accuracy than uniform averaging (29.06% versus 28.70%), while all four comparative rules improve accuracy over their baselines in most settings.Uniform averaging has lower macro-average ECE; Copeland is stronger on InternLM2-1.8B but weaker on the other models, and Maximin is consistently less accurate.Source: Table 23 · p. 37, Effect of Comparative Advantage Rules · p. 37, Table 24 · p. 38

Computational Overhead

  • At K = 128, sorting reduces median aggregation time from 108.3 to 60.3 µs, a 1.80× speedup; direct evaluation remains faster at K = 32 and 64.The exact sorting-and-prefix-sum method takes O(K log K + U) time and O(K + U) working memory; comparisons reuse path confidence scores and require no additional reasoning paths or per-pair model calls.Source: Table 26 · p. 39, Computational Overhead · p. 39

Connection to Rank-Based Borda Aggregation

  • PGLA applied to Borda raises macro-average accuracy from 59.17% to 61.21% on Qwen’s AIME24 and HMMT25 comparisons, and from 25.89% to 27.76% across six InternLM settings.Under SC, the correction has an exact rank representation, and SC-PGLA selects by log nu + (2 R̄u − K − 1)/(K − nu).Source: Table 27 · p. 40, Connection to Rank-Based Borda… · p. 40, Table 28 · p. 41

Impact Statement

  • PGLA’s gains do not eliminate underlying model or confidence-signal errors, and agreement among sampled responses does not guarantee correctness.The authors report using LLMs for sampled responses and process reward models for external-reward scoring, with generative AI also assisting implementation, analysis, and manuscript work; the authors reviewed the content and take responsibility for it.Source: Impact Statement · p. 42, AI Use Statement · p. 42

Related papers

Loading 2610.18583v1…