PaperlayerView exact revision on arXiv

Source-linked AI summary

Reliable Chain-of-Thought via Prefix Consistency

Naoto Iwase, Yuki Ichihara, Mohammad Atif Quamar, Junpei Komiyama

arXiv:2605.07654v1stat.MLcs.CLcs.LG

TL;DR

Standard majority voting can fail when correct reasoning traces are not the most frequent, motivating a reliability signal for weighting sampled answers. The paper introduces prefix consistency, which regenerates CoT continuations from truncated prefixes and weights candidates by answer reproducibility. Across five reasoning models and four benchmarks, it was the strongest correctness predictor in most settings and reached the Standard MV plateau with substantially fewer tokens.

  • Problem

    Standard MV treats sampled traces equally and can fail when the correct answer is not the mode, especially on difficult problems.

  • Method

    Prefix consistency truncates each CoT, regenerates continuations, and uses candidate-level reproducibility to form PC-WMV without token log-probabilities.

  • Results

    Prefix consistency had the highest AUROC on 15 of 20 model-benchmark cells, typically around 0.7, outperforming weighted-voting baselines.

  • Takeaways & Limitations

    Prefix consistency is most effective when its discrimination gap D is positive and Pass@1 remains below the Standard MV plateau.

  • Takeaways & Limitations

    The cost comparison treats log-probability access as free, an implementation-dependent assumption that favors several baselines.

Abstract

from arXiv · show

Large Language Models often improve accuracy on reasoning tasks by sampling multiple Chain-of-Thought (CoT) traces and aggregating them with majority voting (MV), a test-time technique called self-consistency. When we truncate a CoT partway through and regenerate the remainder, we observe that traces with correct answers reproduce their original answer more often than traces with wrong answers. We use this difference as a reliability signal, prefix consistency, that weights each candidate answer by how often it reappears under regeneration. It requires no access to token log-probabilities or self-rating prompts. Across five reasoning models and four math and science benchmarks, prefix consistency is the best correctness predictor in most settings, and reweighting votes by it reaches Standard MV plateau accuracy at up to 21x fewer tokens (median 4.6x). Our code is available at https://github.com/naoto-iwase/prefix-consistency.

1 Introduction

Majority voting improves reasoning accuracy but treats all traces equally, limiting gains when correct answers are not the mode. The paper introduces prefix consistency as a reliability signal for weighted voting and reports stronger prediction and token efficiency across benchmarks.

  • Majority voting, also called self-consistency, samples diverse CoT traces and returns the most frequent answer.
  • Standard MV treats all CoT outputs equally and fails when the correct answer is in the minority.
  • Prefix consistency truncates each CoT and regenerates continuations, using answer reproducibility to weight votes without token log-probabilities.
  • Prefix consistency achieved the best macro-averaged AUROC on 15 of 20 model-benchmark cells, ranging from .63 to .80.
  • Median 4.6× fewer tokens and up to 21× fewer tokens let PC-WMV reach the Standard MV plateau across the evaluated settings.

2 Preliminary

The preliminary setup defines test-time answer aggregation from independently sampled reasoning traces. It motivates weighted majority voting because unweighted MV can fail on difficult problems, while existing reliability signals may not distinguish correct from wrong traces.

  • For each problem, an LLM generates a CoT trace and final answer, and benchmark-level Pass@1 is the macro-average of per-problem single-sample success probabilities.
  • Standard MV aggregates N independently sampled answers by returning the most frequent answer.
  • On FrontierScience-Olympiad with GPT-OSS-120B, DeepConf tail and P(True) nearly overlap for correct and wrong traces, whereas prefix consistency separates them.
  • Standard MV fails when the correct answer is not the mode, typically when Pass@1 is below 50%.
  • WMV assigns nonnegative weighted votes using reliability signals extracted from traces and, for some methods, token-level log-probabilities.

3 Prefix Consistency

Prefix consistency measures whether regenerated continuations reproduce an initial answer, then uses candidate-level reproducibility to weight votes. Its reproduction-rate asymmetry supports theoretical gains over Standard MV when correct traces are more reproducible.

  • 3.1 Prefix Consistency as a Reliability Signal: Prefix consistency truncates each CoT at a fraction τ and regenerates continuations from the shared prefix.
  • 3.1 Prefix Consistency as a Reliability Signal: For K=1, each group contains the initial answer and one regenerated answer, with arbitrary K handled by extension.
  • 3.1 Prefix Consistency as a Reliability Signal: The reproduction probabilities rC(τ) and rW(τ) are conditioned on whether the initial trace is correct or wrong.
  • 3.1 Prefix Consistency as a Reliability Signal: Correct initial answers tend to reappear under regeneration, while incorrect answers more often change to different incorrect answers.
  • 3.2 Prefix-Consistency-Weighted Majority Voting (Algorithm 1): PC-WMV weights every distinct initial or regenerated candidate, allowing answers absent from the initial samples to receive positive votes.
  • 3.2 Prefix-Consistency-Weighted Majority Voting (Algorithm 1): When rC(τ) > rW(τ), PC-WMV strictly improves over Standard MV in the binary-answer limit and can recover the correct answer where Standard MV converges to the wrong majority.

4 Experiments

Experiments across five reasoning models and four math and science benchmarks evaluate prefix consistency as a correctness signal and as a weighting mechanism for majority voting. Prefix consistency consistently separates correct from wrong traces, while PC-cubic substantially reduces tokens needed to approach Standard MV performance, especially when the discrimination gap and room above Pass@1 are large.

  • Experimental setup: Experiments cover five reasoning LLMs across one science benchmark and three math benchmarks, comparing prefix-consistency methods with Standard MV, WMV baselines, and adaptive-stopping rules.The evaluated models are GPT-OSS-120B, GPT-OSS-20B, Nemotron3-30B, Nemotron2-9B, and Ministral3-14B; the benchmarks are FrontierScience-Olympiad, HMMT Feb 2026, AIME 2025, and Brumo 2025.
  • Prefix Consistency as a Correctness Predictor: D > 0 on every model-benchmark cell, with correct traces reproduced more often than wrong traces under prefix regeneration.Table 1 defines D = rC − rW and reports that rC ≥ rW holds on every cell.
  • Prefix Consistency as a Correctness Predictor: Prefix consistency has the highest correctness-discrimination AUROC on 15 of 20 model-benchmark cells, typically around 0.7.Baseline AUROC often approaches 0.5 on harder cells, while prefix consistency separates correct and wrong traces more clearly.
  • Token Efficiency: PC-cubic’s advantage tracks D and the Pass@1-to-plateau gap, while it offers little advantage when Pass@1 is already close to the Standard MV plateau.Two small-gap Nemotron3-30B cells show little or negative advantage, whereas large-D cells yield the largest reductions.
  • Token Efficiency: PC-cubic is competitive with or better than AC sweep on most cells and outperforms ESC sweep at α=99% on every cell except Nemotron3-30B AIME 2025.AC performs better on the two cells where Pass@1 is close to the Standard MV plateau; AC also misses the α=99% target on two cells, while PC-cubic reaches it on all 12.
  • How Problem Difficulty Affects the Discrimination Gap D: rC rises strongly with Pass@1, whereas rW varies more weakly and inconsistently with problem easiness across models and categories.Across the six displayed model-category pairs, β(rC) ranges from +2.7 to +5.1, while |β(rW)| ≤1.14; four of six rW slopes are not statistically distinguishable from zero.

5 Conclusion

The paper presents prefix consistency as a practical signal for weighted vote aggregation and finds that it improves correctness prediction and cost-equivalent performance. It also situates the method among test-time scaling, robustness, and reasoning-error research.

  • Conclusion: Prefix consistency measures whether truncated reasoning traces reproduce their answers under regeneration, supporting weighted majority voting without token log-probabilities.The method combines parallel samples with structured truncation and regeneration to probe reasoning robustness.
  • Conclusion: Prefix consistency is a stronger correctness predictor than existing weighted-voting baselines across the evaluated benchmarks and model scales.The conclusion reports stronger prediction performance across benchmarks and five model scales.
  • Conclusion: PC-WMV improves over existing weighted majority voting methods under cost-equivalent comparison, especially on more difficult benchmarks.The method targets settings where Standard MV and related aggregation approaches struggle.
  • Reasoning Robustness: Prefix consistency exploits the asymmetry that correct reasoning traces are more reproducible than incorrect ones under regeneration.This connects the method to evidence on internal answer determination, error propagation, and reasoning robustness.
  • Test-Time Scaling: The method uses parallel samples with truncation and regeneration, positioning it between sequential chain extension and ordinary parallel aggregation.The same perturbation is suggested as a possible quality estimate for partial reasoning states in richer search procedures.
  • Overthinking and Length Scaling: Aggregating multiple short samples addresses overthinking concerns associated with extending a single long reasoning trace.The cited work reports substantial token overhead from overthinking and motivates the shorter-sample setting targeted here.

B Limitations and Future Work

The paper identifies operational, theoretical, and scope limitations for prefix consistency. Its asymptotic analysis depends on explicit assumptions about regeneration and sampling, while practical gains depend on signal separation, available accuracy headroom, and visible reasoning traces.

  • Practical Limitations: PC-WMV helps most when regeneration behavior meaningfully separates correct from incorrect traces.Its advantage shrinks when wrong answers are stable or correct answers reproduce weakly.
  • Practical Limitations: PC-WMV has limited room to improve when Pass@1 is already close to the Standard MV plateau.In such settings, the wrong-answer vote is already small and gains over Standard MV or simple confidence baselines may be limited.
  • Practical Limitations: Prefix consistency incurs additional inference cost and uses fixed defaults for the truncation fraction, regeneration count, and weighting function.The trade-off may be less favorable when log-probabilities are readily available and inexpensive.
  • Practical Limitations: The method requires an explicit Chain-of-Thought trace that can be truncated and continued from a prefix.It therefore cannot be applied directly to systems that hide internal reasoning or expose only summaries.
  • Scope and Future Work: The observed reproducibility asymmetry is a behavioral signal rather than a mechanistic explanation, and its validity beyond reasoning-oriented math and science evaluations remains unestablished.The paper leaves its origin and transfer to other domains, model families, and API settings for future work.
  • Asymptotic Analysis: The asymptotic analysis assumes i.i.d. sample pairs and characterizes convergence through a population objective with a unique maximizer.The analysis decomposes the objective into pooled-mass and self-reproduction terms and states convergence under explicit assumptions.

C.3 Proof of Theorem 1

The binary-case proof shows PC-WMV can converge to the correct answer in a region where Standard MV converges to the wrong one, provided correct traces reproduce more reliably than wrong traces. Empirical checks support the theorem’s assumptions and show prefix consistency generally discriminates correctness better than baselines.

  • Proof: In the binary answer case, PC-WMV converges to the correct answer whenever π(a⋆) exceeds rW /(rC + rW ).This threshold follows from the specialized theorem and the reproduction rates for correct and wrong answers.
  • Proof: The weighting function affects the binary proof margin only through w(1), with w(0)=0 and w(1)>0 selecting reproduced candidates.The proof derives the margin from self-reproduction and pooled-mass terms, which share the same bracket.
  • Proof: PC-WMV converges where Standard MV does not when rW /(rC + rW ) < π(a⋆) ≤ 1/2.The interval has positive length exactly when rC > rW.
  • Empirical verification: Across evaluated cells, the theorem’s assumptions occur frequently: Pr[A2 | A1] is at least 87.5% everywhere and reaches 100% in 40% of cells.The positive gap predicted for convex weights also occurs with overwhelmingly high probability conditional on A1.
  • Correctness prediction: Prefix consistency has the highest AUROC on 15 of 20 model–benchmark cells, while baselines often remain near the random-classifier level on harder cells.The macro ROC curves average per-problem ROC curves over problems containing both correct and wrong initial samples.
  • Cost–accuracy: Across nearly all 20 cells, PC-cubic reaches the Standard MV plateau at a smaller token budget, with the largest discrimination gap on FrontierScience-Olympiad.Figure 5 presents cost–accuracy curves with confidence bands and operating-point error bars.

D.3 Full Baseline Comparison

The full baseline comparison finds that PC variants form a consistently strong accuracy band, especially on the difficult science benchmark, while other confidence methods become competitive when Standard MV is already near its ceiling.

  • Full baseline set: The three PC variants form a tight band near the plateau across nearly every cell, whereas DeepConf variants, especially filtered variants, vary widely.Figure 6 includes the full baseline set and unattainable oracle upper bounds.
  • Difficult benchmark: PC-linear ≤ PC-quadratic ≤ PC-cubic holds in 11 of 12 FrontierScience-Olympiad model–budget cells.The sole inversion is the Nemotron3-30B 5M cell, where PC-quadratic narrowly exceeds PC-cubic.
  • Difficult benchmark: Nemotron2-9B is an exception: PC-cubic reaches .199 versus .203 for Standard MV at 1M tokens, consistent with its smallest discrimination gap.The reported gap is D = 7.4% on FrontierScience-Olympiad.
  • Easier benchmarks: On easier benchmarks, verbalized confidence and DeepConf tail become competitive or lead in some cells as Standard MV approaches its peak.For example, Verbal binary reaches .951 versus .931 for PC-cubic on GPT-OSS-20B Brumo at 1M tokens.
  • Additional baseline: SubthoughtReasoner does not consistently improve over Standard MV in the evaluated GPT-OSS-20B cells.It exceeds Standard MV on HMMT but falls below it on FrontierScience-Olympiad and AIME 2025; other cells were not evaluated.

D.4 Pool Coverage vs. Reweighting

PC-WMV’s advantage is attributed to reweighting candidates by prefix consistency rather than to adding substantially more correct candidates through regeneration. The default low-cost configuration is broadly effective, while deeper or repeated regeneration trades budget for discrimination.

  • Pool coverage versus reweighting: PC-WMV’s advantage over non-PC baselines comes from reweighting, not an enlarged pool of correct candidates.The oracle using initial and regenerated pools nearly coincides with the Standard MV oracle across most cells.
  • Pool coverage versus reweighting: Regenerations do not raise per-answer correctness rates: per-problem initial and regenerated correctness marginals cluster near the y=x diagonal, and differing cells can favor initial answers.This rules out improved marginal correctness as the explanation for PC-WMV’s gain.
  • Sensitivity to τ and K: At 250k tokens, τ=0.25, K=3 PC-cubic costs about 3.25× and underperforms Standard MV on three of four benchmarks.The high regeneration cost reduces the number of new groups sampled under a fixed budget.
  • Sensitivity to τ and K: The low-cost τ=0.75, K=1 configuration costs about 1.25× and outperforms Standard MV on all four benchmarks.At 1M–5M tokens it remains close to the per-cell best on the three math benchmarks.
  • Sensitivity to τ and K: On FrontierScience-Olympiad, deeper truncation and more regenerations can repay their higher cost: τ=0.25, K=3 reaches .547 at 1M versus .537 for the default.At 5M tokens the corresponding accuracies are .562 and .545.

D.6 Robustness to Judge Choice

The robustness analysis recomputes correctness and performance with an external judge rather than each model’s own judge. Prefix consistency’s relative predictive advantage and the method ordering remain stable, with judge-dependent shifts concentrated on FrontierScience-Olympiad.

  • AUROC robustness: PC’s AUROC advantage is preserved under external judging: 9 of 12 cells favor PC and 3 favor the best baseline, with no cell flipping quadrant.The external judge is Claude Sonnet 4.6, and AIME 2025 is omitted because it uses exact match.
  • Cost–accuracy robustness: WMV cost–accuracy curves nearly overlap across judges, and the per-cell method ordering is preserved at operating points.On FrontierScience-Olympiad, external-judge curves shift upward by a roughly uniform offset while PC-cubic retains its relative position.
  • Judge-dependent shifts: Judge disagreement is below 4% on Brumo and HMMT but reaches 9%–17% per-sample flips on FrontierScience-Olympiad.Pass@1 rises by up to 0.09 on three FrontierScience-Olympiad cells under the external judge.
  • Reproduction-rate analysis: The reproduction-rate analysis finds that rC rises with Pass@1, while rW has a smaller, problem-dependent slope and is lower on Science than Math.Figure 11 extends the GLM panels to GPT-OSS-20B and Nemotron2-9B; slopes use logistic regressions with cluster-bootstrap inference.
  • Inference: The GLM analysis uses problem-level cluster bootstrap resampling to account for correlation among regenerations sharing a prefix.Trial-level Bernoulli standard errors would otherwise be underestimated by a factor of 2 to 4.

D.7.2 Robustness under Per-Benchmark Pooling

Per-benchmark robustness checks support the paper’s main relationships: correct-trace reproduction rises with Pass@1, while wrong-trace reproduction has smaller, problem-dependent slopes. Alternative estimators recover the same qualitative pattern across all five models.

  • Visualization: Figure 12 separates the three Math benchmarks into individual curves and displays per-problem reproduction rates against Pass@1 with cluster-bootstrap confidence intervals.The figure directly visualizes the per-benchmark refit rather than the pooled Math relationship.
  • Per-benchmark refit: β(rC) > 0 on all 20 model–benchmark cells, with p < 0.05 in every cell and p < 0.001 in 17.The per-benchmark refit rules out the pooled Math curve as the source of the rising-rC pattern.
  • Per-benchmark refit: |β(rW)| < β(rC) on all 20 model–benchmark cells, confirming the smaller wrong-trace slope within benchmarks.The comparison remains within-benchmark rather than arising from pooling across datasets.
  • Benchmark-specific effects: GPT-OSS-120B’s pooled Math β(rW) = +1.14 reflects positive effects on AIME 2025 (+1.92) and Brumo 2025 (+3.46), but not HMMT Feb 2026 (−0.14).The wrong-trace relationship is therefore benchmark-specific rather than uniform across Math.
  • Alternative estimators: All three estimators agree qualitatively across all five models: rC rises for every model–category pair, while rW has a smaller, problem-dependent slope.The checks include binned trial-pooled rates and LOWESS in addition to the logistic GLM.
  • Baseline comparison: DeepConf tail and P(True) are likewise plotted against Pass@1 by correct versus wrong trace classes using Gaussian linear fits.These panels provide a comparable Pass@1-conditioned view for baseline confidence signals.

E Baseline Implementation Details

The appendix standardizes baseline confidence scores, weighting rules, adaptive stopping procedures, and prefix-consistency variants. Evaluations use a fixed pre-generated pool, with prefix-consistency methods drawing additional regenerations.

  • Confidence signals: Baseline methods differ in whether their reliability signal uses trace text, token log-probabilities, or a separate self-rating call.The documented baselines include response probability, verbal confidence, P(True), Self-certainty, and DeepConf.
  • Prefix consistency: PC-linear, PC-quadratic, and PC-cubic instantiate PC-WMV with power-family weights w(n)(c) = c^n for n = 1, 2, and 3.PC-cubic denotes the n = 3 variant.
  • DeepConf: DeepConf applies identity weighting to five trace-level signals, including first-token, Mean, bottom-10%, block-min, and tail confidence.Filtered variants retain selected traces before summing confidence-weighted votes.
  • Evaluation setting: The DeepConf online early-termination mechanism is excluded because evaluation is offline over a pre-generated pool under a shared token budget.This distinguishes the evaluated offline baseline from its original online stopping use.
  • Adaptive stopping: Adaptive Consistency stops when a Beta-binomial posterior favors the leading answer over the runner-up by the chosen threshold, then returns the running mode.The AC sweep varies Cthresh from 0.5 through 0.999 and adds no per-sample weighting.
  • Pool construction: Each cell uses N = 128 initial generations per problem, or N = 64 for Ministral3-14B, plus K regenerations per sample for prefix-consistency methods.All methods analyze the same fixed generated pool, so comparisons share generated tokens.

G.3 Cost-Accuracy Evaluation (Sections 4.2 and 4.3)

Cost–accuracy evaluation samples from fixed generation pools under token budgets, compares methods using monotone accuracy envelopes, and measures efficiency relative to a finite Standard MV plateau.

  • Trial design: At each token budget B, samples are drawn with replacement from the fixed pool until cumulative cost reaches B, then mean accuracy is computed over M trials.The same trial design supports fixed-budget accuracy comparisons and Monte Carlo confidence intervals.
  • Reference accuracy: The Standard MV plateau is its bootstrap-saturated accuracy at the finite pool budget B = 10^7 tokens, rather than an unbounded i.i.d. asymptote.This keeps the target reachable by Standard MV for every α between 0 and 1.
  • Interpolation: Each method’s budget–accuracy points are reduced to a running-max envelope before log-budget interpolation estimates the minimum budget reaching a target accuracy.AC and ESC use their natural-stopping point lists instead of continuous budget grids.
  • Uncertainty: Efficiency confidence intervals use a parametric bootstrap that perturbs fixed-budget accuracies and natural-stopping costs with Gaussian noise derived from stored confidence intervals.The procedure recomputes envelopes, targets, and budget ratios for each bootstrap draw.
  • Answer processing: Final answers are extracted from \boxed{} using regex and normalization before aggregation.This extraction rule underlies the answer values consumed by the cost–accuracy pipeline.

H.1 Answer Extraction

Answer extraction combines boxed-answer parsing with benchmark-specific equivalence judging, while verbal-confidence parsers introduce model-dependent missingness. The evaluated model serving as its own judge remains a stated limitation.

  • Boxed-answer extraction: Failed boxed responses receive no vote and only reduce the effective sample count; 17 of 20 cells remain below 1% failure for both generations and continuations.The three continuation cells above 1% all involve Nemotron3-30B.
  • Verbal confidence: Verbal-confidence extraction failure ranges from under 1% for Nemotron2-9B and Ministral3-14B to 19%–56% for GPT-OSS and Nemotron3-30B on Verbal 0–100.AUROC analysis drops failures, whereas WMV imputes the per-problem median.
  • Cost accounting: Verbal-confidence calls are charged for secondary-call tokens regardless of parse success, so failures raise per-vote cost rather than reducing the effective sample count.This cost treatment differs from the handling of failed boxed answers.
  • Parser behavior: Verbal 0–100 parsing is fragile because commentary or parenthetical tokens can precede the expected digit, especially in longer reasoning-model completions.Successful completion medians range from 30–200 tokens for failing models, versus 5 tokens for Nemotron2-9B.
  • Equivalence judging: AIME 2025 uses exact normalized integer match, while HMMT Feb 2026, Brumo 2025, and FrontierScience-Olympiad use LLM-based equivalence judgments.Equivalent answers are grouped through transitive closure over judged equivalence edges.
  • Judging limitation: Using the evaluated model as its own judge may shift absolute accuracies, although Claude Sonnet 4.6 preserves relative AUROC and WMV orderings on the tested subset.The external re-scoring check covers 12 cells across four models and three benchmarks.

I Reproducibility Statement

The study releases its analysis pipeline, reproduction scripts, answer pool, and supporting model, dataset, sampling, prompt, and token-count specifications. Aggregation and voting are bitwise reproducible from the released pool, while initial generation remains stochastic.

  • Reproducibility resources: The analysis pipeline, reproduction scripts, and answer pool are publicly released for reproducing the reported numbers.The releases are provided through GitHub and Zenodo.
  • Reproducibility resources: Aggregation and voting use seed 42, making every reported number bitwise reproducible from the answer pool despite stochastic initial generation.The initial generation does not use a vLLM seed.
  • Inference configuration: Inference runs through vLLM on 4× NVIDIA A100 80GB GPUs with a 131,072-token context window and 100,000-token maximum output.Models use their recommended sampling settings listed in Table 25.
  • Inference configuration: Prompts request step-by-step reasoning and boxed final answers, and regenerated continuations share the same prompt as initial samples.Math and science tasks use different domain-specific system prompts.
  • Experimental scope: Experiments use five reasoning LLMs across one science benchmark and three math benchmarks, with model and dataset details listed in Table 24.The full released test split is used for every benchmark.
  • Token accounting: Table 26 reports average output tokens for original generations, truncation continuations, and secondary verbal queries, with N denoting answers per problem.The default truncation fraction is τ=0.75, with additional τ values for GPT-OSS-20B.
Loading 2605.07654v1…