PaperlayerView exact revision on arXiv

Source-linked AI summary

Confidence Improves Self-Consistency in LLMs

Amir Taubenfeld, Tom Sheffer, Eran Ofek, Amir Feder, Ariel Goldstein, Zorik Gekhman, Gal Yona

arXiv:2502.06233v2cs.CLcs.AI

TL;DR

Self-consistency is effective for reasoning but computationally expensive because it requires many lengthy reasoning paths. CISC uses model-derived confidence to weight paths, reducing required paths while motivating within-question confidence evaluation and evidence that LLMs can assess output quality.

  • Problem

    Self-consistency requires generating many lengthy reasoning paths to increase the chance that the correct answer becomes most frequent, creating computational expense.

  • Method

    CISC extends self-consistency with a weighted majority vote based on confidence scores obtained directly from the model, and WQD evaluates within-question confidence discrimination.

  • Results

    CISC outperformed self-consistency in virtually all examined configurations across nine LLMs and four datasets, reducing required reasoning paths by over 40% on average.

  • Takeaways & Limitations

    Model-derived confidence scores provide practical evidence that LLMs can judge their own output quality, while WQD evaluates confidence for distinguishing responses to the same question.

  • Takeaways & Limitations

    The confidence-prompting approach may not be universally supported by implementation frameworks because it relies on prefix caching.

Abstract

from arXiv · show

Self-consistency decoding enhances LLMs' performance on reasoning tasks by sampling diverse reasoning paths and selecting the most frequent answer. However, it is computationally expensive, as sampling many of these (lengthy) paths is required to increase the chances that the correct answer emerges as the most frequent one. To address this, we introduce Confidence-Informed Self-Consistency (CISC). CISC performs a weighted majority vote based on confidence scores obtained directly from the model. By prioritizing high-confidence paths, it can identify the correct answer with a significantly smaller sample size. When tested on nine models and four datasets, CISC outperforms self-consistency in nearly all configurations, reducing the required number of reasoning paths by over 40% on average. In addition, we introduce the notion of within-question confidence evaluation, after showing that standard evaluation methods are poor predictors of success in distinguishing correct and incorrect answers to the same question. In fact, the most calibrated confidence method proved to be the least effective for CISC. Lastly, beyond these practical implications, our results and analyses show that LLMs can effectively judge the correctness of their own outputs, contributing to the ongoing debate on this topic.

1 Introduction

Self-consistency improves reasoning by selecting the most frequent answer across diverse sampled paths, but its computational cost motivates CISC, which weights paths by model-generated confidence. Across broad evaluations, CISC matches or exceeds self-consistency while reducing required reasoning paths, and supports within-question confidence analysis and LLM self-assessment.

  • Self-consistency samples diverse reasoning paths and selects the most frequent answer to improve LLM reasoning performance.
  • CISC assigns confidence scores to generated paths and uses them in a weighted majority vote to reduce self-consistency’s computational cost.
  • Over 40% fewer reasoning paths are required on average for CISC to achieve comparable performance to self-consistency across nine models and four datasets.
  • CISC outperforms self-consistency in virtually all examined configurations spanning mathematical and commonsense reasoning tasks.
  • Within-question confidence evaluation addresses the poor predictive value of standard methods for distinguishing correct and incorrect answers to the same question.
  • Qualitative analysis found significant agreement between model confidence scores and human assessments of reasoning-path quality.

2 Notations

The paper represents an autoregressive language model through its next-token distribution and describes each response as an answer paired with a reasoning path.

  • The model M is autoregressive, with parameters θ and a next-token distribution denoted pθ(·|x).
  • For a question q, the response is represented as (r, a), where r is the reasoning path and a is the final answer.
  • A reasoning path is a sequence of logical steps purportedly leading to the answer.

3 Confidence-Informed Self-Consistency

CISC extends self-consistency by scoring each sampled reasoning path with model-derived confidence, normalizing those scores, and selecting an answer through confidence-weighted voting. The temperature controls whether confidence weighting approaches ordinary frequency voting or prioritizes the highest-confidence response.

  • CISC reduces self-consistency’s computational costs by generating a confidence score for each reasoning path and performing a weighted majority vote.
  • A weighted vote can require fewer samples than ordinary majority voting when correct answers receive higher weights than incorrect answers.In the illustrative example, 40 samples are needed for 90% accuracy with standard voting, versus fewer than 10 with two-to-one weighting.
  • CISC incorporates the model’s self-assessment of each reasoning path into final answer selection.
  • Confidence-Informed Self-Consistency is defined through confidence extraction, confidence normalization, and confidence-weighted aggregation.
  • The temperature parameter T controls the relative importance of answer frequency versus confidence scores.
  • As T approaches infinity, CISC approaches vanilla self-consistency; as T approaches zero, it prioritizes the single highest-confidence response.

4 Experimental Setup

The experiments compare CISC with self-consistency across confidence extraction methods, reasoning benchmarks, and nine open-weight LLMs. Performance is estimated through bootstrap sampling, while cost and accuracy are measured against matched self-consistency baselines.

  • Evaluation Scope: CISC is evaluated across multiple confidence extraction methods, reasoning tasks, and model families.The study compares response probability, verbal confidence, and P(True) across four benchmarks and nine models from three families.
  • Datasets: The evaluation uses four reasoning benchmarks spanning mathematics, general knowledge, and commonsense-style reasoning tasks.The benchmarks are GSM8K, MATH, MMLU-Pro, and Big-Bench-Hard.
  • Models: The model pool contains nine instruction-tuned open-weight LLMs from the GEMMA2, QWEN2.5, and Mistral families.The selected models range from 0.5B to 123B parameters.
  • Metrics: CISC is compared with self-consistency using cost reduction and accuracy improvement metrics.Cost reduction measures responses saved by CISC at fixed budgets of 5 or 10 responses, while accuracy improvement compares equal response budgets.
  • Confidence Calibration: Confidence scores are softmax-rescaled with a temperature tuned separately for each model and confidence method using a 10% held-out set.The tuning data are aggregated across all four datasets, using one dataset-agnostic hyper-parameter per configuration.
  • Evaluation Procedure: Performance scores are computed by applying each decoding strategy to 500 bootstrap sets for each sample budget and averaging the resulting accuracies.The procedure samples 30 reasoning paths first and evaluates budgets from 1 through 30 paths.

5 Main Results

CISC outperforms self-consistency across virtually all examined models and datasets, improving both efficiency and accuracy. Its best results use P(True), while confidence normalization further improves performance when temperature settings are appropriate.

  • Overall Performance: CISC outperforms self-consistency across virtually all models and datasets and with every examined confidence method.The results are macro-averaged across models and datasets.
  • Efficiency: 46% average Cost Reduction is achieved by P(True) with a budget of 10 responses per question.At this budget, self-consistency requires 18.6 sampled responses on average to match CISC’s accuracy using 10 samples.
  • Efficiency: 41% average Cost Reduction is achieved by P(True) with a budget of 5 responses per question.Some configurations exceed 67% cost reduction.
  • Statistical Evidence: CISC improvements are strongly statistically significant for each examined confidence method.The significance claim is based on additional micro-averaged accuracy-improvement results and confidence intervals.
  • Confidence Normalization: Confidence normalization improves CISC performance compared with using unnormalized confidence scores.Normalization uses softmax with a tunable temperature, and its effectiveness depends on appropriate temperature hyper-parameters.
  • Confidence Normalization: P(True) confidence values require lower temperatures because they have high similarity and therefore need stronger separation.Optimal temperatures vary across confidence extraction methods because their score scales differ.

6 Within-Question Confidence Evaluation

Standard confidence metrics evaluate behavior across questions, but CISC requires distinguishing correct from incorrect responses to the same question. The proposed Within-Question Discrimination metric better predicts CISC performance and increases with confidence gaps.

  • Motivation: Verbal confidence methods can be better calibrated yet less effective for CISC than P(True).Traditional calibration evaluates confidence across different questions, whereas CISC compares responses to the same question.
  • Metric: WQD measures whether the higher-confidence response is correct within pairs of responses to the same question.The pairs contain exactly one correct response.
  • Evaluation: WQD perfectly predicts the relative CISC performance of confidence methods, unlike ECE-t and Brier-t.Across datasets and models, verbal methods had the best ECE-t and Brier-t but the worst CISC performance.
  • Analysis: Within-question discrimination increases smoothly with the confidence gap.The relationship is shown for P(True), Gemma2-9B, and MATH.
  • Implication: The findings support that LLMs can reassess the correctness of their own responses.The paper links this conclusion to confidence scores that distinguish correct and incorrect responses for the same question.

7 Qualitative Analysis

A human evaluation tested whether model confidence tracks recognizable flaws in reasoning paths. Low-confidence responses were substantially more likely than high-confidence responses to contain human-identified low-quality indicators.

  • Human Evaluation: 90 responses from MMLU-Pro were reviewed by two evaluators unaware of confidence scores and ground-truth labels.The sample included balanced cases selected around disagreements between CISC and self-consistency.
  • Results: 67% of model-relative-low-confidence samples contained human-identified low-quality indicators, compared with 33% of relative-high-confidence samples.The evaluators judged logical patterns that reduced confidence in answer correctness.
  • Results: The evaluated low-quality indicators were absent options, deliberation between options, and omitted necessary calculations.Only absent options and omitted calculations strongly correlated with low-confidence scores.

8 Related Work

The paper builds on research about confidence extraction, reasoning verification, and efficient self-consistency. It distinguishes CISC by applying model-derived confidence to entire reasoning paths and using those scores in answer selection.

  • Confidence Signals: Existing confidence methods derive signals from sample agreement, internal representations, or verbalized confidence.Prior work in this line focused on fact-seeking tasks.
  • Novelty: This work applies confidence scoring to entire reasoning paths rather than final answers alone.The paper describes this as a first application of these approaches to full reasoning paths.
  • Reasoning Verification: Research on LLM self-verification and self-correction remains contested, with positive and negative findings.The related literature reports disagreement about the source of observed gains.
  • Efficiency: Prior efforts to reduce self-consistency overhead face limitations including throughput–latency trade-offs and have not matched its adoption.The paper positions CISC as another approach to reducing computational overhead.
  • Self-Consistency with Confidence: Earlier confidence-weighted self-consistency approaches attributed failures to confidence scores being too similar, whereas this work finds useful signal in them.CISC uses confidence-weighted voting despite that prior concern.

9 Discussion

CISC consistently improves self-consistency efficiency across diverse settings, while WQD addresses shortcomings in conventional confidence evaluation. The discussion connects these findings to LLM self-assessment and proposes broader uses of model confidence.

  • Findings: CISC outperformed self-consistency across diverse models, datasets, and confidence extraction methods, reducing computation costs by over 40% on average.The method is presented as a lightweight extension of self-consistency.
  • Self-Assessment: Model-derived confidence scores provide practical evidence that LLMs can judge the quality of their own outputs.Human assessments of response quality showed significant agreement with model confidence.
  • Confidence Evaluation: WQD evaluates whether confidence methods distinguish response quality within the same question, which traditional calibration metrics do not capture.The paper demonstrates WQD’s effectiveness for evaluating confidence methods.
  • Future Work: Future work could integrate model self-confidence into Tree of Thoughts or Graph of Thoughts frameworks.The discussion also proposes training models to produce more accurate intrinsic or verbal confidence.
  • Future Work: CISC and WQD can be used to assess improvements in confidence generation, including signals derived from model internal states.The paper notes that recent evidence suggests internal-state signals can outperform P(True).

10 Limitations

The study identifies implementation, evaluation-scope, and experimental-coverage limitations that constrain how broadly CISC can be applied or interpreted.

  • Implementation scope: Confidence prompting uses short, appended prompts and prefix caching, but this implementation may not be supported universally.A one-step alternative is discussed, but the study evaluates the two-step approach.
  • Implementation scope: The preferred P(True) method requires access to the model’s token probabilities, which may not be available across all frameworks.The method is available in both open-weights and closed-weights frameworks described by the authors.
  • Evaluation scope: Human evaluation was conducted only on MMLU, so extending it to other datasets could provide additional evidence.The authors identify broader dataset coverage as a possible source of further insight.
  • Experimental coverage: The ablations cover confidence extraction and normalization, while additional comparisons remain unexplored.Suggested extensions include zero-shot versus few-shot prompting, alternative normalization techniques, and trainable confidence methods.

11 Ethics Statement

The paper states that CISC introduces no new ethical issues, while emphasizing that LLM deployment can perpetuate bias and create societal impacts.

  • Ethics Statement: CISC itself introduces no new ethical issues, according to the paper’s ethics statement.The statement distinguishes the decoding strategy from broader risks associated with LLMs.
  • Ethics Statement: LLMs can perpetuate biases and have societal impacts, making responsible development and deployment important.The paper specifically mentions bias mitigation as part of responsible practice.

C Additional Results

Additional results reinforce CISC’s efficiency and accuracy advantages, clarify important design choices, and connect model confidence with human judgments of reasoning quality.

  • Human evaluation: The human evaluation sampled paths from Qwen2.5 3B, Gemma2 9B, and Mistral Large 123B, using bootstrap sampling across distinct questions.Evaluators examined logical patterns that reduced confidence in answer correctness; the MMLU setting required substantial domain knowledge.
  • Human evaluation: Three low-quality reasoning categories covered 49% of samples, and missing options or incomplete calculations were strongly associated with low model confidence.Exploring multiple plausible solutions without selecting one was not specifically associated with either confidence level.
  • Human evaluation: Alignment between human-identified low-quality indicators and low-confidence scores supports the use of model confidence to prioritize reasoning paths.The paper presents this alignment as additional evidence that LLMs can self-assess their reasoning.
  • Sample efficiency: Only 13 responses were sufficient on average to achieve 90% of the maximum effect observed with 30 responses.The authors report logarithmic improvement curves across models and datasets.
  • Aggregated results: All confidence methods outperform standard self-consistency, while P(True) achieves the best aggregated results and up to 53% computational-cost reduction.The result is reported across datasets and confidence-extraction methods.
  • Aggregated results: Micro-averaged results show statistically significant improvements over self-consistency for every confidence method, with P(True) significantly superior to the others.These results provide statistical verification of the reported efficiency and accuracy gains.
  • Normalization: Temperature-scaled softmax is optimal for all confidence methods, whereas softmax without temperature scaling should be avoided.This conclusion comes from the normalization ablation.
  • Ablations: CISC significantly outperforms max-confidence selection and tie-only use, while the max-confidence ablation degenerates performance.All ablations use P(True) confidence and aggregate results across models and datasets.
Loading 2502.06233v2…