PaperlayerView exact revision on arXiv

Source-linked AI summary

Is Discrete Difficulty Sufficient? Leveraging Continuous Difficulty for Efficient Self-Consistency in LLMs

Sihyeong Yeom, Geon Park, Geunyeong Jeong, Taewoong Yoon, Jaewook Lee, Harksoo Kim

arXiv:2608.24590v1cs.CL

TL;DR

SC’s multiple reasoning paths improve complex reasoning but consume many tokens, while existing difficulty-based methods use coarse discrete categories. FSC predicts output entropy with a lightweight probe and dynamically allocates reasoning paths, achieving comparable accuracy with substantially lower token usage across benchmarks and models. The method is limited by its dependence on accessible hidden representations and validation on models no larger than 14B.

  • Problem

    Existing methods for efficient Self-Consistency often use discrete difficulty categories that do not capture fine-grained variation in reasoning requirements.

  • Method

    FSC predicts an input question’s output entropy from its last-token embedding using a lightweight linear probe, then allocates a sampling budget accordingly.

  • Results

    Across various models and benchmarks, FSC maintains accuracy comparable to SC while reducing token consumption by up to 76.7% compared to SC.

  • Takeaways & Limitations

    Output entropy can serve as a continuous difficulty signal for more fine-grained and efficient allocation of Self-Consistency reasoning resources.

  • Takeaways & Limitations

    FSC was evaluated only on models no larger than 14B and requires last-token hidden representations, limiting application to some proprietary models.

Abstract

from arXiv · show

Self-Consistency (SC) is a decoding strategy that samples diverse reasoning paths and selects the most consistent answer, demonstrating strong performance on complex reasoning problems. However, the excessive token consumption incurred by generating multiple reasoning paths has been identified as a major limitation of SC. To improve computational efficiency, several studies have proposed strategies that adjust the number of reasoning paths or allocate resources differentially according to problem difficulty. Nevertheless, most existing methods categorize difficulty into a few fixed levels, failing to fully capture the continuously varying nature of reasoning complexity. In this work, we propose Flexible Self-Consistency (FSC), which estimates problem difficulty as a continuous signal and dynamically adjusts the number of generated reasoning paths accordingly. FSC predicts the output entropy of an input question using a pre-trained probe and leverages it as an indicator of model uncertainty to flexibly control the sampling budget. Experimental results show that, across various models and benchmarks, FSC maintains accuracy comparable to SC while achieving token savings of up to 76%.

1 Introduction

Self-Consistency improves complex reasoning by sampling multiple paths, but fixed or coarse resource allocation can waste tokens or underexplore difficult questions. FSC uses predicted output entropy as a continuous difficulty signal to allocate reasoning paths flexibly, reducing token usage while preserving comparable accuracy.

  • Motivation: SC samples multiple reasoning paths and aggregates them to determine an answer, but generating more paths increases token usage.This creates a separate resource-allocation problem: how much reasoning should each question receive?
  • Prior Work: Existing efficiency methods include early stopping and difficulty-adaptive allocation, but commonly represent difficulty with discrete categories and predefined path counts.These approaches improve efficiency by stopping when responses stabilize or adjusting sampling according to estimated difficulty.
  • Research Gap: Coarse difficulty categories can overallocate computation to some questions and provide insufficient exploration for others because same-category problems may require different reasoning amounts.For example, assigning one path to easy problems and 40 paths to hard problems does not capture fine-grained differences.
  • FSC: FSC predicts each question’s output entropy with a lightweight linear probe and uses that continuous signal to determine its sampling budget.The method allocates less computation to easy problems and more computation to difficult ones.
  • Results: FSC substantially reduces token usage while maintaining accuracy comparable to SC across various benchmarks.The paper presents FSC as a framework for balancing accuracy and inference efficiency through adaptive reasoning-path allocation.

2 Motivation

The motivation experiments show that reasoning requirements and answer diversity increase with problem difficulty, supporting continuous difficulty estimation. FSC therefore uses output entropy to adapt the sampling budget to each question.

  • Experimental Setup: The study generates 40 reasoning chains per problem across MATH difficulty levels using four instruction-tuned language models.It measures the minimum chain count at which majority voting first matches the correct answer, assigning the maximum count to unresolved cases.
  • Reasoning Requirements: The average number of chains required to reach the correct answer increases with difficulty across all tested models, rising sharply at Level 5.The trend is consistent across models.
  • Continuous Difficulty: Because reasoning requirements vary gradually, resources should be allocated in proportion to problem difficulty rather than uniformly across inputs.Accurately identifying difficulty is therefore important for adaptive reasoning allocation.
  • Answer Diversity: Answer diversity also increases with difficulty, with incorrect cases showing particularly higher diversity.This pattern suggests greater uncertainty for more difficult problems.
  • Continuous Difficulty: Discrete difficulty categorization cannot precisely reflect the amount of reasoning required when model-perceived difficulty differs among problems at the same level.The study consequently investigates whether this perceived difficulty can be quantified continuously.
  • FSC: FSC uses output entropy to adaptively adjust the sampling budget, treating response-distribution uncertainty as a continuous difficulty signal.The observed relationship between answer diversity and difficulty motivates this design.

3 Methodology

FSC models problem difficulty continuously through predicted output entropy, then converts that signal into an adaptive reasoning budget. Its two-stage pipeline trains an entropy probe and uses the prediction to determine how many chains to sample for majority voting.

  • Probe training: FSC trains a lightweight probe to predict each question’s output entropy from the LLM’s last-token embedding.The probe is implemented as linear regression to preserve a continuous, fine-grained difficulty signal.
  • Probe training: For each training question, FSC generates reasoning trajectories, extracts unique answers, and computes answer-distribution entropy as the supervision label.The resulting question–entropy pairs form the synthetic dataset used for probe learning.
  • Difficulty representation: Unlike discrete difficulty labels, entropy-based labeling provides continuous supervision that captures finer-grained variation in problem difficulty.This signal is intended to distinguish problems requiring different amounts of reasoning within the same coarse category.
  • Adaptive sampling: FSC clips predicted entropy for stability because the theoretical answer-distribution range is [0, log2 N], whereas regression predictions may fall outside it.The answer distribution is constructed from at most N reasoning paths.
  • Adaptive sampling: Predicted entropy is normalized into a relative difficulty score and mapped to an adaptive sampling budget for each input.The budget ranges from one reasoning path for low-entropy questions to N paths for high-entropy questions.

4 Experiments

FSC is evaluated across multiple instruction-tuned models and reasoning benchmarks against SC and other adaptive or early-stopping baselines. It generally achieves the lowest token usage while maintaining comparable accuracy, including token reductions of up to 76.7% versus SC.

  • Setup: FSC is evaluated on MATH500, AMC23, AIME2024, AIME2025, and GPQA-Diamond using Qwen2.5 and Gemma-3 models in zero-shot settings.The probe is trained on 7,500 MATH training instances.
  • Setup: The comparison includes SC majority voting, AC and ESC early-stopping methods, and DSC’s discrete difficulty-aware allocation.Performance is assessed using accuracy and total token usage.
  • Results: FSC consistently reduces token consumption across all model–dataset combinations, achieving up to a 76.7% reduction compared to SC.It achieves the lowest token usage in most settings while maintaining accuracy comparable to existing baselines.
  • Results: AC increases token usage by +15.8% on AIME2025 with Qwen2.5-3B and by +6.8% with Qwen2.5-7B relative to SC.AC and ESC can consume more tokens than SC in some settings.
  • Results: Compared with DSC, FSC delivers higher token efficiency while maintaining comparable accuracy in most settings.The reported difference is attributed to continuous output-entropy signals enabling finer-grained resource allocation than discrete difficulty estimation.
  • Results: Overall, FSC balances accuracy and efficiency through more efficient allocation of inference paths across problem difficulty.The paper characterizes FSC as a more stable and efficient test-time scaling strategy than existing approaches.

5 Analysis

The probe’s predicted entropy increases with problem difficulty, while FSC allocates reasoning paths more continuously than DSC and maintains efficient performance across domains.

  • 5.1 Can the Probe Reflect Problem Difficulty via Entropy?: Predicted entropy increases with problem difficulty across MATH500 and other mathematical benchmarks, including AMC23, AIME2024, and AIME2025.The pattern also appears across other models, suggesting the probe captures a generalized continuous difficulty signal.
  • 5.2 Comparison of Inference Path Assignment Distributions by Difficulty Level: FSC distributes reasoning paths continuously across a broader range, whereas DSC concentrates allocation within selected ranges.Both methods allocate more paths as difficulty increases, but FSC avoids bias toward specific path counts.
  • 5.2 Comparison of Inference Path Assignment Distributions by Difficulty Level: FSC’s continuous entropy-based allocation more precisely reflects each question’s intrinsic difficulty and supports flexible resource allocation.DSC estimates difficulty discretely, while FSC allocates resources in proportion to predicted output entropy.
  • 5.3 Generalization under Distribution Shift: FSC achieves the highest token efficiency in most MMLU-Pro domains and remains consistent across diverse academic distributions.The trend persists across other models, indicating generalization beyond a specific model or parameter scale.

6 Related Works

Related work improves Self-Consistency efficiency through early stopping and difficulty-adaptive allocation, but commonly represents difficulty using coarse categories.

  • Efficient Self-Consistency Variants: Early efficiency methods stop sampling when response consistency or stability is sufficient.Later methods estimate question difficulty and dynamically adjust the number of reasoning paths.
  • Difficulty Estimation: Difficulty-estimation research uses either external judgments or internal model signals such as value functions and linear probes.Internal methods aim to estimate difficulty as perceived by the model without relying solely on explicit judgments.
  • Difficulty Estimation: Existing methods often reduce difficulty to discrete or coarse levels, limiting their ability to reflect fine-grained model-perceived difficulty.FSC addresses this limitation by representing difficulty continuously for reasoning-resource allocation.

7 Conclusion

FSC uses predicted output entropy as a continuous difficulty signal to allocate reasoning paths adaptively, improving efficiency while preserving comparable accuracy.

  • 7 Conclusion: FSC predicts output entropy with a lightweight linear probe and dynamically allocates the sampling budget according to the predicted value.The framework replaces conventional discrete difficulty categorization with a continuous signal.
  • 7 Conclusion: Across benchmarks and models, FSC substantially reduces token usage while maintaining accuracy comparable to existing baselines.The method flexibly adjusts computational resources according to problem difficulty.

Limitations

The paper identifies model-scale, model-access, and probe-training constraints that limit FSC’s validation and applicability.

  • Limitations: FSC was not evaluated on models larger than 14B because of resource constraints.Further validation is needed because predicted entropy may vary with model size and family.
  • Limitations: FSC requires the input question’s last-token hidden representation, complicating application to proprietary models such as GPT.This dependency limits deployment across diverse model environments.
  • Limitations: The probe was trained only on mathematical datasets, so more diverse training data may improve generalization and robustness across tasks and domains.The authors identify broader probe training as a direction for future work.
  • Experimental Setup: The experiments use shared prompts for all methods, allowing comparisons to focus on inference-strategy effects.The mathematical and multiple-choice prompts are presented separately in Figures 7 and 8.

B Motivation Analysis on Entropy-Difficulty Relationship

Across diverse models, output-answer diversity increases with problem difficulty and is higher for incorrect predictions, supporting entropy as a continuous difficulty signal.

  • The study tests whether the entropy–difficulty relationship generalizes beyond Qwen2.5-7B to Qwen2.5-3B, Qwen2.5-14B, and Gemma-3-4B.For each model, the authors generate 40 reasoning chains for 7,500 MATH training problems and measure unique answers by difficulty level.
  • Across all evaluated models, incorrect cases exhibit higher answer diversity than correct cases.
  • The number of unique answers gradually increases as problem difficulty increases across models.The passage links this consistent pattern to the diversity of the output distribution and problem difficulty.

C Detailed Experimental Setup

The experiments cover mathematical, STEM, and general-domain reasoning benchmarks, with probe training based on diverse MATH reasoning paths and fixed generation settings for evaluation.

  • Datasets: The evaluation uses benchmarks spanning mathematical reasoning, STEM question answering, and general-domain reasoning across diverse academic fields.Table 2 summarizes each dataset’s domain, answer format, evaluation-sample count, and license information.
  • Implementation Details: The probe-training dataset contains 40 reasoning paths for each of 7,500 samples from the MATH training split.Temperature and top-p are both set to 1.0 to obtain diverse reasoning paths.

D Case Study

The case study contrasts DSC and FSC on easy and difficult problems, while supplementary figures examine diversity, entropy, path allocation, and token efficiency across models and settings.

  • Easy Problems: On an easy MATH500 Level 1 problem, SC paths converge to one answer, but DSC allocates four paths unnecessarily.FSC uses the probe to identify the problem as easy and operates with one inference path.
  • Supplementary Analyses: Figure 9 reports increasing answer diversity with problem difficulty, particularly for incorrect predictions, across the evaluated models.This supports using output entropy as a continuous difficulty signal.
  • Difficult Problems: On challenging AIME2025 problems, FSC continuously estimates model-perceived difficulty and allocates fewer inference paths than DSC.The comparison describes FSC as maximizing efficiency on difficult inputs as well as easy ones.
  • Supplementary Analyses: Figures 10–12 examine predicted entropy distributions, reasoning-path counts, and token efficiency across difficulty levels and model scales.Figure 12 specifically compares FSC with baseline methods on Qwen2.5-3B, Qwen2.5-14B, and Gemma-3-4B using MMLU-Pro.
  • Case Study: Figure 13 compares DSC and FSC for dynamically allocating inference paths on mathematical problems.
Loading 2608.24590v1…