PaperlayerView exact revision on arXiv

Source-linked AI summary

ConCISE: Confidence-guided Compression in Step-by-step Efficient Reasoning

Ziqing Qiao, Yongheng Deng, Jiali Zeng, Dong Wang, Lai Wei, Guanbo Wang, Fandong Meng, Jie Zhou, Ju Ren, Yaoxue Zhang

arXiv:2505.04881v2cs.LGcs.AIcs.CL

TL;DR

LRMs produce strong reasoning but often generate unnecessarily long chains, while existing compression methods can leave redundancy or disrupt coherence. ConCISE uses confidence-guided data construction with Confidence Injection and Early Stopping to suppress redundant reflection during generation. Across reasoning benchmarks, it improves the compression–performance trade-off, reducing average response length by approximately 50% under SimPO while maintaining high accuracy.

  • Problem

    LRMs often generate verbose reasoning chains, while existing sampling-based and post-hoc compression methods can retain redundancy or disrupt reasoning coherence.

  • Method

    ConCISE constructs concise reasoning data by using Confidence Injection to reduce underconfident reflection and Early Stopping to end reasoning after sufficient post-answer confidence.

  • Results

    Approximately 50%: fine-tuning on CONCISE-generated data reduces average response length under SimPO while maintaining high accuracy across multiple reasoning benchmarks.

  • Takeaways & Limitations

    Combining both mechanisms enables comprehensive redundancy elimination while preserving essential reasoning content and task performance.

  • Takeaways & Limitations

    CONCISE lacks a direct mechanism to model confidence before the first answer, relying instead on reflection steps as an indirect proxy.

Abstract

from arXiv · show

Large Reasoning Models (LRMs) perform strongly in complex reasoning tasks via Chain-of-Thought (CoT) prompting, but often suffer from verbose outputs, increasing computational overhead. Existing fine-tuning-based compression methods either operate post-hoc pruning, risking disruption to reasoning coherence, or rely on sampling-based selection, which fails to remove redundant content thoroughly. To address these limitations, this work begins by framing two key patterns of redundant reflection in LRMs--Confidence Deficit, wherein the model reflects on correct intermediate steps, and Termination Delay, where reflection continues after a verified, confident answer--through a confidence-guided perspective. Based on this, we introduce ConCISE (Confidence-guided Compression In Step-by-step Efficient Reasoning), a framework designed to generate concise reasoning chains, integrating Confidence Injection to boost reasoning confidence, and Early Stopping to terminate reasoning when confidence is sufficient. Extensive experiments demonstrate that compared to baseline methods, fine-tuning LRMs on ConCISE-generated data yields a better balance between compression and task performance, reducing length by up to approximately 50% under SimPO, while maintaining high task accuracy.

1 Introduction

Large Reasoning Models achieve strong complex-reasoning performance but often generate verbose chains that increase computational overhead. ConCISE addresses this by constructing concise training data through confidence-guided suppression of redundant reflections, improving the compression–performance trade-off.

  • LRMs often generate excessively verbose reasoning chains, increasing computational overhead and complicating deployment in resource-constrained settings.
  • Existing compression methods either select or prune reasoning after generation, potentially retaining unnecessary steps or disrupting coherence and performance.
  • Confidence Deficit and Termination Delay explain redundant reflection on correct intermediate steps and after repeatedly verified answers.
  • ConCISE combines Confidence Injection with Early Stopping to suppress redundant reflection during reasoning-data generation.
  • Approximately 50%: CONCISE reduces average response length under SimPO while maintaining high accuracy across multiple reasoning benchmarks.Experiments under both SFT and SimPO show a superior compression–task-performance trade-off compared with baselines.

2 Related Work

Prior LRM compression methods modify inputs, decoding, or model training, but can add overhead, depend on sensitive rewards, or harm coherence. ConCISE instead identifies and actively suppresses redundant reflections throughout generation while preserving reasoning coherence.

  • Input-based methods promote concise reasoning by modifying prompts or imposing token limits, but may add computational overhead and disrupt coherence.
  • Model-based methods include reinforcement learning with verbosity penalties, but their effectiveness can depend on reward formulation and incur substantial costs.
  • ConCISE is a model-based method that identifies and actively suppresses redundant reflection steps throughout reasoning while preserving coherence.It constructs training data that fine-tunes models to adopt concise reasoning without compromising performance.

3 Method

ConCISE models redundant reflection through confidence, targeting Confidence Deficit and Termination Delay with Confidence Injection and Early Stopping. It constructs concise reasoning chains by selectively increasing confidence, detecting sufficient post-answer confidence, and stopping only after answer verification.

  • 3.1 Confidence-guided Formulation: Confidence Deficit occurs when LRMs reflect on correct intermediate steps because internal confidence falls below a dynamic threshold.The framework represents each reasoning step with confidence c_i and compares it with threshold t_i; reflection occurs when c_i < t_i.
  • 3.2 Two Key Patterns of Redundancy: Termination Delay occurs when LRMs continue reflecting after producing a confident answer, partly because the post-answer confidence threshold increases.This motivates controlling generation after the answer rather than relying on confidence injection alone.
  • 3.3 The CONCISE Framework: CONCISE coordinates injection, confidence detection, verification, and stopping while discarding chains that ultimately produce an incorrect answer.The workflow generates reasoning step by step, applies injection to detected reflection steps, and appends a final-answer summary after successful stopping.
  • 3.3.1 Confidence Injection: Confidence Injection selectively appends a confidence phrase when a reflection step is detected, then regenerates the next step to suppress unnecessary reflection.The phrase is sampled from a curated pool and applied retroactively at critical low-confidence points rather than indiscriminately.
  • 3.3.1 Confidence Injection: The phrase pool contains 20 candidates selected for low reflection rates, while the best phrases still retain approximately 20% reflection.This preserves necessary verification while reducing redundant reflections.
  • 3.3.2 Early Stopping: Early Stopping estimates confidence from probabilities of confidence-indicative continuations and halts only after a prompted final answer is verified correct.The detected score averages confidence after the First Answer Step and subsequent reflection steps; experiments set the stopping threshold t_e = 0.5.

4 Experiments

Experiments evaluate CONCISE across multiple models, benchmarks, and fine-tuning settings, showing that it compresses reasoning while preserving task performance. Analyses attribute this balance to selectively reducing redundant reflection, and ablations show that Confidence Injection and Early Stopping are complementary.

  • Settings: Experiments evaluate four LRMs on GSM8K, Math-500, AIME24, and GPQA_diamond using accuracy, response-token length, and compression rate.AIME24 and GPQA_diamond use eight sampled evaluations, with mean values reported.
  • Overall results: CONCISE consistently balances compression and task performance across four LRMs under both SFT and SimPO.The method also generalizes from mathematical datasets to the out-of-domain GPQA_diamond benchmark.
  • Overall results: ~50% compression under SimPO is achieved with minimal impact on original-model performance, including on AIME24 and GPQA.OverThink compresses less, while Spirit reaches comparable compression but degrades performance, especially on AIME24.
  • Reasoning-chain analysis: CONCISE generates the fewest reasoning steps while maintaining a step length comparable to the original model.This indicates that its compression primarily reduces the number of steps rather than shortening each step.
  • Reasoning-chain analysis: CONCISE retains the fewest reflection steps while preserving non-reflection steps at counts similar to OverThink.The analysis reports that pruning reflection steps is central to compression, whereas reducing non-reflection steps can impair performance.
  • Reflection analysis: CONCISE controls reflection before and after the final answer more precisely than comparison methods, avoiding unnecessary pruning of essential reasoning.OverThink retains more pre-answer reflection, while Spirit and SpiritSFT show distinct weaknesses in pruning reflection.
  • Ablation study: Confidence Injection and Early Stopping each preserve reasoning accuracy but provide weaker compression than the full CONCISE framework.Confidence Injection targets earlier unnecessary reflections, whereas Early Stopping targets excessive post-answer reasoning; combining them addresses both patterns.
  • Training-free methods: Training-free methods provide unstable or partial compression and often reduce accuracy, whereas CONCISE-SIMPO and CONCISE-Decoding achieve stronger overall trade-offs.DEER introduces a 5–8% accuracy drop, while TALE performs inconsistently across datasets and models.

5 Conclusion

The paper frames redundant reflection through Confidence Deficit and Termination Delay, then introduces CONCISE to address both patterns. Its experiments show that CONCISE-generated training data compresses responses while maintaining strong task performance.

  • Conclusion: Confidence Deficit and Termination Delay are identified as two patterns responsible for redundant reflection in LRM reasoning.The confidence-guided perspective models reflection generation around these two patterns.
  • Conclusion: CONCISE combines Confidence Injection and Early Stopping to suppress redundant reflection during reasoning generation.The two mechanisms respectively address the identified redundancy patterns.
  • Conclusion: Fine-tuning on CONCISE-generated data significantly compresses responses while maintaining strong task performance.The conclusion presents this as the overall experimental outcome.

Limitations

CONCISE reduces redundant reflection but leaves unresolved questions about compressing non-reflection content, estimating pre-answer confidence, and extending validation to RLVR settings.

  • Room for Further Compression: Further compression remains possible for non-reflection steps and average step length.The authors propose integrating additional techniques while maintaining model performance.
  • Confidence Estimation During Reasoning: CONCISE lacks a direct mechanism to model confidence before the first answer.It instead uses reflection occurrence as an indirect proxy and suggests a lightweight confidence model for finer control.
  • Integration with RLVR methods: The study validates concise reasoning traces under SFT and SimPO, leaving integration with RLVR methods for future work.The paper specifically identifies incorporating confidence-guided signals into RLVR setups as a future direction.

A.2 Early Stopping

Early Stopping estimates confidence from model-generated confidence expressions and uses a threshold to determine when reflection should end, with threshold selection showing robust performance across several settings.

  • Confidence probing: Alternative probing prompts often produce high confidence before Final Answer Serialization, causing unnecessary computational overhead.The authors therefore select “So, I’m” as the fixed probing prompt.
  • Confidence probing: The confidence detector uses the probing prompt “So, I’m” and confidence-indicative continuations such as “confident,” “sure,” and their “pretty” variants.The confidence score sums the probabilities of these token sequences conditioned on the reasoning state and probing context.
  • Threshold design: The threshold is designed to allow no post-answer reflection for simple problems, one round for moderate problems, and multiple rounds for difficult problems.Threshold evaluation uses te ∈ {0.4, 0.5, 0.6, 0.7}.
  • Threshold behavior: The lightweight detector rarely signals high confidence before FAS, reducing the overhead of prematurely judging answer correctness.The observed confidence distributions generally exceed the threshold around the first reflection.
  • Threshold selection: Thresholds te = 0.5, 0.6, and 0.7 produce similar compression rates and accuracy under SFT, leading the authors to select te = 0.5.The threshold choice is therefore relatively robust within the evaluated range.
  • Reflection detection: Rule-based reflection detection identifies reflection onsets efficiently but typically misses subsequent steps in the same reflection sequence.This remains sufficient for Confidence Injection because preventing the first redundant reflection step can prevent the entire reflection.

A.3.2 LLM-as-a-Judge

The LLM-as-a-Judge procedure uses Qwen-Max to annotate answer and reflection structure, supporting both baseline implementation and reflection-based evaluation, but its reliability declines on very long chains.

  • Procedure: When complete reflection behavior must be identified, Qwen-Max receives the problem and reasoning chain and returns annotated FAS and reflection-step indices.The approach is used for the OverThink baseline, where the full first post-FAS reflection sequence must be isolated.
  • Reliability boundary: Qwen-Max performs reliably on moderate-length reasoning chains but degrades on chains exceeding 5000 tokens.Chains longer than 3000 tokens were manually annotated for the OverThink comparison.
  • Evaluation use: LLM-as-a-Judge annotations support RefNum, Non-RefNum, pre-FAS RefNum, and post-FAS RefNum metrics.These metrics are reported in the reflection analyses associated with Figure 5b and Figure 5c.

B.1 Datasets

The datasets are constructed and filtered to support diverse reasoning examples and dependable judge-based comparisons, including a common correctly reasoned training subset and a 340-problem evaluation subset.

  • Question set: The question set spans varied difficulty and reasoning-chain lengths.Difficulty is derived from MATH levels, while length reflects reasoning chains generated by DeepSeek-R1-Distill-Qwen-7B.
  • Training dataset: Approximately 1900 training samples remain after intersecting correctly reasoned outputs from CONCISE, OverThink, and Spirit.This filtering accounts for non-greedy generation and CONCISE’s removal of incorrect reasoning chains.
  • Verification dataset: A separate Verification Dataset contains 1000 MATH training reasoning chains for analyses requiring FAS and reflection annotations.It was created because LLM-as-a-Judge reliability decreases on long reasoning chains.
  • Evaluation dataset: The comparative evaluation uses a common subset of 340 MATH-500 problems whose chains remain shorter than 5000 tokens across all twelve configurations.The subset supports judge-based analysis across methods, model sizes, and training techniques.
  • Evaluation dataset: Table 6 reports accuracy and compression ratio for training-free baselines and CONCISE variants on GSM8K, Math-500, and AIME24.Lower compression ratio indicates shorter outputs.

B.2 Training Details

The study evaluates four open-source reasoning models and constructs training data with ConCISE, OverThink, and Spirit before applying SFT or SimPO.

  • Models: Four open-source models—DeepSeek-R1-Distill-Qwen-7B, DeepSeek-R1-Distill-Qwen-1.5B, Skywork-OR1-7B-Preview, and Qwen3-8B—are used for training and evaluation.The models are selected because they have strong Chain-of-Thought reasoning capabilities but exhibit reasoning redundancy.
  • Models: All models and datasets are used according to their original licenses and intended purposes.
  • Training: Training datasets are constructed from the question set using CONCISE, OverThink, and Spirit.ConCISE generation uses Temperature=0.6 and Top_p=0.95, while OverThink samples eight reasoning chains at Temperature=1.0 and Top_p=0.95.
  • Training: SFT directly trains on compressed reasoning traces paired with questions, whereas SimPO additionally requires reject samples.For SimPO, CONCISE uses the same reject-sample strategy as OverThink: the longest correct chain among eight samples.

B.3 Analysis on DeepSeek-1.5B

The analysis on DeepSeek-R1-Distill-Qwen-1.5B reproduces the main trends observed for the 7B model, using reflection detection and answer-derivation analysis.

  • Analysis on DeepSeek-1.5B: The DeepSeek-R1-Distill-Qwen-1.5B analysis shows trends and relationships among the three methods largely consistent with those for the 7B model.The paper states that the resulting conclusions are also consistent.
  • Analysis on DeepSeek-1.5B: The reflection-step analysis asks an LLM judge to identify whether each reasoning step belongs to a reflection process.The judge prompt also asks for the earliest step where the final answer is first derived.
  • Analysis on DeepSeek-1.5B: Figures 10–12 present the LLM-as-a-Judge prompt, an example input, and an example output for reflection-step detection.
Loading 2505.04881v2…