PaperlayerView exact revision on arXiv

Source-linked AI summary

Dynamic Early Exit in Reasoning Models

Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Qiaowei Li, Minghui Chen, Zheng Lin, Weiping Wang

arXiv:2504.15895v3cs.CLcs.AI

TL;DR

Long chain-of-thought reasoning can waste computation and sometimes hurt accuracy through redundant thinking. DEER dynamically tests trial answers at reasoning transitions and exits when confidence is high, while DEER-Pro adds parallel calibrated confidence. Across 10 benchmarks and 11 models, the approach reduces CoT length while improving accuracy.

  • Problem

    Overlong and redundant chain-of-thought reasoning increases latency and computational cost and can degrade accuracy.

  • Method

    DEER monitors reasoning transitions, induces trial answers, evaluates their confidence, and stops further reasoning when confidence is sufficiently high without additional training.

  • Results

    Across 10 reasoning benchmarks and 11 models, CoT length decreases by 19.1% to 80.1% while accuracy improves by 0.3% to 5.0%.

  • Takeaways & Limitations

    The method offers a plug-and-play approach for improving reasoning-model efficiency and accuracy across varied tasks and model sizes.

Abstract

from arXiv · show

Recent advances in large reasoning language models (LRLMs) rely on test-time scaling, which extends long chain-of-thought (CoT) generation to solve complex tasks. However, overthinking in long CoT not only slows down the efficiency of problem solving, but also risks accuracy loss due to the extremely detailed or redundant reasoning steps. We propose a simple yet effective method that allows LLMs to self-truncate CoT sequences by early exit during generation. Instead of relying on fixed heuristics, the proposed method monitors model behavior at potential reasoning transition points and dynamically terminates the next reasoning chain's generation when the model exhibits high confidence in a trial answer. Our method requires no additional training and can be seamlessly integrated into existing o1-like reasoning LLMs. Experiments on 10 reasoning benchmarks (e.g., GSM8K, MATH-500, AMC, GPQA, AIME and LiveCodeBench) show that the proposed method is consistently effective on 11 cutting-edge reasoning LLMs of varying series and sizes, reducing the length of CoT sequences by an average of 19.1% to 80.1% while improving accuracy by 0.3% to 5.0%.

DYNAMIC EARLY EXIT IN REASONING MODELS

Figure 1 reports correctness statistics for early exits at different reasoning steps.

  • Correctness statistics are measured across various early-exit reasoning steps.

1 INTRODUCTION

Large reasoning models improve complex-task solving through long chain-of-thought generation, but overlong reasoning can increase latency, waste computation, and reduce accuracy. The paper proposes DEER, a training-free dynamic early-exit method that uses reasoning transitions and trial-answer confidence to stop thinking earlier, with broad benchmark gains.

  • Longer chain-of-thought generation enables large reasoning models to produce diverse reasoning paths for complex tasks.
  • Overlong reasoning increases computational overload and latency while redundant steps can derail correct reasoning and degrade accuracy.
  • Approximately 75% of samples contain a critical point where early exit still yields a correct answer, and 36.7% need fewer than half the original reasoning paths.
  • DEER monitors reasoning transitions, induces a trial answer, evaluates its confidence, and exits when confidence is sufficiently high; otherwise, reasoning continues.
  • 19.1% to 80.1% average CoT-length reduction accompanies 0.3% to 5.0% accuracy improvement across 11 models and 10 reasoning benchmarks.

2 MOTIVATIONS AND OBSERVATIONS

The observations identify sufficient intermediate reasoning and show why fixed early-exit positions are inadequate. Correct answers often remain available after substantially shorter reasoning, but the best exit point varies with task and problem difficulty.

  • Approximately 75% of samples contain pearl reasoning, where early exit produces a correct answer.
  • 60.8% of correctly answered MATH-500 samples and 35.1% of GPQA samples retain accuracy after only 20% of reasoning steps.
  • Exiting at 40% of reasoning steps yields the highest correction rate for MATH, while 50% is optimal for GPQA.
  • The optimal early-exit point varies by problem and relates closely to inherent problem difficulty, making fixed heuristics suboptimal.

3 METHOD

DEER dynamically identifies reasoning transitions, elicits trial answers, and uses confidence to decide whether the model has enough information to conclude. DEER-Pro improves robustness through parallel answer inductions and calibrated confidence, while branch-parallel decoding addresses inference overhead.

  • Reasoning structure: Large reasoning models separate slow thinking from conclusions and use action transitions to organize reasoning steps.
  • DEER pipeline: DEER monitors reasoning transitions, induces trial answers, evaluates confidence, and stops generation when the trial answer is sufficiently confident.
  • Reasoning transition monitor: Linguistic markers or entropy-based monitoring identify potential early-exit points during long chain-of-thought generation.
  • Answer inducer: At a potential exit, the answer inducer generates an intermediate answer from the prompt and thoughts produced so far.
  • Confidence evaluator: The confidence evaluator aggregates token-level confidence for multi-token trial answers, using a geometric-mean calculation that is more sensitive to low probabilities.
  • Exit decision: If confidence exceeds the empirical threshold λ, DEER concludes; otherwise, it returns to the transition point and generates further thoughts.
  • DEER-Pro: DEER-Pro uses varied prompts, mean absolute deviation, and a conservative penalty to reduce errors from overestimated confidence caused by prompt noise.
  • Acceleration: Branch-parallel decoding mitigates the latency overhead introduced by trial-answer induction and confidence evaluation.

4 EXPERIMENTS

Across reasoning models, tasks, sizes, token budgets, and efficiency settings, DEER consistently reduces reasoning length while preserving or improving accuracy. Its robustness extends to difficult models and tasks, with early-exit quality remaining high and latency improving further through branch-parallel decoding.

  • Overall Performance: DEER improves accuracy by 0.9 to 4.8 points while reducing sequence length by 19.1% to 42.9% versus vanilla models across 10 datasets and 11 models.DEER-Pro yields higher accuracy with a 2.8% to 6.2% generation-length increase over DEER.
  • Comparison with Efficient Reasoning SoTAs: DEER consistently outperforms efficient-reasoning baselines across tasks and base models, avoiding their accuracy-efficiency trade-offs and generalization failures.Unlike methods with dramatic compression but degraded reasoning, DEER preserves quality while reducing length.
  • Performance on Programming Tasks: 19.9% average compression on programming tasks is smaller than the 61.5% average for mathematics and science tasks.The paper attributes this difference to verbose code segments containing substantial redundant tokens at each reasoning step.
  • Performance Trends across Model Sizes and Reasoning Difficulty: DEER enhances accuracy while reducing token consumption across model sizes, with smaller models showing more severe overthinking and longer reasoning sequences.The evaluation covers MATH-500 and AIME 2024 across varying DeepSeek-R1-Distill-Qwen model sizes.
  • Performance Trends across Token Budgets: DEER dynamically adjusts token budgets across scenarios, and its performance is robust to λ values from 0.9 to 0.97.Token-budget plots use generated CoT length on the x-axis and accuracy on the y-axis; top-left curves indicate a better accuracy-efficiency balance.
  • Efficiency Improvement: 27.9% to 40.1% latency reduction from DEER increases to 36.3% to 58.6% with branch-parallel decoding.The branch-parallel variant further reduces latency for trial-answer induction and confidence evaluation; latency speedup has a superlinear relationship with length savings.

5 RELATED WORK

Related work is organized into post-training methods that shorten reasoning through supervised fine-tuning or length-based rewards.

  • Post-training methods use variable-length chain-of-thought data with supervised fine-tuning or incorporate explicit length rewards.

6 CONCLUSION

The paper proposes DEER, a training-free dynamic early-exit algorithm that stops reasoning when the model has thought enough. Across varied models and ten benchmarks, it improves performance while using fewer tokens.

  • DEER is a training-free dynamic early-exit algorithm that withdraws from subsequent thinking when the reasoning amount is sufficient.
  • DEER was evaluated across reasoning models of varying sizes on ten classical reasoning benchmarks.
  • DEER demonstrates superior performance with fewer tokens, addressing the trade-off between reasoning efficiency and accuracy.

7 ETHICS STATEMENT

The paper states that its research followed the ICLR Code of Ethics and that the work was conducted with integrity, honesty, and respect for ethical principles.

  • The research adhered to the ICLR Code of Ethics throughout all stages of the work.
  • The authors report that findings were presented accurately and honestly, without fabricated, falsified, or misrepresented data or results.
  • The paper states that methods and experimental procedures were described transparently to support reproducibility.

8 REPRODUCIBILITY STATEMENT

The paper reports experimental details, implementation information, and theoretical assumptions concerning confidence-based early-exit decisions. It also describes analyses of decision errors under confidence noise.

  • The reproducibility materials include detailed experimental setups, hyperparameters, implementation details, code, and supplementary materials where possible.
  • Experiments used AIME2024 for exploratory analysis and MATH-500 and GPQA-Diamond for quantitative analysis with DeepSeek-R1-Distill-Qwen-14B.
  • The early-exit decision compares observed confidence C_i = µ + ε_i with threshold λ, executing early exit when confidence exceeds the threshold.
  • The analysis compares decision error rates for calibrated, single-observation, and averaged confidence under risk scenarios where true confidence µ < λ.
  • For averaged confidence, the noise term is modeled as Gaussian with variance σ^2/N, reducing error relative to a single observation.

B.1.3 PROBABILITY OF ERROR FOR MAD-CALIBRATED CONFIDENCE (DEER-PRO):

The MAD-calibrated confidence strategy analyzes false-positive early exits under Gaussian prompt noise and uses self-normalization to reduce sensitivity to the model’s noise level.

  • Calibration setup: The calibrated confidence is evaluated through false-positive probability under multiple independent answer-inducing prompts and a fluctuation penalty based on CMAD.The analysis considers approximate and exact CMAD computation scenarios.
  • Adaptive threshold: The MAD penalty makes the effective threshold scale with noise, while reducing to standard thresholding when noise is minimal.When noise is minimal, λeffective approaches λ and the calibrated method maintains high efficiency.
  • Noise robustness: The resulting SNR formulation is independent of the model noise standard deviation σ, leaving false-positive behavior dependent on prompt count N and threshold α.Larger N and α lead to lower error rates.
  • False-positive bound: The MAD-calibrated strategy has false-positive probability at most ρ times that of simple averaging, with ρ = O(exp(−Θ(N))).The analysis states that this is an exponentially decaying improvement in the number of prompts.
  • Noise-pattern interpretation: MAD filters probable outlier-driven noise patterns, leaving only exponentially rare coherent patterns capable of triggering false positives.Pattern A keeps MAD small enough to satisfy the early-exit condition, whereas outlier patterns do not.
  • Computational implication: DEER reduces the quadratic computation term from O(L^2) to O(α^2L^2) while adding only O(k·m·αL) overhead.Because k and m are much smaller than L for long CoT, the quadratic savings dominate.

I INVESTIGATION OF REASONING TRANSITION MONITORS

The study compares linguistic-marker and entropy-based reasoning-transition monitors, finding that linguistic markers identify positions associated with uncertainty and state transitions.

  • Entropy-based evidence: Linguistic markers have significantly higher token entropy than ordinary tokens across datasets and models.This indicates that marker-based monitoring targets positions with multiple candidate actions.
  • Representation-based evidence: Lower hidden-state cosine similarity around linguistic markers indicates reasoning transitions rather than continuous reasoning.High similarity denotes coherent continuation, whereas low similarity signals a transition.
  • Monitor choice: The experiments support the view that LRMs express uncertainty through language, making external linguistic markers observable manifestations of internal state transitions.The paper recommends linguistic markers for English LRMs and reports an alternative internal-state strategy for non-English settings.
  • Comparative results: Figures compare DEER and Vanilla CoT on Qwen3 models of varying sizes and provide detailed correct-versus-incorrect answer outcomes.The figures cover MATH-500 and AIME 2024, with √ indicating correctness and × indicating incorrectness.

J INVESTIGATION INTO THE REASONS BEHIND DEER’S THRESHOLD ROBUSTNESS

DEER’s threshold robustness is linked to polarized trial-answer confidence: scores usually fall near high- or low-confidence extremes, with few intermediate cases.

  • Confidence distribution: Trial-answer confidence has a U-shaped distribution with remarkably little probability mass between 0.9 and 0.97.The intermediate interval is characterized as an error-prone gray zone.
  • Problem difficulty: On GSM8K, high-confidence scores exceed low-confidence scores, whereas on AIME24, low-confidence scores exceed high-confidence scores.The contrast tracks simpler versus more challenging problems.
  • Evaluation scope: Tables report DEER comparisons using accuracy, average length or tokens, and compression ratio across models, datasets, and threshold settings.The supplied table captions define the reported metrics and comparison scope.

K MORE EXPERIMENTAL RESULTS

Additional experiments examine DEER across model sizes, decoding choices, confidence aggregation, and benchmark settings, generally supporting its efficiency and accuracy benefits while exposing model-dependent limits.

  • Model-size trends: DEER’s Qwen3 results across model sizes and reasoning difficulty are consistent with the main experimental findings.The comparison is presented in Figure 10 alongside Vanilla CoT.
  • State-of-the-art models: 10.4%–35.7% sequence-length reductions are reported on two state-of-the-art models while maintaining competitive accuracy.DeepSeek-R1 makes only one additional error on each dataset in the cited comparison.
  • Broader evaluation: DEER remains effective across seven benchmarks and eleven reasoning models, with supplementary comparisons against Vanilla CoT showing consistent conclusions.The experiments extend the main evaluation to additional models and benchmarks.
  • Model-dependent limitation: Llama-3.1-Nemotron-Nano-8B-v1 has a 55% early-stopping rate versus 80% for Qwen3-8B and 85% for R1-Distill-Qwen-7B, limiting its efficiency improvement.The paper nevertheless reports mitigation of overthinking through early stopping.
  • Configuration robustness: The appendix also reports robustness under officially recommended Qwen3 decoding settings and compares alternative reasoning-transition monitors.The monitor comparison includes Wait, Alternatively, and entropy-based transitions.
  • Confidence aggregation: Using arithmetic rather than geometric mean confidence significantly decreases accuracy while marginally improving compression, because low-valued tokens are diluted.The paper recommends geometric mean confidence estimation.
  • Statistical validation: Repeated runs with 95% confidence intervals confirm that the reported accuracy improvements are statistically significant.Four runs were used for GSM8K, MATH, and GPQA, and eight for AMC23 and AIME24.
  • Qualitative behavior: Examples show DEER following Vanilla CoT until a high-confidence trial answer permits early exit, preserving a correct answer.At reasoning switch points, trial-answer confidence changes consistently with the reliability of current reasoning chunks.
Loading 2504.15895v3…