PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Reasoning with Balanced Thinking

Yulin Li, Tengyao Tu, Li Ding, Junjie Wang, Huiling Zhen, Yixin Chen, Yong Li, Zhuotao Tian

arXiv:2603.12372v3cs.AIcs.CLcs.LG

TL;DR

LRMs can overthink by spending redundant steps or underthink by insufficiently exploring valid reasoning paths, while existing mitigations may trade one failure mode for the other. REBALANCE uses confidence-guided dynamic steering to balance reasoning, and experiments show improved efficiency and accuracy across models and tasks.

  • Problem

    LRMs suffer from overthinking and underthinking, while existing mitigation methods can suppress both redundant and valuable reasoning, limiting efficient deployment.

  • Method

    REBALANCE is a training-free framework that uses confidence as a continuous signal to dynamically adjust internal-state steering strength and direction during inference.

  • Results

    REBALANCE improves inference efficiency and accuracy across different models and tasks, reducing reasoning redundancy without compromising performance.

  • Takeaways & Limitations

    REBALANCE offers a plug-and-play strategy for efficient reasoning with balanced thinking in LRMs.

Abstract

from arXiv · show

Large Reasoning Models (LRMs) have shown remarkable reasoning capabilities, yet they often suffer from overthinking, expending redundant computational steps on simple problems, or underthinking, failing to explore sufficient reasoning paths despite inherent capabilities. These issues lead to inefficiencies and potential inaccuracies, limiting practical deployment in resource-constrained settings. Existing methods to mitigate overthinking, such as suppressing reflective keywords or adjusting reasoning length, may inadvertently induce underthinking, compromising accuracy. Therefore, we propose ReBalance, a training-free framework that achieves efficient reasoning with balanced thinking. ReBalance leverages confidence as a continuous indicator of reasoning dynamics, identifying overthinking through high confidence variance and underthinking via consistent overconfidence. By aggregating hidden states from a small-scale dataset into reasoning mode prototypes, we compute a steering vector to guide LRMs' reasoning trajectories. A dynamic control function modulates this vector's strength and direction based on real-time confidence, pruning redundancy during overthinking, and promoting exploration during underthinking. Extensive experiments conducted on four models ranging from 0.5B to 32B, and across nine benchmarks in math reasoning, general question answering, and coding tasks demonstrate that ReBalance effectively reduces output redundancy while improving accuracy, offering a general, training-free, and plug-and-play strategy for efficient and robust LRM deployment. Project page and code are available at https://rebalance-ai.github.io .

1 INTRODUCTION

Large Reasoning Models can waste computation through overthinking, while existing mitigation methods may cause underthinking. ReBalance uses confidence-guided dynamic steering to balance reasoning depth, reducing redundancy while preserving or improving accuracy across models and tasks.

  • Motivation: Overthinking allocates redundant reasoning steps to simple problems, increasing computational costs with marginal gains and potentially introducing hallucinations.These inefficiencies limit LRM deployment in resource-constrained environments.
  • Motivation: Existing methods that shorten reasoning chains or suppress reflective keywords can restrict valuable reasoning and induce underthinking.They may affect redundant and necessary reasoning indiscriminately, or penalize lengthy reasoning even when it is useful.
  • Results: ReBalance outperforms a previous state-of-the-art method across mathematical reasoning datasets and model scales while reducing reasoning length.The comparison spans model scales from 0.5B to 32B.
  • ReBalance: ReBalance aggregates hidden states from a small seen dataset into reasoning-mode prototypes and computes a steering vector that dynamically guides LRM reasoning.A confidence-dependent control function adjusts the vector’s strength and direction during inference, pruning redundancy or promoting exploration.
  • Results: Experiments across four models from 0.5B to 32B and nine benchmarks show reduced output length alongside improved accuracy.The evaluation covers math reasoning, general question answering, and coding tasks.
  • Key observation: Confidence provides a continuous signal for characterizing both overthinking and underthinking, enabling fine-grained behavioral control.This motivates replacing rigid binary path selection with a continuous reasoning-state indicator.

2 BACKGROUND AND MOTIVATION

The paper analyzes reasoning dynamics through stepwise confidence and local confidence variance. High variance is associated with overthinking, whereas consistently high confidence can signal underthinking; these statistics motivate balanced dynamic control.

  • Confidence measures: Stepwise confidence measures how consistently the model follows one reasoning path, while confidence variance measures switching frequency across reasoning steps.Together, they characterize the dynamics of the reasoning process.
  • Stepwise confidence: Tokenwise maximum predicted probabilities are aggregated across tokens in each reasoning step to obtain stepwise confidence.The step confidence is defined using the geometric average of tokenwise maxima.
  • Confidence variance: Confidence variance is computed over a sliding window of recent reasoning steps to capture short-term fluctuations.The window focuses on local variability because long-term history is considered less relevant.
  • Reasoning modes: High confidence variance indicates frequent reasoning-path switching and may produce redundant steps and delayed convergence, reflecting overthinking.This interpretation does not depend on the current absolute confidence level.
  • Reasoning modes: Consistently high confidence with low variance implies premature commitment to potentially incorrect paths, reflecting underthinking.The model may fail to explore sufficiently before committing.
  • Trade-off: Existing mitigation methods reduce inference length but can introduce underthinking, whereas ReBalance maintains a reasoning-length distribution closer to the original model.The comparison examines correctly and incorrectly answered samples before and after mitigation.

3 METHOD

ReBalance models overthinking and underthinking from confidence dynamics, extracts hidden-state prototypes, and applies a confidence-controlled steering vector during decoding. Its dynamic control adjusts steering direction and strength to reduce redundancy or encourage exploration while preserving efficient reasoning.

  • Overview: ReBalance is a training-free framework that dynamically balances overthinking and underthinking during inference.It is designed to improve efficiency without compromising accuracy.
  • Explicit Modeling: Overthinking is identified through high reasoning variance and low confidence, whereas underthinking involves low variance and persistently high confidence.Normal instances outside these sets are excluded from further analysis.
  • Steering Vector Extraction: The method extracts first-token hidden states from a selected deep layer in one offline pass, then averages tagged states into overthinking and underthinking prototypes.Deeper layers are reported to provide stronger mode discrimination and cross-dataset generalization.
  • Steering Vector Extraction: The steering vector represents the direction from the underthinking prototype to the overthinking prototype and is injected into each step’s initial token representation.Its signed weight can stimulate alternative paths for underthinking or encourage commitment to mitigate overthinking.
  • Dynamic Control: The continuous control function computes the steering weight from current confidence and variance, adaptively changing its strength and direction during inference.The procedure injects the controlled vector at each step without extra forward passes beyond standard decoding.
  • Dynamic Control: The soft saturation component makes steering grow gradually near reasoning boundaries, avoiding abrupt changes and supporting numerical stability.The variance-aware amplitude adapts the control across models and thinking states.

SCIENCE COMMONSENSE PROGRAMMING

The supplied passages identify tables for evaluating ReBalance on non-math tasks and for ablations across math difficulty levels. They specify Pass@1 and token-count outcomes, but provide no table entries or observed comparisons.

  • SCIENCE COMMONSENSE PROGRAMMING: Table 2 evaluates generalization on non-math tasks using Pass@1 and generated-token counts.The caption identifies the metrics and indicates colored changes relative to comparison methods.
  • SCIENCE COMMONSENSE PROGRAMMING: Table 3 reports ablations on R1-1.5B across Math500, GSM8K, and Olympiad at medium, easy, and hard difficulty levels.It compares Pass@1 accuracy and generated-token numbers with directional changes relative to original ReBalance.

4 EXPERIMENT

ReBalance improves reasoning efficiency and accuracy across mathematical, scientific, commonsense, and coding tasks, while remaining compatible with varied models, layers, datasets, and hardware. Ablations show that dynamic control is important for balancing token savings against accuracy.

  • ReBalance outperforms baselines on six math benchmarks, improving Pass@1 by up to 10.0 points while reducing generated tokens by up to 35.4%.
  • ReBalance’s intrinsic model signals generally outperform external early-exit criteria, which may not faithfully reflect the target model’s internal reasoning state.
  • 29.9% reasoning-length reduction is maintained across scientific reasoning, programming, and commonsense QA without domain-specific tuning, while preserving Pass@1.
  • On challenging cross-domain tasks, confidence signals improve effective reasoning and reduce redundant computation, while behavior on saturated StrategyQA remains conservative.
  • Static steering weights expose a trade-off: positive αs improves accuracy but increases token use, whereas negative αs reduces tokens at the expense of accuracy.
  • Steering at tested layers reduces token count without harming accuracy, with the strongest trade-off in mid-to-late layers where confidence separability is highest.
  • Vectors from harder datasets favor accuracy, whereas vectors from easier datasets favor token savings through different control surfaces.
  • Larger control windows increase token usage but can improve accuracy on Olympiad, while a small window of |Ws| = 2 is more responsive to local patterns.

5 CONCLUSION

The paper argues that suppressing overthinking can cause underthinking and presents ReBalance as a confidence-aware, training-free approach to balance both behaviors. Its analyses link confidence dynamics to reasoning length and persistence, supporting adaptive control through hidden-state representations.

  • Existing overthinking mitigations can induce underthinking by removing reasoning needed for difficult instances, motivating selective rather than uniform shortening.
  • ReBalance adaptively shortens reasoning when appropriate while preserving longer explorations for challenging instances, mitigating overthinking without inducing underthinking on MATH.
  • Across four models, word count is negatively associated with minimum step-level confidence and positively associated with confidence variance, with all reported correlations significant at p < 0.001.
  • Confidence states show first-order Markov persistence, making a minimal window of size two sufficient and often preferable for inferring the current reasoning state.
  • Confidence distributions differ across model families, motivating behavior-aware control-function fitting instead of one manually tuned hyperparameter set.
  • Hidden-state representations contain separable high- and low-confidence regions, especially in mid-to-late layers, supporting confidence-aware steering.

A.6 CONFIDENCE AS EVIDENCE UNDER VOCABULARY COVERAGE GAPS

Vocabulary-based interventions often suppress lexical correlates of low confidence, but confidence-based analysis covers additional low-confidence modes beyond predefined vocabularies. Probing and distribution analyses support confidence as a broader behavioral signal.

  • Confidence evidence: Mid-to-late layers provide the strongest confidence decodability, motivating their use for steering.The supporting analyses include t-SNE projections colored by confidence across selected layers and models.
  • Confidence evidence: Linear probes estimate step-level confidence from PCA-projected, layer-wise hidden states of a frozen language model.The procedure evaluates test-set R2 scores across layers.
  • Vocabulary coverage: Vocabulary-driven methods approximate confidence-based control by targeting frequent lexical markers associated with low-confidence reasoning.NOWAIT suppresses suchive markers, while SEAL steers hidden states away from representations producing low-confidence sentences.
  • Vocabulary coverage: Predefined vocabularies miss a substantial long tail of low-confidence cues that confidence-based approaches can identify regardless of lexical realization.The limitation concerns coverage, not merely the semantics of the selected words.
  • Confidence evidence: KDE comparisons examine confidence trajectories for Origin, NoWait, and SEAL, supporting confidence-based diagnosis beyond keyword heuristics.The passage frames this extraction as a basis for diagnosing and mitigating overthinking.

B METHOD DETAILS

ReBalance models overthinking and underthinking from stepwise reasoning behavior, extracts hidden-state prototypes, and uses a continuous controller to steer trajectories. Experiments report reduced reasoning length, preserved accuracy, cross-setting robustness, and sensitivity to both confidence and variance.

  • Explicit reasoning-mode modeling: ReBalance defines overthinking as reasoning beyond the earliest stable correct decision and underthinking as stopping before a later correct decision is reached.The definitions distinguish redundant continuation from premature commitment.
  • Explicit reasoning-mode modeling: High confidence variance proxies overthinking, whereas persistently high confidence with low variance proxies underthinking.The indicators are derived from stepwise confidence and sliding-window variance.
  • Prototype extraction and steering: Hidden states collected once from a small seen dataset are grouped into overthinking and underthinking prototypes whose difference forms the steering vector.The method saves deep-layer states at the first token of each reasoning step and averages states by mode.
  • Dynamic control: A continuous control function modulates steering strength and direction from online confidence statistics, allocating more intervention near either reasoning-mode boundary.The procedure is training-free and avoids additional computation during deployment after one-pass extraction.
  • Dynamic control: Both confidence and variance are necessary: univariate control variants degrade REBALANCE performance relative to the bivariate controller.This ablation supports finer-grained control from g(c, v).
  • Transferability: ReBalance remains effective across extraction domains and difficulty levels, with easy data favoring token reduction and hard data favoring accuracy improvement.Higher-difficulty extraction data are associated with lower confidence and higher variance.
  • Results: ReBalance significantly reduces reasoning length without degrading accuracy, while relative gains remain stable across large- and few-sample evaluations.Few-sample settings can produce larger gains under 16-sample evaluation.
  • Transferability: Across decoding temperatures, ReBalance adapts its behavior to differing confidence regimes.The supplied passage introduces this temperature-based generalizability analysis, while the accompanying figure describes lower- and higher-temperature distribution shifts.

C.5 SEMANTIC CHANGE AND CREATIVITY ANALYSIS

Semantic and creativity analyses examine whether ReBalance preserves useful reasoning patterns and expressive quality while reducing redundancy. The reported comparisons favor retaining reflection and transition behaviors rather than suppressing them indiscriminately.

  • Semantic change: ReBalance preserves a non-negligible amount of transition and reflection patterns, whereas NoWait and NoThinking show noticeable accuracy drops.The analysis links retained reasoning patterns with maintaining, and sometimes improving, accuracy.
  • Creativity analysis: Creative-writing evaluation reports that ReBalance generally maintains or improves creativity and expression naturalness across four models.The reported percentages of metrics at least matching the original models are 71%, 88%, 100%, and 65% for the four evaluated models.
  • Creativity analysis: Qwen3-14B improves across all reported creative-writing metrics after applying ReBalance.The table reports rubric, Elo, and fine-grained ability measures.
  • Creativity analysis: The authors associate reduced linguistic diversity with distillation and task-specific fine-tuning, while describing ReBalance as partially restoring creative expressiveness.This interpretation is presented as an analysis of the observed creativity results.
  • Confidence interpretation: Across models, overthinking corresponds to lower confidence and higher variance, while underthinking corresponds to higher confidence and lower variance.These patterns align the semantic analysis with the paper’s confidence-based reasoning-mode characterization.
  • Efficiency comparison: Compared with FlashThink and TrimR, ReBalance is described as reducing token use and latency with minimal overhead and uninterrupted decoding.The comparison highlights the absence of frequent verification interruptions and a separate verifier.
  • Prototype construction: Stability-index-based steering performs worse than confidence-based extraction in both accuracy and efficiency, with a smaller steering-vector norm.The authors interpret the smaller norm as blurred separation between reasoning modes.

D DETAILS ON EXPERIMENTAL SETTINGS

The evaluation spans mathematics, scientific, commonsense, and code reasoning across multiple open-source model families and parameter scales. ReBalance is measured primarily by answer accuracy and generated-token count under standardized decoding and fixed per-model extraction.

  • Benchmarks: Nine benchmarks cover mathematics, scientific reasoning, commonsense reasoning, and code reasoning.The suite includes MATH-500, AIME24, AIME25, AMC23, GSM8K, OLYMPIADBENCH, GPQA DIAMOND, STRATEGYQA, and LIVECODEBENCH.
  • Metrics: Evaluation uses Pass@1 for accuracy and average generated tokens Tok for efficiency.Higher Pass@1 is preferred, while lower Tok is preferred.
  • Models: Five backbone models span four architectures and four parameter scales, including models from 1.5B to 32B parameters.The listed backbones are DeepSeek-R1-Distill-Qwen, Qwen3-14B, QwQ-32B, and OpenPangu-Embedded-7B-V1.1.
  • Extraction protocol: For each backbone, the steering vector and control surface are estimated once from 500 randomly sampled MATH problems and fixed across benchmarks.This design tests cross-benchmark reuse of per-model extraction statistics.
  • Baselines: Comparisons include prompt-based, output-based, dynamic early-exit, and steering baselines that do not use auxiliary models.The listed baselines include COD, NoThinking, NOWAIT, DynaSor-COT, DEER, FlashThink, TrimR, SEAL, and Manifold Steering.
  • Implementation: Unless otherwise specified, decoding uses temperature 0.7, top_p 0.95, and a 16,000-token maximum across both Transformers and vLLM.Experiments run on a single server with 8 NVIDIA RTX PRO 6000 GPUs.

E DETAILS ON BENCHMARKS

The evaluation spans mathematics, science, commonsense, and code reasoning benchmarks, with prompt modifications and prompt-based comparisons included as supplementary analyses.

  • Benchmark coverage: LIVECODEBENCH evaluates runnable program generation with execution-based unit tests, emphasizing algorithmic reasoning, data structures, and implementation fidelity.The evaluation uses version v1 with 400 problems.
  • Prompt-based comparisons: Prompt modifications are evaluated as complementary to ReBalance, with QwQ-32B generating 3064 rather than 3662 tokens on MATH-500 without compromising accuracy.The passage reports this as an example of prompt refinement combined with the proposed method.
  • Prompt-based comparisons: Prompt-based methods can reduce redundant reasoning but are not consistently followed by the model, limiting their reliability.The discussion frames combining training-free strategies with stronger control methods as future work.

H DETAILED DISCUSSION OF RELATED WORKS

Related work spans chain-of-thought, latent reasoning, post-training, early exit, and steering methods. ReBalance differs by using bidirectional confidence-controlled steering to address overthinking and underthinking together without extra verifiers or inference stages.

  • Reasoning paradigms: Chain-of-thought methods elicit intermediate rationales, while self-consistency, search, verification, process rewards, and reinforcement learning extend reasoning beyond a single chain.The related approaches include Tree-of-Thoughts, Stream-of-Search, Graph-of-Thoughts, Process Reward Models, and RL-based self-correction.
  • Reasoning paradigms: Latent reasoning replaces discrete chain-of-thought tokens with continuous hidden representations to reduce tokenized traces and sampling while preserving intermediate signals.Representative methods use gated hidden-state signals, continuous latent spaces, or dense vectors.
  • Efficiency methods: Post-training methods improve the accuracy–latency/token trade-off through supervised conditional brevity, rationale compression, reward shaping, and preference learning.The surveyed methods include SFT- and RFT-based approaches that teach models when concise reasoning suffices.
  • Steering methods: ReBalance uses real-time confidence to adjust steering direction and intensity, targeting both overthinking and underthinking without discarding reasoning paths or adding verifiers.This bidirectional mechanism is presented as a fine-grained alternative to one-directional steering and early exit.
  • Efficiency methods: Early-exit methods use external instruction-following monitors to halt reasoning when further computation appears unnecessary.TRIMR and FLASHTHINK are cited as representative approaches.

I EFFICIENCY ANALYSIS

ReBalance reduces reasoning length while maintaining token-generation efficiency and reports faster reasoning with minimal additional memory, though difficult datasets can incur efficiency overheads.

  • Scope boundary: ReBalance preserves exploration on difficult reasoning problems, making efficiency overheads more likely on challenging datasets.The efficiency evaluation therefore adopts AIME24 as its setting.
  • Inference efficiency: ReBalance’s confidence computation and dynamic control logic have negligible impact on single-token generation time relative to the baseline.Confidence is obtained directly from each decoded token’s log probability, and the dynamic function is lightweight.
  • Inference efficiency: 1.5× and 1.4× TPR speedups are reported over DeepSeek-R1-Distill-Qwen-7B and QwQ-32B, respectively.The speedups accompany significantly shorter reasoning sequences without compromising token-generation efficiency.
  • Memory overhead: The QwQ-32B steering vector occupies only 22 KB of GPU memory, whereas early-exit methods require additional verifiers that introduce memory and communication costs.The comparison concerns additional GPU memory usage during inference.

L REPRODUCIBILITY STATEMENT

The reproducibility setup fixes sampling and hardware conditions, while steering vectors and control surfaces are extracted once and reused across benchmarks.

  • Experimental controls: All sampling and evaluation use random seed 42, eight NVIDIA RTX PRO 6000 GPUs, and unified decoding settings.The settings include temperature=0.7, top_p=0.95, and max_generated_tokens=16000.
  • Experimental controls: Steering vectors and control surfaces are extracted once from 500 randomly sampled MATH problems and then held fixed across all benchmarks.The setup also standardizes evaluation with Pass@1 and average token length.
  • Case-study context: The case-study figures contrast redundant checking, incorrect answers after insufficient verification, and a correct succinct response with judicious verification.The examples use DeepSeek-R1-Distill-Qwen-1.5B inference cases.
Loading 2603.12372v3…