PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Reasoning with Balanced Thinking

Yulin Li, Tengyao Tu, Li Ding, Junjie Wang, Huiling Zhen, Yixin Chen, Yong Li, Zhuotao Tian

arXiv:2603.12372v3cs.AIcs.CLcs.LG

TL;DR

Large Reasoning Models can overthink by spending redundant steps or underthink by failing to explore sufficient reasoning paths, limiting efficient deployment. ReBalance is a training-free framework that uses confidence-guided dynamic steering to balance reasoning, reducing redundancy while improving or preserving accuracy across models and tasks.

  • Problem

    Existing overthinking-mitigation methods may suppress valuable reasoning and induce underthinking, motivating balanced control of both behaviors.

    Source: §1 · pp. 1–2

  • Method

    ReBalance models reasoning states from confidence signals, extracts hidden-state steering vectors, and dynamically adjusts their strength and direction to balance thinking.

    Source: §1 · pp. 2–3, §3.1 · p. 4

  • Results

    ReBalance reduces reasoning redundancy while improving accuracy across diverse models and datasets, outperforming prior methods across mathematical reasoning datasets and model scales.

    Source: §1 · pp. 2–3, §5 · p. 10

  • Takeaways & Limitations

    ReBalance offers a general, training-free, plug-and-play strategy for efficient reasoning with balanced thinking.

    Source: §1 · p. 3, §5 · p. 10

Abstract

from arXiv · show

Large Reasoning Models (LRMs) have shown remarkable reasoning capabilities, yet they often suffer from overthinking, expending redundant computational steps on simple problems, or underthinking, failing to explore sufficient reasoning paths despite inherent capabilities. These issues lead to inefficiencies and potential inaccuracies, limiting practical deployment in resource-constrained settings. Existing methods to mitigate overthinking, such as suppressing reflective keywords or adjusting reasoning length, may inadvertently induce underthinking, compromising accuracy. Therefore, we propose ReBalance, a training-free framework that achieves efficient reasoning with balanced thinking. ReBalance leverages confidence as a continuous indicator of reasoning dynamics, identifying overthinking through high confidence variance and underthinking via consistent overconfidence. By aggregating hidden states from a small-scale dataset into reasoning mode prototypes, we compute a steering vector to guide LRMs' reasoning trajectories. A dynamic control function modulates this vector's strength and direction based on real-time confidence, pruning redundancy during overthinking, and promoting exploration during underthinking. Extensive experiments conducted on four models ranging from 0.5B to 32B, and across nine benchmarks in math reasoning, general question answering, and coding tasks demonstrate that ReBalance effectively reduces output redundancy while improving accuracy, offering a general, training-free, and plug-and-play strategy for efficient and robust LRM deployment. Project page and code are available at https://rebalance-ai.github.io .

1 INTRODUCTION

ReBalance is a training-free framework that uses confidence to distinguish overthinking from underthinking and dynamically steer LRM reasoning. Across models and tasks, it reduces reasoning length while improving accuracy.

Problem

  • LRMs can overthink by spending redundant steps on simple problems, increasing computational costs and potentially introducing hallucinations.These inefficiencies limit practical deployment in resource-constrained environments.Source: §1 · p. 1
  • Methods that shorten reasoning or suppress reflective behavior may remove valuable reasoning, while difficulty-based length control can induce underthinking through overly short chains.Underthinking occurs when models fail to explore valid reasoning paths despite possessing the capability to solve the problem.Source: §1 · p. 1

Results

  • Across four models from 0.5B to 32B, ReBalance reduces reasoning length while improving accuracy across multiple mathematical reasoning datasets and nine benchmarks spanning math, general QA, and coding.The framework is presented as generalizing across diverse models and unseen data while offering plug-and-play deployment.Source: §1 · pp. 2–3

Key observations

  • Confidence provides a continuous signal: high variance indicates overthinking, whereas consistent overconfidence indicates underthinking and premature commitment to incorrect paths.This signal supports fine-grained control of internal reasoning states represented by hidden states.Source: §1 · pp. 2–3

Method

  • ReBalance aggregates hidden states from overthinking and underthinking steps in a small seen dataset into prototypes and derives a steering vector between the reasoning modes.The steering vector encodes the model’s reasoning dynamics and is intended to generalize to unseen data.Source: §1 · p. 2
  • A confidence-conditioned control function amplifies steering to prune redundancy during overthinking and reverses it to promote alternative-path exploration during underthinking.This dynamic mechanism modulates both the strength and direction of hidden-state steering without additional training.Source: §1 · pp. 2–3

2 BACKGROUND AND MOTIVATION

The section defines stepwise confidence and confidence variance to characterize reasoning dynamics, then shows that existing overthinking mitigation can induce underthinking. It motivates confidence-guided control by linking overthinking to high variance and underthinking to persistent overconfidence.

2.1 PRELIMINARIES

  • Stepwise confidence measures adherence to a reasoning path, while confidence variance over a sliding window measures switching among different reasoning paths.Stepwise confidence is computed from tokenwise maximum predicted probabilities aggregated across each reasoning step; variance captures local fluctuations.Source: §2.1 · p. 3
  • High confidence variance may drive redundant continuation and overthinking, while high confidence with low variance suggests premature commitment and potential underthinking.These statistics provide the signals for the dynamic control mechanism introduced later.Source: §2.1 · p. 3

2.2 KEY OBSERVATIONS

  • Existing methods and ReBalance both reduce overthinking, but only ReBalance preserves reasoning-length distributions while avoiding the notable underthinking introduced by existing methods.Figure 2(a) compares original, existing-mitigation, and proposed-method conditions using ground-truth reasoning length as an ideal-length proxy.Source: §2.2 · pp. 3–4, Figure 2 · p. 4
  • The analysis categorizes shortened correct reasoning as reduced redundancy and lost correctness after mitigation as restricted exploration, motivating explicit modeling of both reasoning modes.These categorizations guide comparisons of confidence and confidence variance against normal reasoning.Source: §2.2 · p. 4
  • Overthinking coincides with high confidence variance, whereas underthinking reflects consistently high confidence and premature commitment to insufficiently explored reasoning paths.These patterns support confidence as a continuous indicator for fine-grained control of reasoning states.Source: §2.1 · p. 3, §2.2 · p. 4, Figure 2 · p. 4

3 METHOD

REBALANCE is a training-free framework that balances overthinking and underthinking through confidence-based state modeling, hidden-state steering, and adaptive control. Its components provide dynamic, efficient control of reasoning trajectories.

3.1 OVERVIEW

  • REBALANCE models reasoning states with stepwise confidence and confidence variance, then dynamically balances overthinking and underthinking without additional training.The framework identifies reasoning modes, extracts steering vectors, and adaptively controls their strength and direction during inference.Source: §3.1 · p. 4

3.2 EXPLICIT MODELING OF OVERTHINKING AND UNDERTHINKING

  • Overthinking is characterized by high variance and low confidence, whereas underthinking shows low variance and persistently high confidence; other instances are treated as normal.These confidence-based thresholds classify reasoning steps into overthinking and underthinking sets, excluding instances outside their union.Source: §3.2 · pp. 5–6

3.3 CONFIDENCE-BASED STEERING VECTOR EXTRACTION

  • Using one offline pass over a small seen dataset, REBALANCE extracts deep-layer hidden-state prototypes and defines a steering vector from underthinking toward overthinking.The first token of each reasoning step provides a compact representation; positive steering promotes exploration, while negative steering encourages commitment.Source: §3.3 · p. 6

3.4 MODEL BEHAVIOR–BASED DYNAMIC CONTROL FUNCTION

  • A dynamic function sets steering weight as a continuous function of current confidence and variance, adaptively directing trajectories away from nearby reasoning boundaries.The method combines confidence-based direction selection with variance-aware amplitude and soft saturation for smooth transitions and numerical stability.Source: §3.4 · pp. 6–7

SCIENCE COMMONSENSE PROGRAMMING

This section reports ReBalance’s generalization on non-math tasks and analyzes ablations across math-benchmark difficulty levels. It also directs theoretical details on mode boundaries and gating to Appendix B.

Generalization

  • Generalization results on other non-math tasks are evaluated using Pass@1 and generated-token count.Pass@1 is higher-is-better, while #Tokens is lower-is-better; changes are color-coded in Table 2.Source: Table 2 · p. 8

Ablations

  • Ablations on the R1-1.5B backbone compare performance across medium, easy, and hard math benchmarks.The study uses Math500, GSM8K, and Olympiad, reporting Pass@1 accuracy and generated-token numbers relative to original REBALANCE.Source: Table 3 · p. 8

Additional analysis

  • Additional details, theoretical derivations, and proofs for the mode boundaries and gating function are provided in Appendix B.These materials are deferred because of the page limit.Source: SCIENCE COMMONSENSE PROGRAMMING · p. 8

4 EXPERIMENT

ReBalance improves accuracy while reducing reasoning length across diverse mathematical, scientific, programming, commonsense, and NPU settings without auxiliary models or domain-specific tuning. Ablations show that dynamic control, layer choice, dataset-derived vectors, window size, mode boundaries, and gating design govern the accuracy–efficiency trade-off.

4.1 MAIN RESULTS

  • ReBalance outperforms all baselines across six math benchmarks, improving Pass@1 by up to 10.0 points while reducing generated tokens by up to 35.4%.These gains require neither auxiliary models nor additional inference stages.Source: §4.1 · p. 8
  • Prompt-based suppression harms distilled models more than non-distilled models, while intrinsic-signal methods generally outperform external early-exit criteria.The results indicate that distilled reasoning structures are more sensitive to prompt manipulation and that intrinsic signals provide lighter, more faithful control.Source: §4.1 · pp. 8–9
  • Across scientific reasoning, programming, and commonsense QA, ReBalance preserves Pass@1 while reducing reasoning length by up to 29.9% without domain-specific tuning.Confidence signals extracted from mathematics generalize well, improving effective reasoning on challenging tasks while behaving more conservatively on saturated StrategyQA.Source: §4.1 · p. 9
  • On Ascend 910B with openPangu-Embedded-7B-V1.1, ReBalance consistently improves reasoning accuracy and efficiency over the baseline in slow-thinking mode.The experiment demonstrates compatibility with an NPU-native model and platform.Source: §4.1 · p. 9

4.2 ABLATION STUDY

  • Static positive αs improves accuracy but increases reasoning length, with αs = +3 on QWQ–32B increasing tokens by 147%, whereas negative αs saves tokens at an accuracy cost.These results motivate dynamically adapting αs to instance difficulty.Source: §4.2 · p. 9
  • Steering at every tested layer reduces token count without harming accuracy, with the strongest trade-off in mid-to-late layers where confidence separability is highest.These layers therefore introduce less steering noise according to the probing analysis.Source: §4.2 · p. 9
  • Steering vectors generalize across datasets, while harder-source data favors accuracy and easier-source data favors token savings through different fitted control surfaces.Harder datasets induce conservative surfaces prioritizing correctness, whereas easier datasets produce more aggressive surfaces favoring efficiency.Source: §4.2 · pp. 9–10
  • A small window |Ws| = 2 is responsive and sufficient, moderate underthinking boundaries improve accuracy at modest cost, and all tested gating variants remain effective.Larger windows increase token usage but can improve hard-Olympiad accuracy; removing or overly increasing Bu harms performance, while linear, hard-step, polynomial, and ReLU gates differ only slightly.Source: §4.2 · p. 10

5 CONCLUSION … A.5 ENCODING OF CONFIDENCE SIGNALS IN LATENT REPRESENTATIONS

ReBalance addresses the tension between overthinking and underthinking through confidence-aware, training-free control, supported by evidence on reasoning length, state persistence, model heterogeneity, and latent confidence encoding. Across experiments and analyses, it reduces redundancy while preserving accuracy and adapts its interventions to model behavior.

5 CONCLUSION

  • ReBalance curbs overthinking without inducing underthinking, reducing redundancy while preserving accuracy across diverse models and datasets.The authors identify future extension to multimodal reasoning scenarios.Source: §5 · p. 10

M Case Study

  • ReBalance selectively shortens reasoning for appropriate cases while preserving longer explorations for challenging instances, unlike methods whose short outputs reflect underthinking.On MATH with DeepSeek-R1-Distill-Qwen-1.5B, competing methods collapse error distributions toward short outputs, whereas ReBalance avoids that collapse.Source: §A.1 · p. 20, Figure 6 · p. 20

A.2 CONFIDENCE-LENGTH ASSOCIATION

  • Word count correlates negatively with minimum step-level confidence and positively with confidence variance across four models, supporting confidence trajectories as indicators of reasoning effort and stability.Spearman ρ values are approximately −0.62 to −0.80 for minimum confidence and 0.46 to 0.70 for variance; all 95% bootstrap intervals exclude zero.Source: §A.2 · p. 21, Table 6 · p. 21

A.3 MARKOV PERSISTENCE OF CONFIDENCE STATES

  • Confidence states exhibit strong first-order Markov persistence, with same-state transitions dominating across all four models and supporting a minimal window size of two.SameRate exceeds 0.5, Fisher tests yield p < 0.001, and diagonal odds-ratio intervals exclude one; a two-step window captures adjacent transitions with minimal lag.Source: §A.3 · pp. 21–22, Table 7 · p. 22

A.4 DISTRIBUTIONAL HETEROGENEITY IN MODEL CONFIDENCE

  • Confidence distributions differ across model families, motivating behavior-aware control that derives model-specific parameters instead of relying on manually tuned universal hyperparameters.QWEN2-based models show broadly similar patterns, while QWEN3-14B differs clearly; LLAMA-3.1-NEMOTRON-NANO-8B is reported to remain uniformly low-confidence (Yang et al., 2025b).Source: §A.4 · p. 23, Figure 7 · p. 23

A.5 ENCODING OF CONFIDENCE SIGNALS IN LATENT REPRESENTATIONS

  • Confidence is systematically and predominantly linearly encoded in hidden states, with high- and low-confidence representations forming distinct clusters that become clearer in mid-to-late layers.The analysis maps sentence-level confidence to delimiter-following hidden states and uses dimensionality reduction plus linear probing to assess decodability.Source: §A.5 · p. 23
  • PCA and ridge-based linear probing quantify how readily confidence can be decoded from hidden representations using layer-wise R2, with higher values indicating stronger linear decodability.PCA reduces the several-thousand-dimensional hidden states before probing, and Table 8 summarizes retained variance for k = 64.Source: §A.5 · p. 24, Table 8 · p. 24

A.6 CONFIDENCE AS EVIDENCE UNDER VOCABULARY COVERAGE GAPS

Vocabulary-based reasoning controls are incomplete approximations of confidence-based control: they capture frequent lexical signals of low-confidence reasoning but miss broader uncertainty cues. Confidence analysis explains why NOWAIT and SEAL work and motivates confidence-driven interventions.

Confidence representation

  • Linear probes show that confidence is most decodable in mid-to-late layers, motivating steering-layer selection.Layer-wise hidden states are projected by PCA and passed to a linear predictor to obtain test-set R2 scores.Source: Figure 9 · p. 26, Figure 10 · p. 26

Mechanism and limitation

  • Vocabulary-based strategies capture frequent lexical manifestations of low-confidence reasoning while missing a broader spectrum of uncertainty signals.This reframes keyword suppression and latent steering as incomplete confidence-based control rather than interventions targeting particular semantics.Source: §A.6 · pp. 26–27
  • Most NOWAIT and SEAL vocabulary items correspond to sentence confidences below the overall mean, c̄ = 0.8162, indicating that they suppress low-confidence reasoning modes.The analysis projects step-level confidence onto words in DEEPSEEK-R1-DISTILL-QWEN-7B reasoning on MATH-500.Source: §A.6 · pp. 26–27
  • NOWAIT and SEAL reliably elevate confidence along reasoning trajectories by suppressing or steering away from high-frequency markers and states associated with low-confidence sentences.NOWAIT operates lexically, whereas SEAL modifies hidden representations.Source: §A.6 · p. 27

Implications

  • Confidence-based approaches can surface low-confidence modes regardless of their lexical realization, offering broader coverage than predefined vocabularies.The analysis identifies a substantial long tail of informative cues omitted by existing keyword lists.Source: §A.6 · p. 27, Figure 11 · p. 27

B METHOD DETAILS … C.4 PERFORMANCE VARIATION UNDER DIFFERENT CONFIDENCE DISTRIBUTIONS

This section details ReBalance’s confidence-driven modeling of overthinking and underthinking, latent steering directions, and behavior-adaptive control for balancing exploration and commitment. It also examines the method’s transfer behavior, multi-sample evaluation, and robustness across confidence distributions induced by different decoding temperatures.

B METHOD DETAILS

  • The complete framework combines explicit confidence-based mode modeling, one-pass latent prototype extraction, and behavior-based dynamic control to balance exploration and commitment.Together, these sequential components form an adaptive mechanism for keeping reasoning trajectories between overthinking and underthinking boundaries.Source: B METHOD DETAILS · p. 27

B.1 EXPLICIT MODELING OF OVERTHINKING AND UNDERTHINKING

  • ReBalance formally defines overthinking as reasoning beyond the earliest stable correct decision and underthinking as premature termination before a later correct decision exists.Confidence and sliding-window variance identify high-variance reasoning as a proxy for overthinking and persistent high-confidence, low-variance reasoning as a proxy for underthinking.Source: §B.1 · pp. 27, 29

B.2 CONFIDENCE-BASED STEERING VECTOR EXTRACTION

  • The method extracts overthinking and underthinking prototypes from deep-layer first-token hidden states and uses their difference as a steering vector injected at each reasoning step.One-pass extraction is performed once per model before deployment; the vector’s direction promotes exploration away from underthinking or suppresses overthinking.Source: §B.2 · pp. 29–30

B.3 MODEL BEHAVIOR-BASED DYNAMIC CONTROL FUNCTION

  • The dynamic controller maps real-time confidence and variance to steering magnitude and direction, increasing intervention near overthinking or underthinking boundaries while remaining continuous.It is training-free, uses statistics already computed online, adapts across models through behavior-based distances, and applies smooth variance-aware gates.Source: §B.3 · pp. 30–32

C.1 ABLATION ON INDIVIDUAL AXES

  • Both confidence-only and variance-only control variants degrade performance, indicating that the bivariate controller g(c, v) provides finer-grained and more effective control.The ablation evaluates the R1-1.5B backbone across Math500, GSM8K, and Olympiad with accuracy and generated-token metrics.Source: §C.1 · p. 32, Table 11 · p. 33

C.2 CROSS-DOMAIN AND CROSS-DIFFICULTY TRANSFERABILITY

  • Across domains, ReBalance maintains strong performance despite differing confidence statistics, and StrategyQA-derived extraction can exceed MATH-derived extraction on Pass@1 within the same target domain.The cross-domain study transfers coding and commonsense extraction statistics to mathematics, while cross-difficulty analysis also uses fine-grained in-distribution MATH difficulty levels to reduce dataset-distribution confounds.Source: §C.2 · p. 33
  • Extraction difficulty shapes control behavior: easy datasets prioritize token reduction, whereas hard datasets prioritize accuracy improvement, motivating medium-difficulty datasets for practical use.Harder extraction data produce lower confidence and higher variance, while easier data widen the overthinking range and narrow the underthinking range.Source: §C.2 · pp. 33–34, Table 12 · p. 34

C.3 PASS@K AND AVG@K PERFORMANCE ANALYSIS

  • ReBalance significantly reduces reasoning length without degrading Pass@20 accuracy, pruning redundant steps and enabling lossless sequence compression near the model’s capability ceiling.Multi-sample evaluation further finds that relative performance remains largely consistent across sampling counts on sufficiently large datasets, while few-sample datasets show substantially higher variability.Source: §C.3 · pp. 34–35, Table 13 · p. 35
  • ReBalance’s relative improvement over the baseline remains stable as sampling increases on few-sample datasets, with larger gains under 16-sample evaluation than single-sample evaluation.Both accuracy and sequence length fluctuate more with increased sampling, but the improvement remains robust across multiple datasets and models.Source: §C.3 · p. 35

C.4 PERFORMANCE VARIATION UNDER DIFFERENT CONFIDENCE DISTRIBUTIONS

  • Across decoding temperatures, ReBalance consistently achieves higher accuracy and shorter reasoning traces, demonstrating stable generalization under varying confidence distributions.At low temperatures it counters persistent overconfidence and underthinking; at high temperatures it narrows the search space and accelerates convergence.Source: Table 16 · p. 36, §C.4 · p. 36

C.5 SEMANTIC CHANGE AND CREATIVITY ANALYSIS … C.9 ADDITIONAL PROTOTYPE CONSTRUCTION STRATEGIES

Across supplementary analyses, ReBalance preserves reasoning patterns and creative expression while using confidence to distinguish overthinking from underthinking. It outperforms verifier-based alternatives with lower overhead, generalizes to dynamic temperature control, and validates step-level prototype extraction over trajectory-level alternatives.

C.5 SEMANTIC CHANGE AND CREATIVITY ANALYSIS

  • Semantic analysis finds that all evaluated methods reduce reflection and transition patterns, but ReBalance retains more of these patterns and thereby preserves or sometimes improves accuracy.The analysis uses DeepSeek-R1-Distill-Qwen-1.5B reasoning traces and scales TF and TF–IDF by 1000 for readability.Source: §C.5 · p. 37, Table 17 · p. 37
  • ReBalance maintains or improves creativity metrics for 71%, 88%, 100%, and 65% of evaluated metrics on DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek-R1-Distill-Qwen-7B, Qwen3-14B, and QwQ-32B, respectively.Qwen3-14B improves across all metrics, supporting the claim that continuous guidance balances overthinking and underthinking without suppressing divergent thinking.Source: §C.5 · p. 38, Table 18 · p. 38

C.6 CONFIDENCE CHARACTERISTICS OF OVERTHINKING AND UNDERTHINKING

  • Across model families and sizes, overthinking shows lower confidence with higher variance, whereas underthinking shows higher confidence with lower variance and premature commitment to incorrect paths.These results support confidence as a broadly applicable indicator of reasoning mode.Source: §C.6 · p. 39, Table 19 · p. 39

C.7 PERFORMANCE COMPARISON WITH TRIMR AND FLASHTHINK

  • ReBalance simultaneously reduces token consumption and latency while outperforming TrimR and FlashThink, which require frequent verification, separate verifiers, and additional inference stages.The comparison uses reproduced implementations because official implementations were unavailable; detailed results are reported for QwQ-32B and across three model sizes.Source: §C.7 · pp. 39–40, Table 20 · p. 40, Table 21 · p. 40

C.8 BALANCED THINKING WITH DYNAMIC TEMPERATURE

  • Using confidence to adjust temperature achieves significant performance improvements and shorter reasoning even with binary settings of 1.2 for underthinking and 0.7 for overthinking.The preliminary experiment suggests continuous, behavior-based control could improve this simpler approach without manual tuning.Source: §C.8 · p. 40, Table 21 · p. 40

C.9 ADDITIONAL PROTOTYPE CONSTRUCTION STRATEGIES

  • Eq. (3) is unsuitable as a practical steering indicator because it operates at trajectory level, requires ground truth or an external verifier, and rigidly assumes identical correct future answers.These limitations conflict with ReBalance’s need for fine-grained real-time control and complex reasoning dynamics.Source: §C.9 · p. 41
  • Stability indices concentrate near the beginning and end of reasoning sequences, forming a bimodal distribution that empirically supports distinct underthinking and overthinking behaviors.The validation uses 500 MATH training samples and yields 29,701 overthinking and 24,710 underthinking samples.Source: §C.9 · p. 41, Table 22 · p. 42
  • Step-level extraction using Eq. (5) significantly outperforms trajectory-level Eq. (3) extraction in accuracy and efficiency, while producing a substantially smaller steering-vector norm under the stability-index variant.The smaller norm indicates blurred overthinking–underthinking boundaries and helps explain the performance gap.Source: §C.9 · pp. 41–42, Table 23 · p. 42

D DETAILS ON EXPERIMENTAL SETTINGS

The experimental evaluation spans nine reasoning benchmarks, five open-source backbone models, and multiple training-free baselines, using Pass@1 and generated-token count as primary metrics. ReBalance’s steering components are extracted once per backbone and evaluated under standardized decoding and hardware settings.

Benchmarks

  • Evaluation covers nine benchmarks across mathematics, science, commonsense, and code reasoning tasks.The suite includes MATH-500, AIME24, AIME25, AMC23, GSM8K, OLYMPIADBENCH, GPQA DIAMOND, STRATEGYQA, and LIVECODEBENCH.Source: D DETAILS ON EXPERIMENTAL SETTINGS · p. 42

Evaluation metrics

Backbone reasoning models

  • Experiments use five open-source backbone models spanning four architectures and four parameter scales.The backbones are DEEPSEEK–R1–DISTILL–QWEN at 1.5B and 7B, QWEN3–14B, QWQ–32B, and OPENPANGU-EMBEDDED-7B-V1.1.Source: D DETAILS ON EXPERIMENTAL SETTINGS · p. 42

Steering extraction and dynamic function fitting

Baseline methods

Decoding and hardware

  • Unless otherwise noted, decoding uses temperature = 0.7, top_p = 0.95, and max_generated_tokens = 16000 on a single server with 8× NVIDIA RTX PRO 6000 GPUs.The same decoding configuration is applied across the Transformers and vLLM backends.Source: D DETAILS ON EXPERIMENTAL SETTINGS · pp. 42–43

E DETAILS ON BENCHMARKS · F DETAILS ON PROMPTS · G DETAILS ON PROMPT-BASED APPROACHES

The paper evaluates reasoning across math, commonsense, and coding benchmarks, using task-specific prompts and a separate answer-detector prompt for steering-vector extraction. Prompt-based methods can reduce redundancy and complement ReBalance, but their effectiveness is inconsistent.

E DETAILS ON BENCHMARKS

  • The benchmark suite covers six math datasets spanning simple to hard problems, including MATH-500, AIME24, AIME25, AMC23, GSM8K, and OLYMPIADBENCH.The datasets range from short school-level arithmetic and high-school competition questions to advanced Olympiad-style mathematical and physics reasoning.Source: E DETAILS ON BENCHMARKS · p. 43
  • GPQA DIAMOND tests graduate-level biology, physics, and chemistry knowledge, STRATEGYQA tests implicit multi-hop commonsense reasoning, and LIVECODEBENCH tests executable competitive-programming solutions.LIVECODEBENCH v1 contains 400 problems judged with execution-based unit tests, emphasizing algorithms, data structures, and implementation fidelity.Source: E DETAILS ON BENCHMARKS · p. 43

F DETAILS ON PROMPTS

  • Math prompts request step-by-step reasoning with the final answer in \boxed{}, while multiple-choice prompts require only the boxed choice letter.The math evaluation includes MATH-500, AIME 2024, AIME 2025, AMC23, GSM8K, and Olympiad.Source: F DETAILS ON PROMPTS · p. 44
  • StrategyQA prompts require binary answers in \boxed{Yes} or \boxed{No}, coding prompts require only a correct Python program, and steering-vector extraction uses an answer detector that returns only a JSON yes/no judgment.The detector labels a sentence yes when it explicitly states, semantically expresses, or uniquely determines the gold answer.Source: F DETAILS ON PROMPTS · p. 44

G DETAILS ON PROMPT-BASED APPROACHES

  • 3662 to 3064 generated tokens on MATH-500 without compromising accuracy when replacing the prompt, while QwQ-32B gained further improvements on several datasets.These results indicate that prompt refinement and ReBalance provide complementary benefits, although the experiments also examine prompt-based approaches as a simple way to alter model behavior.Source: G DETAILS ON PROMPT-BASED APPROACHES · p. 45
  • Prompt-based methods mitigate redundant reasoning to some extent, but models do not consistently follow instructions, limiting their reliability.The paper notes that prompt positioning can also affect model accuracy and efficiency, and identifies combining training-free strategies with stronger control methods as future work.Source: G DETAILS ON PROMPT-BASED APPROACHES · pp. 45–46

H DETAILED DISCUSSION OF RELATED WORKS

Prior work improves reasoning efficiency through test-time search, latent reasoning, post-training, steering, early exit, inference acceleration, and smaller models. ReBalance addresses a gap in existing steering and early-exit methods by dynamically adjusting both steering direction and strength to mitigate overthinking without inducing underthinking.

Chain-of-Thought

  • Chain-of-thought methods improve multi-step reasoning through intermediate rationales, self-consistency, search, verification, and reinforcement-based correction, with test-time compute scaling often rivaling parameter scaling.Representative methods include Tree-of-Thoughts, Stream-of-Search, Graph-of-Thoughts, Process Reward Models, and RL-based Self-Correction.Source: H DETAILED DISCUSSION OF RELATED WORKS · p. 46

Latent Reasoning and Other Efficiency Directions

  • Other efficiency directions replace discrete traces with latent representations, accelerate inference through speculative decoding and serving optimization, or transfer reasoning into smaller models via quantization, distillation, and pruning.These approaches reduce tokenized reasoning, memory or scheduling overhead, sampling cost, or backbone compute, but compression can be sensitive to calibration and where it is applied.Source: H DETAILED DISCUSSION OF RELATED WORKS · pp. 46, 48

Post-Training Methods for Efficient Reasoning

  • Post-training methods improve the accuracy–latency/token trade-off by teaching conditional brevity, rationale compression, or adaptive reasoning length through supervised or reinforcement fine-tuning.RFT approaches use reward shaping, preference learning, PPO, GRPO, control tokens, and explicit reasoning-length objectives.Source: H DETAILED DISCUSSION OF RELATED WORKS · pp. 46–47

Steering-Based Methods for Efficient Reasoning

  • Existing steering methods mainly target overthinking, use static or one-directional updates, and may amplify underthinking when reducing redundant reasoning.Prior approaches include phase-, dataset-, token-, and difficulty-based steering, while CONTROLLING THINKING SPEED dynamically changes magnitude using sliding-window difficulty estimates.Source: H DETAILED DISCUSSION OF RELATED WORKS · p. 47

Early-Exit Methods for Efficient Reasoning

  • Early-exit methods reduce token consumption by terminating reasoning based on external monitors or internal confidence and entropy signals, but their binary, coarse-grained decisions can discard valuable reasoning steps.TRIMR and FLASHTHINK also rely on handcrafted keyword triggers, limiting adaptability across models and tasks.Source: H DETAILED DISCUSSION OF RELATED WORKS · p. 47

Steering-Based Methods for Efficient Reasoning

  • ReBalance uses real-time confidence to detect overthinking or underthinking and bidirectionally adjust steering strength and direction, without additional verifiers or inference stages.This fine-grained control keeps reasoning within a balanced boundary rather than discarding reasoning paths.Source: H DETAILED DISCUSSION OF RELATED WORKS · p. 47

I EFFICIENCY ANALYSIS · J THE USE OF LARGE LANGUAGE MODELS · K ETHICS STATEMENT

ReBalance delivers efficient reasoning by shortening generated sequences while preserving token-generation efficiency, with low memory overhead and improved reasoning time. The paper also states limited LLM use, ethical compliance, and no significant ethical concerns.

I EFFICIENCY ANALYSIS

  • The efficiency comparison covers TPS, time per request, and additional GPU memory against baselines and prompt-based, early-exit, and latent-steering methods, using AIME24.ReBalance promotes exploration on difficult problems to avoid underthinking, which can create overhead on challenging datasets.Source: I EFFICIENCY ANALYSIS · pp. 48–49
  • ReBalance’s dynamic adjustment logic has negligible impact on single-token generation time because confidence comes directly from token log probabilities and the control function is lightweight.Token generation efficiency is evaluated using tokens per second (TPS).Source: I EFFICIENCY ANALYSIS · p. 49
  • ReBalance achieves 1.5× and 1.4× TPR speedups over DeepSeek-R1-Distill-Qwen-7B and QwQ-32B, respectively, by significantly shortening reasoning length without compromising token-generation efficiency.The efficiency evaluation uses TPS and #Tokens to assess these gains.Source: I EFFICIENCY ANALYSIS · p. 49
  • ReBalance uses minimal additional GPU memory: the extracted steering vector for QwQ-32B is only 22 KB, unlike early-exit methods requiring verifiers usually at least 7B.The comparison includes FlashThink and TrimR, whose verifiers also introduce communication load.Source: I EFFICIENCY ANALYSIS · p. 49

J THE USE OF LARGE LANGUAGE MODELS

  • The authors state that large language models were used only to polish writing and check grammar, not for ideation, experimentation, analysis, or conclusions.They attribute all substantive intellectual contributions to the authors.Source: J THE USE OF LARGE LANGUAGE MODELS · p. 49

K ETHICS STATEMENT

  • The research adheres to the ICLR Code of Ethics and involves no human subjects, personal data, or real-world deployment that could cause harmful, biased, or discriminatory outcomes.The authors report that experiments used publicly available datasets and identify no significant ethical concerns after considering fairness, transparency, and societal impact.Source: K ETHICS STATEMENT · p. 49

L REPRODUCIBILITY STATEMENT · M CASE STUDY

The reproducibility statement standardizes sampling, hardware, decoding, steering-vector extraction, and evaluation settings. The case study illustrates how ReBalance targets redundant reasoning while preserving verification needed for correctness.

L REPRODUCIBILITY STATEMENT

  • Experiments fix the random seed at 42 and use identical hardware and decoding configurations across evaluations.Runs use 8× NVIDIA RTX PRO 6000 GPUs, temperature=0.7, top_p=0.95, and max_generated_tokens=16000.Source: L REPRODUCIBILITY STATEMENT · p. 49
  • Steering vectors and control surfaces are extracted once from 500 randomly sampled MATH problems and reused across all benchmarks.This reuse supports consistent cross-benchmark evaluation without repeated extraction.Source: L REPRODUCIBILITY STATEMENT · p. 49
  • Standardized Pass@1 and average token length metrics, together with fixed settings, are intended to make results reproducible.The statement explicitly links these choices to reliable reproduction of the experiments.Source: L REPRODUCIBILITY STATEMENT · p. 49

M CASE STUDY

  • A DeepSeek-R1-Distill-Qwen-1.5B case reaches the correct solution despite substantial redundancy and repetition in its reasoning.The example includes unnecessary inspections beyond required boundary checks, creating considerable token overhead.Source: Figure 14 · p. 50
  • Existing overthinking-mitigation techniques reduce token usage but omit verification steps, producing an incorrect answer in the case study.The figure contrasts lower token consumption with the loss of verification needed for correctness.Source: Figure 15 · p. 51
  • Adding judicious verification steps yields a correct and succinct response in the DeepSeek-R1-Distill-Qwen-1.5B example.The case demonstrates the target balance between avoiding redundant reasoning and retaining necessary verification.Source: Figure 16 · p. 52

Related papers

Loading 2603.12372v3…