PaperlayerView exact revision on arXiv

Source-linked AI summary

MixReasoning: Switching Modes to Think

Haiquan Lu, Gongfan Fang, Xinyin Ma, Qi Li, Xinchao Wang

arXiv:2510.06052v2cs.AIcs.CL

TL;DR

Uniformly extended reasoning can waste effort on routine steps, even though difficult subproblems may need deeper deliberation. MixReasoning switches reasoning depth within a response using local difficulty signals, and experiments report shorter reasoning with maintained task performance.

  • Problem

    Extended reasoning across every step can add inference cost and verbosity, while subproblems within one response vary in difficulty and some routine steps need less elaboration.

    Source: §1 · p. 1

  • Method

    MixReasoning trains a concise LoRA adapter and uses sliding-window token confidence to switch between detailed and concise reasoning within a response.

    Source: §1 · p. 2

  • Results

    13%–49% fewer reasoning tokens across AIME24, MATH-500, GPQA, and GSM8K while maintaining task performance.

    Source: §5 · p. 9

  • Takeaways & Limitations

    The results support concentrating detailed reasoning on difficult steps while compressing routine substeps.

    Source: §5 · p. 9

Abstract

from arXiv · show

Reasoning models enhance performance by tackling problems in a step-by-step manner, decomposing them into sub-problems and exploring long chains of thought before producing an answer. However, applying extended reasoning to every step introduces substantial redundancy, as sub-problems vary widely in difficulty and complexity: a small number of pivotal steps are genuinely challenging and decisive for the final answer, while many others only involve straightforward revisions or simple computations. Therefore, a natural idea is to endow reasoning models with the ability to adaptively respond to this variation, rather than treating all steps with the same level of elaboration. To this end, we propose MixReasoning, a framework that dynamically adjusts the depth of reasoning within a single response. The resulting chain of thought then becomes a mixture of detailed reasoning on difficult steps and concise inference on simpler ones. Experiments on GSM8K, MATH-500, and AIME show that MixReasoning shortens reasoning length and substantially improves efficiency without compromising accuracy.

1. Introduction

Uniformly elaborate reasoning can waste inference time and obscure pivotal steps, while global compression and problem-level routing do not adapt to difficulty within a response. MixReasoning switches between concise and detailed reasoning using local uncertainty, reducing reasoning length while maintaining performance.

1. Introduction

  • Uniformly elaborate chains of thought add inference cost, latency, and compute overhead, and can reduce readability through filler and redundant self-checks.The passage notes that autoregressive decoding time scales roughly linearly with sequence length.Source: §1 · p. 1
  • Global compression can truncate pivotal steps, while problem-level hybrid routing misses the heterogeneous difficulty of substeps within one response.Some steps need deliberate analysis or planning; others involve routine operations such as arithmetic carry-outs or straightforward transformations.Source: §1 · p. 1
  • MixReasoning trains a lightweight LoRA concise adapter and uses sliding-window token confidence to switch reasoning depth within a response.Low-confidence regions tend to include analysis, planning, verification, and key derivations; high-confidence regions tend to involve routine follow-through.Source: §1 · p. 2

Contributions

  • Across AIME24, MATH-500, GPQA, and GSM8K, MixReasoning shortens reasoning length by 13%–49% and improves inference efficiency while maintaining performance.The authors describe this result across models and benchmarks.Source: §1 · p. 2

2. Related Work

Prior efficiency approaches compress reasoning traces or route between short and long responses based on problem difficulty, while speculative decoding targets token-generation speed. MixReasoning instead adjusts detail within a reasoning trace according to local difficulty.

Efficient Reasoning

  • MixReasoning's overview depicts inference-time switching among thinking modes based on local uncertainty, balancing conciseness and detail.Its approach contrasts with long-to-short compression and hybrid reasoning.Source: Figure 2 · p. 3
  • Trace compression and problem-level hybrid routing can reduce redundancy, but neither addresses routine substeps inside long chains of thought.Problem-level routing can also be brittle when a seemingly simple problem contains a localized difficult segment.Source: §2 · p. 3

Speculative Decoding and Reasoning

  • Speculative decoding can reduce time per output token without shortening the chain of thought, and typically requires co-serving two models.MixReasoning shortens rationales through adaptive detail selection and can be combined with speculative decoding.Source: §2 · p. 3

3. Method

MixReasoning adapts reasoning depth within one response by controlling a concise LoRA adapter and estimating local difficulty from sliding-window token confidence. It reuses cached computation when switching modes to reduce serving overhead.

3. Method

  • MixReasoning combines inference-time control of a LoRA concise adapter with sliding-window confidence to switch reasoning modes within one response.The method targets detailed deliberation on difficult steps and concise inference on easier ones.Source: §3 · p. 3

3.1. Mode Switching via LoRA Strength Control

  • A lightweight LoRA adapter distills concise reasoning while keeping the base model's thinking mode intact; scaling α interpolates between detailed and concise modes.The authors describe the update as primarily controlling chain-of-thought length and state that α can be adjusted at any decoding position while serving a single model.Source: §3.1 · pp. 3–4

3.2. Difficulty-Aware Switching via Sliding-Window Token Confidence

  • Sliding-window confidence provides a segment-level difficulty signal, with low-confidence regions associated with analysis and verification and high-confidence regions with routine follow-through.The method uses token confidence derived from the model's internal token distribution; the reported pattern is described as semantically meaningful and robust.Source: §3.2 · p. 4

3.3. KV-Cache Reuse and Serving Efficiency.

  • A dwell constraint requires at least one window of decoding before another switch, and concise-to-thinking transitions roll back one window for regeneration.For serving, mode switches reuse the destination mode's KV cache and prefill only tokens generated in the other mode, avoiding recomputation over the shared prefix.Source: §3.2 · p. 5, §3.3 · p. 5

4. Experiments

Experiments evaluate MixReasoning across multiple reasoning models and benchmarks, compare its efficiency with existing methods, and analyze its switching behavior, design choices, and inference cost.

4.1. Experimental Setups

  • Experiments cover four reasoning-model backbones and AIME24, MATH-500, GSM8K, and GPQA, using a LoRA-trained concise adapter.The adapter is trained with short, correct solutions from DeepScaleR-40K; results are averaged over five runs.Source: §4.1 · pp. 5–6

4.3. More Baseline Comparisons

  • On AIME24, MixReasoning cuts generation by 25% while retaining 67.33 accuracy, and remains effective on challenging tasks where competing methods falter.Table 2 compares it with long-to-short compression and hybrid reasoning methods.Source: Table 2 · p. 6, §4.3 · p. 7

4.2. Main Results

  • Across model backbones and benchmarks, MixReasoning reduces token usage by approximately 13%–49% while maintaining or sometimes improving performance over the base reasoning model.The experiments use the base model for reasoning mode and α = 64 for concise mode.Source: §4.2 · p. 6

4.4. Ablation Studies

  • Confidence distributions overlap across benchmarks and models, supporting transferable thresholds; increasing τlow assigns more steps to thinking mode and lengthens responses.Sliding-window averaging reduces noisy, ungrounded switches; moderate window sizes improve accuracy and stability over m=1, while extreme sizes can over- or under-smooth.Source: §4.4 · p. 7, Figure 4 · p. 7, Figure 5 · p. 8
  • Confidence-based switching performs best: random and reverse-confidence strategies fare substantially worse, while cue-triggering remains less effective than sliding-window confidence.The comparison includes random switching, reverse confidence, and keyword-triggered switching.Source: §4.4 · pp. 7–8, Table 3 · p. 7

4.5. Further Analysis

  • Harder benchmarks use more thinking-mode tokens and switches; MixReasoning allocates detailed reasoning more to challenging datasets and concise generation to simpler ones.AIME24 and GPQA have thinking-mode ratios of 53.82% and 40.83%, with 66 and 38 switches, respectively.Source: §4.5 · p. 8, Table 4 · p. 8
  • Concise adapters shorten outputs but can lose performance on difficult benchmarks, and scaling distillation data from 40K to 300K yields only marginal improvements.The adapter study compares GSM8K, DeepScaleR, and OpenMathReasoning data.Source: §4.5 · pp. 8–9, Table 5 · p. 9
  • MixReasoning reduces end-to-end latency across evaluated datasets despite switching overhead, and qualitative examples show detailed reasoning on uncertain steps and concise follow-through.The latency results use vLLM; examples describe detailed reasoning for problem interpretation, planning, and key derivations, then concise mode for routine steps.Source: §4.5 · p. 9, Figure 6 · p. 9

5. Conclusion

MixReasoning adjusts reasoning depth within a response, reducing reasoning tokens by 13%–49% across benchmarks while maintaining task performance.

5. Conclusion

  • 13%–49% fewer reasoning tokens across AIME24, MATH-500, GPQA, and GSM8K while maintaining task performance.The authors conclude that models should think deeply only when it matters.Source: §5 · p. 9
  • MixReasoning adaptively switches reasoning depth within a single response, allocating computation to difficult steps and compressing routine ones.The framework is described as model-agnostic.Source: §5 · p. 9
  • The reported results support a principle of thinking deeply only when it matters.Source: §5 · p. 9

Impact Statement

MixReasoning adapts reasoning depth during generation, while its impact statement describes potential efficiency benefits and recommends safeguards for broader deployment.

Impact Statement

  • MixReasoning may lower latency, compute cost, and energy use in interactive and resource-constrained settings.Source: Impact Statement · p. 9
  • The impact statement cautions that wider deployment may increase overall usage and recommends monitoring, filtering, rate limiting, and verification when reasoning is concise.Source: Impact Statement · p. 9

A. Algorithm

  • The algorithm starts in thinking mode and uses a sliding-window confidence signal with low and high thresholds to switch between thinking and concise modes.It scales the concise LoRA adapter online and returns an answer with adaptive reasoning depth.Source: A. Algorithm · p. 13

B. More Implementation Details

The implementation details describe evaluation settings, prompt formats, and the training data and procedure used for the concise-mode adapter.

B. More Implementation Details

  • Results for Tables 1 and 2 are means over five independent runs, reporting pass@1 and token counts with a 16,384-token generation limit.Generation used temperature 0.6, top-p 0.95, and top-k 20.Source: B. More Implementation Details · p. 13
  • Math benchmarks use a step-by-step prompt with the answer in boxed format, while multiple-choice QA prompts request one explicit option letter.Source: B. More Implementation Details · pp. 13–14
  • The concise adapter is trained on DeepScaleR-derived prompts using the shortest correct response among ten sampled short chains as supervision.Candidates without a correct answer are discarded; training uses one epoch on four H100 GPUs.Source: B. More Implementation Details · p. 14

C. Further Theoretical and Empirical Analysis

The theoretical analysis interprets token-level confidence as a measure of concentration in the model’s top-k distribution and characterizes how it responds to increasing concentration.

C.1. Theoretical Justification of Difficulty-Aware Switching Signals

  • Token-level confidence increases when the normalized top-k distribution becomes more concentrated in the sense of majorization.Source: §C.1 · p. 14
  • The confidence measure equals a reverse-KL divergence from uniform, penalizing near-zero top-k probabilities and indicating distributional self-certainty rather than sampled-token surprisal.Source: §C.1 · pp. 14–15

C.2. Relation to other Distribution-Derived Metrics

This section contrasts sliding-window confidence with other top-k distribution measures and describes the latency cost of switching modes. It presents the alternatives’ conceptual differences and bounds switching overhead under stated assumptions.

C.2. Relation to other Distribution-Derived Metrics

  • Unlike entropy, which decreases as distributions become more peaked, ct increases with peakedness and is especially sensitive to nearly impossible top-k alternatives.The passage attributes this sensitivity to reverse-KL weighting each candidate equally, which penalizes small probabilities more strongly.Source: §C.2 · p. 15
  • Max probability ignores how probability mass is distributed among competitors, whereas ct uses the entire top-k set to capture their ambiguity.Source: §C.2 · p. 16
  • ct generalizes a two-token margin by aggregating multiple competitors, capturing whether probability mass collapses beyond the first few tokens.For k = 2, larger margins correspond monotonically to higher ct.Source: §C.2 · p. 16
  • Sliding-window confidence is compared with entropy, max probability, and margin as switching criteria using Pass@1 and generation length across three datasets.The supplied passages describe the comparison setup but do not report its results.Source: §C.2 · p. 16, Table 6 · p. 16

C.3. Switching Overhead

  • After a mode switch, the system prefills tokens generated since last visiting the destination mode; the overhead is one prefill per switched span.The KV cache for the destination mode can be reused on the shared prefix.Source: §C.3 · p. 16
  • With moderate dwell/window size m, relative switching latency is bounded approximately by γ/m, making overhead small in practice.The bound uses P(ℓ) ≤ γD(1), with γ ≈ 1–2; switches occur roughly once every m tokens, except rare rollback events.Source: §C.3 · p. 17

C.4. Additional Reasoning Benchmarks

Additional code-generation and commonsense benchmarks broaden evaluation beyond the paper’s primarily math- and science-focused tasks. MixReasoning reduces generation length while preserving or improving performance on the reported tasks.

C.4. Additional Reasoning Benchmarks

  • The evaluation covers HumanEval and LiveCodeBench for code generation, plus StrategyQA and CommonsenseQA for commonsense reasoning.The authors use the same evaluation settings and decoding hyperparameters as in the main paper.Source: §C.4 · p. 17
  • CommonsenseQA uses five-option questions requiring commonsense knowledge and multi-hop associative reasoning rather than mathematical derivations.StrategyQA tests implicit multi-step commonsense reasoning through yes/no questions.Source: §C.4 · p. 17
  • HumanEval Pass@1 rises from 91.95 to 93.29 as average generation length falls from 2881 to 2643 tokens; CommonsenseQA accuracy rises from 83.39 to 83.90 as length falls from 1183 to 895 tokens.Source: §C.4 · p. 17, Table 7 · p. 17

C.5. Performance Evaluation under Pass@k

The paper evaluates repeated-sampling performance on AIME24 and MATH-500 using Pass@2–5. MixReasoning improves Pass@k on AIME24, with gains increasing at higher k, and mostly improves it on MATH-500.

C.5. Performance Evaluation under Pass@k

  • Pass@k measures whether at least one of k sampled reasoning trajectories is correct, complementing Pass@1 with a view of candidate-path diversity and quality.The reported Pass@2–5 evaluation uses Qwen3-8B on AIME24 and MATH-500.Source: §C.5 · p. 17
  • MixReasoning consistently improves Pass@k on AIME24, with gains increasing from Pass@2 to Pass@5; on MATH-500 it improves Pass@2–4 and is comparable at Pass@5.Source: §C.5 · p. 17, Table 8 · p. 18

D. Qualitative Examples: Thinking Where It Matters

Qualitative examples show MixReasoning concentrating detailed thought on challenging or uncertain steps while keeping straightforward steps concise. The mixture between modes can be adjusted to trade efficiency against rationale detail and robustness.

D. Qualitative Examples: Thinking Where It Matters

  • MixReasoning produces shorter, more focused traces than Long CoT while retaining answer correctness in the examples, concentrating deliberation on ambiguous or decision-critical steps.Long CoT examples contain repeated self-checks, redundant restatements, and excessive reflection; Figures 7 and 8 illustrate the comparison.Source: D. Qualitative Examples: Thinking… · p. 18, Figure 7 · p. 18, Figure 8 · p. 19
  • Adjusting confidence thresholds τlow and τhigh and window size m trades efficiency and robustness: aggressive settings allocate more tokens to thinking, while conservative settings keep responses compact.More aggressive settings also produce more detailed rationales.Source: D. Qualitative Examples: Thinking… · p. 18

Related papers

Loading 2510.06052v2…