PaperlayerView exact revision on arXiv

Source-linked AI summary

SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs

Dachuan Shi, Abedelkadir Asi, Keying Li, Xiangchi Yuan, Leyan Pan, Wenke Lee, Wen Xiao

arXiv:2510.05069v3cs.CLcs.AI

TL;DR

Explicit reasoning collapses uncertainty into one discrete trajectory, while latent reasoning can diffuse probability mass and sustain costly exploration. SWIREASONING alternates between explicit and latent thinking using entropy-based confidence and limits switches; it improves accuracy and token efficiency across evaluated settings.

  • Problem

    Explicit chain-of-thought collapses the full token distribution into one trajectory, while training-free latent reasoning preserves alternatives but can diffuse probability mass and impede convergence.

  • Method

    SWIREASONING dynamically switches between explicit and latent reasoning using block-wise entropy trends, while a switch-count controller limits transitions and enables early answering.

  • Results

    +1.92% average pass@1 accuracy over standard CoT at 32B scale, with additional gains across coding, multi-hop QA, and commonsense reasoning tasks.

  • Takeaways & Limitations

    Confidence-aware switching combines latent exploration with explicit-path consolidation, while switch limits curb overthinking and improve token efficiency.

Abstract

from arXiv · show

Recent work shows that, beyond discrete reasoning through explicit chain-of-thought steps, which are limited by the boundaries of natural languages, large language models (LLMs) can also reason continuously in latent space, allowing richer information per step and thereby improving token efficiency. Despite this promise, latent reasoning still faces two challenges, especially in training-free settings: 1) purely latent reasoning broadens the search distribution by maintaining multiple implicit paths, which diffuses probability mass, introduces noise, and impedes convergence to a single high-confidence solution, thereby hurting accuracy; and 2) overthinking persists even without explicit text, wasting tokens and degrading efficiency. To address these issues, we introduce SwiReasoning, a training-free framework for LLM reasoning which features two key innovations: 1) SwiReasoning dynamically switches between explicit and latent reasoning, guided by block-wise confidence estimated from entropy trends in next-token distributions, to balance exploration and exploitation and promote timely convergence. 2) By limiting the maximum number of thinking-block switches, SwiReasoning curbs overthinking and improves token efficiency across varying problem difficulties. On widely used mathematics, STEM, coding, and general benchmarks, SwiReasoning consistently improves average accuracy by 1.8%-3.1% across reasoning LLMs of different model families and scales. Furthermore, under constrained budgets, SwiReasoning improves average token efficiency by 57%-79%, with larger gains as budgets tighten.

1 INTRODUCTION

SWIREASONING addresses limitations of explicit and purely latent reasoning with a training-free framework that switches modes using confidence signals and limits transitions to improve accuracy and efficiency.

  • 1 INTRODUCTION: Explicit chain-of-thought collapses each next-token distribution into one discrete trajectory, discarding uncertainty and potentially useful reasoning paths.This motivates retaining richer distributional information during reasoning.
  • 1 INTRODUCTION: Training-free latent reasoning avoids costly retraining and preserves multiple implicit hypotheses, but distributional mismatch can cause drift and diffuse probability mass across paths.These issues can impede convergence and hurt accuracy.
  • 1 INTRODUCTION: SWIREASONING reintroduces diversity through explicit-block sampling while switching modes rather than injecting randomness into latent distributions.The authors position this as distinct from related stochasticity-based approaches.
  • 1 INTRODUCTION: SWIREASONING dynamically alternates between explicit and latent thinking using confidence signals, combining latent expressivity with explicit stability.The framework uses block-wise confidence inferred from entropy trends to guide switching.
  • 1 INTRODUCTION: The framework caps thinking-block transitions, enabling early answers from partial reasoning trajectories and suppressing overthinking under limited budgets.This switch-count control is intended to improve token efficiency across problem difficulties.
  • 1 INTRODUCTION: The framework is validated across mathematics, STEM, coding, and general reasoning benchmarks, model families, and model sizes, with gains in accuracy and token efficiency over training-free baselines.The supplied contribution statement describes broad empirical validation but does not provide aggregate values here.

2 RELATED WORK

Prior work studies explicit reasoning through text and latent reasoning through continuous representations, with latent methods divided into training-required and training-free approaches.

  • 2 RELATED WORK: Explicit reasoning methods use intermediate text, including chain-of-thought, self-consistency, and search- or tool-augmented variants, to expand or aggregate reasoning trajectories.The passage identifies these as major directions in explicit LLM reasoning.
  • 2 RELATED WORK: Latent reasoning operates in continuous representation space and includes both training-required methods that supervise hidden trajectories and training-free methods that avoid retraining.This categorization frames SWIREASONING as a training-free latent-reasoning approach.

3 METHODOLOGY

SWIREASONING combines explicit and latent inference, switches modes from entropy-based confidence trends, and bounds transitions to control overthinking and token use.

  • 3.1 SWIREASONING OVERVIEW: SWIREASONING alternates between explicit and latent reasoning while regulating switch counts to suppress overthinking and improve token efficiency.The overview separates dynamic switching, switch-count control, and their implementation details.
  • 3.2 PRELIMINARY: EXPLICIT AND TRAINING-FREE LATENT THINKING: Explicit decoding samples discrete tokens from the predictive distribution, externalizing reasoning but discarding probability information beyond each chosen token.The reasoning trace is followed by answer-token decoding after a termination condition.
  • 3.2 PRELIMINARY: EXPLICIT AND TRAINING-FREE LATENT THINKING: Latent decoding replaces the hard token policy with a probability-weighted soft embedding fed back as the next input, preserving first-order uncertainty.The soft embedding remains within the token-embedding hull, reducing information discards and increasing robustness to local noise.
  • 3.3 DYNAMIC SWITCH BETWEEN EXPLICIT AND LATENT THINKING: The mode-switch criterion compares current block entropy with reference entropy: lower uncertainty favors explicit convergence, whereas higher uncertainty favors latent exploration after a dwell window.The reference entropy is initialized and refreshed at switches; asymmetric dwell windows avoid oscillations.
  • 3.3 DYNAMIC SWITCH BETWEEN EXPLICIT AND LATENT THINKING: Signal-token mixing biases block entry toward thinking and block exit toward ending thinking, aligning switches with the model’s learned reasoning patterns.The supplied passages describe schedules for switch-time mixing coefficients.
  • 3.4 OVERTHINKING SUPPRESSION BY SWITCH COUNT CONTROL: A switch counter creates convergence and termination triggers that force thinking termination or inject a concise answer prefix after the configured transition budget.Injection queues deterministically overwrite future tokens, and termination allows at most B additional answer tokens.

4 EXPERIMENTS

Experiments across multiple reasoning models, benchmarks, and domains show that SWIREASONING improves accuracy and token efficiency over single-mode baselines, with stronger accuracy gains on difficult tasks and efficiency gains under constrained budgets.

  • Reasoning Accuracy Under Unlimited Token Budgets: On AIME24/AIME25, accuracy gains reach 3.34%/2.50% for Qwen3-8B and 5.00%/5.00% for Qwen3-1.7B, exceeding gains on easier benchmarks.The results associate larger improvements with benchmarks requiring deeper deductive reasoning.
  • Reasoning Accuracy Under Unlimited Token Budgets: +2.17% average accuracy improvement over CoT baselines is reported on mathematics and STEM benchmarks.Table 1 compares SWIREASONING with CoT sampling, CoT greedy decoding, and Soft Thinking.
  • Token Efficiency Under Limited Budgets: SWIREASONING achieves peak efficiency gains of 4.6×–6.8× over CoT across models, with the advantage persisting across small to moderate budgets.The reported Pareto-frontier advantage spans more than a single token budget.
  • Pass@k Accuracy: SWIREASONING reaches peak Pass@k accuracy with 13 versus 46 samples on AIME24 and 16 versus 22 on AIME25, compared with CoT.These correspond to about 72% and 27% fewer samples, respectively.
  • Maximum Switch Count: Reducing the maximum switch count improves token efficiency by curbing prolonged latent exploration, while difficult problems benefit from allowing more switches.A small number of confidence-aware blocks usually suffices for easy-to-moderate problems.
  • Larger Models and Broader Domains: SWIREASONING improves average accuracy by +1.92% on Qwen3-32B and +2.70% across coding, multi-hop QA, and commonsense benchmarks.On broader domains, the largest coding gain is +18.18% on the hard-level subset; multi-hop QA and commonsense gains are +2.50% and +1.39%.

5 CONCLUSION

SWIREASONING is a training-free inference framework that alternates explicit and latent thinking using entropy-based confidence signals and limits switches to curb overthinking. These mechanisms improve accuracy and token efficiency, yielding stronger Pareto frontiers across reasoning budgets.

  • SWIREASONING consistently improves maximum accuracy under unlimited budgets and token efficiency under limited budgets, producing improved Pareto frontiers for reasoning LLMs.The framework combines entropy-trend mode switching with a switch-count controller.

6 ETHICS STATEMENT

The paper reports no safety concerns or sensitive-data collection and uses publicly available models and benchmarks under their original licenses. The experiments use a training-free implementation without model or architecture changes.

  • The work reports no safety concerns and no collection of sensitive data.
  • All evaluations use publicly available models and benchmarks under their original licenses.

B.2 BENCHMARK DETAILS

The paper evaluates SWIREASONING across mathematics, STEM, coding, and general reasoning benchmarks using multiple models and extensive efficiency analyses. Results show consistent accuracy and efficiency benefits, especially under tighter budgets.

  • SWIREASONING consistently improves accuracy and token efficiency across mathematics, STEM, coding, and general reasoning benchmarks, model families, and sizes.The evaluation covers 11 reasoning benchmarks.
  • Latent-mode token distributions have 2.08× higher average entropy than explicit mode on Math500 with Qwen3-8B, indicating less concentration.The reported averages are 0.444 in latent mode versus 0.213 in explicit mode.
  • Latent mode also produces more semantically diverse token candidates, complementing its higher-entropy exploration.The figure analyzes average pairwise similarity among top-5 token candidates.
  • At 90% pass@1 accuracy, SWIREASONING uses 1.36× fewer TFLOPs and is 1.36× faster than CoT on MATH500 with Qwen3-8B.At 80% accuracy, the framework uses 2.18× fewer TFLOPs than CoT; the advantage grows as budgets tighten.
  • Switching adapts to difficulty: AIME 2024 and AIME 2025 have median switch counts of 44–69 and maximum counts reaching 151.

C.4 ANALYSIS OF CHALLENGES IN SPECIFIC SCENARIOS

The analyzed gecko problem illustrates a limitation of latent-based reasoning on spatial tasks with rigid topological constraints. Although SWIREASONING identifies room unfolding as the high-level strategy, latent transitions can blur constraints and produce invalid paths.

  • The shortest-path setup requires unfolding the left wall, ceiling, and right wall so the surface path becomes a straight line in a 2D plane.The gecko starts at (y = 1, z = 7) on the left wall, while the fly is at (y = 9, z = 1) on the right wall.
  • SWIREASONING may struggle on spatial reasoning tasks requiring rigid geometric constraint satisfaction.The paper identifies these tasks as a specific challenging category for latent-based methods.
  • The gecko example correctly identifies unfolding the 3D room into a 2D plane but fails to preserve the restriction to ceiling and wall surfaces.The reasoning trace may instead calculate a direct Euclidean path through prohibited space.
  • Latent smoothing supports semantic exploration but can blur exact topological relationships, such as which wall edges connect.This trade-off can interfere with precise geometric execution after switching back to explicit reasoning.

C.5 ABLATION STUDY ON THE CONVERGENCE TRIGGERS W.R.T. TERMINATION ONES

The ablation shows that convergence timing trades accuracy against token efficiency, with the default 1/2T trigger providing the intended balance.

  • C.5 ABLATION STUDY ON THE CONVERGENCE TRIGGERS W.R.T. TERMINATION ONES: The default 1/2T convergence trigger balances accuracy and token efficiency for general-purpose reasoning under budget constraints.It avoids the accuracy loss of earlier triggers while using fewer tokens than later triggers.
  • C.5 ABLATION STUDY ON THE CONVERGENCE TRIGGERS W.R.T. TERMINATION ONES: At Cmax = 20, moving convergence from 1/2T to 1/4T reduces accuracy from 87.80% to 84.00%.Earlier convergence shortens generation but can prevent sufficient exploration on harder problems.
  • C.5 ABLATION STUDY ON THE CONVERGENCE TRIGGERS W.R.T. TERMINATION ONES: +1.0% accuracy from 1/2T to 3/4T at Cmax = 20 comes with increased token consumption.Later convergence offers only marginal accuracy gains while reducing efficiency.

C.6 ABLATION STUDY ON ISOLATING THE EFFECT OF LATENT UNDER SMALL BUDGETS

This ablation isolates latent reasoning under small token budgets by comparing full SwiReasoning with an explicit-only variant that retains the switch controller.

  • C.6 ABLATION STUDY ON ISOLATING THE EFFECT OF LATENT UNDER SMALL BUDGETS: +3.24% average accuracy: full SwiReasoning outperforms the explicit-only ablation under lower or equal token usage.The comparison spans all evaluated benchmarks under small token budgets.
  • C.6 ABLATION STUDY ON ISOLATING THE EFFECT OF LATENT UNDER SMALL BUDGETS: +4.16% accuracy on AIME2025 highlights the larger contribution of latent reasoning on a challenging benchmark.Disabling latent reasoning consistently degrades performance across benchmarks.

C.7 SUPPLEMENTARY COMPONENT-WISE ABLATION STUDY

The component ablations support asymmetric switching windows, signal mixing, and switch-count control as contributors to performance and efficiency.

  • C.7 SUPPLEMENTARY COMPONENT-WISE ABLATION STUDY: 91.92% versus 89.10% average accuracy: removing the window constraint causes a significant performance drop.Rapid mode oscillations prevent coherent reasoning chains from forming.
  • C.7 SUPPLEMENTARY COMPONENT-WISE ABLATION STUDY: The default asymmetric window outperforms all symmetric configurations.The design uses an explicit consolidation period and immediate latent exit upon confidence recovery.
  • C.7 SUPPLEMENTARY COMPONENT-WISE ABLATION STUDY: 91.92% versus 91.32% average accuracy: disabling signal mixing consistently decreases accuracy.The ablation attributes the difference to improved internal-state orientation during mode transitions.
  • C.7 SUPPLEMENTARY COMPONENT-WISE ABLATION STUDY: Generation length falls from 4924 to 1189 as Cmax decreases from unlimited to 4.Moderate caps such as 20–32 remain competitive with the unlimited setting while consuming fewer tokens.

C.8 DETAILED EVALUATION RESULTS UNDER VARYING TOKEN BUDGETS

The detailed evaluations report results for three model configurations across five benchmarks under varying token budgets.

  • C.8 DETAILED EVALUATION RESULTS UNDER VARYING TOKEN BUDGETS: The supplementary tables sort rows by generation length in descending order for each model and benchmark.This layout exposes accuracy behavior across progressively varying token budgets.
Loading 2510.05069v3…