PaperlayerView exact revision on arXiv

Source-linked AI summary

Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models

Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, Hanjie Chen, Xia Hu

arXiv:2503.16419v4cs.CL

TL;DR

Long Chain-of-Thought reasoning can improve complex-task performance but may generate excessive, redundant steps that increase computational cost and latency. This survey systematically organizes efficient-reasoning research across model, reasoning-output, and input-prompt approaches, alongside data, small-model, evaluation, and benchmarking topics. It also covers reported evaluation evidence, including a mitigation that improved performance by 30% while reducing computational overhead by 43%.

  • Problem

    Long Chain-of-Thought reasoning can produce excessive and redundant steps, increasing computational cost and latency and limiting deployment in computation-sensitive settings.

  • Method

    The paper presents a structured survey categorizing efficient reasoning into model-based, reasoning output-based, and input prompts-based approaches, with related coverage of data, small models, evaluation, and benchmarking.

  • Results

    A surveyed mitigation using lower overthinking scores improved performance by 30% while reducing computational overhead by 43%.

  • Takeaways & Limitations

    Efficient reasoning research targets concise reasoning for lower-cost, lower-latency deployment while retaining reasoning capabilities across practical applications.

  • Takeaways & Limitations

    Distilling complex reasoning into small language models faces a Small Model Learnability Gap, while compression alone may not ensure instruction following.

Abstract

from arXiv · show

Large Language Models (LLMs) have demonstrated remarkable capabilities in complex tasks. Recent advancements in Large Reasoning Models (LRMs), such as OpenAI o1 and DeepSeek-R1, have further improved performance in System-2 reasoning domains like mathematics and programming by harnessing supervised fine-tuning (SFT) and reinforcement learning (RL) techniques to enhance the Chain-of-Thought (CoT) reasoning. However, while longer CoT reasoning sequences improve performance, they also introduce significant computational overhead due to verbose and redundant outputs, known as the "overthinking phenomenon". In this paper, we provide the first structured survey to systematically investigate and explore the current progress toward achieving efficient reasoning in LLMs. Overall, relying on the inherent mechanism of LLMs, we categorize existing works into several key directions: (1) model-based efficient reasoning, which considers optimizing full-length reasoning models into more concise reasoning models or directly training efficient reasoning models; (2) reasoning output-based efficient reasoning, which aims to dynamically reduce reasoning steps and length during inference; (3) input prompts-based efficient reasoning, which seeks to enhance reasoning efficiency based on input prompt properties such as difficulty or length control. Additionally, we introduce the use of efficient data for training reasoning models, explore the reasoning capabilities of small language models, and discuss evaluation methods and benchmarking. Project website: https://github.com/Eclipsess/Awesome-Efficient-Reasoning-LLMs

1 Introduction

The survey addresses the overthinking phenomenon, where lengthy and redundant Chain-of-Thought reasoning increases inference cost and latency, by organizing efficient-reasoning research into three directions and related topics.

  • Motivation: Long Chain-of-Thought reasoning can improve accuracy but also produces redundant steps, increasing computational overhead, latency, and deployment difficulty.The problem is especially acute when smaller reasoning models generate sequences spanning thousands of tokens.
  • Motivation: Efficient reasoning reduces reasoning length and unnecessary thinking steps to lower costs while preserving accuracy for practical deployments.The survey distinguishes efficient reasoning from model-compression methods that primarily reduce model size.
  • Survey scope: The survey organizes existing work into model-based, reasoning output-based, and input prompts-based efficient reasoning.These directions respectively optimize or train models, dynamically shorten inference reasoning, and use prompt properties such as difficulty or length control.
  • Model-based methods: Model-based approaches include reinforcement learning with length-based rewards and supervised fine-tuning with variable-length Chain-of-Thought data.The listed methods cover both reward design and data-driven fine-tuning strategies.
  • Reasoning output-based methods: Reasoning output-based approaches include compressing reasoning into fewer latent representations and dynamically adjusting reasoning during inference.These methods target the generated reasoning process rather than only the underlying model.
  • Input and related topics: Input prompt-based approaches include prompt-guided efficient reasoning and routing prompts to optimize reasoning efficiency.The survey also covers efficient training data, small-model reasoning and compression, and evaluation and benchmarking.

2 Background: Long CoT Reasoning Models and Overthinking Phenomenon

CoT-based reasoning models generate structured intermediate steps to improve reasoning, but excessive or redundant thinking can increase cost, latency, and errors. The survey frames efficient reasoning as a response to this overthinking problem while noting that training objectives often encourage longer reasoning.

  • Chain-of-Thought Reasoning: CoT prompting guides models to generate explicit, step-by-step reasoning sequences before producing a final answer.
  • Large Reasoning Models: Reasoning-capable models internalize iterative generation and refinement through extensive training rather than relying only on explicit test-time prompting.
  • Overthinking Phenomenon: The overthinking phenomenon occurs when models continue producing unnecessarily elaborate reasoning after reaching a correct answer, reducing efficiency.
  • Overthinking Phenomenon: Long reasoning sequences create substantial computational overhead and increased reasoning time, limiting deployment in computation-sensitive applications.
  • Challenge: Inference efficiency is difficult because reasoning-model training recipes often encourage extended reasoning, with longer responses sometimes correlating with improved benchmark performance.

3 Model-based Efficient Reasoning

Model-based efficient reasoning makes LLM reasoning more concise by modifying training objectives, reasoning data, or model parameters. The surveyed approaches use length-aware RL, variable-length CoT supervision, fine-tuning, and model merging to reduce unnecessary reasoning while preserving capabilities.

  • 3.1 RL with Length Reward Design: Length-aware RL rewards short, correct answers more highly and penalizes lengthy or incorrect reasoning paths.This directly targets overthinking by optimizing the length of the reasoning path.
  • 3.1 RL with Length Reward Design: RL-based length-control methods mitigate overthinking and can achieve nearly lossless alignment with original reasoning capabilities under computational budgets.The paper frames this as improving deployment in resource-constrained scenarios.
  • 3.2 SFT with Variable-Length CoT Data: Variable-length CoT SFT constructs long and short reasoning data, then trains models to learn compact reasoning chains that preserve effective knowledge.Data can be created after full reasoning by compression or during reasoning through prompts and sampling.
  • 3 Model-based Efficient Reasoning: Standard and progressive fine-tuning adapt LLMs to shorter reasoning, while model merging transforms lengthy-CoT models into more concise variants.Standard methods include LoRA or full fine-tuning; progressive methods gradually reduce reasoning steps.

4 Reasoning Output-based Efficient Reasoning

Reasoning output-based methods improve inference efficiency by changing how models generate, select, terminate, or represent reasoning. The survey covers latent reasoning, dynamic criteria, and optimized test-time scaling to reduce unnecessary computation.

  • 4.1 Compressing Reasoning Steps into Fewer Latent Representations: Latent-reasoning methods replace explicit CoT steps with compact hidden-state or thinking-token representations.These approaches include training the LLM to reuse latent representations and augmenting a frozen LLM with auxiliary soft thought tokens.
  • 4.1 Compressing Reasoning Steps into Fewer Latent Representations: SoftCoT keeps the underlying LLM frozen while an auxiliary model generates instance-specific soft thought tokens in its embedding space.Experiments report consistent performance improvements, supporting auxiliary latent reasoning tokens.
  • 4.1 Compressing Reasoning Steps into Fewer Latent Representations: Latent reasoning is associated with improved speed, adaptive inference, parallel backtracking, and partial interpretability through compressed non-textual reasoning.The paper presents these as broader possibilities of compact latent CoT paradigms.
  • 4.2 Dynamic Reasoning Paradigm during Inference: Dynamic reasoning selects inference strategies using reward, confidence, consistency, or summarization criteria.The central design question is which criterion should guide efficient inference.
  • 4.2 Dynamic Reasoning Paradigm during Inference: Test-time scaling improves difficult math and code reasoning by allocating more inference computation, but generating many responses remains expensive.Efficient methods therefore optimize Best-of-N sampling and other test-time scaling procedures.
  • 4.2 Dynamic Reasoning Paradigm during Inference: Speculative Rejection filters low-quality sampled outputs with a reward model, while reward-guided speculation accepts strong draft outputs and refines weaker ones with a larger model.Both methods reduce the computational overhead of inference-time reasoning.
  • 4.2 Dynamic Reasoning Paradigm during Inference: Confidence-based methods terminate reasoning when further steps are unlikely to change the answer, dynamically reallocating resources toward harder queries.DPTS also adjusts search expansion and memory use according to available GPU resources.
  • 4.2 Dynamic Reasoning Paradigm during Inference: ST-BoN reduces Best-of-N computation by early-terminating paths according to consistency among latent embeddings rather than reward-model scores.The most consistent thinking path is treated as more likely to reach the correct answer.

5 Input Prompts-based Efficient Reasoning

Input-prompt methods control reasoning efficiency through explicit length constraints or query-level routing. They estimate suitable reasoning budgets, study accuracy–compression trade-offs, and assign queries to models according to complexity or uncertainty.

  • 5.1 Prompt-guided Efficient Reasoning: Prompt-guided efficient reasoning explicitly instructs models to generate fewer reasoning steps through concise-output prompts.The section asks which prompts can accurately control reasoning length.
  • 5.1 Prompt-guided Efficient Reasoning: Token-Budget uses prompting to estimate a reasonable token budget, then incorporates that estimate into a constrained reasoning prompt.TALE-EP performs this estimation training-free and zero-shot.
  • 5.1 Prompt-guided Efficient Reasoning: Prompt-based compression studies identify a universal trade-off between reasoning length and accuracy across different compression instructions.The analysis hypothesizes an intrinsic token complexity for each task.
  • 5.1 Prompt-guided Efficient Reasoning: Short-CoT data collected through prompting can be followed by SFT, producing efficient reasoning models that often perform more promisingly on complex reasoning challenges.This combines prompt-based data construction with model training.
  • 5.2 Prompts Attribute-Driven Reasoning Routing: Routing strategies assign simpler queries to faster weaker models and complicated queries to slower stronger models based on prompt complexity or uncertainty.The goal is query-level efficiency rather than uniform allocation of reasoning resources.
  • 5.2 Prompts Attribute-Driven Reasoning Routing: Hybrid reasoning models can expose quick-answer and step-by-step modes while allowing users to control the time spent thinking, although the routing criterion may remain unspecified.Claude 3.7 Sonnet is presented as an example of this approach.
  • 5.2 Prompts Attribute-Driven Reasoning Routing: RouteLLM trains a query router on preference data to dispatch simple questions to low-latency models and complex questions to more powerful models.The paper reports significantly accelerated overall reasoning efficiency.
  • 5.2 Prompts Attribute-Driven Reasoning Routing: Self-Ref uses intrinsic uncertainty scores to let an LLM decide whether to answer confidently or route a query to a stronger model.This supports more efficient and reliable query-level decisions in online LLM services.

6 Reasoning Abilities via Efficient Training Data and Model Compression

Efficient reasoning also depends on selecting training data carefully and transferring reasoning ability to smaller models. The surveyed evidence highlights minimal high-quality datasets, self-verification signals, distillation challenges, and differing effects of quantization versus pruning.

  • 6.1 Efficient Training Data: Efficient training-data methods seek to reduce data requirements while maintaining or improving reasoning performance through careful selection and structuring.The key question is how to construct less but high-quality training data.
  • 6.1 Efficient Training Data: LIMO elicits sophisticated reasoning from minimal curated examples selected for difficulty, generality, knowledge diversity, structural organization, scaffolding, and verification.Its selection criteria cover both questions and solutions.
  • 6.1 Efficient Training Data: S2R uses supervised initialization followed by outcome- and process-level RL to teach self-verification and self-correction.The method reports consistent improvements from 3,100 initialization samples across base models.
  • 6.2 Reasoning Capabilities of Small Language Models: Small Language Models offer a resource-efficient alternative for edge, mobile, and real-time settings, with distillation and compression as the main approaches.The section examines whether SLMs retain reasoning under strict computational and memory constraints.
  • 6.2 Reasoning Capabilities of Small Language Models: The Small Model Learnability Gap describes SLMs’ difficulty emulating the reasoning depth of larger models through distillation.Mixed distillation and counterfactual distillation are proposed responses to this challenge.
  • 6.2 Reasoning Capabilities of Small Language Models: Quantization preserves reasoning performance remarkably well, whereas pruning severely degrades multi-step logical reasoning quality.Compression can retain inherited reasoning structures, but instruction following may still require additional fine-tuning or adaptation.

7 Evaluation and Benchmark

The survey highlights standardized evaluation of reasoning ability, inference-time computation, overthinking, long CoT effects, and compression. Findings show that evaluation outcomes depend on strategy and that shorter reasoning can improve performance while reducing overhead.

  • Evaluation Frameworks: Sys2Bench evaluates LLMs across arithmetic, logical, commonsense, algorithmic, and planning reasoning categories using eleven datasets.The benchmark includes datasets such as GSM8K, AQuA, StrategyQA, HotPotQA, and ProntoQA.
  • Inference-time Computation: Compute-optimal test-time scaling depends on policy models, process reward models, and problem difficulty.The survey emphasizes that test-time scaling effectiveness is sensitive to these factors.
  • Evaluating Overthinking: An overthinking framework analyzes 4,018 agentic-task trajectories and links higher overthinking scores to decreased task performance.Selecting lower-overthinking solutions improved performance by 30% while reducing computational overhead by 43%.
  • Effect of Long CoT Reasoning: Longer reasoning chains often improve model outputs, while reward design can reduce CoT length and stabilize reasoning ability during training.The survey presents these findings as complementary evidence about the performance–length relationship.
  • Effect of Compression on Reasoning Models: CompressionReasoning finds that parameter count affects knowledge retention more than reasoning ability, while shorter outputs generally improve performance.The benchmark covers quantization, distillation, and pruning techniques.

8 Applications and Discussion

The survey discusses efficient reasoning across autonomous driving, embodied AI, healthcare, recommender systems, agentic AI, reasoning improvement, and safety. It also notes trade-offs among efficiency, reasoning quality, safety, and training-method control.

  • Applications: Efficient reasoning can help autonomous vehicles process multimodal sensor data, make decisions in difficult situations, and react quickly to unexpected events.The cited applications combine cameras, LiDAR, radar, and other sensors.
  • Applications: Efficient reasoning supports embodied AI by helping robots and smart devices interpret sensor inputs, navigate obstacles, and adapt to changing situations.The passage also associates these capabilities with reliability, safety, and usefulness in daily environments.
  • Improving Reasoning Ability: Meta-Reasoner selects guidance strategies with contextual multi-armed bandits, while ITT allocates extra processing to difficult tokens through adaptive routing.SyzygyoT instead decomposes complex tasks into logically complete and minimal subproblems using Minimal Free Resolution.
  • Safety of Efficient Reasoning: Safety and efficiency can conflict because stronger safety checks may require longer reasoning, whereas minimizing computation may reduce self-reflection and verification.The survey frames this as a trade-off requiring balance rather than a universally optimal setting.
  • Efficient LLMs for Agentic AI: Combining multiple planning trees and optimizing internal reasoning processes are presented as approaches for improving agentic AI efficiency.The stated targets include decision-making speed, resource utilization, and effectiveness in real-world applications.
  • Discussion: RL offers trial-and-error adaptability but can be unpredictable and costly, whereas SFT provides more consistent control but may struggle outside its training examples.The survey identifies combining RL and SFT as a possible direction.

9 Conclusion

The paper concludes that efficient reasoning is organized around model-based, reasoning output-based, and input prompts-based methods. It also covers efficient data, small language models, evaluation, benchmarking, and practical applications.

  • Conclusion: The survey categorizes efficient reasoning approaches into model-based, reasoning output-based, and input prompts-based methods.It additionally discusses efficient data utilization, smaller reasoning models, evaluation techniques, and benchmarking.
  • Conclusion: The survey presents efficient reasoning as practically relevant to healthcare diagnostics, autonomous driving, embodied AI, and financial algorithmic trading.The paper connects these applications with reduced costs, faster decisions, safety, reliability, and risk assessment.
Loading 2503.16419v4…