Source-linked AI summary
Efficient Reasoning Models: A Survey
Sicheng Feng, Gongfan Fang, Xinyin Ma, Xinchao Wang
TL;DR
Reasoning models achieve strong performance through extended Chain-of-Thought generation, but this creates substantial computational overhead. The survey synthesizes efficient reasoning research into shorter, smaller, and faster approaches, and discusses evaluation, safety, sustainability, and deployment implications.
Problem
Extended reasoning chains, large model sizes, and decoding strategies create substantial computational, memory, and latency overhead for reasoning models.
Method
The survey categorizes efficient reasoning research into shorter reasoning chains, smaller reasoning-capable models, and faster decoding strategies.
Results
Efficient reasoning methods are associated with reduced computational costs and resource usage, while token count closely tracks latency with a Pearson correlation coefficient of 0.9998 on Qwen2.5-7B using MAHT-500.
Takeaways & Limitations
The survey presents efficient reasoning as a foundation for scalable and practical deployment across real-time systems and resource-constrained environments.
Takeaways & Limitations
Safety and efficiency remain in tension because shorter reasoning may skip critical safety checks, while longer reasoning can help self-correction.
Abstract
from arXiv · showhide
Reasoning models have demonstrated remarkable progress in solving complex and logic-intensive tasks by generating extended Chain-of-Thoughts (CoTs) prior to arriving at a final answer. Yet, the emergence of this "slow-thinking" paradigm, with numerous tokens generated in sequence, inevitably introduces substantial computational overhead. To this end, it highlights an urgent need for effective acceleration. This survey aims to provide a comprehensive overview of recent advances in efficient reasoning. It categorizes existing works into three key directions: (1) shorter - compressing lengthy CoTs into concise yet effective reasoning chains; (2) smaller - developing compact language models with strong reasoning capabilities through techniques such as knowledge distillation, other model compression techniques, and reinforcement learning; and (3) faster - designing efficient decoding strategies to accelerate inference of reasoning models. A curated collection of papers discussed in this survey is available in our GitHub repository: https://github.com/fscdc/Awesome-Efficient-Reasoning-Models.
1 Introduction
The survey frames efficient reasoning as a response to redundancy in reasoning length, model size, and decoding, organizing methods into shorter, smaller, and faster directions.
- Long Chain-of-Thought reasoning improves complex-task performance but creates substantial overhead and redundancy compared with standard language models.
- Shorter: Shorter methods address overthinking and the finding that longer reasoning can produce negative returns, including reduced performance when excessively verbose.
- Smaller: Smaller methods apply compression techniques such as distillation, quantization, and pruning to reduce the costs of large reasoning models.
- Faster: Faster methods target decoding-stage inefficiency through strategies such as test-time-scaling optimization, parallel decoding, and problem decomposition.
- The survey categorizes research by whether it shortens reasoning paths, builds compact reasoning-capable models, or reduces decoding latency.
2 Background
The background introduces Chain-of-Thought reasoning, model training and compression techniques, and test-time computation before motivating efficiency through excessive tokens, model scale, and decoding demands.
- Chain-of-Thought prompting generates intermediate steps before the final answer and improves performance on complex reasoning tasks.
- Recent reasoning models internalize step-by-step reasoning through supervised fine-tuning and reinforcement learning using correctness, format, and process-based rewards.
- Test-time computation improves reasoning by generating multiple samples or scaling computation across complementary horizontal and vertical dimensions.
- Model compression reduces computational overhead through quantization, which lowers parameter precision, and pruning, which removes less significant parameters.
- DeepSeek R1-685B uses 619 tokens to answer “What is the answer of 1 plus 2?”, illustrating excessive reasoning length even on simple tasks.
- Efficient reasoning supports real-time processing and decision-making in embodied AI, agent systems, and autonomous-driving platforms.
3 Efficient Reasoning
The survey organizes efficient reasoning around shorter chains, smaller capable models, and faster decoding, reviewing methods that target each form of redundancy.
- Efficient reasoning is divided into shortening long CoTs, building small reasoning-capable models, and improving decoding efficiency.These categories correspond to chain-length, model-size, and decoding-stage redundancy.
- Shorter: Length-reduction methods include RL with penalties, variable-length-CoT SFT, prompt-driven techniques, and latent reasoning.The survey also discusses model merging and prompt-based routing as related approaches.
- Shorter: RL-based methods reward concise reasoning while preserving accuracy, with some approaches dynamically allocating more reasoning steps to harder questions.O1-Pruner uses length and accuracy baselines, whereas DAST introduces a Token Length Budget to quantify difficulty.
- Smaller: Small language models improve reasoning efficiency through compression, distillation, and reinforcement learning, reducing memory and computational requirements.The survey notes that progressively refined or synthesized training data tends to produce greater performance improvements.
- Smaller: Small models can match or surpass larger models under suitable test-time scaling, but effectiveness varies with architecture, reward design, and task complexity.The survey also identifies instruction following and self-reflection as adaptation challenges for small models.
- Smaller: Compression results indicate that quantization often preserves reasoning better than pruning, while aggressive pruning or low-bit quantization can substantially degrade performance.High-bit quantization is described as nearly lossless, whereas 4-bit settings especially harm complex-task performance.
3.3 Let Decoding More Efficient
Decoding-efficiency research reduces the overhead of test-time scaling and other reasoning procedures through adaptive sampling, self-consistency, decomposition, and routing.
- Evaluation and scope: Many decoding methods achieve efficiency gains and can improve model performance without additional training, although results may be skewed by concentration on Qwen2.5 models.The survey identifies Qwen2.5's instruction-following and self-reflection abilities as potential sources of skew.
- Efficient decoding: Decoding methods target test-time-scaling overhead through efficient sampling and efficient self-consistency techniques.The survey frames both directions as adaptive allocation of computation during inference.
- Efficient sampling: Efficient sampling terminates low-quality candidates early using partial rewards, early consistency, or structured path evaluation.These methods aim to avoid redundant computation spent on underperforming reasoning chains.
- Efficient self-consistency: Efficient self-consistency stops sampling when confidence, unanimity, or sufficient high-quality evidence is reached.Difficulty awareness and score-weighted voting further adjust sample budgets or select final answers.
- Other faster methods: Problem decomposition reduces redundant token generation by solving simplified sub-problems and merging their results.AoT models dependencies in a DAG and iteratively decomposes and merges sub-problems.
- Other faster methods: Compute-optimal scaling estimates prompt difficulty and adaptively selects among test-time-scaling strategies.This supports detailed reasoning for complex inputs and concise responses for simpler ones.
4 Evaluation and Benchmark
The survey evaluates reasoning efficiency across computational costs, model performance, and resource use, while emphasizing the trade-off between accuracy and efficiency. It reviews general metrics, reasoning-specific metrics, composite measures, datasets, and benchmarks.
- General efficiency metrics: Efficiency evaluation extends beyond accuracy to token count, model size, inference latency, memory, computation, power, and carbon emissions.These metrics capture computational cost and resource demands alongside model capability.
- Reasoning performance metrics: Accuracy variants measure deterministic performance, stability across runs, potential success, and agreement among repeated outputs.Examples include greedy accuracy, minimummaximum spread, Pass@k, Pass∧k, G-Pass@kτ, mG-Pass@kτ, and TAR@N.
- Reasoning-specific metrics: Reasoning-specific measures quantify overthinking, reasoning boundaries, underthinking, and the minimum tokens needed to preserve correctness.These measures target inefficiencies and capability limits that conventional resource metrics may not capture.
- Unified metrics: Composite metrics jointly assess performance and efficiency instead of reporting accuracy and resource costs separately.ACU combines accuracy with parameter count and token count, while AES considers solution length and accuracy and penalizes accuracy degradation more than improvement.
- Datasets and benchmarks: Datasets and benchmarks span mathematical, logical, commonsense, algorithmic, planning, and multi-hop reasoning under diverse resource constraints.Specialized suites also assess overthinking, test-time optimization, and over-reasoning behaviors.
5 Discussions and Future Directions
The discussion identifies open directions for extending efficient reasoning across safety, multimodal systems, memory, training, compression, and sustainability. It also highlights the need to connect intrinsic efficiency measures with deployment-facing costs.
- Safety and efficiency: Efficient reasoning creates a safety-efficiency tension because shorter paths may skip safety checks, whereas longer reasoning can support self-correction.The survey identifies latent reasoning and representation alignment as promising directions for reducing safety risks.
- Multimodal reasoning: Efficient reasoning methods can extend to multimodal models through task decomposition, latent reasoning, and unified discrete-continuous latent language modeling.LatentLM uses a VAE and next-token diffusion for autoregressive multimodal generation.
- Memory limitation: Memory-focused methods erase outdated reasoning steps, segment reasoning into summarized fragments, or otherwise address the long-context burden.PENCIL and INFTYTHINK exemplify progressive erasure and chunk-wise thinking strategies.
- Training efficiency: Curriculum learning and reinforcement-learning optimization are being explored to reduce the computational burden of training long reasoning models.Light-R1 and FASTCURL progressively increase task complexity to facilitate stable learning.
- Traditional model compression: Distillation transfers reasoning abilities from larger models to smaller ones and can also compress explicit CoTs into implicit reasoning paths.The survey identifies reasoning-path granularity, presentation format, and teacher choice as factors affecting CoT distillation.
- Traditional model compression: Quantization successfully maintained reasoning performance while improving efficiency, whereas preliminary pruning experiments were not promising.The reported benefit specifically includes improved memory usage.
- Sustainability: Efficient reasoning can reduce computational costs, resource use, energy requirements, and associated carbon footprint as reasoning models scale.The survey frames this as supporting more environmentally friendly deployment without significant negative effects.
- Intrinsic versus hard metrics: Token count closely tracks latency in the reported Qwen2.5-7B MAHT-500 experiment, with a Pearson correlation coefficient of 0.9998.However, PEFT-based reductions in memory and computation during SFT or RL do not reduce inference-time deployment memory.
6 Conclusion
The survey organizes efficient reasoning into shorter, smaller, and faster approaches addressing reasoning-chain length, model size, and decoding latency. It presents these directions as a foundation for scalable and practical deployment across diverse applications.
- Conclusion: The survey categorizes efficient reasoning into shorter reasoning chains, smaller capable models, and faster decoding.These directions target redundancy in reasoning paths, model size, and the decoding stage, respectively.
- Conclusion: The survey positions efficient reasoning as a foundation for scalable and practical deployment in real-time systems and resource-constrained environments.The stated scope covers diverse applications of reasoning models.
A.1 Details for Model Compression
The survey outlines quantization, pruning, and knowledge distillation as traditional model-compression techniques relevant to efficient reasoning. These methods reduce precision, remove parameters, or transfer behavior from larger teachers to smaller students.
- Quantization: Quantization lowers parameter bit precision to improve model efficiency and reduce memory usage.It includes post-training quantization and quantization-aware training, targeting weights, activations, or both.
- Pruning: Pruning reduces model size and inference latency by eliminating redundant or less important parameters.Unstructured, structured, and semi-structured pruning differ in sparsity structure and hardware compatibility.
- Knowledge distillation: Knowledge distillation transfers a large teacher’s behavior to a smaller student by aligning outputs, representations, or attention patterns.Black-box and white-box variants differ in whether the student accesses only teacher outputs or also internal states.
A.2 Overthinking Example
The survey illustrates overthinking with a simple addition problem: DeepSeek R1 generates a lengthy reasoning response even though the correct answer is short. This demonstrates inefficiency on simple tasks.
- The example shows that current models’ overthinking can create inefficiency when solving simple problems.
- The displayed reasoning repeatedly verifies that 1 plus 2 equals 3 using apples, fingers, a number line, and books.
- The concise answer is simply “The answer is 3.”
A.3 Latency Comparison of Efficient Reasoning Methods
Table 5 provides a practical overview of representative efficient reasoning methods on GSM8K across different categories.
- Table 5 summarizes representative efficient reasoning methods on GSM8K across different categories.It is intended as a practical overview for users.
A.4 Metric Formulas
This appendix defines metrics for evaluating reasoning performance, stability, efficiency, difficulty boundaries, underthinking, and accuracy–length trade-offs. The formulas use quantities such as sampled outputs, correctness, tokens, diversity, and accuracy thresholds.
- A.4.2 Pass@k: Pass@k measures the probability of obtaining at least one correct answer among k outputs.Its formula uses n sampled outputs and c correct ones.
- A.4.3 Pass∧k: Pass∧k measures whether all k generations are correct, while G-Pass@k uses τ to require a minimum correct-response proportion.
- A.4.5 Outcome and Process Efficiency Metric: The outcome efficiency metric uses total tokens, tokens until the first correct answer, and correctness across N instances.σ_i indicates whether at least one solution is correct.
- A.4.5 Outcome and Process Efficiency Metric: The process efficiency metric represents solution diversity through tokens contributing to distinct reasoning strategies.τ_m indicates whether a solution introduces a new reasoning strategy.
- A.4.6 Reasoning Boundary (RB): Reasoning Boundary defines the greatest task difficulty at which model accuracy reaches a predefined threshold K1.The metric takes the supremum over difficulty levels satisfying the accuracy condition.
- A.4.7 Underthinking Metric: The underthinking metric averages, over incorrect responses, the token position through the first correct thought relative to total response tokens.
- A.4.8 Accuracy Efficiency Score: Accuracy Efficiency Score combines baseline–model length reduction with accuracy change using separate penalties for accuracy gains and drops.Its default weights α = 1, β = 3, and γ = 5 penalize accuracy drops more heavily than rewarding improvements.
A.5 Complete List of Datasets and Benchmarks
The appendix provides an organized reference for evaluating efficient reasoning through a complete list of datasets and benchmarks.
- Table 6 presents the full list of datasets and benchmarks used for efficient reasoning evaluation.The list is offered as an organized reference for researchers.