PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models

Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, Wanxiang Che

arXiv:2503.09567v5cs.AIcs.CL

TL;DR

Long CoT lacks a comprehensive survey clarifying its distinction from Short CoT and organizing debates about overthinking and inference-time scaling. This survey defines Long CoT’s core characteristics, synthesizes its paradigms and phenomena, and identifies challenges and future directions. It concludes that deep reasoning, extensive exploration, and feasible reflection provide a clearer account of Long CoT’s advantages and support continued research.

  • Problem

    Long CoT lacks comprehensive systematic coverage distinguishing it from Short CoT and resolving debates about inference-time scaling, overthinking, and the relationship between length and accuracy.

  • Method

    The survey defines Long CoT against Short CoT, organizes reasoning paradigms through a taxonomy, analyzes associated phenomena, and reviews challenges and future research directions.

  • Results

    The survey identifies deep reasoning, extensive exploration, and feasible reflection as core Long CoT features and provides a comprehensive overview of related advances, phenomena, and challenges.

  • Takeaways & Limitations

    The framework clarifies Long CoT’s advantages and offers resources and research frontiers for continued study of logical reasoning.

  • Takeaways & Limitations

    Multilingual Long CoT remains constrained by cross-lingual knowledge transfer and limited training data for low-resource languages.

Abstract

from arXiv · show

Recent advancements in reasoning with large language models (RLLMs), such as OpenAI-O1 and DeepSeek-R1, have demonstrated their impressive capabilities in complex domains like mathematics and coding. A central factor in their success lies in the application of long chain-of-thought (Long CoT) characteristics, which enhance reasoning abilities and enable the solution of intricate problems. However, despite these developments, a comprehensive survey on Long CoT is still lacking, limiting our understanding of its distinctions from traditional short chain-of-thought (Short CoT) and complicating ongoing debates on issues like "overthinking" and "inference-time scaling." This survey seeks to fill this gap by offering a unified perspective on Long CoT. (1) We first distinguish Long CoT from Short CoT and introduce a novel taxonomy to categorize current reasoning paradigms. (2) Next, we explore the key characteristics of Long CoT: deep reasoning, extensive exploration, and feasible reflection, which enable models to handle more complex tasks and produce more efficient, coherent outcomes compared to the shallower Short CoT. (3) We then investigate key phenomena such as the emergence of Long CoT with these characteristics, including overthinking, and inference-time scaling, offering insights into how these processes manifest in practice. (4) Finally, we identify significant research gaps and highlight promising future directions, including the integration of multi-modal reasoning, efficiency improvements, and enhanced knowledge frameworks. By providing a structured overview, this survey aims to inspire future research and further the development of logical reasoning in artificial intelligence.

1 Introduction

Long CoT research has expanded alongside reasoning large language models, but the field lacks a comprehensive account of its distinctions, mechanisms, and debated phenomena. This survey addresses that gap by defining Long CoT characteristics, examining associated phenomena, and identifying research frontiers.

  • Reasoning large language models have expanded research into Long CoT for mathematical, programming, and multidisciplinary knowledge reasoning.
  • The field lacks a comprehensive survey of Long CoT, amid debates over inference-time scaling, overthinking, and the relationship between reasoning length and accuracy.
  • The survey distinguishes Long CoT from Short CoT through deep reasoning, extensive exploration, and feasible reflection.
  • It examines the emergence of Long CoT, overthinking, inference-time scaling during testing, and the “Aha Moment.”
  • The survey identifies emerging challenges and research frontiers while highlighting open resources and datasets for future investigations.

2 Discussion of Long CoT v.s. Short CoT

Long CoT differs from Short CoT within System 2 reasoning by combining greater depth, broader exploration, and reflection. Short CoT follows a shallow, mostly linear path, whereas Long CoT supports more complex and adaptive reasoning.

  • Short CoT: Short CoT proceeds through a limited sequence of logical nodes with minimal revisitation or exploration of alternatives.Its sequential structure favors rapid, straightforward reasoning but restricts generalizability.
  • Deep Reasoning: Long CoT expands reasoning across more logical nodes, enabling deeper analysis of intricate relationships and reducing unresolved or hallucinated responses.Its larger reasoning boundary Bl accommodates more intricate nodes than Short CoT’s Bs.
  • Extensive Exploration: Long CoT explores uncertain or unknown nodes in parallel, opening additional reasoning paths for ambiguous, incomplete, or multi-solution problems.Parallel exploration supports more systematic consideration of previously unconsidered paths.
  • Feasible Reflection: Long CoT reflection revisits prior nodes to verify connections, correct inconsistencies, and redirect reasoning toward alternative paths.Feedback evaluates intermediate and overall outputs, while refinement adjusts the logical flow based on that feedback.
  • Unified Application: Together, Long CoT’s deeper, broader, and more reflective process enhances accuracy and coherence, while Short CoT remains better suited to simpler, well-defined problems.The distinction emphasizes Long CoT’s scalability and adaptability for complex reasoning.

3.1 Analysis & Explanation for Long CoT

The survey analyzes Long CoT through external behaviors and internal mechanisms, covering its emergence, boundaries, overthinking, scaling, supervision, and reinforcement-learning dynamics. It synthesizes evidence on how these phenomena relate to reasoning performance and model limits.

  • Analytical Perspectives: The survey organizes Long CoT explanations into external behavior analysis and internal mechanism analysis.The external-behavior stream identifies six key phenomena for discussion.
  • Long CoT Emergence: Contextual examples, decoding changes, prompts, training data, and rule-based reinforcement learning are reported as routes for activating or developing Long CoT.Some studies attribute Long CoT to pretraining and others examine influential training data or reinforcement learning.
  • Reasoning Boundaries: RLLMs exhibit reasoning boundaries: exceeding their capacity can reduce performance, with limits varying across task complexity and input length.These boundaries have been examined in code generation, arithmetic, and benchmarked reasoning tasks.
  • Overthinking and Scaling: Longer reasoning chains can improve performance up to a threshold and then cause decline, although some studies report no significant relationship between length and accuracy.Sequential scaling can suffer from error accumulation beyond an optimal path length.
  • Overthinking and Scaling: Inference-time scaling extends reasoning length or trials, and reports on o1 and R1 associate direct inference-length scaling with improved final performance.The survey also discusses scaling laws and their limitations.
  • Supervision and Reinforcement Learning: The survey covers supervision choices between process reward models and outcome reward models, alongside entropy mechanisms that influence exploration and reasoning improvement.Early entropy decline can cause premature convergence and limit further reasoning improvement without entropy control.

3.2 Long CoT Evaluations

Long CoT evaluation spans outcome and process benchmarks across mathematical, scientific, medical, logical, exploratory, reflective, agentic, embodied, multimodal, and research-oriented tasks. The survey also catalogs metrics and decoding strategies for assessing outcomes, consistency, feedback, and exploration.

  • Metrics: RLLM evaluation uses Accuracy, Pass@k, and Cons@k for mathematical and code tasks, with each metric targeting a distinct aspect of correctness or consistency.Pass@k measures whether at least one correct solution appears within k attempts, while Cons@k evaluates repeated correctness or logical coherence.
  • Metrics: Exact Match and Accuracy are commonly used for scientific and commonsense question answering, based on regex-extracted outputs.Exact Match checks whether the output exactly matches the expected solution.
  • Metrics: Rank and Best-of-N evaluate reward-model effectiveness by prioritizing strong reasoning processes or selecting the highest-scoring trajectory.Rank assesses ordering among top-k candidates, whereas Best-of-N uses final outcomes indirectly.
  • Decoding Strategies: Greedy Decoding and Beam Search reduce randomness, while Major@k selects the most consistent solution from k candidates.These strategies control inference by limiting sampling or aggregating candidate reliability.
  • Benchmark Taxonomy: Outcome benchmarks assess holistic reasoning, whereas process benchmarks target local Long CoT processes or individual capabilities.The benchmark taxonomy includes broad outcome evaluations and specialized deep-reasoning, exploration, and reflection evaluations.
  • Benchmark Coverage: Evaluation coverage extends to complex mathematics, scientific and medical reasoning, agentic and embodied interaction, multimodal inputs, and AI-for-research applications.These benchmarks test reasoning across diverse domains, formats, environments, and research workflows.

4 Deep Reasoning for Long CoT

The survey organizes deep reasoning for Long CoT around reasoning formats and learning strategies. It covers natural-language, structured-language, and latent-space formats, alongside imitation and self-learning approaches.

  • Deep Reasoning Format: Deep reasoning formats comprise natural language, structured language, and latent-space reasoning, with the latent category subdivided into token-, vector-, and manager-driven paradigms.Their performance is presented in Table 1, which is sorted primarily by GSM8K scores.
  • Deep Reasoning Format: Natural-language approaches support intuitive reasoning, while structured-language methods use programming or symbolic formats to enhance logical and mathematical reasoning.The survey describes both formats as established approaches for extending reasoning beyond basic chains of thought.
  • Deep Reasoning Format: Latent-space approaches perform reasoning through operations in continuous representations, including planning tokens, recurrent depth, and continuous reasoning managers.These paradigms implicitly scale inference-time computation or iteratively govern recurrent thought blocks.
  • Deep Reasoning Learning: Deep reasoning learning uses supervised fine-tuning for imitation and preference-based reinforcement learning with implicit rewards for self-learning.The survey presents these as the two key approaches in Figure 6 and evaluates them in Table 2.
  • Deep Reasoning Imitation: Distilling reasoning from advanced RLLM APIs significantly improves smaller language models, and some studies report that minimal imitation samples can activate deep reasoning.Reported examples include distillation from O1 and R1, as well as LIMO, S1, and RedStar findings on sample efficiency.
  • Deep Reasoning Self-Learning: Current imitation and distillation methods still rely heavily on human annotations or outputs from more advanced models.This reliance motivates research into self-play and self-learning techniques.

5 Feasible Reflection for Long CoT

Feasible Reflection strengthens Long CoT through feedback that evaluates reasoning and refinement that corrects it. The survey organizes feedback into overall, process, and hybrid forms, while highlighting annotation costs, robustness limits, and reward hacking.

  • 5 Feasible Reflection for Long CoT: Feasible Reflection uses iterative feedback and refinement to support complex-task reasoning.Feedback generates signals to correct prior nodes, and refinement adjusts subsequent reasoning nodes.
  • 5.1 Feedback: Feedback evaluates both final outputs and the processes that produce them, using overall, process, or hybrid signals.Signals may use natural language or structured data and can support tree-search methods.
  • 5.1.1 Overall Feedback: Overall feedback draws on Outcome Reward Models, rule extraction, and RLLM-generated feedback to assess complete reasoning trajectories.These sources provide global rather than step-by-step evaluations.
  • 5.1.2 Process Feedback: Process feedback supplies step-level guidance through process reward models or prompted RLLMs, supporting automated reasoning improvement.Such methods can combine process feedback with MCTS or RL rewards, reducing reliance on labor-intensive annotations.
  • 5.1.2 Process Feedback: Process-reward integration can reduce over 99% of PRM parameters while enabling unified policy-model and reward-model interfaces.The cited approach evaluates rewards at each step without additional process annotations.
  • 5.1.3 Hybrid Feedbacks: Hybrid feedback combines overall and stepwise evaluations to improve reasoning accuracy and support error correction through reflection.Examples combine Monte Carlo estimation with LLM judging or use PRM and ORM binary feedback.

6 Extensive Exploration for Long CoT

Exploration is a core Long CoT capability for navigating complex problem spaces beyond linear reasoning. The survey distinguishes exploration scaling from internal and external exploration.

  • 6 Extensive Exploration for Long CoT: Exploration enables Long CoT models to navigate complex problem spaces through branching and iterative refinement.Hypothesis branching and reflective error backtracking help overcome linear reasoning constraints.
  • 6.1 Exploration Scaling: Exploration scaling studies how the breadth and depth of reasoning paths affect downstream applications.Its focus is improving the exploration-path size m.
  • 6.2 Internal Exploration: Internal exploration trains models to generate multiple exploration paths efficiently, whereas external exploration uses outside systems to select among them.The taxonomy contrasts model-developed exploration with externally assisted path selection.

6.1 Exploration Scaling

Exploration scaling expands Long CoT through sequential or parallel reasoning and improves the selection of useful paths through verification and sampling optimization.

  • 6.1 Exploration Scaling: Exploration scaling follows sequential and parallel paradigms for increasing reasoning length or the number of generated reasoning paths.Sequential scaling connects reasoning processes through reflection, while parallel scaling generates multiple processes for comparison.
  • Sequential Scaling: Sequential scaling extends a single generation or iteratively feeds prior answers into later reasoning rounds.Prior work reports that longer reasoning paths can substantially improve performance.
  • Parallel Scaling: Parallel scaling generates multiple reasoning results and verifies them before producing the final output.Self-consistency established majority voting over multiple sampled reasoning processes as an exploration strategy.
  • Verification Optimization: Verification optimization replaces or supplements majority voting with self-verification to improve scaled reasoning.The survey also notes parallel Long CoT scaling in code-oriented settings.
  • Sampling Optimization: Sampling optimization seeks diverse, efficient reasoning paths by adjusting path selection, prompts, or sampling temperature.Some methods aggregate short yet varied paths to improve scalability.
  • Takeaways: Exploration Scaling: The survey identifies branching, backtracking, sequential and parallel scaling, verification, and sampling diversity as central exploration mechanisms.Together, these directions target reasoning depth and efficiency.

6.2 Internal Exploration

Internal exploration is advanced through SFT–RL complementarity, reward-free and reward-based RL strategies, and correctness- or model-based reward design. Reported directions target better exploration efficiency and reasoning performance, while shorter accurate reasoning may improve efficiency.

  • 6.2 Internal Exploration: SFT stabilizes output format, whereas RL improves generalization and can increase learning efficiency by up to eight times in mathematical reasoning.The survey presents their combination as a basis for improving internal exploration.
  • 6.2 Internal Exploration: Internal exploration research includes reward-free RL, reward-based RL, and reward strategies designed to guide reasoning without external assistance.Figure 10 groups optimization around RL strategy improvements and rule-based or model-based rewards.
  • 6.2.1 RL Strategies: Recent RL strategies target improved exploration across mathematical reasoning, multi-step reasoning, agent control, and distribution-shift settings.The surveyed approaches include offline RL, critic-based optimization, and modified KL penalties.
  • 6.2.1 RL Strategies: Reward-based RL uses a reward model or verifier to guide learning and decision-making when explicit rewards are unavailable.The survey discusses PPO, ReMax, GRPO, and related simplifications for reasoning optimization.
  • 6.2.1 RL Strategies: Weakness-driven problem synthesis concentrates reinforcement learning on challenges tailored to an RLLM’s specific weaknesses.The survey describes this as producing more focused reasoning improvements.
  • Reward Strategies: Rule-rewarded RL emphasizes correctness, format, and scaling rewards to guide exploration and reasoning accuracy.The survey also notes that shorter, accurate reasoning may improve efficiency and performance.
  • Model-rewarded RL: Model-rewarded RL uses additional reward models to guide exploration and decision-making, including outcome- and process-level reward models.The approach extends reward guidance beyond explicit task rewards.

6.3 External Exploration

External exploration in Long CoT is organized around human-managed fixed pipelines and model-driven adaptive search, with research also combining exploration structures and feedback.

  • Human-Driven Exploration: Human-driven methods use prompt-, tree-, and graph-structured search frameworks to support long-term exploration across datasets.
  • Model-Driven Exploration: Model-driven exploration develops search structures iteratively through beam search, A* search, and Monte Carlo Tree Search.
  • External exploration divides into Human-Driven Exploration using fixed human-defined pipelines and Model-Driven Exploration using dynamic, adaptive model-based search structures.
  • Model-Driven Exploration: Exploration-path feedback improves reasoning exploration and output quality by integrating optimized value models, reward models, or self-refinement with search.
  • Unified Improvements and Path Feedback: Unified approaches jointly optimize exploration strategies and feedback models, including iterative learning, reward-guided trees, and stepwise guidance in large search spaces.

7 Training Resources

The survey catalogs open-source frameworks and datasets that support Long CoT training, inference optimization, replication, and evaluation across multiple domains.

  • Open-source frameworks provide tools for Long CoT training and inference, spanning R1 replication, reinforcement learning, external exploration, and efficiency improvements.
  • SimpleRL and DeepScaler replicated R1’s technology stack, while Open-Reasoner-Zero reproduced the DeepSeek-R1-zero scheme with a 32B model and similar performance.
  • Open-source Long CoT datasets are organized into manual annotation, direct distillation, search-based distillation, and validated distillation categories.
  • The datasets cover Mathematics, Science, Medicine, Code, and General domains, providing resources for broader Long CoT implementation.

8 Frontiers & Future Direction

The survey identifies six Long CoT frontiers—multimodal, multilingual, agentic and embodied, efficient, knowledge-augmented, and safe reasoning—alongside concrete challenges for each.

  • Future directions include Multimodal, Multilingual, Agentic & Embodied, Efficient, Knowledge-augmented, and Safety-focused Long CoT.
  • Multimodal Long CoT: Multimodal Long CoT aims to integrate visual content into reasoning and extend longer inference processes through methods such as reinforcement learning or MCTS.
  • Multilingual Long CoT: Multilingual Long CoT must improve cross-lingual knowledge transfer and preserve strong reasoning in low-resource languages despite limited training data.
  • Agentic & Embodied Long CoT: Agentic Long CoT faces challenges in reliable decision-making under evolving uncertainty and efficient scaling across extended multi-agent interactions.
  • Efficient Long CoT: Efficient Long CoT research seeks faster reasoning through KV-cache optimization, token compression, efficient structures, dynamic patterns, and adaptive reasoning strategies.
  • Knowledge-augmented Long CoT: Knowledge-augmented Long CoT requires scalable retrieval that integrates timely information with historical knowledge while maintaining coherence across long reasoning chains.
  • Safety in Long CoT: Safety research addresses unstable, misleading, offensive, or otherwise unsafe outputs, including risks associated with overthinking and adversarial attacks.

9 Related Work

Related work has surveyed reasoning emergence, inference-time scaling, System 2 thinking, and broader reasoning tasks, but has not comprehensively distinguished Long CoT from Short CoT.

  • Earlier studies examined reasoning emergence in RLLMs as models scale, including in-context and few-shot learning across tasks.
  • Recent reviews address mathematical reasoning, complex optimization and multi-step reasoning, inference-time search scaling, and modular reasoning-system design.
  • A separate survey focused on System 2 thinking and methods for distinguishing it from System 1 thinking.
  • The literature has limited discussion of Long CoT versus Short CoT, motivating this survey’s capability-based re-examination of their differences.

10 Conclusion

The survey closes by consolidating its framework for Long CoT and outlining resources and directions for continued research.

  • The survey distinguishes Long CoT from Short CoT and defines deep reasoning, extensive exploration, and feasible reflection as core features.
  • It introduces a novel taxonomy and summarizes current advancements to provide a comprehensive overview of Long CoT research.
  • The survey highlights emerging challenges and opportunities while providing resources intended to support ongoing Long CoT studies.
Loading 2503.09567v5…