Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models

Introduction

• Long CoT uses detailed exploration and reflection with inference-time scaling, improving mathematical, programming, and multidisciplinary reasoning beyond traditional LLM approaches.It is associated with RLLMs such as OpenAI o1 [307] and DeepSeek R1 [227], while SFT and RL support learning and exploration of extended reasoning chains.Source: §1 · p. 2
• Long CoT remains contested because inference-time scaling may help, overthinking may add complexity, and greater length does not always improve accuracy.These debates motivate a systematic account of Long CoT’s effectiveness and limitations [864] [486] [103] [142] [357] [886].Source: §1 · p. 2
• The survey distinguishes Long CoT from Short CoT through deep reasoning, extensive exploration, and feasible reflection, enabling more intricate and coherent reasoning.Deep reasoning processes many logical nodes, extensive exploration transitions from known to unknown logic, and feasible reflection provides feedback and refinement.Source: §1 · p. 2
• The survey investigates Long CoT’s emergence, overthinking, inference-time scaling, and the “Aha Moment” as key phenomena in complex reasoning.It presents itself as the first comprehensive survey dedicated to these topics.Source: §1 · p. 3
• The work highlights emerging challenges and research frontiers, including open-resource frameworks and datasets to support future Long CoT investigations.These directions are intended to advance Long CoT methodologies across the growing literature.Source: §1 · p. 3

2.1 Overview of Short CoT

• Short CoT uses shallow, linear reasoning with limited logical nodes, minimal revisitation, and little alternative-path exploration, prioritizing rapid and efficient conclusions.This structure can restrict generalizability and oversimplify reasoning for complex problems [683].Source: §2.1 · pp. 3–4

2.2 Overview of Long CoT

• Long CoT expands reasoning across broader logical structures by integrating deep reasoning, extensive exploration, and feasible reflection as complementary capabilities.Deep reasoning supports rigorous steps, exploration identifies non-obvious pathways, and reflection enables iterative reassessment and evolving conclusions.Source: §2.2 · p. 4

2.2.1 Deep Reasoning for Long CoT

• Long CoT accommodates substantially more interconnected logical nodes than Short CoT, enabling thorough deductions for complex problems and reducing unresolved or hallucinated answers.Its larger reasoning boundary alleviates insufficient depth in short-form reasoning.Source: §2.2.1 · pp. 4–5

2.2.2 Extensive Exploration for Long CoT

• Long CoT explores uncertain or unknown logical nodes in parallel, expanding possible reasoning paths for ambiguous, incomplete, or multi-solution problems.Parallel exploration systematically considers previously unexamined paths and improves understanding of possible solutions [1016] [871].Source: §2.2.2 · pp. 5–6

2.2.3 Feasible Reflection for Long CoT

• Long CoT revisits prior nodes through feedback and refinement, correcting errors or redirecting reasoning to alternative paths and thereby minimizing error propagation.Verification evaluates intermediate and overall outputs, while refinement adjusts steps or logical flow when inconsistencies or gaps are found.Source: §2.2.3 · p. 6

2.2.4 Unified Application and Development History of Three Capabilities

• Modern Long CoT emerged gradually by unifying deep reasoning, reflection, and exploration, rather than appearing suddenly through isolated systems such as o1 [307] and R1 [227].Earlier approaches developed capabilities separately: ToT emphasized exploration, while GoT [48] added self-reflection but initially lacked robust deep reasoning.Source: §2.2.4 · pp. 6–7
• Long CoT’s deeper, broader, and more reflective process enhances accuracy and coherence for complex reasoning, while Short CoT better suits simpler, well-defined problems.Traditional Short CoT optimization typically focuses on only one of the three capabilities, whereas Long CoT integrates all three.Source: §2.2.4 · p. 7

3.1 Analysis & Explanation for Long CoT

• Long CoT research is organized into external behavior analysis and internal mechanism analysis, covering its mechanisms, principles, and characteristic phenomena.The external stream identifies six phenomena, while the internal stream investigates mechanisms underlying Long CoT-related RLLMs.Source: §3.1 · p. 7, §3.1.1 · p. 7, §3.1.2 · p. 10

3.1.1 Long CoT External Behavior Analysis

• Long CoT can emerge through contextual logical-structure imitation, developmental generalization, decoding or prompt activation, and reinforcement of verification, backtracking, sub-target setting, and backward chaining.Rule-based reinforcement readily triggers these behaviors in Qwen models, whereas Llama models require example-based reinforcement to improve significantly [65].Source: §3.1.1 · p. 7
• RLLMs have reasoning boundaries: exceeding task, input-length, or model-capacity limits can reduce performance, while additional reasoning steps increase capability requirements for harder problems.These boundaries motivate formal limits, tight lower bounds, and reinforcement-learning approaches to expand reasoning capacity [90] [12] [483].Source: §3.1.1 · p. 8
• Overthinking occurs when longer reasoning improves performance only up to a threshold, after which excessive length or complexity causes performance decline, although some studies find no length–accuracy correlation.Proposed explanations include avoiding snowball errors, exceeding reasoning boundaries, feasible-length constraints, and Ohm’s law of Long CoT [21] [441] [192] [90] [851] [867] [93].Source: §3.1.1 · p. 8
• Inference-time scaling improves performance through longer inference or repeated reasoning, but sequential paths can accumulate errors and parallel sampling cannot overcome model-internal reasoning limits.o1 [307] and R1 [227] demonstrate gains from directly scaling inference length; sequential scaling has an optimal path length, while parallel scaling has verifier and computational bounds.Source: §3.1.1 · p. 9
• PRM-versus-ORM selection remains unresolved because process supervision requires fine-grained trajectory coverage and can face reward hacking, while theory shows outcome supervision can suffice for online reinforcement learning.Outcome supervision is not statistically harder than process supervision under standard coverage assumptions, aside from polynomial factors [317] [247].Source: §3.1.1 · p. 9
• Rule-based reinforcement can trigger the “aha moment” and self-reflection, but alternative evidence attributes longer responses to reward optimization and finds superficial self-reflection that does not ensure correct answers.The phenomenon has also been extended to multimodal settings [1119] [547], while external markers include anthropomorphic language and uncertainty adjustment [939].Source: §3.1.1 · p. 9

3.1.2 Long CoT Internal Mechanism Analysis

• Internal analyses link coherent Long CoT to selective attention, stable reasoning layers, finite-state dynamics, consistency heads, and mechanisms that connect stored knowledge across entities and concepts.These studies use white-box analyses of neural substructures [992] [169] and propose PMM, ITC, Concept Depth, and knowledge-loop evolution for characterizing reasoning and knowledge integration [630] [572].Source: §3.1.2 · p. 10

3.2.1 Metrics

• Metrics quantify correctness, repeated-sample performance, consistency, exact matching, and reward-model selection across reasoning tasks.Accuracy, Pass@k, and Cons@k are used for mathematical and coding tasks; EM and Accuracy support scientific or commonsense question answering, while Rank and Best-of-N evaluate feedback models.Source: §3.2.1 · pp. 10–11

3.2.2 Decoding Strategies

• Decoding strategies regulate inference randomness and consistency through Greedy Decoding, Beam Search, and Major@k selection.Greedy Decoding and Beam Search restrict sampling, whereas Major@k chooses the most consistent solution among k candidates.Source: §3.2.2 · p. 11

3.2.3 Benchmarks

• Benchmarks divide into outcome evaluations of holistic reasoning and process evaluations of local capabilities, spanning mathematics, coding, commonsense, science, and medicine.Outcome benchmarks include increasingly difficult mathematics and coding tasks, commonsense puzzles, and knowledge benchmarks for scientific and medical reasoning, including out-of-distribution evaluation.Source: §3.2.3 · pp. 11–12

3.2.4 Process Evaluations

• Process evaluations target deep reasoning, exploration, and reflection by testing stepwise reasoning, generalization, interaction, error detection, and correction.Deep-reasoning benchmarks inspect reasoning steps, exploration benchmarks assess scaling and interactive problem solving, and reflection benchmarks cover feedback and refinement across general, coding, multimodal, commonsense, and medical tasks.Source: §3.2.4 · p. 12

3.2.5 Advanced Evaluation

• Advanced evaluations assess agentic adaptation and tool use, multimodal reasoning, and RLLMs’ ability to support scientific research and generate feasible solutions.Agentic benchmarks cover physical understanding, planning, multi-agent decisions, tools, and changing digital environments; multimodal benchmarks span mathematics, code, science, and puzzles; AI-for-research evaluations examine experiments, creativity, and engineering solutions.Source: §3.2.5 · pp. 12–13

4 Deep Reasoning for Long CoT

• Deep reasoning methods either maximize reasoning-step length within a boundary through execution formats or intrinsically extend that boundary through learning.Insufficient deep reasoning can cause significant performance declines in RLLMs [758] [823].Source: §4 · p. 13

4.1 Deep Reasoning Format

• Deep reasoning formats comprise natural language, structured language, and latent-space reasoning, with latent methods divided into token-, vector-, and manager-driven paradigms.Figure 5 presents this taxonomy, while Table 1 reports performance across the formats.Source: §4.1 · p. 13, Figure 5 · p. 14, Table 1 · p. 15

4.1.1 Natural Language Deep Reasoning

• Natural-language methods deepen reasoning through free-flowing or structured analysis, code-to-language reorganization, and mixtures of code and natural-language formats [1118] [303] [205].Natural Program [460], CodeI/O [401], and CoRT [387] exemplify these approaches.Source: §4.1.1 · p. 14

4.1.2 Structured Language Deep Reasoning

• Structured-language methods use programming or symbolic formats to strengthen reasoning, especially mathematical reasoning, while neural-symbolic training and quasi-symbolic abstraction improve efficiency and structural precision [100] [464] [687] [591] [198] [1044].SKIntern reduces computational costs and accelerates inference, while QuaSAR promotes higher-level quasi-symbolic reasoning.Source: §4.1.2 · p. 14

4.1.3 Latent Space Deep Reasoning

• Latent-space methods perform reasoning through continuous operations using planning tokens, thought vectors, or recurrent managers, enabling alternative-path maintenance, recurrent-depth scaling, adaptive routing, and computational savings [151] [640] [324].Manager-driven approaches can enhance performance without specialized training data, while dynamic shortcuts allocate depth across critical tokens and decoding steps.Source: §4.1.3 · pp. 14–15

4.2 Deep Reasoning Learning

• Deep reasoning learning combines supervised fine-tuning, which stabilizes outputs through imitation, with reinforcement learning, which supports generalization and self-learning [1058] [883] [227] [137] [276] [898].The survey distinguishes deep reasoning imitation from deep reasoning self-learning and reports their performance in Table 2.Source: §4.2 · pp. 15–16, Figure 6 · p. 16, Table 2 · p. 17

4.2.1 Deep Reasoning Imitation

• Deep reasoning imitation learns complex reasoning patterns from human annotations, advanced RLLMs, or scaling-augmented systems, and can generalize across tasks [61] [115] [403] [227] [58] [957] [370] [102].Distilling O1 [307] and R1 [227] can substantially improve smaller models, while LIMO [967], S1 [560], and RedStar [902] show that few samples may activate deep reasoning.Source: §4.2.1 · p. 16

4.2.2 Deep Reasoning Self-Learning

• Self-learning reduces reliance on human or advanced-model outputs through direct sampling or tree search, using implicit rewards, verifiers, entropy, reinforcement learning, and MCTS to refine reasoning paths [502] [1077] [409] [624].Combining imitation with self-learning strengthens RLLM reasoning and supports strong performance on complex tasks.Source: §4.2.2 · pp. 16–18

5 Feasible Reflection for Long CoT

• Feasible Reflection consists of feedback that generates correction signals and refinement that adjusts subsequent reasoning nodes [406] [192].This two-stage process supports iterative correction and refinement for complex tasks.Source: §5 · p. 18

5.1 Feedback

• Feedback evaluates outputs and their generating processes through natural language or structured formats, comprising overall, process, and hybrid feedback [394] [396] [838] [220] [862].These feedback forms provide foundations for tree-search methods [113].Source: §5.1 · p. 18

5.1.1 Overall Feedback

• Overall feedback assesses complete processes and results through outcome reward models, rule extraction, and RLLM-based feedback, improving reasoning and reward modeling.Outcome reward models provide quantifiable rewards when tasks lack direct accuracy metrics, whereas rule-based rewards can improve accuracy, output length, and reward-hacking resistance.Source: §5.1.1 · pp. 18–20

5.1.2 Process Feedback

• Process feedback supplies automated step-level guidance through process reward models or prompted LLMs, combining with MCTS or RL rewards to reduce annotation demands [749] [344].Process reward models use human- or outcome-based supervision, while RLLM and environment-driven methods generate or verify feedback; one unified PRM-policy interface reduced PRM parameters by over 99%.Source: §5.1.2 · pp. 20–21

5.1.3 Hybrid Feedbacks

• Hybrid feedback combines overall and stepwise signals, including consensus filtering and joint PRM-ORM guidance, to improve reasoning accuracy and error correction.These approaches integrate Monte Carlo estimation, LLM-as-judge evaluation, and binary process- and outcome-level feedback.Source: §5.1.3 · pp. 21–22

5.2 Refinement

• Refinement corrects reasoning errors using prompt-based generation, SFT-based imitation, or RL-based learning.These categories organize methods that iteratively improve reasoning from prior feedback.Source: §5.2 · p. 22

5.2.1 Prompt-based Refinement Generation

• Prompt-based refinement iteratively applies self-feedback to improve outputs, reduce hallucinations, and simplify reasoning, but unstable self-refinement without oracle feedback can bias answers and fail.Methods also refine prompts, support multi-agent reasoning, and adjust strategies through backtracking and confidence changes.Source: §5.2.1 · pp. 22–23

5.2.2 SFT-based Refinement Imitation

• SFT-based refinement teaches error correction from advanced models through iterative feedback, step-wise correction data, rationale tags, MCTS backtracking, and multimodal visual-textual reasoning.These methods improve reflective capabilities and can enhance smaller language models through self-play or supervised correction trajectories.Source: §5.2.2 · p. 23

Overview

• Exploration enables Long CoT models to navigate complex problem spaces through strategic branching, iterative refinement, hypothesis branching, and reflection-based error backtracking [1019] [381] [784] [227].These mechanisms address the limitations of linear reasoning paths.Source: §6 · p. 24

Exploration Scaling and Internal Exploration

• Exploration scaling studies how the breadth and depth of exploration affect downstream applications, including the size of the exploration path m.Internal exploration trains models to generate m exploration paths more efficiently and effectively.Source: §6 · p. 24

External Exploration

• External exploration uses external systems to improve exploratory abilities and select the most effective path from m exploration paths.This work focuses on choosing among candidate paths generated during exploration.Source: §6 · p. 24

6.1 Exploration Scaling

• Exploration scaling follows sequential and parallel paradigms: sequentially connected reasoning processes use reflection, whereas parallel processes use unified verification or feedback to select effective results.The paradigms are compared respectively to series and parallel resistors [93].Source: §6.1 · p. 24

6.1.1 Sequential Scaling

• Sequential scaling extends reasoning within one generation or latent space, using iterative prompting, tree search, budget forcing, recurrent depth, and curricula to improve performance.Its core mechanisms include linking asymmetric skills, extending reinforcement-learning search through error trajectories, and matching task difficulty with training token budgets [653].Source: §6.1.1 · p. 24

6.1.2 Parallel Scaling

• Parallel scaling increases reasoning iterations and verifies candidate results, with self-consistency initiating exploration through multiple samples and majority voting.Subsequent approaches replace or refine voting with overall, meta-, code-based, uncertainty-based, or step-level verification; however, these strategies cannot surpass Best-of-N methods [93].Source: §6.1.2 · p. 25
• Parallel scaling also improves efficiency by generating diverse reasoning paths through varied prompts, temperatures, reward mechanisms, languages, expressions, and response lengths.Some methods aggregate short diverse paths or select the shortest correct responses across inference efforts [1020] [943].Source: §6.1.2 · p. 25

6.2 Internal Exploration

• RL improves generalization and can increase learning efficiency by up to eight times in mathematical reasoning, while SFT stabilizes output format.The survey presents SFT as a memory process and RL as a mechanism for generalization.Source: §6.2 · pp. 25–26

6.2.1 RL Strategies

• RL exploration methods improve credit assignment, feedback density, targeted Long CoT exploration, sample efficiency, training stability, and large-scale reasoning optimization.Reward-free approaches include offline RL, critic-based advantage optimization, preference optimization, replay and sample prioritization, asynchronous training, and SFT-RL combinations.Source: §6.2.1 · pp. 26–27
• Reward-model-based RL streamlines policy optimization through improved sampling, variance reduction, and value-model alternatives, while PRIME provides implicit process rewards from SFT models.The surveyed methods include ReMax, GRPO, REINFORCE++ and Dr. GRPO, KL-penalty modifications, DVPO, and PRIME.Source: §6.2.1 · pp. 26–27

6.2.2 Reward Strategies

• Rule-based rewards guide exploration through correctness, format, and scaling objectives, but excessive length scaling can cause inefficiency and overcomplicated reasoning.Long2Short methods instead favor shorter, more accurate reasoning to improve efficiency and performance.Source: §6.2.2 · p. 28
• Model-based rewards enhance exploration through outcome- and step-level correctness, entropy rewards, dynamic regularization, trajectory adjustment, and multimodal PRM-based reinforcement learning.These approaches guide decision-making and deepen reasoning by identifying suboptimal actions and refining trajectories.Source: §6.2.2 · p. 28

6.3.1 Human-driven Exploration

• Human-driven methods use prompt-, tree-, and graph-structured searches, with multi-agent execution and feedback improving planning, refinement, scalability, and task-solving accuracy [479] [422].CodeTree [400], Tree-of-Code [565], SPAR [118], Forest-of-Thought [54] [448], and Python-tool integration [388] extend these structures for multi-stage decisions, preference learning, complex tasks, and test-time scaling.Source: §6.3.1 · p. 29

6.3.2 Model-driven Exploration

• Model-driven exploration enhances search logic through Beam Search, A* Search, and MCTS, while addressing inefficient search and large action spaces in Long CoT.These methods include self-evaluation, deductive reasoning, predicted A* dynamics, Levin tree search, collective learning, Nash-equilibrium balancing, constrained action spaces, and interactive-environment extensions [378] [338] [1095] [323] [896] [1046].Source: §6.3.2 · pp. 29–31

6.3 External Exploration

• External exploration divides into Human-Driven Exploration with fixed human-defined pipelines and Model-Driven Exploration with dynamic, adaptive search structures.This taxonomy is presented in Figure 11, with performance comparisons summarized in Table 6.Source: §6.3 · p. 28, Figure 11 · p. 29

6.3.2 Model-driven Exploration

• Model-driven methods also improve exploration by combining path-feedback models with search, and by jointly optimizing exploration, reward modeling, and response refinement.PPO-augmented MCTS, LLaMA-Berry, AtomThink, QLASS, and DQO improve feedback, controllability, stepwise guidance, or output reliability [1034] [879] [458] [471].Source: §6.3.2 · pp. 30–31

7.1 Open-Sourced Training Framework

• Frameworks such as SimpleRL, DeepScaler, X-R1, TinyZero, and Open-Reasoner-Zero replicate R1 technologies, with Open-Reasoner-Zero matching performance using a 32B model.X-R1 and TinyZero target an intuitive “Aha moment” experience under $50, while Open-Reasoner-Zero reproduces the R1-zero scheme.Source: §7.1 · p. 31
• Open-source training frameworks reproduce and extend R1-style Long CoT reinforcement learning for text, efficiency optimization, external exploration, and multimodal settings.Frameworks include OpenR, OpenRLHF, OpenR1, and Logic-RL for text; DAPO and VAPO for finer-grained efficiency; and multiple R1-derived multimodal systems.Source: §7.1 · p. 31
• Open-source sharing accelerates academic research and industry application of large-scale language models, inference algorithms, and multimodal Long CoT processing.The frameworks provide technical support for both deep-learning-based inference and multimodal processing.Source: §7.1 · p. 31

7.2 Open-Sourced Training Data

• Open-source Long CoT datasets span manual annotation, direct distillation, search-based distillation, and validated distillation across mathematics, science, medicine, code, and general domains.Manual-annotation datasets contain 8K–250K examples, whereas direct-distillation datasets provide millions of examples generated mainly in language by large pretrained models.Source: §7.2 · p. 31
• Validated and search-based datasets such as STILL-1 and KodCode-V1 combine structured data with validation techniques to provide higher-quality training resources.The collection is intended to improve model performance across different domains.Source: §7.2 · p. 31

8 Frontiers & Future Direction

• Six future directions are identified: multimodal, multilingual, agentic and embodied, efficient, knowledge-augmented, and safe Long CoT.These directions target diverse modalities, cross-lingual applications, real-world interaction, faster reasoning, external knowledge integration, and reliable outputs.Source: §8 · pp. 31–32, Figure 12 · p. 33

8.1 Multimodal Long CoT

• Multimodal Long CoT combines prompting, imitation, and reward-model exploration to improve visual reasoning, but multimodal inference-time scaling remains underexplored.Challenges include grounding visual content for spatial reasoning and extending longer reasoning through methods such as reinforcement learning or MCTS.Source: §8.1 · pp. 32–33

8.2 Multilingual Long CoT

• Multilingual Long CoT uses prompting, multilingual training, and inference-time scaling to align reasoning across languages, especially between high- and low-resource languages.Future work must improve cross-lingual knowledge transfer and preserve strong reasoning in low-resource languages despite limited training data.Source: §8.2 · pp. 33–34

8.3 Agentic & Embodied Long CoT

• Agentic and embodied Long CoT advances through tree search, environmental memory, and multi-agent cooperation, but robust decisions and scalable interactions remain challenging.Dynamic environments introduce uncertainty, noisy feedback, large memory histories, and real-time interaction costs.Source: §8.3 · p. 34

8.4 Efficient Long CoT

• Efficient Long CoT research compresses reasoning chains or moves CoT into hidden continuous spaces to reduce computation while preserving accuracy.Key directions include adaptive reasoning based on task difficulty or intermediate quality and multimodal or latent formats that reduce lengthy textual reasoning.Source: §8.4 · pp. 34–35

8.5 Knowledge-Augmented Long CoT

• Knowledge-augmented Long CoT combines retrieval-augmented generation with model knowledge injection, but must align external knowledge with reasoning and scale timely retrieval.These challenges require coherent integration of domain-specific information and fast, contextually relevant updates from large knowledge stores.Source: §8.5 · pp. 35–36

8.6 Safety and Stability for Long CoT

• Safety research addresses unstable, hallucinated, biased, and harmful Long CoT outputs through attacks, prompting, search, self-reflection, and training interventions.The central challenge is balancing performance gains with cognitive-load management and resistance to adversarial or unsafe behavior.Source: §8.6 · p. 36

Prior reasoning surveys

• Early studies examine reasoning emergence in scaled RLLMs, including in-context and few-shot learning, while surveys review LLM advances across reasoning tasks [603] [285] [138] [208] [473] [696].Chu-Carroll et al. [139] additionally call for hybrid architectures to address reliance on statistical patterns over structured reasoning.Source: §9 · p. 36

Recent reasoning methods

• Recent work targets mathematical and complex reasoning through inference-time search, reinforcement learning, thought sequences, prompting, deeper analysis, and self-evolution [795] [1096] [33] [588] [440] [419] [899] [546] [473] [557].These studies address limitations of standard LLMs on optimization and multi-step reasoning and investigate strategies such as Monte Carlo Tree Search.Source: §9 · p. 36

System design and System 2

• Other research proposes modular system design integrating structure, strategy, and training, while surveying System 2 thinking and its distinction from System 1 thinking [50] [432].Source: §9 · pp. 36–37

Research gap

• Despite these technical reviews, prior work has limited discussion of how Long CoT differs from Short CoT, although Short CoT technologies have not matched Long CoT effectiveness.This survey therefore re-examines their core capability differences to guide future optimization.Source: §9 · p. 37

• The survey distinguishes Long CoT from Short CoT and defines deep reasoning, extensive exploration, and feasible reflection as its core features.These definitions clarify Long CoT’s advantages and address key gaps in the field.Source: §10 · p. 37
• A novel taxonomy organizes current Long CoT advancements and provides a comprehensive overview of the research landscape.Source: §10 · p. 37
• The survey highlights emerging challenges and opportunities while offering resources intended to support and inspire future Long CoT research.Source: §10 · p. 37