Source-linked AI summary
Reasoning Beyond Language: A Comprehensive Survey on Latent Chain-of-Thought Reasoning
Xinghao Chen, Anhao Zhao, Heming Xia, Xuan Lu, Hanlin Wang, Yanjun Chen, Wei Zhang, Jian Wang, Wenjie Li, Xiaoyu Shen
TL;DR
Conventional CoT relies on explicit language, creating redundancy and a semantic bottleneck for reasoning beyond discrete symbols. This survey synthesizes latent CoT methods and organizes them into a unified taxonomy spanning token-wise and layer-wise computation. It concludes that latent CoT offers a structured foundation for more abstract, efficient, and scalable inference, while interpretability, alignment, and empirical validation remain open challenges.
Problem
Explicit language-based CoT can introduce expressive redundancy and a semantic bottleneck, limiting reasoning beyond discrete linguistic representations.
Method
The paper surveys latent CoT research, proposes a unified taxonomy, and analyzes methods, training strategies, architectures, supervision signals, applications, and challenges.
Results
The survey synthesizes evidence that latent CoT supports more abstract, efficient, and scalable inference while organizing the field's methods and analyses.
Takeaways & Limitations
Latent CoT provides a structured foundation for reasoning that is not strictly confined to language and for further exploration of the emerging field.
Takeaways & Limitations
Many surveyed works rely on small-scale models or limited benchmarks, leaving a need for more up-to-date and rigorous empirical validation.
Abstract
from arXiv · showhide
Large Language Models (LLMs) have shown impressive performance on complex tasks through Chain-of-Thought (CoT) reasoning. However, conventional CoT relies on explicitly verbalized intermediate steps, which constrains its broader applicability, particularly in abstract reasoning tasks beyond language. To address this, there has been growing research interest in \textit{latent CoT reasoning}, where the reasoning process is embedded within latent spaces. By decoupling reasoning from explicit language generation, latent CoT offers the promise of richer cognitive representations and facilitates more flexible, faster inference. This paper aims to present a comprehensive overview of this emerging paradigm and establish a systematic taxonomy. We analyze recent advances in methods, categorizing them from token-wise horizontal approaches to layer-wise vertical strategies. We then provide in-depth discussions of these methods, highlighting their design principles, applications, and remaining challenges. We hope that our survey provides a structured foundation for advancing this promising direction in LLM reasoning. The relevant papers will be regularly updated at https://github.com/EIT-NLP/Awesome-Latent-CoT.
1 Introduction
Latent CoT addresses limitations of language-based reasoning by moving intermediate computation into latent spaces. This survey organizes the rapidly growing field, analyzes its methods, and identifies unresolved supervision, evaluation, and interpretability challenges.
- Natural-language CoT improves transparency and enables human intervention by verbalizing intermediate reasoning steps.
- Language-based reasoning suffers from expressive redundancy and a semantic bottleneck that can inflate computation and lose information.Redundant tokens may slow inference and encourage stylistic overfitting, while fixed vocabularies constrain continuous or multi-conceptual reasoning.
- Latent CoT performs reasoning within continuous latent spaces rather than through explicit language tokens.This shift is motivated by the limitations of natural-language reasoning and replaces linguistic articulation with internal computation.
- Latent reasoning can reduce token-level computation, support richer compact representations, and enable parallel exploration of reasoning trajectories.
- Latent CoT remains difficult to supervise and evaluate because its reasoning process is unobservable.Latent trajectories may drift or lack structured reasoning, and evaluation may not distinguish genuine reasoning from input–output correlations.
- The survey provides a unified taxonomy and analyzes methods, training strategies, architectural paradigms, supervision signals, efficiency trade-offs, and future challenges.
2 Preliminaries
The survey defines CoT as intermediate reasoning steps connecting an input to an answer, distinguishing explicit linguistic traces from latent computational traces. Its taxonomy separates token-wise sequence reasoning from layer-wise iterative computation.
- A Chain-of-Thought is a sequence of intermediate reasoning steps connecting an input query x to its final answer y.
- Explicit CoT verbalizes each reasoning step as discrete vocabulary tokens, making the thought process transparent and human-readable.
- Latent CoT represents each reasoning step as a compact computational structure in a non-verbal, high-dimensional latent space.Examples include dense vectors and transformations of hidden states.
- The taxonomy distinguishes token-wise horizontal methods, which generate latent thoughts across sequence positions, from layer-wise vertical methods, which deepen computation across layers.
3 Token-wise Horizontal Level
Token-wise horizontal methods generate intermediate latent thoughts along the sequence dimension before producing the final answer. The survey traces this approach to extra-computation tokens and internalized reasoning, then examines latent representation initialization.
- Token-wise Horizontal Level: Token-wise horizontal methods generate intermediate latent thoughts along the sequence dimension to guide reasoning before the final answer.
- Token-wise Horizontal Level: These methods let models form a compact latent reasoning plan instead of verbalizing every reasoning step.
- Token-wise Horizontal Level: Early research added designated thinking or planning tokens, while parallel work embedded explicit CoT logic directly into latent representations.
- Token-wise Horizontal Level: The survey defines horizontal methods as generating latent thoughts before final-answer tokens and organizes them around representation initialization and subsequent stages.The supplied passage introduces this staged organization but truncates its continuation.
LLaMA 3-1B-it
The survey organizes latent CoT methods by representation initialization, optimization, and inference exploration, while comparing their performance and efficiency. It distinguishes pre-training from post-training approaches and supervised from reinforcement-learning strategies.
- Representation Initialization: Representation initialization is categorized by hidden states, weighted vocabulary embeddings, and newly introduced special vectors.This choice shapes the latent space and influences reasoning capacity.
- Representation Initialization: Latent thoughts can be initialized from hidden states, projected representations, or weighted combinations in embedding space.Hidden-state methods may pass prior hidden states through projection networks, while weighted embeddings remain tied to the learned vocabulary.
- Optimization: Optimization methods are divided into pre-training approaches and post-training techniques based on specialized datasets and objectives.Representative methods are compared by performance and computational efficiency in Table 1.
- Post-Training: Post-training methods use either supervised fine-tuning or reinforcement learning to train latent reasoning.SFT may supervise only outputs or directly target latent representations, whereas RL maximizes reward signals and can include efficiency objectives.
- Inference Exploration: Inference exploration includes sequential scaling, which generates latent thoughts one after another to refine a single reasoning chain.The survey presents sequential and parallel scaling as distinct test-time exploration strategies.
4 Layer-wise Vertical Level
Layer-wise vertical latent CoT repeatedly applies network computation to refine hidden states before output, creating a flexible think-then-speak mechanism. The survey covers encoder and decoder architectures, adaptive looping, and latent-semantic supervision.
- Overview: Vertical latent CoT uses looping units to iteratively refine hidden states before producing the next token.The mechanism consolidates a latent thought through layer-wise computation while preserving concise internal traces.
- Encoder-based Models: Encoder-based models repeatedly apply an encoder to refine latent representations across vertical loop iterations.Examples include coarse-to-fine reasoning and aligning loop iterations with CoT steps for long-horizon reasoning.
- Decoder-based Models: Decoder-based models reapply part of the decoder stack before emitting each output token.They include adaptive-depth looping and latent-semantic supervision inside the loop.
- Decoder-based Models: Latent-semantic supervision uses probability-weighted embeddings or classifier-based latent rewards to guide otherwise opaque trajectories.These signals provide intermediate targets or direct rewards for latent reasoning.
- Conclusion: Vertical approaches strengthen reasoning by repeatedly applying the same parameters rather than enlarging the parameter set.The survey frames depth as a flexible computational resource bridging verbalized and silent latent reasoning.
5 Analysis
The survey analyzes latent CoT through behavioral observations, theoretical comparisons, and hidden-state investigations. Evidence supports deeper computation and potential efficiency benefits, but empirical findings about internal latent reasoning remain mixed.
- Behavioral Observations: Behavioral studies examine task accuracy and scaling without directly inspecting hidden states.Repeated application of transformer layers improved multi-step reasoning while keeping parameter counts nearly constant.
- Behavioral Observations: Some studies found that models internalize redundant reasoning steps or develop shortcut-like implicit reasoning under regular training patterns.These findings suggest latent reasoning can unfold within hidden representations without emitting every intermediate step.
- Theoretical Analysis: Theoretical analyses report that token-wise horizontal methods can solve certain structured problems with fewer steps than explicit CoT.This comparison concerns reasoning-step requirements rather than a universal performance claim.
- Theoretical Analysis: Layer-wise vertical methods remain effective across varied reasoning tasks and support a degree of parallel computation.The survey presents this as more efficient than the sequential nature of explicit CoT.
- Empirical Analysis: Empirical hidden-state analyses disagree about when latent CoT emerges and whether deeper layers show systematic progression toward higher-level reasoning.Reported conditions include extended training, fixed-pattern data, and training rather than prompting alone.
6 Applications
Latent CoT has been applied beyond textual reasoning to multimodal systems, autonomous driving, information retrieval, and recommendation. These applications place intermediate computation in visual, spatial, document, or user-history representations.
- Multimodal Reasoning and Generation: Multimodal methods embed intermediate reasoning in latent visual or spatial representations.The survey reports applications that enhance depth perception and fine-grained understanding.
- Multimodal Reasoning and Generation: Latent world models for autonomous driving reason over action-aligned representations to predict future states and optimize planning.This extends latent reasoning from perception toward action-oriented prediction and planning.
- Information Retrieval and Recommendation: In information retrieval, latent deliberation creates richer document embeddings than conventional single-pass encoding.The motivation is to capture complex document semantics more effectively.
- Information Retrieval and Recommendation: Recommendation systems recurrently refine user interaction histories or reason over noisy cross-domain signals.Examples include search histories and evolving user preferences.
7 Challenges and Future Directions
Latent CoT faces instability, weak generalization, and interpretability concerns, while future work targets alternative architectures, improved verification, training strategies, and agentic applications.
- Challenges: Training instability remains a key challenge, with current latent CoT methods still underperforming explicit CoT approaches.
- Challenges: Latent CoT models often struggle with novel problem structures or reasoning patterns absent from training.
- Challenges: Unfaithful hidden reasoning can remain unreflected in verbalized CoTs, complicating interpretation of model conclusions.
- Future Directions: Future directions include recurrent, recursive, and diffusion language-model architectures beyond conventional Transformers.
- Future Directions: Researchers prioritize probing latent representations, reinforcement learning, curriculum learning, and latent reasoning for autonomous and multi-agent systems.
8 Conclusion
The survey consolidates latent CoT methods, challenges, and future directions, arguing that latent-space reasoning can support more abstract, efficient, and scalable inference. It also acknowledges limits from the field’s rapid evolution, interpretability and alignment issues, and insufficient large-scale empirical validation.
- The survey summarizes latent CoT methods, identifies major challenges, and highlights promising future directions.
- Latent-space reasoning is presented as enabling more abstract, efficient, and scalable inference beyond surface-level language.
- The survey aims to provide a structured foundation for further exploration and actionable insights into latent reasoning model design and deployment.
- The review may omit valuable contributions because the field is broad and rapidly evolving.
- Many surveyed works rely on small-scale models or limited benchmarks, requiring more up-to-date and rigorous empirical validation.
Appendix
Discrete reasoning tokens have progressed from simple markers to tools for encoding more complex cognitive structures, improving reasoning efficiency, coherence, precision, and interpretability.
- Discrete tokens can improve task performance and efficiency by serving as symbolic representations of intermediate reasoning steps or cognitive operations.
- Early approaches used pause markers and ellipses to segment reasoning, while learnable pause tokens allocated additional internal computation before prediction.
- Planning tokens were introduced to encode complex reasoning structures.
- Heuristic- or VAE-derived tokens target greater coherence and precision, while memory and reason tokens isolate cognitive operations for interpretability.
- Overall, discrete tokens evolved into versatile tools for abstract cognitive modeling that improve reasoning efficiency and interpretability.
A.2 Implicit CoT
Implicit CoT methods internalize explicit reasoning within latent representations, using supervised alignment, self-supervision, or initialized latent states. They are evaluated across mathematical, commonsense, logical, benchmark, and code-centric tasks, although application-aligned benchmarks remain scarce.
- ICoT aligns teacher-generated CoT hidden states with student direct answers, providing supervision for implicit reasoning.
- System 2 Distillation uses self-supervision to compile multi-step reasoning strategies into latent representations.
- Special Vector: Special-vector methods initialize reasoning with learnable scratchpads, inserted latent tokens, or other learnable starting states.
- Latent CoT is applied to mathematical, commonsense, logical multi-hop, high-bar reasoning, and code-centric datasets.
- Benchmarks aligned with real-world applications and designed to demonstrate latent CoT’s advantages remain lacking.