PaperlayerView exact revision on arXiv

Source-linked AI summary

Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization

Jiecong Wang, Hao Peng, Chunyang Liu

arXiv:2601.21358v2cs.AIcs.CL

TL;DR

Discrete and prior latent reasoning methods face path collapse, opacity, or fixed inference steps. PLaT separates continuous latent planning from language verbalization, and mathematical evaluations show lower greedy accuracy but stronger scaling in reasoning diversity. The framework therefore offers a broader, explorable solution space, while remaining below explicit CoT performance.

  • Problem

    Existing reasoning methods face discrete path collapse and high token costs, while latent methods are often opaque and require fixed numbers of inference steps.

  • Method

    PLaT decouples a deterministic latent Planner that evolves planning states from a Decoder that verbalizes them when needed, enabling dynamic termination and interpretable intermediate states.

  • Results

    PLaT has lower greedy accuracy than baselines but superior Pass@k scaling and reduced diversity saturation, indicating broader reasoning diversity.

  • Takeaways & Limitations

    PLaT learns a broad, explorable solution manifold rather than a narrow memorized path, providing a transparent basis for inference-time search.

  • Takeaways & Limitations

    Explicit CoT remains the performance upper bound, with a persistent gap from PLaT because compressed latent reasoning incurs information loss relative to full-text reasoning.

Abstract

from arXiv · show

Chain-of-Thought (CoT) empowers Large Language Models (LLMs) to tackle complex problems, but remains constrained by the computational cost and reasoning path collapse when grounded in discrete token spaces. Recent latent reasoning approaches attempt to optimize efficiency by performing reasoning within continuous hidden states. However, these methods typically operate as opaque end-to-end mappings from explicit reasoning steps to latent states, and often require a pre-defined number of latent steps during inference. In this work, we introduce PLaT (Planning with Latent Thoughts), a framework that reformulates latent reasoning as planning by fundamentally decouple reasoning from verbalization. We model reasoning as a deterministic trajectory of latent planning states, while a separate Decoder grounds these thoughts into text when necessary. This decoupling allows the model to dynamically determine when to terminate reasoning rather than relying on fixed hyperparameters. Empirical results on mathematical benchmarks reveal a distinct trade-off: while PLaT achieves lower greedy accuracy than baselines, it demonstrates superior scalability in terms of reasoning diversity. This indicates that PLaT learns a robust, broader solution space, offering a transparent and scalable foundation for inference-time search. Our code can be found in https://github.com/yunsaijc/PLaT.

1 Introduction

PLaT reframes latent reasoning as planning over continuous states, separating reasoning from verbalization to address path collapse, opacity, and fixed inference budgets. Its evaluations reveal a trade-off: lower greedy precision but stronger scaling in reasoning diversity.

  • Motivation: CoT can collapse reasoning paths because each generation step commits to a discrete token, pruning alternative valid strategies and causing irrecoverable errors.Discrete token generation also incurs high costs when producing long sequences of intermediate reasoning tokens.
  • Motivation: Prior latent reasoning methods use opaque intermediate states and often require a fixed number of latent inference steps regardless of problem difficulty.This fixed budget lacks the flexible computation associated with human System 2 thinking.
  • PLaT: PLaT models reasoning as a trajectory of latent planning states and decouples a latent Planner from a Decoder that verbalizes thoughts when needed.The Planner evolves states in a continuous manifold, while the Decoder grounds plans in language through reconstruction.
  • PLaT: The decoupled architecture enables interpretable intermediate reasoning and dynamic termination instead of relying on static inference hyperparameters.Latent states can be decoded into text without disrupting the continuous reasoning flow.
  • Empirical findings: PLaT sacrifices greedy precision but achieves superior Pass@k scaling and reduced diversity saturation under search-based inference.The result frames a trade-off between deterministic accuracy and exploration potential.

2 Related Work

Related work develops explicit multi-path reasoning and latent reasoning methods, while PLaT is presented through a two-stage Planner–Decoder framework for latent planning and verbalization.

  • Explicit reasoning: CoT explicitly decomposes complex problems into intermediate reasoning steps, and self-consistency samples multiple CoT trajectories before marginalizing final answers.These approaches use explicit language-space reasoning paths to improve problem solving or robustness.
  • Explicit reasoning: Tree-of-Thought extends CoT into explicit tree search by branching, evaluating, and backtracking over intermediate states.
  • PLaT framework: PLaT’s SFT stage generates latent states with a Planner and verbalizes them with a Decoder, while its RL stage samples Decoder outputs and reinforces valid equations and correct answers.
  • Latent reasoning: Recent latent reasoning methods shift part of reasoning into continuous hidden states while retaining natural-language final answers to reduce explicit CoT costs and path collapse.Some approaches introduce pause or planning tokens for internal processing before surface-level generation.

3 Method

PLaT separates latent planning from textual verbalization through a Planner–Decoder architecture, enabling deterministic latent trajectories, flexible termination, and decoding only when needed. Training combines reconstruction with reinforcement-learning policy refinement while preserving the learned latent structure.

  • Problem formulation: PLaT reformulates reasoning as a latent autoregressive process, replacing discrete token-by-token reasoning with continuous latent planning states.Standard CoT commits to specific tokens at each step, potentially pruning alternative reasoning paths.
  • Architecture: The Planner generates deterministic latent vectors, while the Decoder conditions on aggregated planning states to verbalize each reasoning step.The Decoder receives the current aggregated state as a soft prefix and must rely on that state for historical context.
  • Architecture: Exponential moving-average aggregators stabilize fine-grained latent states by accumulating information separately across latent slots.The final planning state concatenates the stabilized aggregators, with the smoothing coefficient set to 1 for the first step.
  • Training: Supervised fine-tuning uses reconstruction loss for intermediate reasoning steps and final answers, with Gaussian noise injected into accumulated states.This removes the need for mode switching and a fixed number of latent steps while encouraging the Decoder to learn manifold structure.
  • Inference: Lazy Decoding generates latent trajectories without full text and uses a first-token semantic probe to decide whether to continue reasoning or decode the final answer.Intermediate probe tokens are discarded, while the answer is fully decoded only after the conclusion token is detected.
  • Policy refinement: Reinforcement learning refines the Decoder’s search policy while deterministic latent states decouple planning stability from exploration.The Decoder samples multiple verbalization paths from fixed latent states and applies a group-relative policy objective based on normalized rewards.

4 Experiments

Experiments show that PLaT trades lower greedy precision for broader and more scalable reasoning diversity, while retaining interpretability and improving efficiency over explicit CoT. Its exploration benefits are strongest under search-based inference, but reinforcement learning improves in-domain accuracy at the cost of diversity and OOD performance.

  • Effectiveness: On GSM8k, PLaT-2 reaches 74.2% Pass@128, versus 66.7% for Coconut and 70.1% for CODI.PLaT also surpasses both baselines on Pass@128 across GSM8k, GSM-HARD, and SVAMP, while baseline diversity curves show saturation.
  • Effectiveness: PLaT-2 improves GSM8k Pass@128 over PLaT-1, 74.2% versus 72.8%, but PLaT-1 is slightly better or comparable on SVAMP and GSM-HARD.Increasing latent sequence length raises capacity in-domain but may introduce optimization challenges or source-domain overfitting.
  • Reinforcement Learning: RL consistently improves GSM8k greedy accuracy but decreases Pass@128 and degrades performance on SVAMP and MultiArith.The reported pattern is attributed to collapsing the planning distribution toward high-likelihood in-domain trajectories and overfitting the training-domain reward.
  • Efficiency and Interpretability: PLaT-1 reduces inference latency to 152.6ms from CoT’s 349.6ms, an approximately 56% reduction, while remaining slower than Coconut.PLaT’s Decoder adds forward-pass overhead for termination checks, but intermediate latent states remain inspectable unlike Coconut’s opaque states.
  • Reasoning Diversity: PLaT maintains an average branching factor approximately 1.0 higher than CoT, while its valid-branch count decays more slowly and eventually surpasses CoT.Both methods exhibit an inverted-U exploration pattern: the search space expands initially and narrows later.
  • Reasoning Diversity: PLaT shifts samples toward higher branching and moderate validity, accumulating more valid samples than CoT, including differences of +77, +71, and +59.This distribution prioritizes solution-space coverage over precision of a single trajectory, supporting search-based inference such as Tree-of-Thoughts or rejection sampling.
  • Ablation Study: Removing full question context lowers greedy accuracy from 28.66% to 21.30% but produces the highest Pass@128.The result illustrates a precision–exploration trade-off: fewer contextual constraints may encourage broader exploration.
  • Ablation Study: Removing EMA aggregation or training noise degrades all metrics, while the residual variant performs worst in exploration and second worst in greedy accuracy.The ablations support EMA and denoising as stabilizers of the latent trajectory and manifold, whereas parameter sharing regularizes the latent space.

5 Conclusion

PLaT decouples latent thought from verbalization by modeling latent states as explorable planning trajectories. It enables dynamic termination and trades some greedy accuracy for broader, more scalable reasoning diversity.

  • PLaT reformulates latent reasoning as planning over continuous latent space rather than implicit end-to-end pattern matching.
  • Its glass-box architecture models latent states as explorable planning trajectories anchored to the language manifold.
  • Dynamic termination allows latent reasoning to stop without relying on static inference hyperparameters.
  • PLaT sacrifices some deterministic accuracy but scales better in reasoning diversity than baselines, which experience diversity saturation.
  • The framework offers transparent, continuous, dynamic, and interpretable internal thoughts for inference-time scaling and search-based reinforcement learning.

A Notations

Table 1 summarizes the notations and special tokens used in PLaT.

  • Table 1 provides a summary of PLaT’s notations and special tokens.

B.1 More Implementation Details

The implementation uses LoRA-based fine-tuning and separates Planner and Decoder parameters during reinforcement learning. Rule-based rewards evaluate semantic validity and mathematical correctness at intermediate and final steps.

  • LoRA uses rank 128 and α = 32, while the additional Planner layers are fully trained.
  • During RL, Planner LoRA weights are frozen to preserve the latent manifold, while Decoder LoRA weights remain trainable.
  • The KL penalty constrains the policy relative to the frozen SFT policy through LKL = βDKL(πθ||πref).
  • The rule-based reward combines semantic validity and mathematical correctness for intermediate reasoning steps and final answers.
  • Intermediate steps receive rewards for containing mathematically extractable equations and for equation correctness.
  • Final answers receive 0.2 for valid formatting, 1.0 for correctness, and −0.2 for an incorrect extracted answer.

B.2 Agreement of LLM Judgments

The agreement study compares human and LLM judgments using accuracy, Cohen’s Kappa, and a confusion matrix. The automated evaluator shows high agreement, with discrepancies occurring only as false positives.

  • The agreement analysis reports human–LLM judgment metrics and a confusion matrix.
  • The study manually labeled 200 PLaT and CoT reasoning steps to assess GPT-4o-mini’s evaluator reliability.
  • 95.0% overall accuracy and Cohen’s Kappa κ = 0.8721 indicate strong agreement between the automated evaluator and the human annotator.
  • The confusion matrix shows 10 discrepancies, all false positives, with zero false negatives.

B.3 Entropy Analysis

PLaT’s reasoning entropy measures uncertainty in the Decoder’s token distribution at each latent reasoning step. Compared with baselines, PLaT preserves higher entropy across most of the reasoning process, indicating less early pruning of alternative paths.

  • Entropy definition: Reasoning entropy H(S_k) measures the Shannon entropy of the Decoder’s first-token distribution for latent state S_k at step k.The distribution assigns probabilities P(v | s_k) over vocabulary tokens.
  • Entropy evolution: PLaT maintains significantly higher entropy than Explicit CoT and CODI across most of reasoning progress, from 20% to 90%.Explicit CoT and CODI rapidly lose entropy after the initial 10%-30% of progress, indicating early concentration on narrow probability paths.
  • Latent sequence length: NL = 1 or 2 generally provides the best balance between accuracy and diversity across datasets.This suggests that compact latent trajectories are sufficient for current reasoning tasks.

B.4 Detailed Analysis of Hyperparameters

The sensitivity analysis shows that PLaT’s performance depends non-monotonically on latent sequence length, while EMA smoothing and latent dimension regulate information stability and capacity.

  • Sensitivity analysis: The analysis supports hyperparameter choices through sensitivity experiments over latent sequence length, EMA coefficient, and latent dimension.The reported figures examine α_EMA for NL = 1 and NL = 2, and d_s with NL = 2 and α_EMA = 0.5.
  • Sensitivity to Latent Sequence Length: NL = 2 provides the optimal GSM8k balance, with higher greedy accuracy than NL = 1 and superior Pass@128.Increasing NL beyond 2 consistently degrades both precision and diversity.
  • Impact of EMA Coefficient: α_EMA = 0.9 is preferred for NL = 1, whereas α_EMA = 0.5 yields more robust results for NL = 2.Sparse planning benefits from retaining more raw state information, while longer horizons require stronger smoothing to stabilize information flow.
  • Effect of Latent Dimension: A latent dimension of d_s = 2048 is a robust sweet spot across all benchmarks.Lower dimensions restrict planning-state expressivity, while d_s = 4096 adds computational cost without substantial gains.

D Limitations and Future Work

The paper identifies limitations in reinforcement-learning exploration, latent-state scaling, and evaluation scope. It proposes joint optimization, improved training, and broader task validation as future directions.

  • Reinforcement Learning: Reinforcement-learning exploration is preliminary because the Planner is frozen while optimization is restricted to the Decoder.This preserves latent-manifold semantic stability but prevents the Planner from learning new reasoning topologies or correcting logic errors through trial and error.
  • Latent-state scaling: Performance saturates beyond NL = 2 despite the theory that more latent states should increase information capacity.The authors characterize this as likely an optimization challenge requiring advanced training techniques.
  • Evaluation scope: Evaluation is concentrated on mathematical reasoning, leaving efficacy in creative writing, common-sense reasoning, and complex code generation empirically unvalidated.The authors identify broader task evaluation as a key future objective.
Loading 2601.21358v2…