PaperlayerView exact revision on arXiv

Source-linked AI summary

LLM Latent Reasoning as Chain of Superposition

Jingcheng Deng, Liang Pang, Zihao Wei, Shicheng Xu, Zenghao Duan, Kun Xu, Yang Song, Huawei Shen, Xueqi Cheng

arXiv:2510.15522v2cs.CL

TL;DR

Latent reasoning can reduce the length and cost of explicit chains, but current approaches face performance degradation and challenges in token representation, chain definition, and learning. Latent-SFT addresses these issues with a unified token, chain, and learning framework. Across six mathematical benchmarks, it outperforms explicit SFT while shortening reasoning chains.

  • Problem

    Latent reasoning methods can reduce chain length but often underperform explicit reasoning, with gaps in representation, chain definition, and step-by-step learning guidance.

    Source: §1 · pp. 1–2

  • Method

    Latent-SFT combines vocab-space-constrained Latent-Vocab, Induction-Supervision Masking for Latent-Chains, and stochastic Gumbel-Softmax optimization.

    Source: §1 · p. 2

  • Results

    Across six mathematical benchmarks, Latent-SFT consistently outperforms explicit SFT while reducing reasoning-chain length by 2.7x to 5.5x.

    Source: §1 · p. 2, §8 · p. 9

  • Takeaways & Limitations

    The authors conclude that the learned latent process is a superposition of multiple reasoning chains, suggesting an evolution of chain-of-thought into multi-path latent reasoning.

    Source: §8 · p. 9

    The Latent-Vocab design assumes latent tokens should preserve vocab-space distributional geometry, an assumption motivated by analysis of embedding spectra.

    Source: §3.2 · p. 3

Abstract

from arXiv · show

Latent reasoning offers a computation-efficient alternative to Chain-of-Thought but often suffers from performance degradation due to distributional misalignment and ambiguous chain definitions. Ideally, latent reasoning should function as a superposition of multiple reasoning paths. To realize this, we introduce Latent-SFT, a unified framework addressing challenges at three levels: token, chain, and learning. First, we define the Latent-Vocab to constrain hidden states within the pre-trained vocab-space. Second, we construct the Latent-Chain via Induction-Supervision Masking to ensure semantic compactness and sufficiency. Third, we employ Latent-Optim with stochastic Gumbel-Softmax to guide the model toward generalizable solutions. Empirical results demonstrate that Latent-SFT consistently outperforms explicit SFT across six mathematical benchmarks (e.g., GSM8k, AIME24) while achieving a 2.7x to 5.5x reduction in reasoning length. Analysis confirms that our method effectively captures a superposition of diverse reasoning trajectories rather than merely compressing a single path.

1. Introduction

Latent reasoning could reduce the redundancy and computational cost of long explicit chains, but existing approaches face performance degradation and challenges in token representation, chain definition, and learning. Latent-SFT addresses these challenges and reports stronger results than explicit SFT with shorter reasoning chains.

1. Introduction

  • Latent reasoning offers a shorter-chain alternative to natural-language reasoning, but current methods often suffer performance degradation relative to explicit reasoning.Long token chains are also described as redundant and computationally costly.Source: §1 · p. 1
  • Current latent methods use unconstrained hidden states, ambiguous chains that may decode only the final answer, and indirect final-answer supervision without step-by-step guidance.The authors identify distributional mismatch, neglected intermediate semantic fidelity, and unstable reasoning trajectories as concerns.Source: §1 · pp. 1–2
  • Latent-SFT unifies token, chain, and learning strategies: Latent-Vocab constrains representations to vocab-space, Induction-Supervision Masking constructs Latent-Chains, and stochastic Gumbel-Softmax guides optimization.The chain design targets semantic compactness, compatibility, and sufficiency; the stochastic optimization is described as guiding the model toward flatter, more generalizable minima.Source: §1 · p. 2
  • Across six mathematical benchmarks, Latent-SFT consistently outperforms explicit SFT while reducing reasoning-chain length by 2.7x to 5.5x.The authors also report that its latent reasoning represents a superposition of multiple chains rather than compression of one explicit path.Source: §1 · p. 2

2. Related Work

The related-work discussion groups latent-token methods by how they represent tokens and contrasts unconstrained hidden-state approaches with constrained representations. It identifies limits in alignment, performance, and reasoning-chain compression among existing approaches.

2. Related Work

  • Prior latent-token work is categorized by representation, while chain definition and optimization strategies are described as comparatively overlooked.The authors identify token representation as the primary differentiator in the field.Source: §2 · p. 2
  • The cited visual analyses cover hidden-state and token-embedding distribution differences, and singular-value decay curves with effective-rank legends across LLMs.The captions specify the analyses' scope but do not provide numerical outcomes.Source: Figure 2 · p. 2, Figure 3 · p. 2
  • COCONUT and CCOT use last-layer hidden states, while CODI and PCCOT add mapping or distillation but remain continuous-hidden-state methods.The authors associate these unconstrained representations with optimization instability and limited gains relative to explicit reasoning.Source: §2 · p. 2
  • CoLaR uses Gaussian latent tokens, while Soft Thinking and MoI form linear combinations of vocabulary embeddings.The authors describe the embedding-merge methods' performance gains as marginal and say they do not shorten reasoning chains.Source: §2 · p. 2
  • Training-free latent methods are characterized as greedy latent-space decoding, which the authors say cannot realize learned multi-path superposition or the reasoning-length reduction central to their work.The authors exclude these methods from primary baselines to focus on training-based chain compression.Source: §2 · p. 3

3. Latent-Vocab: Superposition of Vocab-Space

This section motivates constraining latent tokens to the model's vocabulary space: hidden states and token embeddings differ distributionally, while embedding matrices have non-full effective rank. It defines a latent token as a superposition within the vocab-space.

3. Latent-Vocab: Superposition of Vocab-Space

  • The section asks what representation latent tokens should use, contrasting its investigation with prior approaches that use raw, unconstrained hidden states.The stated focus is the reasonable representation for latent tokens.Source: §3 · p. 3

3.1. The Distributional Discrepancy in Vector Spaces

  • Last-layer hidden states and token embeddings show substantial distributional divergence, including differences in means and variances and large inter-distribution distances.The authors suggest this mismatch can contribute to distribution shift and training instability because shallow parameters are calibrated on token embeddings during pre-training.Source: §3.1 · p. 3

3.2. The Low Effective Rank of Vocab-Space

  • Embedding matrices have non-full effective rank, with singular values dropping sharply; the authors infer that the semantic manifold is low-dimensional.They propose constraining latent tokens to the embedding subspace rather than modeling them as unconstrained continuous vectors.Source: §3.2 · p. 3
  • A Superposition Embedding defines a latent token as a linear combination of token embeddings, keeping it within the vocab-space's linear span.The coefficients govern the semantic mixture.Source: §3.2 · p. 3

4. Latent-Chain: Forming Latent Chain via Induction-Supervision Masking

The Latent-Chain is designed to retain intermediate reasoning information while meeting semantic compactness, compatibility, and sufficiency criteria. Induction-Supervision Masking builds and aligns latent representations, then trains the model to decode subsequent reasoning and answers from them.

4. Latent-Chain: Forming Latent Chain via Induction-Supervision Masking

  • The Latent-Chain is intended to avoid a black-box chain optimized only for final-answer reconstruction and to preserve intermediate semantic fidelity.Its three criteria are compactness, compatibility with explicit-token semantics, and sufficiency to deduce the correct answer.Source: §4 · p. 3
  • Figure 4 presents two phases: generating latent chains through Induction-Supervision Masking, then training the LLM to generate chains autonomously with weighted KL and CE losses.The mask components use standard autoregressive attention mechanisms.Source: Figure 4 · p. 4
  • The process segments explicit reasoning into subsegments and inserts a latent placeholder after each segment, using LTIM to restrict each placeholder to preceding reasoning context.Segmentation may use a fixed compression ratio or semantic boundaries; the resulting latent token summarizes cumulative reasoning up to that point.Source: §4 · pp. 3–4
  • For semantic compatibility, each placeholder's hidden state is projected onto the vocabulary simplex and reconstructed as a mixture of token embeddings.The overview describes the Latent Token Encoder as sharing structure and initialization with the base LLM.Source: §4 · p. 4, Figure 4 · p. 4
  • Under LTSuM, each latent representation conditions decoding of subsequent explicit reasoning through the final answer, while attention to future latent tokens and past explicit reasoning is blocked.The target includes later reasoning subsegments, the end-of-thought token, and the answer.Source: §4 · pp. 4–5
  • Training alternates encoder optimization with a frozen LLM, LLM optimization with a frozen encoder, and joint fine-tuning.The authors state that this progressive strategy stabilizes training and lets the model adapt to the distributional shift introduced by latent tokens.Source: §4 · p. 5

5. Latent-Optim: Training LLMs for Latent Reasoning

Latent-Optim trains the model to generate latent chains autonomously, addressing the overfitting risk of static encoder-derived targets with stochastic Gumbel-Softmax perturbations. The resulting noise-averaged objective is described as an implicit regularizer that promotes generalization.

5. Latent-Optim: Training LLMs for Latent Reasoning

  • Latent-Optim trains LLMs to generate high-quality latent chains autonomously, rather than relying on indirect backpropagation from final supervision.The authors identify that indirect supervision lacks explicit guidance for the intermediate latent process.Source: §5 · p. 5
  • Static encoder-derived latent targets can overfit to encoder artifacts and limit generalization.Source: §5 · p. 5
  • Each training epoch samples a perturbed latent target using Gumbel-Softmax, treating static targets as parameters of a generative distribution rather than immutable labels.Source: §5 · p. 5
  • The noise vector is independently resampled at each training step, so the model does not encounter the exact same latent target twice.The experiments set temperature τ = 1.Source: §5 · p. 5
  • The objective averages over Gumbel noise, and the authors prove this injection acts as an implicit regularizer that penalizes sharp loss-Hessian curvature and enhances generalization.Source: §5 · p. 5

6. Experiments

Experiments report that Latent-SFT outperforms explicit CoT-SFT across nearly all mathematical reasoning datasets while shortening reasoning chains, including on high-difficulty tasks. The study evaluates multiple model sizes and difficulty settings, and ablations report drops when masking, representation alignment, or Gumbel noise is removed.

6.1. Experimental Setups

  • Low-difficulty evaluation uses GSM8k-Aug, GSM-Hard, SVAMP, and MultiArith; high-difficulty evaluation uses Math500 and AIME24.High-difficulty training is restricted to 57,544 Open-R1 examples with reasoning chains shorter than 4k tokens, a restriction the authors say constrained performance.Source: §6.1 · p. 6
  • The study compares against COT-SFT, iCOT, COCONUT, CODI, and CoLaR, matching 1B/3B LLaMA models to low-difficulty tasks and a 7B DeepSeek model to high-difficulty tasks.Evaluation reports Pass@1 and reasoning-chain token count, with Pass@1/# L as a joint accuracy-efficiency ratio.Source: §6.1 · p. 6
  • All reported results use fixed-length segmentation, which the authors found performed better than semantic-based segmentation.Training uses frozen vocabulary parameters and LoRA with rank 64; evaluation uses different inference libraries and maximum sequence lengths for low- and high-difficulty datasets.Source: §6.1 · p. 6

6.2. Main Results

  • Latent-SFT achieves state-of-the-art performance on all three out-of-distribution datasets.Source: §6.2 · p. 7
  • At 1B, Latent-SFT(2) improves average Pass@1 by 2.0% while reducing average reasoning chain length by approximately 5.5×.Across model sizes, it outperforms explicit CoT-SFT in Pass@1 on nearly all mathematical reasoning datasets.Source: Table 1 · p. 6, §6.2 · p. 7
  • On the 7B model, Latent-SFT surpasses explicit CoT-SFT in Pass@1 while reducing reasoning chain length by 2.7–3.6× on high-difficulty tasks.The text reports that Superposition Embedding outperforms Hidden State on these datasets and that unconstrained vectors struggle to maintain precise semantic alignment.Source: §6.2 · p. 7, Table 2 · p. 7

6.3. Ablation Study

  • Removing either LTIM or LTSuM causes an approximately 5% accuracy drop.LTIM enforces causal independence, while LTSuM retains sufficient semantic information for decoding.Source: §6.3 · p. 7
  • Replacing Superposition Embedding with raw hidden states degrades performance, and removing Gumbel noise also causes a decline.The authors describe stochasticity as a regularizer that helps prevent overfitting and enhance generalization.Source: §6.3 · p. 7

7. Essence of Latent Reasoning

The analysis tests whether latent reasoning only compresses a single path or also superposes multiple paths. ECR and Neff findings support both behaviors.

Hypothesis 1. Latent Reasoning as the Compression of a Single Reasoning Chain

  • ECR@K measures how much explicit-chain information falls within the top K probabilities at corresponding latent steps; values above 1 indicate compression.Larger ECR@K values indicate stronger horizontal compression.Source: §7 · p. 7
  • ECR exceeds 1 for almost all samples, and doubling training compression ratio nearly doubles inference ECR, supporting compression of a single reasoning path.The ECR curves also surpass the uncompressed single-chain baseline.Source: §7 · p. 8, Figure 5 · p. 8

Hypothesis 2. Latent Reasoning as the Superposition of Multiple Reasoning Chains

  • Neff quantifies global evidence for multiple parallel paths, while Top-2 Score is the probability ratio of the two most likely paths.The analysis uses alternative reasoning chains corresponding to one explicit path in GSM8k-Aug.Source: §7 · p. 8
  • In the illustrated Latent-SFT(2) example, Neff stays above 1.7 at each latent step, and the heatmap shows supported paths by step.Path0 is the original chain; the other paths are additional candidates.Source: Figure 7 · p. 8
  • Across compression ratios, most samples exceed the 1.7 parallelism threshold, with evidence spread across roughly 3–4 paths and average Top-2 ratio above 0.6.Parallelism increases as the compression ratio rises.Source: §7 · p. 8, Figure 6 · p. 8

8. Conclusion

The conclusion presents Latent-SFT as a structured approach to multi-path latent reasoning, reporting benchmark gains alongside shorter reasoning chains. It also discloses the use of GPT-5 to polish manuscript language.

8. Conclusion

  • Across six mathematical benchmarks, Latent-SFT consistently outperforms explicit SFT while reducing reasoning-chain length by 2.7x to 5.5x.The conclusion characterizes the learned process as a superposition of multiple chains rather than compression of one explicit path.Source: §8 · p. 9

A. Use of LLMs

  • The authors used GPT-5 to refine drafted manuscript text and incorporated the polished version after checking logical and semantic consistency.Source: A. Use of LLMs · p. 12

B. Training Computational Costs

Training comprises Latent-Chain Construction followed by Latent-Optim. The appendix reports a manageable total training cost despite the multi-stage framework.

B. Training Computational Costs

C. Theoretical Analysis of Stochastic Latent Optimization

The analysis attributes an expected Hessian-based regularization effect to Gumbel-noise injection combined with Softmax translation invariance. The derivation relies on a small-perturbation regime.

C. Theoretical Analysis of Stochastic Latent Optimization

D. Construction of the Multi-Chain GSM8k-Aug Dataset

The appendix describes construction of a multi-path GSM8k-Aug dataset and metrics for assessing whether latent steps support single-chain compression or parallel reasoning paths.

D. Construction of the Multi-Chain GSM8k-Aug Dataset

  • 1,310 of 1,319 GSM8k-Aug examples contain multiple paths, generated with GPT-5, manually checked, and filtered to pairwise edit-distance similarity below 0.9.Figure 9 illustrates rejecting candidate paths too similar to the standard solution or previously accepted paths.Source: D. Construction of the Multi-Chain… · p. 13, Figure 9 · p. 14

E.1. Notation and Preliminaries

  • The setup models a latent chain as T steps with vocabulary distributions and assumes that grouping every r explicit tokens per latent step yields uniform information granularity at inference.The assumption is motivated by expected consistency between training and inference.Source: §E.1 · pp. 14–15

E.2. ECR@K (Effective Compression Rate)

  • ECR@K tests single-chain compression by counting aligned reference-chain tokens in each latent step’s Top-K probabilities; values above 1 mean each step covers over one reference token on average..Source: §E.2 · p. 15

E.3. Neff and Top-2 Score

  • For multiple candidate chains, sequence evidence scores become path posteriors, while Neff and Top-2 indicate the extent of parallel support rather than collapse to one path.A larger Neff and higher Top-2 indicate stronger parallel support across multiple chains.Source: §E.3 · p. 15
  • Neff above 1.7 is treated as evidence of parallel inference, using 1.7 as a conservative baseline for partially overlapping prefixes; K=100 mitigates tail noise.The baseline corresponds to approximately 70% probability mass on path 1 and 30% on path 2.Source: §E.3 · p. 15
  • Nearly 50% shorter reasoning chains than the reference solution still yield the correct GSM8k-Aug answer, 18, in the illustrated Latent-SFT(2) example.The example displays latent tokens rather than explicit natural-language steps.Source: Figure 10 · p. 16

Related papers

Loading 2510.15522v2…