PaperlayerView exact revision on arXiv

Source-linked AI summary

Latent Thoughts Tuning: Bridging Context and Reasoning with Fused Information in Latent Tokens

Weihao Liu, Dehai Min, Lu Cheng

arXiv:2602.10229v2cs.CL

TL;DR

Existing latent reasoning methods face alignment, distribution-mismatch, feature-collapse, and static-allocation problems. LT-Tuning combines contextual and predictive information through Context-Prediction-Fusion, confidence-driven switching, and progressive curriculum training. It outperforms prior latent reasoning baselines across model scales with robust scaling behavior.

  • Problem

    Existing latent reasoning methods can suffer from distribution mismatch and feature collapse when reusing hidden states, or representational misalignment when using assistant models.

  • Method

    LT-Tuning fuses contextual hidden states with predictive vocabulary-embedding guidance, dynamically switches between latent and explicit reasoning, and applies a progressive three-stage curriculum.

  • Results

    LT-Tuning consistently outperforms existing latent reasoning baselines across 1B–8B models, achieving up to a 4.3% average improvement over the strongest prior method.

  • Takeaways & Limitations

    The framework provides robust latent reasoning without external assistants while mitigating feature collapse and supporting dynamic reasoning allocation.

  • Takeaways & Limitations

    Improved reasoning could be misused for convincing misinformation or harmful-content creation, although the authors do not view these risks as uniquely exacerbated by their contributions.

Abstract

from arXiv · show

While explicit Chain-of-Thought (CoT) equips Large Language Models (LLMs) with strong reasoning capabilities, it constrains the model's thoughts to a discrete vocabulary space. Recently, reasoning in continuous latent space has emerged as a promising alternative, but current paradigms suffer from feature collapse and instability due to distribution mismatch when recurrently reusing hidden states, or alignment issues when relying on assistant models. To address this, we propose Latent Thoughts Tuning (LT-Tuning), a post-training framework that redefines how latent thoughts are constructed and deployed. Instead of relying solely on raw hidden states, our method introduces a Context-Prediction-Fusion mechanism that jointly leverages contextual hidden states and predictive semantic guidance from the vocabulary embedding space. Combined with a progressive three-stage curriculum learning pipeline, LT-Tuning also enables dynamic switching between latent and explicit thinking modes. Experiments demonstrate that our method outperforms existing latent reasoning baselines, effectively mitigating feature collapse and achieving robust reasoning accuracy.

1. Introduction

Explicit CoT supports multi-step reasoning but confines thoughts to discrete tokens, motivating latent-space alternatives. LT-Tuning addresses latent reasoning’s alignment and allocation challenges with fused latent tokens, confidence-driven switching, and progressive training, outperforming prior baselines across scales.

  • Explicit CoT enables multi-step reasoning by verbalizing intermediate steps, but constrains reasoning to a discrete token sequence.
  • Latent-space reasoning instead operates directly on high-dimensional hidden states to decouple internal reasoning from explicit text generation.
  • Latent reasoning must construct semantically expressive representations compatible with the model’s embedding space.
  • Static reasoning schedules waste computation on trivial steps while providing insufficient depth for complex reasoning.
  • LT-Tuning combines hidden-state context with probability-weighted vocabulary guidance, dynamically switches between explicit and latent reasoning, and uses three-stage curriculum learning.
  • LT-Tuning consistently outperforms existing latent reasoning baselines across 1B–8B models, reaching up to a 4.3% average improvement over the strongest prior method.

2. Related Work

Related work spans explicit reasoning, latent-token reasoning, assistant-based representations, recurrent transformers, and training-free switching. LT-Tuning differs as a post-training approach for off-the-shelf models with token-level latent computation and fused latent-token construction.

  • Explicit reasoning decomposes complex problems into verbalizable intermediate steps and has been extended with program-aided, self-consistency, and tree-structured methods.
  • Coconut recurrently feeds hidden states as latent inputs, while Soft-Thinking uses probabilistic mixtures but discards contextual hidden-state information.
  • Assistant-based methods avoid training large models but risk representational misalignment between assistant outputs and the reasoning model’s embedding space.
  • Recurrent transformers support iterative refinement but typically require pretraining from scratch, limiting applicability to existing LLMs.
  • LT-Tuning is a post-training framework for off-the-shelf models that enables token-level recurrent latent computation and stabilizes latent-token construction through Context–Prediction Fusion.

3. Preliminaries

Standard decoding couples each next input to a discrete vocabulary token, restricting the reasoning trace. Latent reasoning replaces this constraint with recurrent continuous vectors, but directly reusing hidden states creates a distribution mismatch that motivates fused construction.

  • A decoder-only Transformer produces next-token probabilities from contextual hidden states through a linear projection and softmax.
  • Standard decoding couples the next input embedding to a selected or sampled discrete token, restricting reasoning to the vocabulary grid.
  • Latent reasoning processes continuous vectors as recurrent inputs, allowing a thought process to evolve without collapsing into discrete tokens at every step.
  • Coconut directly reuses preceding hidden states as latent inputs, creating distribution mismatch because output contextual states differ from the input embedding manifold.
  • A constructive latent-token mapping can fuse hidden-state history with predictive vocabulary guidance to stabilize the latent reasoning trajectory.

4. Methodology

LT-Tuning progressively trains explicit reasoning, confidence-based latent insertion, and Context-Prediction Fusion. The method combines predictive embedding guidance with contextual hidden states to create compatible latent inputs while supporting dynamic reasoning allocation.

  • Framework overview: LT-Tuning is a post-training framework that dynamically chooses latent reasoning or explicit text generation instead of enforcing a static latent-token schedule.
  • Stage 1: Explicit Reasoning Warm-up: Stage 1 fine-tunes the pretrained model on explicit CoT data to establish step-by-step reasoning capabilities.
  • Stage 2: Confidence-driven latent insertion: Stage 2 inserts latent steps at low-confidence positions, initializes them from hidden states, and trains subsequent explicit-token prediction on the mixed sequence.
  • Stage 3: Context-Prediction Fusion: Stage 3 addresses output–input distribution mismatch by fusing a probability-weighted predictive embedding with a contextual hidden state.
  • Stage 3: Context-Prediction Fusion: The fused representation becomes the latent input, preserving contextual information while improving compatibility with the input embedding space.
  • Stage 3: Context-Prediction Fusion: Removing fusion reduces average accuracy by 23.5% at 8B and 8.0% at 3B, supporting a larger fusion benefit for the larger model.

5. Experiments

LT-Tuning is evaluated across three model scales and four mathematical reasoning benchmarks, with comparisons against explicit, intrinsic, and assistant-based baselines. It shows consistent scaling and adaptive latent-token allocation, while ablations and visualizations examine the roles of curriculum learning, fusion, and latent reasoning.

  • Experimental Setup: Experiments use Llama-3.2-1B, Llama-3.2-3B, and Llama-3.1-8B, trained on GSM8K and evaluated on GSM8K-NL, ASDiv-Aug, MultiArith, and SVAMP.
  • Main Results: LT-Tuning achieves the best average performance at all scales: 36.4% (1B), 52.4% (3B), and 68.8% (8B).
  • Main Results: Coconut’s average accuracy drops from 50.3% to 41.5% at 8B, whereas LT-Tuning reaches 68.8% and shows healthy scaling.
  • Baseline Analysis: Assistant-based methods show erratic performance, including SemCoT’s 73.5% on ASDiv-Aug versus 6.6% on MultiArith for the 3B model.
  • Adaptive Computation: Latent-token counts generally increase with question difficulty, particularly for the 8B model, indicating difficulty-aware allocation of latent reasoning effort.
  • Ablation Study: Removing Stage 2 reduces average accuracy by 3.9% at 3B and 6.7% at 8B, while removing Stage 3 causes the largest 8B drop of 23.5%.

6. In-Depth Analyses of LT-Tuning

In-depth analyses examine generation dynamics, feature diversity, and hidden-layer selection. LT-Tuning reduces uncertainty, attends more to latent tokens, mitigates feature collapse, and remains relatively robust to layer choice.

  • Generation Dynamics: LT-Tuning reduces uncertainty during generation and produces fewer uncertainty peaks than the pause-token variant.
  • Attention Allocation: LT-Tuning allocates substantially more attention to latent <thinking> tokens than the pause-token baseline, indicating active use of their encoded information.
  • Feature Collapse: PCA visualizations show Coconut collapsing after two reasoning steps, while LT-Tuning maintains semantic diversity through six latent tokens.
  • Layer Selection: Performance is relatively robust to hidden-layer choice for context extraction, although the last layer is better for Llama-3.1-8B.
  • Layer Selection: Fusion learning in Stage 3 may compensate for suboptimal layer choices and plays a more important role than the selected context layer.

7. Conclusion

The conclusion presents LT-Tuning as a continuous latent-reasoning framework that combines contextual history, predictive vocabulary guidance, dynamic switching, and progressive curriculum learning. Across 1B–8B models, it reports stronger mathematical reasoning performance and reduced feature collapse.

  • LT-Tuning combines contextual hidden states with predictive vocabulary-distribution guidance through Context-Prediction-Fusion.
  • The framework couples fusion with confidence-driven dynamic switching and a progressive three-stage curriculum learning strategy.
  • Evaluations from 1B to 8B demonstrate stronger performance than existing baselines while mitigating feature collapse, especially in larger models with untied embeddings.

Impact Statement

The impact statement emphasizes efficiency and robustness benefits from reducing reliance on verbose intermediate text generation, while acknowledging possible misuse and interpretability considerations.

  • LT-Tuning reduces reliance on verbose intermediate text generation, with potential benefits for computational cost and inference latency in deployed systems.
  • The dynamic insertion mechanism explicitly marks positions of model uncertainty, which may provide signals for understanding model behavior.
  • Improved reasoning could be misused for convincing misinformation or harmful-content automation, although the authors state these risks are not uniquely exacerbated.

A. Dataset Statistics

The supplied passages describe mathematical evaluation data and provide a theoretical analysis of LT-Tuning’s predictive embedding and fusion rule, including conditions under which fusion reduces mismatch and improves estimation.

  • Dataset Statistics: All evaluation datasets focus on mathematical word problems requiring multi-step arithmetic reasoning.
  • Predictive Embedding: The predictive embedding is the unique conditional mean-squared-error-optimal estimate of the next-token embedding under the model’s predictive distribution.
  • Fusion and Mismatch: Because the predictive embedding lies in the convex hull of token embeddings, fusion provides a representation compatible with a conservative embedding-space surrogate.
  • Fusion and Mismatch: For any α < 1, the fused representation strictly reduces off-manifold mismatch relative to directly reusing the hidden state.
  • Optimal Fusion: Under a complementary-observation model, the MSE-optimal estimator has the fusion form αh + (1 −α)epred, with interior fusion strictly outperforming both endpoint estimators when the condition holds.
  • Assumptions and Scope: The analysis assumes zero-mean observation errors, while allowing nonzero bias through an affine shift term that can be absorbed during Stage 3 training.

B.5. Unifying Results 2 and 3: Mismatch-Aware Fusion

The unified objective balances manifold compatibility and estimation accuracy, yielding a mismatch-aware fusion weight whose behavior matches observed model-scale effects. Additional experiments indicate transfer across model families and beyond arithmetic reasoning.

  • Unified objective: The unified fusion objective jointly optimizes manifold compatibility and estimation accuracy through a convex quadratic in α.It combines the desiderata addressed separately by the preceding results and produces a closed-form minimizer.
  • Mismatch-aware behavior: Larger hidden-state mismatch predicts a smaller optimal α, increasing reliance on the predictive component.This prediction is associated with models using untied input–output embeddings, such as Llama-3.1-8B.
  • Mismatch-aware behavior: −23.5% without fusion at 8B versus −8.0% at 3B, showing that fusion is most critical at the larger model scale.The comparison is reported as an ablation result for Stage 3 fusion.
  • Complementary information: Complementary errors favor interior fusion, while removing either the hidden-state or predictive component degrades performance.The ablation contrasts removing Stage 3 with the Coconut configuration.
  • Scope: The theoretical results do not establish that the specific fixed α is globally optimal.They instead support the fusion form and its qualitative behavior across model scales.
  • Transfer: LT-Tuning delivers consistent gains over the strongest baseline on Qwen2.5-0.5B and Qwen2.5-1.5B.The cross-family evaluation uses the same Stage-1 checkpoint and GSM8K training data across methods.
  • Transfer: LT-Tuning preserves its advantage on GPQA-Diamond after training on non-arithmetic Bespoke-Stratos-17k data.This evaluates transfer beyond arithmetic reasoning with a Llama-3.2-1B backbone.

E. Compute Efficiency Analysis

LT-Tuning reduces generated text while improving accuracy, indicating that latent tokens can replace verbose intermediate reasoning rather than add inference overhead.

  • Token-level efficiency: 24.1% fewer text tokens on average while simultaneously improving accuracy by +7.1 points on four mathematical reasoning benchmarks.The analysis compares LT-Tuning with a fine-tuned Explicit CoT baseline on Llama-3.1-8B.

F.1. Training Hyperparameters

The implementation uses staged training, confidence-based latent-token insertion, scale-sensitive layer selection, and fusion-specific components. Evaluation employs deterministic decoding and standardized baseline settings, alongside qualitative examples.

  • Training configuration: Training hyperparameters vary by model scale across all three LT-Tuning stages to accommodate memory requirements and optimization dynamics.Table 7 reports the scale-specific learning rates and batch sizes.
  • Training stages: Stage 1 performs supervised fine-tuning on GSM8K Chain-of-Thought data using step-by-step natural-language solutions.The prompt includes the question, CoT reasoning, and final answer.
  • Dynamic latent tokens: The confidence threshold τ controls latent-token insertion granularity, with lower thresholds producing fewer insertions and higher thresholds producing more frequent latent reasoning.The number of inserted latent tokens is randomly sampled from 0 ∼k according to model confidence.
  • Layer selection: For Llama-3.1-8B, the method generally uses the last or penultimate hidden-state layer because intermediate layers are harder to interpret under distribution misalignment.Layer selection is more consequential for the 8B model than for smaller models.
  • Fusion implementation: The 8B adapter projects hidden states down and back to the embedding dimension and is trained jointly during Stages 2 and 3.It addresses mismatch caused by untied input and output embeddings.
  • Evaluation: Inference uses greedy decoding, dynamically generating <thinking> tokens from confidence patterns learned during Stages 2 and 3.Final numerical answers are extracted with a pattern identifying the last number in the output.
  • Baseline comparison: Baselines share the same Stage-1 checkpoint and are trained on GSM8K with their official code under matched experimental settings.All methods are evaluated on the test sets of four benchmarks.
  • Qualitative analysis: Qualitative examples compare LT-Tuning and regular CoT reasoning trajectories for Llama-3.1-8B.The passage states that incorporating latent tokens enhances reasoning capabilities and achieves higher accuracy.
Loading 2602.10229v2…