PaperlayerView exact revision on arXiv

Source-linked AI summary

Parallel Test-Time Scaling for Latent Reasoning Models

Runyang You, Yongqi Li, Meng Liu, Wenjie Wang, Liqiang Nie, Wenjie Li

arXiv:2510.07745v4cs.CLcs.AIcs.LG

TL;DR

Parallel test-time scaling in latent reasoning remains an open question because continuous vectors lack inherent sampling mechanisms and trajectory scores. The paper introduces uncertainty-inspired stochastic sampling and a step-wise contrastive Latent Reward Model; experiments report compute-scalable sampling and effective trajectory selection. The findings point to parallel inference as a route for scaling latent reasoning.

  • Problem

    Whether latent reasoning models can benefit from parallel test-time scaling remains open because continuous latent vectors lack inherent sampling mechanisms and trajectory-scoring signals.

    Source: §1 · p. 2

  • Method

    The framework uses Monte Carlo Dropout and Additive Gaussian Noise for stochastic sampling, and a step-wise contrastively trained Latent Reward Model to score and guide latent trajectories.

    Source: §1 · p. 2

  • Results

    Both sampling strategies scale effectively with increased compute and show distinct exploration dynamics, while LatentRM enables consistent gains with best-of-N and beam search across compute budgets.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings demonstrate that parallel test-time scaling can transfer to latent reasoning through redesigned sampling and aggregation.

    Source: §1 · p. 2, §8 · p. 9

    Future work should explore integrating sampling and aggregation into reinforcement learning and adapting ensemble techniques to continuous latent representations.

    Source: §8 · p. 9

Abstract

from arXiv · show

Parallel test-time scaling (TTS) is a pivotal approach for enhancing large language models (LLMs), typically by sampling multiple token-based chains-of-thought in parallel and aggregating outcomes through voting or search. Recent advances in latent reasoning, where intermediate reasoning unfolds in continuous vector spaces, offer a more efficient alternative to explicit Chain-of-Thought, yet whether such latent models can similarly benefit from parallel TTS remains open, mainly due to the absence of sampling mechanisms in continuous space, and the lack of probabilistic signals for advanced trajectory aggregation. This work enables parallel TTS for latent reasoning models by addressing the above issues. For sampling, we introduce two uncertainty-inspired stochastic strategies: Monte Carlo Dropout and Additive Gaussian Noise. For aggregation, we design a Latent Reward Model (LatentRM) trained with step-wise contrastive objective to score and guide latent reasoning. Extensive experiments and visualization analyses show that both sampling strategies scale effectively with compute and exhibit distinct exploration dynamics, while LatentRM enables effective trajectory selection. Together, our explorations open a new direction for scalable inference in continuous spaces. Code and checkpoints released at https://github.com/ModalityDance/LatentTTS

1 Introduction

The introduction asks whether parallel test-time scaling can benefit latent reasoning, whose continuous vectors lack token-based sampling and trajectory-scoring signals. It presents stochastic sampling and LatentRM as solutions, reporting compute-scalable sampling and effective trajectory selection.

1 Introduction

  • Parallel test-time scaling samples multiple reasoning paths and aggregates outcomes through voting or search, using additional inference compute without parameter updates or retraining.The introduction positions latent reasoning as an alternative to explicit token-by-token reasoning.Source: §1 · pp. 1–2
  • Latent reasoning uses continuous vectors instead of tokens for intermediate steps and may be more compact and computationally efficient than explicit Chain-of-Thought.The introduction describes it as a promising alternative for reasoning that can capture abstract patterns difficult to express in natural language.Source: §1 · p. 2
  • Latent models lack an inherent stochastic sampling mechanism because their continuous vectors do not provide the explicit probability distributions used by token-based sampling.They also lack likelihoods or stepwise scores for evaluating and aggregating sampled paths.Source: §1 · p. 2
  • Monte Carlo Dropout and Additive Gaussian Noise introduce stochasticity, while step-wise contrastive training equips LatentRM to score and guide latent reasoning.The methods respectively target epistemic and aleatoric uncertainty, and LatentRM discriminates among candidate thoughts at each step.Source: §1 · p. 2
  • Both sampling strategies scale with increased compute, while LatentRM enables consistent gains with best-of-N and beam search across compute budgets.The methods also show distinct exploration dynamics, and ablations identify contrastive supervision and stochastic rollouts as crucial.Source: §1 · p. 2

2 Related Work

Related work describes test-time scaling through longer reasoning traces or parallel trajectory generation and aggregation, while noting that these approaches rely on token-level sampling. It also traces latent reasoning from architectural modifications to latent autoregressive methods, including early attempts at stochasticity.

2.1 Test-Time Scaling

  • Test-time scaling has sequential and parallel axes: generating longer traces or generating multiple trajectories for aggregation, often combining both for accuracy–compute trade-offs.Aggregation and decision rules include adaptive voting and confidence-based stopping, but these methods rely on token-level sampling.Source: §2.1 · p. 3

2.2 Latent Reasoning

  • Latent reasoning research includes architectural changes and latent autoregressive methods such as COCONUT and CODI, while CoLaR and SoftCoT++ made initial attempts to inject noise.The cited passage contrasts this work with CoT's reliance on natural language, which it describes as inefficient and limiting for abstract insights or intuitive leaps.Source: §2.2 · p. 3

3 Preliminaries

The preliminaries define latent reasoning as autoregressive generation in continuous hidden space, where each latent vector represents an intermediate thought. They describe inference termination and distinguish how latent models control reasoning length.

3 Preliminaries

  • A latent reasoning model autoregressively generates continuous hidden-state vectors as intermediate thoughts, bypassing token-level verbalization for a more compact and efficient alternative to explicit CoT.Source: §3 · p. 3
  • At inference, the model generates a sequence of latent thoughts autoregressively and ends latent thinking with an <|EoT|> token, which triggers explicit answer generation.Source: §3 · p. 3
  • COCONUT and CODI use predetermined reasoning lengths, whereas CoLaR adjusts reasoning length through a prompt-specified compression factor and adaptive termination.The passage gives “thinking speed = 5×” as an example of the prompt specification.Source: §3 · p. 3

4 Method

The method combines stochastic sampling in latent space with a learned scorer for trajectory aggregation. It uses uncertainty-inspired perturbations to generate candidate paths and trains LatentRM with step-wise contrastive comparisons to evaluate them.

4 Method

  • The framework has two components: stochastic sampling of latent thoughts and aggregation of the resulting trajectories.Source: §4 · p. 3

4.1 Stochastic Sampling in Latent Space

  • Monte Carlo Dropout samples weight masks at inference to capture epistemic uncertainty, while Additive Gaussian Noise perturbs latent thoughts to simulate aleatoric uncertainty.The method describes arbitrary noise as a risk to reasoning quality and motivates uncertainty-based sampling as a way to make it controllable and meaningful.Source: §4.1 · p. 4
  • Additive Gaussian Noise applies zero-mean Gaussian perturbations with standard deviation σ at each reasoning step, after which autoregressive inference continues from the perturbed trajectory.Its variance is controlled by σ independently of model parameters.Source: §4.1 · p. 4

4.2 Latent Trajectories Aggregation

  • LatentRM addresses the lack of explicit scores for continuous trajectories with a scoring head that maps hidden states to scalar estimates of the promise of continuing from the current thought.Existing process reward models cannot interpret or score abstract latent thoughts, motivating a dedicated scorer.Source: §4.2 · p. 4
  • At inference, LatentRM sums logits over a generated sequence as a proxy for the relative quality of its latent reasoning trajectory.Source: §4.2 · p. 4
  • LatentRM supervises thought quality using empirical correctness from stochastic rollouts and compares candidate scores at each step with a softmax in a step-wise contrastive objective.The passage contrasts this relative comparison with binary cross-entropy, which it reports performs poorly in practice.Source: §4.2 · pp. 4–5

5 Experiment Setup

Experiments use three arithmetic benchmarks and five latent-reasoning backbones. Sampling is evaluated by solution coverage, while aggregation compares LatentRM-guided selection and search with majority voting under matched compute budgets.

5.1 Benchmarks and Models.

  • Experiments cover GSM8K-Test, GSM8K-Hard, and MultiArith, using five latent-reasoning backbones with their released checkpoints and stated latent-thought settings.COCONUT and CODI use GPT-2; CoLaR uses Llama-3.2-1B; Latent-SFT and Render-of-Thought extend evaluation to larger models.Source: §5.1 · p. 5

5.2 Sampling Evaluation

  • Sampling effectiveness is measured as coverage: the fraction of problems with at least one correct answer among N sampled trajectories.Coverage is equivalent to pass@k when the sampling size N equals cutoff k, and sampling methods are compared at equal N.Source: §5.2 · p. 5

5.3 Aggregation Evaluation

  • Aggregation evaluation compares LatentRM-scored best-of-N and LatentRM-guided beam search with majority voting at matched compute budgets.Best-of-N and majority voting use N samples, while beam search uses beam size N; MC-dropout with p = 0.2 is the default sampler.Source: §5.3 · p. 5

6 Results of Sampling

Both stochastic sampling strategies increase solution coverage as the sample budget grows, though MC-dropout generally achieves higher coverage. Their exploration patterns differ, and the analyses identify settings and tasks where each strategy is advantageous.

6.1 Main Results

  • Coverage increases with sample count for both MC-dropout and AGN, with diminishing gains at larger N; MC-dropout leads at nearly all N, while COCONUT and CODI nearly converge by N=64.Figure 2 compares coverage across COCONUT, CODI, and CoLaR on GSM-Test, GSM-Hard, and MultiArith.Source: §6.1 · p. 6, Figure 2 · p. 6

6.2 Generalization to Larger Backbones

  • Parallel test-time scaling improves performance across larger backbones and scales; on MATH500, RoT-4B rises from 20.3% to 22.0% with N = 16.Table 1 reports generalization results for larger backbones and scales, with MC-dropout as the default.Source: §6.2 · p. 7, Table 1 · p. 7

6.3 Analysis on Diversity

  • Coverage peaks at moderate diversity, while at high diversity AGN tends to maintain or improve coverage and MC-dropout declines sharply.Figure 3 plots coverage against diversity while varying sampling method and N.Source: Figure 3 · p. 6, §6.3 · p. 7
  • MC-dropout maintains similar diversity across reasoning steps, whereas AGN fluctuates because fixed-scale noise affects latent vectors of varying scales unevenly.The analysis attributes MC-dropout's stepwise consistency to uncertainty-responsive stochasticity and AGN's variable influence to its fixed σ.Source: Figure 4 · p. 6, §6.3 · p. 7

6.4 Visualization on Stochastic Latent Landscape

  • On easy questions, AGN keeps more probability mass near the deterministic latent thought; on hard questions, dropout's larger displacement and denser exploration increase the chance of reaching a correct solution.Figure 5 visualizes sampled latent thoughts for an easy and a hard question.Source: §6.4 · pp. 7–8, Figure 5 · p. 8

7 Results of Aggregation

Accuracy rises with compute across the three evaluation datasets, and LatentRM-guided best-of-N and beam search outperform majority voting. Ablations indicate that contrastive supervision, stochastic rollouts, and learned scoring contribute to effective aggregation.

7.1 Main Results

  • Accuracy increases monotonically with N across all three datasets, and LatentRM-guided best-of-N and beam search consistently outperform majority voting.Beam search is comparable to best-of-N on GSM-Test and MultiArith but trails on GSM-Hard; best-of-N is the most effective route overall.Source: §7.1 · p. 8, Figure 6 · p. 9

7.2 Ablation Studies

  • Removing step-wise contrastive loss or stochastic rollouts reduces aggregation performance, while an untrained random scalar head performs worse than majority voting.These ablations indicate the value of relative supervision, Monte Carlo estimates for thought-level annotation, and learned trajectory evaluation.Source: §7.2 · pp. 8–9, Table 2 · p. 9

7.3 More Analysis

  • Further analyses of latent versus explicit reasoning, wall-clock performance, and variable-length settings are reported in Appendices F.1–F.3.The main text directs readers to the appendices for these extended analyses.Source: §7.3 · p. 9

8 Conclusion and Future Work

The paper frames parallel test-time scaling for latent reasoning as a new direction, with stochastic sampling and learned trajectory scoring addressing challenges specific to continuous latent spaces. It also distinguishes pure latent reasoning from token-embedding approaches and outlines theoretical interpretations and aggregation procedures.

8 Conclusion and Future Work

  • The framework combines MC-dropout and additive Gaussian noise with a step-wise contrastive Latent Reward Model to scale parallel inference in latent reasoning.The conclusion presents these components as addressing sampling and trajectory aggregation in continuous latent spaces.Source: §8 · p. 9
  • Future work should integrate sampling and aggregation into reinforcement learning and adapt ensemble techniques to continuous latent representations.The latter direction faces the challenge of applying token-based ensemble methods in latent space.Source: §8 · p. 9

A Soft and Latent Reasoning

  • Unlike Soft Thinking's mixtures of token embeddings, latent reasoning uses pure latent vectors unconstrained by vocabulary structure, which the proposed parallel TTS explores with sampling and LatentRM.The paper connects this distinction to discovering reasoning patterns beyond vocabulary structure.Source: A Soft and Latent Reasoning · p. 14

B Theoretical Perspectives on Sampling in Latent Space

  • MC-dropout approximates epistemic uncertainty by sampling model weights, while AGN simulates aleatoric variability by perturbing latent inputs.The MC-dropout predictive distribution is approximated by stochastic forward passes; AGN's output variance depends on local network sensitivity.Source: B Theoretical Perspectives on Sampling… · p. 15

C Derivation of Cumulative Scoring

  • Cumulative log probability across steps serves as a proxy for trajectory quality because shared stepwise normalizers preserve relative ordering.The derivation compares candidate trajectories using their cumulative log probabilities.Source: C Derivation of Cumulative Scoring · p. 15

D Inference Procedures for Aggregation

  • LatentRM supports best-of-N selection and beam search through cumulative rewards, while majority voting selects the most frequent decoded answer without latent scores.The majority-voting baseline isolates the benefit of learned aggregation.Source: D Inference Procedures for Aggregation · pp. 15–16

E Additional Results on Harder Benchmarks

Additional evaluations report gains on harder benchmarks and compare latent reasoning with explicit CoT under parallel TTS. The comparisons describe both computational efficiency and wall-clock latency, while noting limits in absolute performance.

E Additional Results on Harder Benchmarks

  • Figure 7 reports consistent gains on GPQA and AIME, though absolute performance remains limited by small models and the evolving latent reasoning framework.The authors cite the challenges of advanced mathematics and Ph.D.-level questions as further factors.Source: E Additional Results on Harder… · p. 16

F.1 Comparison of Latent and Explicit Reasoning with TTS

  • COCONUT with parallel TTS outperforms explicit CoT on the same backbone with fewer decoding steps, while LatentRM improves over the baseline but remains below Coverage@N.The comparison reports a favorable accuracy-computation balance for latent reasoning.Source: §F.1 · p. 17, Table 3 · p. 17

F.2 Wall-Clock Comparison of Latent and Explicit Reasoning with TTS

  • At N=32, aggregation contributes about 20% of total latency, so parallel sampling rather than reranking dominates cost.The latency comparison reports a favorable wall-clock profile for COCONUT relative to explicit CoT at matched N.Source: §F.2 · p. 17, Table 4 · p. 17

F.3 Analysis on LatentRM under Variable-Length Setting

The variable-length evaluation samples latent reasoning paths with different numbers of steps and normalizes their scores by length. Under this setup, best-of-N reranking with average reward outperforms the deterministic baseline.

F.3 Analysis on LatentRM under Variable-Length Setting

  • Best-of-N reranking by average reward consistently outperforms the deterministic baseline with variable-length latent reasoning.The evaluation samples reasoning lengths uniformly from 4 to 6 steps and divides total reward by step count to mitigate length bias.Source: §F.3 · p. 18

G Training Configuration for LatentRM

The LatentRM training configuration uses sampled COCONUT trajectories and rollout-based step labels, excluding uniformly easy or difficult examples. The accompanying sampling algorithms specify inference-time generation of multiple latent trajectories using MC-dropout or AGN.

G Training Configuration for LatentRM

  • LatentRM is trained for COCONUT using a step-wise contrastive objective, with its model initialized from a GPT-2 COCONUT checkpoint.The checkpoint has 124 million parameters.Source: G Training Configuration for LatentRM · p. 18
  • Training examples come from GSM8K's 385K-example training set, with eight MC-dropout trajectories sampled per problem at dropout probability 0.2.For each intermediate step, 128 stochastic rollouts estimate correctness; examples where all trajectories are correct or all are incorrect are excluded.Source: G Training Configuration for LatentRM · p. 18

H Algorithmic Specification of Latent Sampling Procedures

  • The sampling algorithms generate multiple latent reasoning trajectories using MC-dropout or AGN at inference time.The algorithms cover both sampling procedures for latent test-time scaling.Source: H Algorithmic Specification of Latent… · p. 18

Related papers

Loading 2510.07745v4…