PaperlayerView exact revision on arXiv

Source-linked AI summary

Stabilizing Recurrent Dynamics for Test-Time Scalable Latent Reasoning in Looped Language Models

Xiao-Wen Yang, Ziyu Han, Xi-Hua Zhang, Wen-Da Wei, Jie-Jing Shao, Lan-Zhe Guo, Yu-Feng Li

arXiv:2605.26733v1cs.LGcs.AI

TL;DR

Looped language models can lose performance as recurrence increases, and existing designs face a trade-off between latent-state stability and reasoning effectiveness. The paper proposes STARS, combining spectral-radius regularization with random loop sampling to encourage stable fixed points. It reports reliable arithmetic scaling and reduced degradation with deeper recurrence on mathematical reasoning tasks, while noting remaining limits in task generality and monotonic scaling.

  • Problem

    LoopLM performance can peak and then collapse as recurrent depth increases, while current designs face a trade-off between reasoning effectiveness and latent-trajectory stability.

    Source: §1 · pp. 1–2

  • Method

    STARS combines Jacobian Spectral Radius Regularization with random loop sampling to train latent states toward asymptotically stable fixed points across recurrent depths.

    Source: §1 · p. 2, §5 · p. 7

  • Results

    STARS achieves fully reliable arithmetic scaling and more robust mathematical-reasoning performance; on GSM8K, its 8-step drop is 8.26% versus Ouro-1.4B’s 20.47%, with peak performance improved 4.01%.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Experiments on synthetic tasks and mathematical reasoning benchmarks show more reliable test-time scaling over greater recurrent depths than prior methods.

    Source: §7 · p. 9

    Evaluation focuses on mathematical reasoning, leaving generalizability to other complex reasoning domains open; performance also does not always improve monotonically with more steps.

    Source: A. Limitations and Future Work · p. 11

Abstract

from arXiv · show

Looped Language Models (LoopLMs) enable efficient latent reasoning through depth recurrence, yet exhibit unreliable test-time scaling behavior: performance often peaks at a certain iteration depth and then collapses with further recurrence. Through latent dynamics analysis, we find an inherent trade-off between stability and effectiveness in existing architectures and strategies. By conceptualizing reasoning as uncertainty reduction, we propose that convergence toward stable fixed points while preserving effectiveness represents a promising way. To this end, we propose STARS (STAbility-driven Recurrent Scaling), a training framework that constrains latent states to approach asymptotically stable fixed points. This is realized via efficient Jacobian Spectral Radius Regularization with random loop sampling, enabling STARS to maximize effectiveness while ensuring rigorous stability. Experiments on arithmetic tasks show that STARS achieves reliable test-time scaling, and on complex mathematical reasoning it substantially mitigates performance degradation as recurrence depth increases while also improving peak performance.

1. Introduction

Looped language models offer latent test-time scaling, but their performance can peak and collapse as recurrence grows. The paper diagnoses a stability–effectiveness trade-off and proposes STARS, which improves scaling reliability on arithmetic and mathematical reasoning tasks.

1. Introduction

  • Existing LoopLMs often peak at a particular iteration depth and then deteriorate sharply or collapse, as illustrated by Ouro-1.4B performance across GSM8K recurrent steps.The paper attributes this behavior to overfitting to the specific recurrent iteration used during training.Source: §1 · p. 1, Figure 1 · p. 1
  • Current designs face an effectiveness–stability trade-off: internal normalization preserves information flow but can cause exponential state growth, while external normalization bounds states but often limits deep reasoning.Prelude/Coda layers, L2 regularization, and random loop sampling alone do not fully resolve this deadlock.Source: §1 · p. 2
  • STARS combines Jacobian Spectral Radius Regularization with random loop sampling to constrain latent states toward asymptotically stable fixed points while preserving effectiveness.The framework uses single-step power iteration with Jacobian-vector products to estimate the spectral radius efficiently.Source: §1 · p. 2
  • STARS achieves fully reliable test-time scaling on arithmetic tasks and reduces GSM8K degradation after 8 recurrent steps from Ouro-1.4B’s 20.47% to 8.26%, while improving peak performance by 4.01%.The method also demonstrates more robust performance on mathematical reasoning tasks.Source: §1 · p. 2

2. Related Work

Prior work scales reasoning through longer generated sequences or latent computation, while recurrent language models offer depth scaling through parameter sharing. The paper identifies unreliable scaling and limited latent-dynamics analysis in modern LoopLMs as open concerns.

Test time scaling and latent reasoning

  • Conventional explicit reasoning and search-based methods explore natural-language reasoning paths, but remain constrained by language-generation bandwidth and efficiency.Examples include Chain-of-Thought, majority voting, Tree-of-Thoughts, and Monte Carlo Tree Search.Source: §2 · p. 2
  • Latent reasoning shifts computation from discrete tokens into latent representations, but many approaches still scale along the sequential dimension.Coconut uses continuous thought tokens derived from previous hidden states.Source: §2 · p. 2

Recurrent Transformers and LoopLM

  • Recurrent Transformers scale depth through recurrence and parameter sharing, but LoopLM performance can decline beyond a specific iteration depth, and modern LoopLM latent dynamics remain under-analyzed.Prior DEQ dynamics studies use architectures and training algorithms that differ from standard language models.Source: §2 · p. 2

3. Preliminaries

A LoopLM repeatedly applies a shared transformer block, making inference a trajectory through latent space. In this view, test-time scaling extends that trajectory, so its long-term dynamics and convergence matter.

3.1. Looped Language Models

  • A LoopLM uses weight sharing by iteratively applying a recurrent block, with prelude embeddings as input and coda and lmhead components for output.At one iteration, the architecture reduces to a standard non-looped model.Source: §3.1 · p. 3

3.2. Latent Reasoning as a Dynamical System

  • The recurrent computation defines a discrete-time latent dynamical system, and test-time scaling extends its trajectory by increasing iterations without changing parameters.The hidden state evolves under a deterministic nonlinear map parametrized by the model parameters.Source: §3.2 · p. 3
  • A fixed point leaves the representation unchanged under further recurrence, and local asymptotic stability guarantees convergence rather than oscillation or divergence when inference is extended.The paper frames stable fixed points as representations signifying completion of reasoning.Source: §3.2 · p. 3

4. Recurrent Dynamics of LoopLM

This section uses controlled recurrent-model experiments to diagnose latent dynamics, architectural effects, and candidate strategies for scalable latent reasoning. Its central finding is that normalization placement shapes a stability–effectiveness trade-off that the tested interventions do not eliminate.

4.1. Experimental Setup

  • The experiments use recurrent GPT-style Transformers trained on randomly generated 4-digit addition, then sweep test-time iterations beyond the training horizon.The setup uses 100,000 training samples, a single recurrent Transformer block, and Ttrain = 4; latent trajectories are visualized with PCA.Source: §4.1 · pp. 3–4

4.2.1. IMPACTS OF NORM STRUCTURE DESIGN

  • Normalization placement, not operator choice, determines recurrent dynamics: internal normalization can produce divergent states, while external normalization keeps states bounded but degrades performance.Internal normalization preserves information flow but hidden states drift away from the effective manifold; external normalization can settle into poor attractors.Source: §4.2.1 · pp. 4–5, Figure 2 · p. 4

4.2.2. IMPACTS OF PRELUDE AND CODA LAYERS

  • Prelude and Coda layers do not change the systems’ dynamical tendencies: Prelude only marginally slows internal-system degradation, while Coda concentrates external attractors without improving performance.Their effect on system stability is negligible.Source: §4.2.2 · p. 5, Figure 3 · p. 5

4.3.1. RANDOM LOOP SAMPLING

  • Random loop sampling retains Pre-Sandwich performance beyond training-range iterations, but does not remove state drift and can destabilize Post-Sandwich training.A wider sampling range and higher mean are not necessarily better; wide ranges can lead Post-Sandwich models to training collapse.Source: §4.3.1 · pp. 5–6, Figure 3 · p. 5

4.3.2. L2 REGULARIZATION

  • L2 regularization between adjacent hidden states yields only marginal retention and drift improvements for internal normalization, virtually no effect for external normalization, and poor synergy with random loop sampling.The combined strategy underperforms pure random loop sampling.Source: Figure 3 · p. 5, §4.3.2 · p. 6

5. Jacobian Spectral Radius Regularization

STARS combines Jacobian Spectral Radius Regularization with random loop sampling to pursue effective reasoning with stable latent dynamics. It maintains performance as recurrence depth increases on arithmetic and mathematical reasoning tasks, with both components contributing to slower degradation in ablations.

5. Jacobian Spectral Radius Regularization

  • STARS combines JSRR with random loop sampling to pursue effective reasoning while hidden states converge toward a stable, effective fixed point.The paper frames reasoning as iterative uncertainty reduction and thought refinement.Source: §5 · p. 6
  • Because locating the exact fixed point is computationally prohibitive, JSRR regularizes the squared Jacobian spectral radius at current states across sampled recurrent depths.This applies regularization across diverse inference states to promote global stability over the latent-dynamics support.Source: §5 · pp. 6–7

6.1. Experimental Setup

  • Experiments cover multi-digit addition and pretrained Ouro fine-tuned on 400K NuminaMath samples, evaluated zero-shot across five mathematical benchmarks.The mathematical-reasoning setup uses a random subset because of computational resource constraints.Source: §6 · p. 7, §6.1 · pp. 7–8

6.2. Main Results

  • Ouro-1.4B-STARS achieves the highest overall in-distribution performance, averaging 74.18% at 4 recurrent steps and maintaining 65.55% at 8; its post-peak decline is slower as recurrent steps increase.Source: §6.2 · p. 8
  • 100% accuracy across recurrent iterations on multi-digit addition accompanies hidden states converging to a stable fixed point.The passage describes a relatively fast convergence rate and reliable reasoning as recurrent steps increase.Source: §6.2 · p. 8, Figure 5 · p. 8

6.3. Ablation Study

  • Across four ablation benchmarks, STARS declines more slowly than variants using only random loop sampling or only JSRR, with each component contributing to delayed degradation.The comparison also includes the original Ouro-1.4B base model.Source: §6.3 · p. 8, Figure 4 · p. 8

7. Conclusion

The paper diagnoses unreliable scaling in current LoopLMs as a trade-off between reasoning effectiveness and latent-trajectory stability, and proposes STARS to address it. STARS uses Jacobian Spectral Radius Regularization and random loop sampling to promote stability and more reliable scaling at greater recurrence depths.

7. Conclusion

  • Current LoopLMs exhibit unreliable scaling because reasoning effectiveness and latent-trajectory stability are in tension.Source: §7 · pp. 8–9
  • STARS enforces asymptotic stability with Jacobian Spectral Radius Regularization and random loop sampling to support more reliable scaling over greater recurrent depths.The paper identifies an inherent trade-off between reasoning effectiveness and latent-trajectory stability in current LoopLMs.Source: §7 · p. 9

Impact Statement

The work aims to improve the stability and scaling behavior of recurrent language models for reasoning tasks, with evaluation on standard academic mathematical-reasoning benchmarks.

Impact Statement

  • The work focuses on improving stability and scaling behavior in recurrent language models for reasoning tasks.Source: Impact Statement · p. 9
  • The techniques are evaluated on standard academic benchmarks for mathematical reasoning.Source: Impact Statement · p. 9
  • The paper introduces no new applications or datasets and does not directly address fairness, safety, or bias in deployed systems.Source: Impact Statement · p. 9

A. Limitations and Future Work

The evaluation is concentrated on mathematical reasoning, and performance does not always improve monotonically as recurrence steps increase. Future work will extend STARS to more reasoning and planning tasks and explore adaptive guidance toward stable fixed points.

A. Limitations and Future Work

  • The empirical evaluation is concentrated on mathematical reasoning, leaving generalization to commonsense or strategic planning open.This concentration reflects computational-resource limits.Source: A. Limitations and Future Work · p. 11
  • Even on evaluated tasks, STARS prevents catastrophic collapse and scales beyond the training horizon, but performance does not always improve monotonically with more steps.Fully reliable and predictable scaling remains challenging on the most difficult problems.Source: A. Limitations and Future Work · p. 11
  • Future work will extend STARS to more reasoning and planning tasks and investigate adaptive mechanisms for guiding latent trajectories toward optimal, stable fixed points.Source: A. Limitations and Future Work · p. 11

B. Training Algorithm Details

The training algorithm takes a dataset, initial parameters, a loop-depth distribution, regularization weight, learning rate, and power-iteration count; it samples batches and loop depths, runs the model to that depth, and computes a loss.

B. Training Algorithm Details

C. More Results

The supplementary results examine random-loop configurations, regularization-weight effects, training efficiency, and AMC23 comparisons. They report a constraint on regularization strength and relative training-efficiency values.

Random loop sampling on PreNorm and PostNorm

  • Random-loop results cover PreNorm and PostNorm with LN across Log-Normal, Poisson, and Uniform distributions, each with two configurations.Figure 6 shows results across distributions and parameter sets; Table 3 lists settings and clipping bounds.Source: C. More Results · p. 11, Figure 6 · p. 12, Table 3 · p. 12

Hyperparameter analysis

  • When λ is too large, including 0.15 and 0.20, attractor convergence can coexist with a significant accuracy drop on the original multi-digit task.The analysis cautions that excessive regularization can impede effective learning.Source: C. More Results · pp. 11, 13, Figure 7 · p. 13

Efficiency analysis during training phase

  • Ouro-Random Loop, Ouro-JSRR, and Ouro-STARS have training-phase efficiency values of 1.4976, 1.5317, and 2.0439 relative to Ouro-SFT, respectively.Table 4 presents these values relative to Ouro-SFT as the base.Source: Table 4 · p. 13, C. More Results · p. 13

Comparative Analysis and Ablation Study on AMC23

  • The AMC23 supplementary analysis compares the method with baselines and ablation variants, complementing the mathematical-reasoning benchmark analysis in the main text.Figure 8 presents the AMC23 comparison; the passage does not report its outcomes.Source: C. More Results · p. 13, Figure 8 · p. 13
Loading 2605.26733v1…