PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling Latent Reasoning via Looped Language Models

Rui-Jie Zhu, Zixuan Wang, Kai Hua, Tianyu Zhang, Ziniu Li, Haoran Que, Boyi Wei, Zixin Wen, Fan Yin, He Xing, Lu Li, Jiajun Shi, Kaijing Ma, Shanda Li, Taylor Kergan, Andrew Smith, Xingwei Qu, Mude Hui, Bohong Wu, Qiyang Min, Hongzhi Huang, Xun Zhou, Wei Ye, Jiaheng Liu, Jian Yang, Yunfeng Shi, Chenghua Lin, Enduo Zhao, Tianle Cai, Ge Zhang, Wenhao Huang, Yoshua Bengio, Jason Eshraghian

arXiv:2510.25741v5cs.CL

TL;DR

The paper addresses whether recurrent latent computation can improve reasoning and scaling beyond conventional parameter and data growth. It presents Ouro, a pre-trained LoopLM family with adaptive depth learned through entropy-regularized training, and reports parameter-efficient performance, stronger knowledge manipulation, and more faithful reasoning traces. The study also identifies training-stability constraints and open questions around sample efficiency and depth extrapolation.

  • Problem

    Prior LoopLM gains at modest scales left frontier-level scaling behavior and comparisons with non-recursive transformers unproven.

  • Method

    Ouro applies shared-weight Transformer blocks recurrently, uses entropy-regularized adaptive exits, and scales pre-training to 7.7T tokens.

  • Results

    Ouro’s 1.4B and 2.6B models match or exceed 4B and 8B standard transformers, with gains attributed to knowledge manipulation rather than increased storage.

  • Takeaways & Limitations

    Iterative latent computation is presented as a third scaling axis beyond model parameters and data, with causally faithful traces and safety improving at greater recurrent depth.

  • Takeaways & Limitations

    Recurrent architectures require smaller learning rates, and exhaustive learning-rate sweeps were not run because of compute constraints.

Abstract

from arXiv · show

Modern LLMs are trained to "think" primarily via explicit text generation, such as chain-of-thought (CoT), which defers reasoning to post-training and under-leverages pre-training data. We present and open-source Ouro, named after the recursive Ouroboros, a family of pre-trained Looped Language Models (LoopLM) that instead build reasoning into the pre-training phase through (i) iterative computation in latent space, (ii) an entropy-regularized objective for learned depth allocation, and (iii) scaling to 7.7T tokens. Ouro 1.4B and 2.6B models enjoy superior performance that match the results of up to 12B SOTA LLMs across a wide range of benchmarks. Through controlled experiments, we show this advantage stems not from increased knowledge capacity, but from superior knowledge manipulation capabilities. We also show that LoopLM yields reasoning traces more aligned with final outputs than explicit CoT. We hope our results show the potential of LoopLM as a novel scaling direction in the reasoning era. Our model is available here: http://ouro-llm.github.io.

1 Introduction

LoopLM introduces recurrent, shared-parameter computation as a parameter-efficient alternative to scaling model size, with adaptive depth and latent reasoning built into pre-training. At scale, Ouro models show stronger capability, safety, and knowledge manipulation than their parameter counts suggest.

  • Architectural motivation: LoopLM recursively reuses shared parameters to provide dynamic computation within a fixed parameter budget.Its learned early-exit mechanism allocates fewer recurrent steps to simple inputs and more to complex ones.
  • Results: LoopLM offers a more efficient path to higher performance, with scaling behavior characterized across capabilities, efficiency, and safety.The study extends prior work through multi-trillion-token training regimes and controlled ablations.
  • Results: 7.7T training tokens support 1.4B and 2.6B LoopLMs matching 4B and 8B standard transformers on most benchmarks.This corresponds to reported 2-3× parameter-efficiency gains for deployment in resource-constrained environments.
  • Adaptive computation: Entropy regularization with a uniform exit-step prior supports unbiased depth exploration before tuning the compute-performance trade-off.The focused stage allocates recurrent steps according to input difficulty.
  • Mechanism: Recurrence leaves raw knowledge storage near 2 bits per parameter but dramatically improves knowledge manipulation for fact composition and multi-hop reasoning.The comparison is between looped and non-looped models under controlled experiments.
  • Safety and faithfulness: LoopLM improves safety as recurrent steps increase and produces reasoning traces more aligned with final outputs than CoT.The paper frames these traces as more causally faithful than post-hoc rationalization.

2 Related Works

Related work frames looping in two complementary ways: parameter sharing for efficiency and iterative latent reasoning through hidden-state refinement. Prior architectures include Universal Transformers, recursive Transformers, recurrent-depth methods, and explicit feedback-loop variants.

  • Prior looped models: Universal Transformers and recursive Transformers established recurrent-depth approaches, while later work connected looping to latent thoughts and reasoning.The cited line includes recurrent depth, looped transformers, and relaxed recursive transformers.
  • Parameter Sharing for Model Efficiency: Looped architectures reuse Transformer blocks or submodules across depth, reducing parameters without changing the computation.ALBERT is identified as a prominent modern example of parameter reuse with embedding factorization.
  • Latent Reasoning and Iterative Refinement: LoopLM iteration can be viewed as latent reasoning, with hidden states forming a non-verbal chain of thought that progressively refines representations.This implicit process differs from approaches that feed hidden states explicitly back into the input.
  • Latent Reasoning and Iterative Refinement: Explicit feedback-loop methods include Coconut’s continuous thought token and CoTFormer’s activation interleaving before reapplying shared layers.These contrast with implicit LoopLM variants whose thought process remains in hidden-state evolution.
  • Synthesis: The literature positions shared-parameter iteration as a route to parameter efficiency, reasoning enhancement, and efficient sequence-length expansion.Prior studies report improved performance on complex reasoning tasks as recurrent steps increase.

3 Learning Adaptive Latent Reasoning with LoopLM

LoopLM reuses a Transformer layer stack recurrently and learns adaptive exit decisions so computation can vary with token or example difficulty. Its training combines losses across recurrent steps with entropy regularization, then focuses gate training on measured performance improvements.

  • 3.2 Adaptive Computation: Adaptive computation lets the model choose recurrent depth per token and example, spending less compute on easy inputs and more on hard inputs.An exit gate runs alongside the language-modeling head at each step up to Tmax.
  • 3.1 LoopLM Architecture: LoopLM reuses the same depth-L layer stack for t recurrent iterations, with t = 1 recovering the non-looped model.The model produces a language-modeling output at each recurrent step.
  • 3.2 Adaptive Computation: The deterministic Q-exit policy terminates at the first step whose cumulative exit probability reaches threshold q, trading earlier exits for lower compute against deeper computation.Smaller q favors earlier exits, while larger q allows deeper computation.
  • 3.3 Stage I: Learning an Entropy-Regularized Objective: During pre-training, the objective weights each step loss by its exit probability and adds entropy regularization to prevent collapse onto the deepest step.The coefficient β controls the exploration–exploitation trade-off in the learned exit distribution.
  • 3.4 Stage II: Focused Adaptive Gate Training: The exit gate is subsequently fine-tuned with the language model frozen, using detached per-step loss improvements to learn whether continuing computation remains beneficial.The adaptive loss penalizes both underthinking and overthinking by matching predicted continuation behavior to improvement-derived targets.

4 Training Looped Language Models

Ouro trains Looped Language Models by repeatedly applying shared Transformer parameters, combining multi-stage data scaling with stability-focused optimization and adaptive recurrent computation. The training pipeline spans 7.7T tokens and includes unsuccessful reinforcement-learning attempts constrained by variable-depth execution.

  • Training pipeline: Ouro trains 1.4B and 2.6B models through four stages totaling 7.7T tokens, followed by Reasoning SFT variants.The stages cover pre-training, continual-training annealing, long-context training, and mid-training.
  • Architecture: The architecture repeatedly applies a stack of weight-tied Transformer blocks, using MHA with RoPE, SwiGLU FFNs, and sandwich normalization.The design prioritizes a clean recurrent implementation and training stability.
  • Data: The corpus combines web text, mathematics, code, and long-context documents, with Stage 1 using Nemotron-CC alongside Chinese, coding, and mathematics datasets.Stage 2 adds higher-quality data with 16K-token sequences, while Stage 3 uses 64K-token ProLong data for long-context training.
  • Training stability: Training stability required reducing recurrent steps from 8 to 4 after loss spikes and gradient oscillations appeared during Stage 1.The authors also increased batch size from 4M to 8M tokens and used conservative optimization settings.
  • Adaptive computation: The adaptive objective uses entropy regularization, with β reduced from 0.1 to 0.05 to stabilize optimization and allow more flexible depth patterns.The sequence length progresses from 4K to 16K, 64K, and 32K tokens across training stages.
  • Reinforcement learning: Reinforcement-learning attempts did not improve over the final SFT checkpoint because fixed execution paths conflict with LoopLM’s variable-depth computation.A fixed four-step setup trained normally but still failed to surpass the SFT checkpoint.

5 Experiments

Experiments evaluate Ouro across general knowledge, reasoning, mathematics, science, coding, multilingual, and safety-related settings. Small LoopLMs match or exceed larger dense baselines, with gains concentrated on multi-step reasoning, while depth extrapolation and cache reuse reveal important operating boundaries.

  • Evaluation setup: Ouro evaluations cover general knowledge, reasoning, mathematics, science, coding, and multilingual capabilities using common evaluation frameworks and matched baselines.Baselines include Qwen2.5, Qwen3, Gemma3, Llama3.1, and Llama3.2 base models.
  • Base model evaluation: The 1.4B Ouro matches the 4B Qwen3-Base on most benchmarks, including BBH at 71.02 versus 70.95 and MATH500 at 82.40 versus 59.60.It also scores 78.92 versus 72.86 on GSM8K.
  • Base model evaluation: The 2.6B Ouro scores 55.73 on MMLU-Pro, 80.46 on BBH, and 90.85 on MATH500, surpassing the 8B Qwen3-Base at 53.72, 77.65, and 62.30.The results show the strongest gains on reasoning-intensive benchmarks.
  • Reasoning model evaluation: The 2.6B Ouro-Thinking scores 76.44 on OlympiadBench versus 75.25 for Qwen3-8B and 39.0 on BeyondAIME versus 38.0.The 1.4B variant reaches 71.55 on OlympiadBench and 34.0 on BeyondAIME.
  • Recurrent depth: Performance generally peaks near the trained depth of T = 4, while extrapolating to T = 5 through T = 8 causes moderate benchmark degradation.For reasoning SFT models, performance is very low at T = 1 and peaks around T = 4 or nearby depths.
  • Safety and depth: Safety improves as recurrent steps increase, including beyond the trained depth, contrasting with the degradation observed on task-specific benchmarks.The authors associate this pattern with iterative refinement continuing to enhance safety alignment.
  • Adaptive computation: At an average exit round of 2.5, specialized Ponder-gate training reaches 66% accuracy versus approximately 64% for the standard gate.The specialized gate performs best across computational budgets, while hidden-state thresholds remain within 1%-2% at moderate budgets.
  • KV-cache reuse: Last-step and averaged KV-cache reuse stay within 0.3 points of the full-cache GSM8K baseline while reducing memory requirements by 4×.Reusing only the first-step cache collapses GSM8K performance to 18.73 from 78.92.

6 Understanding LoopLMs Superiority from a Parametric Knowledge Viewpoint

LoopLM’s advantage is attributed to better manipulation and composition of learned knowledge, not greater atomic-fact storage. Controlled synthetic tasks and theoretical analysis connect recurrence to stronger reasoning and sample efficiency.

  • Knowledge capacity: Looped and non-looped models achieve approximately 2 bits of knowledge per parameter, showing that looping does not increase knowledge capacity.The comparison uses equal-parameter models and finds no improvement in capacity scaling from recurrence.
  • Knowledge manipulation: Looped models outperform equal-parameter non-looped models on Mano across k ∈ {2, 3, 6}, including comparisons at equal FLOPs.Mano requires applying modular-arithmetic rules and composing a binary-tree structure without intermediate thinking.
  • Knowledge manipulation: Looped models learn three-hop multi-hop QA with fewer examples and faster improvement at the same unique-sample budget.The result indicates stronger learning of composition and manipulation of atomic factual knowledge.
  • Interpretation: Across Mano and multi-hop QA, LoopLM shows a stronger inductive bias for flexible knowledge manipulation rather than increased knowledge capacity.The conclusion holds for both reasoning-heavy and knowledge-heavy synthetic tasks; benchmark gains are larger in reasoning-heavy categories.
  • Theoretical perspective: A one-layer transformer with loops O(log^2 D) times can check reachability in a combined knowledge graph, supporting parallelizable latent reasoning.Here D is the diameter of the combined graph, and the claim is contrasted with sequential reasoning approaches.
  • Discussion: Parameter sharing may reduce sample complexity by restricting the hypothesis class that must represent repeated manipulation and recursion.The passage presents this as a possible statistical explanation for improved sample efficiency.

7 Safety, Faithfulness and Consistency

Ouro’s recurrent process improves safety as depth increases and exposes intermediate latent predictions aligned with later outputs. These properties support faithfulness analysis, early screening, and proposal–verification deployment.

  • Safety: Ouro’s safety alignment improves as recurrent steps increase, including evaluation beyond the four steps used during training.Figure 8 reports this pattern for both 1.4B and 2.6B models.
  • Safety: At four recurrent steps, harmfulness rates are 0.003 for Ouro 1.4B Thinking and 0.003 for Ouro 2.6B Thinking, compared with 0.009 for Qwen3-4B-Thinking.HEx-PHI scores harmfulness from 1 to 5, with 5 indicating the least safe output.
  • Faithfulness: LoopLM’s latent trajectory is the causal reasoning substrate, while decoded intermediate text is an instrumented readout of evolving hidden states.Each shared-weight transition is trained to improve the task objective, and the latent states directly precede the answer.
  • Faithfulness: Unlike Qwen3-4B-Thinking, whose final-token probe reaches 0.99 ROC AUC, Ouro’s intermediate predictions change across recurrent steps on ambiguous questions.The comparison is used to distinguish iterative updating from reasoning that merely rationalizes a pre-committed answer.
  • Deployment: Earlier recurrent outputs can propose tokens for verification by the final step, reusing cached activations and KV states without an external draft model.Rejected tokens are rolled back under standard acceptance tests.
  • Deployment: Intermediate predictions provide a usable proxy for final outputs, allowing a threshold to jointly adjust compute, consistency, and safety strictness.The Q-exit threshold shifts average exit depth while a verifier maintains final quality.

8 Conclusion

Ouro integrates iterative latent computation and adaptive depth into pre-training, achieving parameter-efficient performance while improving knowledge manipulation. Its latent process also supports faithful traces and stronger safety with increased recurrence.

  • Conclusion: Ouro integrates iterative computation and adaptive depth into pre-training on 7.7T tokens, with 1.4B and 2.6B models matching or exceeding 4B and 8B standard transformers.The reported efficiency gain is 2-3×.
  • Conclusion: The reported advantage is attributed to superior knowledge manipulation rather than increased knowledge storage.The conclusion is supported by synthetic experiments and theoretical analysis.
  • Conclusion: LoopLM’s iterative refinement provides causally faithful reasoning traces and safety alignment that improves with recurrent depth, including extrapolated depths.The paper presents these properties as distinctive architectural benefits beyond benchmark performance.

Contributions

The paper’s contributors cover pre-training, analysis, data, scaling laws, safety and faithfulness, post-training, theory, adaptive exits, and inference infrastructure.

  • Contributions: Rui-Jie Zhu led the project and Ouro pre-training while optimizing infrastructure and exploring RLVR.
  • Contributions: Zixuan Wang led analyses of LoopLM superiority, experiments, adaptive early exits, training, and safety analysis.
  • Contributions: Kai Hua curated pre-training data mixtures, while Ge Zhang co-led and supervised Ouro and contributed pre-training and post-training insights.
  • Contributions: Tianyu Zhang led consistency, safety, and faithfulness analysis, including faithfulness evaluation, probing, and efficient KV-cache design.
  • Contributions: Ziniu Li led post-training and contributed to supervised fine-tuning and RLVR exploration.
  • Contributions: Haoran Que led scaling-law analysis relating performance to model size and recurrent depth, while Zixin Wen contributed theory and adaptive exits.
  • Contributions: Fan Yin, He Xing, and Lu Li contributed to inference and infrastructure development, including vLLM and SGLang optimization.

A Empirical Validation of Prior Choice

Under matched training conditions, the uniform prior supports stronger convergence, broader depth exploration, and a better accuracy–compute frontier than geometric priors. The section also evaluates knowledge storage and manipulation using synthetic biography and modular-arithmetic tasks.

  • Prior choice: Uniform priors achieve lower training loss and cleaner convergence than geometric priors on a 776M LoopLM with Tmax = 4.Geometric priors plateau higher, with the gap widening as their bias toward early exit increases.
  • Prior choice: Uniform priors maintain exploration across recurrent depths, whereas stronger geometric priors concentrate probability on shallow steps and weaken supervision for deeper iterations.The uniform prior allows later iterations to specialize when maximum depth is available at inference.
  • Prior choice: Uniform-prior models achieve a strictly better accuracy–compute Pareto frontier than geometric-prior models at a fixed average step budget.Inference can still cap steps or apply a halting threshold despite the uniform prior not explicitly favoring early exit.
  • Knowledge evaluation: LoopLM knowledge-capacity evaluation measures reliably stored information in bits per parameter using synthetic biographies and partial correctness.The biographies encode names and five attributes, while the metric credits partially recalled information.
  • Knowledge evaluation: The Mano task tests whether models can manipulate stored knowledge within parameters without intermediate thoughts using modular-arithmetic expressions.The evaluation uses leave-out questions and exact-match accuracy for single-token answers.

B.3.1 Additional experimental results

Additional experiments generally support LoopLM’s advantage in knowledge manipulation, while exposing a limitation in one iso-flop baseline comparison. Real-world MMLU gains are concentrated in reasoning-heavy categories rather than fact-retrieval categories.

  • Additional experimental results: The iso-flop baseline at N = 1.2 × 10^5 does not significantly outperform the shallower main-paper version, possibly because of randomness or insufficient hyperparameter tuning.The authors state that further follow-up experiments are needed to validate this appendix conclusion.
  • Additional experimental results: LoopLM’s largest MMLU improvements occur in logical, mathematical, and procedural categories, while retrieval-heavy subjects show more modest gains.The comparison measures accuracy from Loop 1 to Loop 4 across all 57 MMLU categories.
  • Additional experimental results: The nearly flat global_facts improvement indicates that iterative computation enhances knowledge manipulation rather than simply increasing accessible knowledge.The MMLU pattern corroborates the synthetic-task findings reported in the paper.

B.5 Theory: latent thought with LoopLM

The theory models LoopLM iteration as latent computation for graph reachability, combining learned graph knowledge with contextual graph information. Repeated loops expand reachable distances exponentially, yielding an O(log D)-step construction under stated input and dimensionality assumptions.

  • Task formulation: The theoretical task combines a hidden knowledge graph G stored in parameters with an input context graph Gctx to test whether s reaches t.This formalizes knowledge manipulation involving fixed facts and additional contextual facts.
  • Architecture and input: The LoopLM receives graph adjacency information and uses shared attention and MLP parameters across recurrent loops.The construction uses a one-head transformer, a two-layer gated MLP, and thresholding normalization.
  • Iterative computation: Each loop uses attention to combine reachable nodes and the MLP to add the hidden graph’s adjacency information.This implements a repeated-squaring-style expansion of connectivity in the combined graph.
  • Iterative computation: After i recurrences, the construction tracks vertices reachable within distance 2^i, so each loop doubles the covered distance.The induction establishes distance 1 after the first recursion and distance 2^i thereafter.
  • Theoretical result: With L = ⌈log2 D⌉ + 1 recursion steps, the model correctly solves reachability on the combined graph G + Gctx.The theorem uses hidden dimension de = 2n and recurrent O(log2 D) times.
  • Limitations and assumptions: The construction requires Θ(n) hidden dimension in the worst case to encode Θ(n) nodes’ information, though this requirement can relax when graph degree is bounded.The theorem also assumes adjacency-matrix input; edge-sequence input requires an additional induction-head mechanism.

C.1 Evaluation Settings

The evaluation studies LoopLM and standard models across model sizes, recurrent depths, six benchmarks, and loss-scaling experiments. Performance generally improves with size and depth, while scaling-law fits support predictability across model size, data size, and recurrent depth.

  • Evaluation setup: The evaluation covers 53M–1.36B models, recurrent depths 1, 2, 4, and 8, and six benchmarks trained on 20B FineWeb-Edu tokens.The benchmarks are ARC-Challenge, ARC-Easy, HellaSwag, LAMBADA, OpenBookQA, and PIQA.
  • Benchmark trends: At recurrent step 1, LoopLM and standard models share identical architectures and overlapping curves; performance generally increases with model size and recurrent step.The stated exception is LoopLM at 778M and 1.364B, where the depth trend does not hold.
  • Benchmark trends: Standard models outperform LoopLM under the same conditions, with the performance gap increasing with recurrent step and generally decreasing with model size.The gap is defined as standard-model score minus LoopLM score and remains positive in these experiments.
  • Scaling-law analysis: The total-loss scaling law models loss using model size N, data size D, and maximum recurrent step Tm through a power-law form.Offset terms smooth the fitting curve near zero.
  • Scaling-law analysis: Actual and predicted loss curves are highly consistent, supporting predictability with respect to model size, training data size, and maximum recurrent step.The study quantifies consistency using the coefficient of determination R^2.
  • Scaling-law analysis: Both total loss and step-wise loss strongly correlate with N, D, and T/Tm, supporting the scaling law’s predictability and generalizability.Step-wise loss is the cross-entropy loss measured at each recurrent step.

D.3 RQ3: What is the inherent connection between total loss and step-wise loss?

The paper links total loss and step-wise loss through a shared scaling-law structure, while the learned gating distribution weights step-wise losses in the total objective. Experiments show this relationship remains predictive across model sizes, training data, and recurrent steps, with shallow-loss behavior affected by limited model capacity.

  • The total loss combines expected task loss, a weighted sum of step-wise losses, with entropy regularization on the gating distribution.Entropy regularization prevents the gating mechanism from collapsing onto a single recurrent step.
  • When model capacity is insufficient, shallow step-wise loss can increase with additional training data as gating weights favor deeper steps to reduce total loss.The paper attributes this behavior to limited information capacity combined with entropy-constrained gating.
  • When the maximum recurrent step is fixed, the Step-wise Loss Scaling Law and Total Loss Scaling Law have completely consistent forms.This establishes trend consistency between total loss and step-wise loss under a fixed recurrent-depth budget.
  • {0.0004, 0.0855, 0.3793, 0.5348} is the averaged gating distribution across recurrent steps when the maximum recurrent step is 4.The distribution is substituted into the training objective after ignoring the relatively small stabilized entropy term.
  • 0.9542 average R2 across 10 model-size fits demonstrates Total Loss Scaling Law generalizability across unseen model sizes.The experiments use five model sizes and fit three at a time before evaluating against the full set.
  • 0.9385, 0.9609, and 0.962 are the R2 values for fitting the first 25%, 50%, and 75% of training-data points, respectively.The fitted curves become more consistent with actual curves as the number of data points increases.
  • 0.9581 average R2 demonstrates Total Loss Scaling Law generalizability across maximum recurrent steps 2, 4, and 8.Each experiment fits two maximum-step settings and validates on all data points.

E.2 Generalizability for the Step-wise Loss Scaling Law

The Step-wise Loss Scaling Law is evaluated on unseen model sizes, training-data points, and recurrent steps. Across these settings, fitted curves remain close to full-data results, supporting generalizability, though fit quality varies by maximum recurrent step.

  • Model Size Generalizability: 0.8815 and 0.797 average R2 values for maximum recurrent steps 2 and 4 remain close to full-data values 0.8898 and 0.8146.These results support model-size generalizability using unseen size data points.
  • Training Data Generalizability: 0.8686, 0.8882, and 0.8896 are the R2 values for maximum recurrent step 2 using the first 25%, 50%, and 75% of data points.The corresponding maximum-step-4 values are 0.793, 0.813, and 0.8142, approaching full-data values as fitting data increases.
  • Training Data Generalizability: 0.793, 0.813, and 0.8142 are the R2 values for maximum recurrent step 4 using the first 25%, 50%, and 75% of data points.These values are close to the full-data result of 0.8146.
  • Recurrent Step Generalizability: 0.8118 average R2 across four fits is similar to the full-data value 0.8146, demonstrating recurrent-step generalizability.The maximum recurrent step is 4, with three recurrent-step values used for fitting and the remaining value held out.
Loading 2510.25741v5…