PaperlayerView exact revision on arXiv

Source-linked AI summary

What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling

Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov, Inessa Fedorova, Dmitry Bocharov, Ivan Oseledets, Valerii Ternovskii

arXiv:2609.39967v1cs.LGcs.AI

TL;DR

Existing recursive models vary architecture, gradient propagation, and optimization together, making their performance difficult to attribute. The paper studies these choices in a controlled pipeline across six domains and identifies a stable training recipe. Its model leads evaluated recursive baselines on nearly all metrics, including a large Arithmetic OOD accuracy gain.

  • Problem

    HRM, TRM, and URM vary architecture, gradient propagation, optimization, and regularization simultaneously, making it difficult to identify which choices drive their gains.

    Source: §1 · p. 2

  • Method

    The study retrains recursive baselines under matched data and evaluation, varies factors one at a time across six domains, and tests intermediate gradient horizons, large physical batches, and controlled recurrent-state updates.

    Source: §1 · p. 2, §6 · p. 9

  • Results

    71.16% Arithmetic OOD accuracy versus 36.20% for the strongest recursive baseline; the model has the strongest performance among evaluated recursive baselines on nearly all reported metrics.

    Source: §4.6 · p. 6

  • Takeaways & Limitations

    Within the recursive architectures studied, performance depends strongly on how recurrence is trained and stabilized, while added hierarchy, block size, and test-time computation bring domain-dependent rather than universal gains.

    Source: §6 · p. 9

Abstract

from arXiv · show

Recursive reasoning models apply a small shared Transformer block many times to refine a latent state. This gives them large effective depth with few parameters and makes them strong on algorithmic tasks. Such compact solvers are natural candidates for tools that an LLM can call on narrow algorithmic subproblems. However, existing models such as HRM, TRM and URM differ in architecture, gradient propagation and training procedure simultaneously. This makes it hard to tell what drives their performance, and their optimization is still poorly understood and often unstable. In this work we address both of these gaps. First, we study these questions under a unified experimental pipeline spanning six algorithmic domains. Individual controlled ablations are performed on representative domains, while the resulting recipe is evaluated across the full suite. The study reveals a surprisingly simple recipe for stable and generalizable recursive reasoning: an intermediate gradient horizon, large physical batches and controlled updates of the recurrent state. An explicit hierarchical architecture is not needed. Second, we combine these findings into a stable 13.6M-parameter model that achieves the strongest overall performance among the evaluated recursive baselines, with particularly large gains on out-of-distribution generalization. It raises Arithmetic OOD accuracy to 71.2%, from 36.2% for the strongest baseline, while reaching 98.41% on Sudoku and 59.5% pass@2 on ARC-AGI-1. Our results show that, within the recursive architectures studied here, performance depends strongly on how recurrence is optimized and stabilized. More broadly, it shows how AI systems can be improved by optimizing their components one at a time.

1 INTRODUCTION

Existing recursive reasoning systems couple architectural and optimization changes, obscuring why they work, while recurrence creates distinct optimization challenges. A unified controlled study identifies factors for stable training and a 13.6M-parameter model that leads evaluated recursive baselines on nearly all metrics.

1 INTRODUCTION

  • HRM, TRM, and URM vary architecture, gradient propagation, optimization, and regularization together, making it difficult to attribute their improvements to individual choices.Repeated shared-parameter application also creates optimization challenges unlike those of standard feed-forward Transformers.Source: §1 · p. 2
  • The study retrains recursive baselines under matched data and evaluation, varying one factor at a time across standard benchmarks and controlled out-of-distribution domains.Game of Life and Arithmetic test whether learned computation generalizes beyond the training distribution.Source: §1 · p. 2
  • An intermediate gradient horizon, large physical batches, and controlled recurrent-state updates support stability, while hierarchy, larger blocks, and extra test-time computation yield domain-dependent benefits.Source: §1 · p. 2
  • The stable 13.6M-parameter recursive reasoner achieves the strongest results among evaluated recursive baselines on nearly all metrics, scoring 71.16% on Arithmetic OOD, 98.41% on Sudoku, and 59.50% pass@2 on ARC-AGI-1.Source: §1 · p. 2

2 RELATED WORK

Prior work establishes latent iteration as a useful computational primitive, but leaves its optimization and stabilization open. HRM, TRM, and URM combine different architectural and training choices, motivating a shared framework that varies them separately.

2 RELATED WORK

  • Looped Transformers and recurrent-depth language models demonstrate latent iteration for algorithmic reasoning, length generalization, and test-time computation.Related work also adapts recurrent depth across tokens or searches continuous latent program spaces (Bae et al., 2025); prior demonstrations include Saunshi et al. (2025), Fan et al. (2025), and Geiping et al. (2025).Source: §2 · p. 2
  • HRM used two-timescale modules and truncated credit assignment; TRM removed much of that structure, while URM retained shared modules and added local mixing.HRM also used adaptive computation; TRM extended differentiated recurrence and used weight averaging (Jolicoeur-Martineau, 2025); URM used a different truncated-backpropagation scheme (Gao et al., 2025).Source: §2 · pp. 2–3
  • HRM, TRM, and URM change several design choices simultaneously, so this work places them in a shared framework and varies choices one at a time.Source: §2 · p. 3

3 DATA

The data suite combines established recursive-reasoning benchmarks with controlled out-of-distribution evaluations. Tasks share a padded-sequence prediction interface, while their datasets and splits probe distinct forms of algorithmic generalization.

3 DATA

  • Six domains span constraint satisfaction, search, iterative dynamics, symbolic composition, and abstract rule induction; Sudoku, Maze, and ARC-AGI are established benchmarks, while Game of Life and Arithmetic provide controlled out-of-distribution evaluation.Source: §3 · p. 3, Figure 2 · p. 3
  • All tasks use padded token-sequence inputs and require complete-target prediction without intermediate supervision or chain-of-thought traces.Source: §3 · p. 3
  • Sudoku puzzles become 81-token inputs; mazes become 900-token grids targeting a shortest path; ARC-AGI-1 is combined with ConceptARC, while ARC-AGI-2 is evaluated separately.ARC follows prior preprocessing and augmentation protocols and uses a transductive training setup.Source: §3 · pp. 3–4
  • Game of Life uses 200,000 trajectories on random binary grids up to 17 × 17, with held-out patterns and tests 1, 2, 3, and 10 steps beyond training horizons.Each target is the configuration after exactly k updates, making sequential rule applications controllable.Source: §3 · pp. 3–4
  • Arithmetic masks operators in reverse-Polish expressions with three to eight operands, then splits tests by unseen operand combinations and targets within [0, 101] versus outside it, [102, 201].Expressions use operands from {1, . . . , 9} and operators from {+, −, ×, ÷}.Source: §3 · p. 4

4 STABLE RECURSIVE REASONING

The paper builds a recursive-reasoning recipe by isolating optimization and recurrent-state choices, then evaluates the combined model across algorithmic domains. The recipe emphasizes controlled state refinement and stable training rather than added architectural complexity.

4 STABLE RECURSIVE REASONING

  • The unified recipe combines shared recurrent computation with controlled state updates, intermediate gradient propagation, and recurrence-specific regularization.The final configuration focuses on stabilizing recurrent dynamics rather than increasing architectural complexity.Source: §4 · p. 4, §4.5 · p. 5

4.1 RECURSIVE COMPUTATION

  • The nested recurrence updates the low-timescale state repeatedly within each high-level cycle, then updates the high-timescale state; the default shares recurrent transformation parameters.Source: §4.1 · p. 4

4.2 CONTROLLED RECURRENT-STATE UPDATES

  • Repeated shared-parameter updates can accumulate errors and drift, so the model bounds candidate updates, gates their application, and normalizes after cycles.These mechanisms constrain the trajectory while allowing token positions to refine at different rates.Source: §4.2 · pp. 4–5

4.3 RECURRENCE-CONSISTENT REGULARIZATION

  • The model reuses one dropout mask across recurrent cycles and ACT steps, and adds norm-scaled Gaussian noise to embeddings and recurrent states.Source: §4.3 · p. 5

4.4 TRAINING THE RECURSION

  • Training executes the full recurrent trajectory but backpropagates only through its final two high-level and two low-level cycles by default.Optimization also uses large physical batches rather than equivalent gradient accumulation.Source: §4.4 · p. 5

4.5 FINAL RECURSIVE REASONING RECIPE

  • The final model recurrently applies a shared Transformer block, using four high-level and two low-level updates per ACT step and gradients through the final two updates at each level.Its state updates use bounding, learned gating, and post-cycle normalization.Source: §4.5 · p. 5, Figure 3 · p. 5

4.6 EVALUATION AND MAIN RESULTS

  • Arithmetic OOD accuracy rises from 36.20% for the strongest recursive baseline to 71.16%, and the model leads evaluated recursive baselines on nearly all reported metrics.Source: Table 2 · p. 6, §4.6 · p. 6
  • Recursion's effect varies by domain: recursive models outperform the non-recurrent control on Sudoku and Maze, while ARC-AGI-2 remains difficult for all systems.The largest gains appear on explicit generalization tests rather than uniformly across domains.Source: §4.6 · pp. 6–7

5 CONTROLLED ABLATIONS AND SCALING LIMITS

Controlled ablations show that gradient flow, physical batch size, and recurrent-state stabilization materially affect recursive reasoning performance. Explicit hierarchy, greater model capacity, and more test-time computation do not yield uniform gains.

5 CONTROLLED ABLATIONS AND SCALING LIMITS

  • The ablations vary architectural or optimization factors while holding the rest of the training pipeline fixed, then test scaling with capacity, task breadth, and inference-time computation.The gradient-horizon comparison fixes forward recursion and varies only the backward horizon.Source: §5 · p. 7, §5.1 · p. 7

5.1 GRADIENT PATH LENGTH

  • 71.16% Arithmetic OOD accuracy at (KL, KH) = (2, 2) contrasts with 17.88% at (1, 3), which has the highest ID accuracy at 99.60%.The two horizons differentiate the same number of recurrent applications, so both the extent and placement of gradient flow matter.Source: Figure 4 · p. 7, §5.1 · p. 7

5.2 PHYSICAL BATCH VS. GRADIENT ACCUMULATION

  • Game of Life OOD accuracy falls from 63.16% with physical batch 8192 to 37.71% with four accumulation steps, while mean pre-clipping gradient norm rises from 0.38 to 65.46.At the same effective batch size, gradient accumulation substantially changes stability and final accuracy, making physical batch size an optimization parameter in this setting.Source: §5.2 · pp. 7–8, Figure 5 · p. 8

5.3 IS THE HIERARCHY NECESSARY?

  • Neither shared nor separate recurrent modules consistently dominate: shared modules do better on Maze and ARC-AGI-1, separate modules gain slightly on Sudoku and Game of Life, and Arithmetic is nearly tied.A shared recurrent module remains competitive across the studied domains, so gains do not depend on explicit hierarchical decomposition.Source: §5.3 · p. 8

5.4 STABILIZATION MECHANISMS

  • With the complete stabilization recipe, Game of Life reaches 66.08% ID and 65.80% OOD accuracy, while seed variability falls from 3.32/2.90 to 0.11/0.13 points.On Sudoku, variability falls from 1.06 points in the base configuration to approximately 0.2 points in stabilized variants while mean accuracy remains near 98%.Source: Table 3 · p. 8, §5.4 · pp. 8–9

5.5 LIMITS OF SCALING RECURSIVE COMPUTATION

  • Recursive reasoning does not improve simply by adding parameters or recurrent steps; capacity and test-time compute help selectively, while optimization and stabilization yield the most consistent gains.Increasing recurrent-block depth, hidden size, or attention heads did not consistently improve accuracy; additional ACT budget helped most on Arithmetic and Sudoku.Source: §5.5 · p. 9, Figure 6 · p. 9

6 CONCLUSION

Across six algorithmic domains, stable recursive computation depends strongly on how recurrence is trained and stabilized; the conclusion identifies gradient propagation, physical batch size, and recurrent-state updates as key factors.

6 CONCLUSION

  • Controlled state updates, an intermediate gradient horizon, and large physical batches underpin reliable recursive training, while added capacity and test-time computation bring domain-dependent rather than universal gains.Neither aggressive truncation nor full backpropagation provides the best generalization; bounded, gated, and normalized recurrent-state updates support reliable training.Source: §6 · p. 9

A.1 COMMON REPRESENTATION

  • All domains use padded token sequences, with complete-target prediction and no intermediate supervision or chain-of-thought traces.Grid tasks are flattened into token sequences.Source: §A.1 · p. 12

A.2 SUDOKU

  • Sudoku examples are 9×9 boards flattened into 81 tokens, with zeros for empty cells and the completed board as target.The study uses the sudoku-extreme corpus and trains without symmetry-based augmentation.Source: §A.2 · p. 12

A.3 MAZE

  • Maze examples are 30×30 grids encoded as 900-token sequences, with the target marking a shortest path from start to goal.The maze-30x30-hard corpus represents walls, free cells, start, goal, and path.Source: §A.3 · p. 12

A.4 GAME OF LIFE

  • Game of Life examples pair an initial configuration with requested step k, and the target is the configuration after exactly k transition-rule applications.Increasing k makes the required sequential computation longer without changing the transition rule; generated patterns exceeding 250 serialized characters are discarded.Source: §A.4 · p. 12

A.5 ARITHMETIC

  • In arithmetic sequences of at most 19 tokens, operators are masked with ?, the final value is appended, and the model reconstructs operators; evaluation uses 80/20 operand-multiset splits and target-value partitions.Targets [0, 101] are in-distribution; [102, 201] are outside the training range and form the out-of-distribution value split.Source: §A.5 · p. 13

A.6 ARC-AGI

  • ARC-AGI-1 is combined with ConceptARC while ARC-AGI-2 is evaluated separately; the transductive protocol trains on task demonstrations but not the held-out test input.Augmentations apply grid transformations and color permutations, which are inverted to map predictions back to the original grid.Source: §A.6 · p. 13

A.7 EVALUATION SPLITS

The evaluation spans standard held-out tasks and controlled distribution shifts, while the implementation specifies the shared backbone, optimization, recurrent-state stabilization, and ACT carry procedure.

A.7 EVALUATION SPLITS

  • Sudoku and Maze use standard held-out splits, ARC-AGI uses a transductive protocol, and Game of Life and Arithmetic add controlled out-of-distribution evaluations.The latter domains isolate different forms of generalization.Source: §A.7 · p. 13, Table 4 · p. 14

B.1 BACKBONE AND OPTIMIZATION

  • The recurrent module uses four post-norm Transformer layers with hidden size 512, and shared high- and low-level parameters yield 13.6M trainable parameters; Adam-atan2, clipping, and EMA support optimization and evaluation.The implementation also specifies warm-up followed by domain-dependent cosine or exponential decay.Source: §B.1 · p. 14

B.2 RECURRENT-STATE REGULARIZATION

  • Recurrent-state stabilization combines bounded relative updates with detached scaling, a gate initialized near 1/2, recurrently reused dropout masks, and norm-scaled state and embedding noise.Dropout masks persist across recurrent cycles and ACT steps until the sequence halts.Source: §B.2 · pp. 14–15

B.3 ACT AND RECURRENT CARRY

  • The model allows up to 16 ACT steps, stores and detaches recurrent states between steps, and trains with output and halting losses; main evaluation disables learned halting and uses the full budget.ARC-AGI-1 uses a reduced budget of 8 ACT steps.Source: §B.3 · p. 15

B.4 DOMAIN-SPECIFIC TRAINING CONFIGURATIONS

The recurrent architecture is fixed across domains, while regularization and optimization settings vary by task; ARC-AGI scoring maps augmented predictions back to original grids before pass@k evaluation.

B.4 DOMAIN-SPECIFIC TRAINING CONFIGURATIONS

  • All domain models share a recurrent Transformer and propagate gradients through the final (2, 2) cycles, while state normalization, dropout, noise, and controlled zL updates are configured across tasks.Optimization settings also vary by domain, including physical batch size, training duration, learning-rate schedule, and weight decay.Source: §B.4 · pp. 15–16

C.1 ARC-AGI EVALUATION

  • ARC-AGI predictions are inverse-transformed, cropped, grouped by exact grid equality, and ranked by votes with halting confidence as a tie-break; pass@k requires the correct output among the top-k for every test input.The main comparison disables adaptive halting and executes the full ACT budget; test-time scaling instead extends that budget.Source: §C.1 · p. 16

C.2 INFERENCE COMPUTE

Recursive inference cost depends on repeated layer applications and ACT steps, so parameter sharing reduces model size more directly than sequential computation.

C.2 INFERENCE COMPUTE

  • One ACT step requires Hcycles(LcyclesLlayers + Hlayers) layer applications, and executing T ACT steps multiplies that cost by T.The table reports inference cost per answer relative to one forward pass of the dense control.Source: §C.2 · p. 16, Table 7 · p. 16
  • Weight sharing reduces parameter storage rather than sequential computation: TRM and this configuration have different parameter counts but similar recurrent inference cost, while the dense control is cheaper per answer.ARC-AGI augmentation and voting add inference cost.Source: §C.2 · pp. 16–17
  • The selected (2, 2) gradient-path model obtains the highest final Arithmetic OOD score among the configurations shown.Figure 7 plots final EMA best full exact accuracy against retained high-level cycles and OOD accuracy over training.Source: Figure 7 · p. 17

D GRADIENT-PATH LENGTH ABLATION

The ablation varies how many terminal high- and low-level recurrent cycles receive gradients while keeping forward recurrence, inference cost, and parameter count fixed. An intermediate gradient path gives the best observed balance between Arithmetic in-distribution fitting and out-of-distribution generalization.

D GRADIENT-PATH LENGTH ABLATION

  • The ablation varies retained terminal high- and low-level cycles across eight settings, without changing forward recurrence, inference cost, or parameter count.The settings retain gH ∈ {1, 2, 3, 4} and gL ∈ {1, 2} cycles in the backward graph.Source: D GRADIENT-PATH LENGTH ABLATION · p. 17

D.1 SETUP.

  • Runs use the same 13.65M-parameter Arithmetic model and evaluate EMA best full exact accuracy on ranges 0–101 and 102–201.The shared setup uses global batch size 4096, 2000 epochs, and 439,600 optimizer updates.Source: §D.1 · p. 17

D.2 THE SELECTED (2, 2) SETTING BALANCES FITTING AND GENERALIZATION.

  • 71.16% final OOD accuracy at (gL, gH) = (2, 2), the highest in the comparison, accompanies 96.34% ID accuracy.The final OOD score is 2.62 percentage points above the next-best (1, 4) setting.Source: Table 8 · p. 18, §D.2 · p. 18

D.3 AN INTERMEDIATE GRADIENT PATH IS OPTIMAL.

  • The (2, 2) path balances fitting and extrapolation: shorter paths can reach 97.35%–99.60% ID but stay below 14% OOD, while (2, 4) falls to 89.08% ID and 59.88% OOD.The authors attribute the intermediate setting's advantage to sufficient gradient reach without the degradation seen at the longest path.Source: §D.3 · p. 18, §D.2 · p. 18

E GAME OF LIFE: SINGLE-SEED TRAINING DYNAMICS

This appendix tracks one seed of the final Game of Life configuration to characterize training dynamics, not to provide seed-averaged model comparisons. Accuracy peaks before the final checkpoint, and continued recurrent steps add little refinement.

F A NEGATIVE RESULT: JOINT TRAINING ACROSS FOUR DOMAINS

A shared model trained across Arithmetic, Sudoku, Game of Life, and Maze learns only a partial Arithmetic solver in the available run. The authors treat this as a negative diagnostic, not evidence that multitask training is intrinsically ineffective.

Loading 2609.39967v1…