PaperlayerView exact revision on arXiv

Source-linked AI summary

Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals

Pengxiang Li, Dilxat Muhtar, Di He, Guinan Su, Lu Yin, Shiwei Liu

arXiv:2610.11570v1cs.AI

TL;DR

Increasing loop iterations can reduce reasoning accuracy when noisy updates overwrite useful intermediate deductions and completed solutions. InfiLoop replaces carry-last with a learned, decayed aggregation of recurrent states and an exact streaming recurrence. Across reasoning benchmarks it outperforms carry-last and fixed-point damping baselines, including 97.9% exact accuracy on Sudoku-Extreme.

  • Problem

    Carry-last looped Transformers pass only the latest recurrent state forward, leaving earlier computations accessible only through information retained in that state.

    Source: §1 · p. 1

  • Method

    InfiLoop combines recurrent outputs using shared content-based scoring and learned exponential temporal decay, with an exact recurrence that keeps persistent aggregation memory constant as loops grow.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    Across Sudoku-Extreme, Maze-Hard, ARC-AGI-1, and ARC-AGI-2, InfiLoop outperforms carry-last and fixed-point damping baselines; it reaches 97.9% exact accuracy on Sudoku-Extreme.

    Source: §4.3 · p. 6, §5 · p. 10

  • Takeaways & Limitations

    The learned connection suppresses proposals with constraint violations, increases its update upon reaching a valid solution, and resists state drift under perturbations.

    Source: §5 · p. 10

Abstract

from arXiv · show

In this paper, we argue that looped Transformers need their own residual connections to prevent performance degradation as the number of iterations grows. We observe that increasing loop iterations can reduce reasoning accuracy: noisy state updates overwrite correct intermediate deductions and even undo completed solutions. This leaves subsequent iterations to recover lost information from an already degraded representation: once an error arises in an earlier loop, often as a result of long-range propagation through the recurrence, later loops find it difficult to correct. In this paper, we introduce InfiLoop, a loop-native residual connection that learns which past computations to retain and how much to accept from each new update. InfiLoop combines content-based weighting with learned temporal decay to maintain a running summary of recurrent states. An exact streaming recurrence keeps its persistent aggregation memory constant as the loop count grows. The resulting adaptive update suppresses unreliable proposals and preserves useful intermediate states. Across extensive reasoning tasks, a 7M-parameter InfiLoop model outperforms existing recursive architectures, reaching 97.9% exact accuracy on Sudoku-Extreme, and 13.6% pass@2 on ARC-AGI-2. Notably, on Sudoku-Extreme, InfiLoop continues to improve with test-time looping beyond 20,000 effective steps, showing that added depth translates directly into stronger reasoning. Our code is available at https://github.com/pixeli99/InfiLoop.

1. Introduction

Looped Transformers gain computational depth by repeatedly applying shared blocks, but carry-last leaves earlier computations dependent on what survives in the latest state. InfiLoop replaces that rule with learned aggregation of recurrent outputs and an exact streaming recurrence.

1. Introduction

  • Carry-last leaves earlier computations accessible only through information retained in the latest state, so noisy iterations can degrade useful representations throughout propagation.Existing decay and layer-scaling remedies regulate update magnitude but still pass only the latest recurrent state to the next loop.Source: §1 · pp. 1–2
  • On Sudoku-Extreme, TRM peaks at 468 layers and then loses accuracy as depth grows, while its state changes again after briefly settling.Figure 1 tracks accuracy by effective depth and, for puzzles solved within 36 updates by both models, state changes and continued correctness after solving.Source: Figure 1 · p. 2
  • InfiLoop replaces carry-last with learned attention over recurrent outputs, combining content-based scoring and exponential temporal decay.Its exact streaming recurrence uses a weighted-state vector and normalization scalar per token, so persistent aggregation memory is independent of loop count.Source: §1 · p. 2
  • The paper reports that InfiLoop preserves solved states, combines recurrent states with constant memory, and outperforms damping baselines with fewer layer evaluations.It also reports that the learned step size discounts unsolved search outputs, accepts the solution-completing output, and freezes the state once solved.Source: §1 · p. 2

2. Deeper Loops Undo Earlier Progress

As depth grows, TRM loses accuracy and can undo correct predictions, while InfiLoop continues improving and retains solved puzzles over the measured updates. InfiLoop replaces carry-last with an aggregated recurrent state that serves as input to the next iteration.

2. Deeper Loops Undo Earlier Progress

  • Figure 2 contrasts TRM's carry-last state, where zt becomes ht, with InfiLoop's aggregation of earlier candidates using content weights and exponential temporal decay.The illustrated candidate contributions are schematic, and the same block parameters are reused at every recurrent step.Source: Figure 2 · p. 3
  • 90.9%: InfiLoop exact accuracy at 1,872 layers, while TRM falls from 62.2% at 468 layers to 58.1% at 1,872 layers.Over updates 36–72, TRM changes 77.0 predictions per puzzle and adds 0.21 incorrect cells per puzzle; InfiLoop makes 24.0 changes and removes 1.25 incorrect cells per puzzle.Source: §2 · p. 3
  • 20.9% of initially empty cells that TRM predicted correctly at update 36 become incorrect at least once during later updates, versus 2.4% for InfiLoop.After solving, TRM's state change rises from 3% to 10% per update and it loses 19.7% of solved puzzles after 36 updates; InfiLoop's change falls to 0.3% and its solved puzzles remain correct.Source: §2 · p. 3
  • InfiLoop's running summary combines earlier outputs and learns how much of each new output to add before passing the summary to the next iteration.A shared scorer weights each output once when produced, while learned decay reduces older contributions; persistent memory does not grow with iteration count.Source: §2 · p. 3

3. Loop-Native Attention Residuals

The section replaces carry-last state replacement with attention over recurrent outputs, so the next iteration can use a learned aggregation rather than relying only on the latest state. It combines content weighting and temporal decay, with an exact streaming recurrence and an adaptive update that keeps aggregation memory independent of loop count.

3.1. Attention Residuals Across Recurrent Steps

  • InfiLoop adapts Attention Residuals (Kimi Team et al., 2026) to recurrent steps, weighting candidates by a shared pseudo-query and learned exponential recency decay.The normalized aggregation becomes the state passed to the next iteration.Source: §3.1 · pp. 3–4

3.2. Exact Recurrence with Constant Memory

  • The shared scorer and geometric decay yield an exact streaming recurrence, retaining only a weighted-state vector and normalization mass instead of the full candidate history.The recurrence is algebraically identical to the attention aggregation and introduces no approximation to its weights.Source: §3.1 · p. 4, §3.2 · p. 4

3.3. Interpretation as an Adaptive Residual Update

  • The adaptive step favors a new candidate when its content weight exceeds retained historical mass, while the convex aggregation bounds state norm by the largest candidate norm.A uniform bound across arbitrarily many iterations additionally requires candidate norms to remain bounded; the rule also applies independently to inner and outer recurrent axes.Source: §3.3 · p. 4

3.4. Computational and Memory Complexity

  • Aggregation uses O(d) persistent memory and O(Td) total cost, versus O(Td) memory and O(T^2d) total cost for Full Attention Residuals with iteration-specific queries.Per-step aggregation requires O(d) computation.Source: §3.4 · p. 5

4. Experiments

The experiments evaluate a compact recursive reasoner across symbolic puzzles and ARC tasks, then examine test-time scaling and the learned connection’s behavior. Results cover benchmark performance, state retention, and design ablations.

4.1. Dataset description

  • InfiLoop evaluates Sudoku-Extreme, Maze-Hard, ARC-AGI-1, and ARC-AGI-2; Sudoku and Maze use exact sequence accuracy, while ARC uses pass@2.The model is a 7M-parameter recursive reasoner with shared Transformer blocks and learned connections across inner and outer loops.Source: §4.1 · p. 5, §4.2 · p. 5

4.2. Evaluation protocol

  • The evaluation benchmarks InfiLoop against looped reasoning baselines and frontier reasoning LLMs; for Sudoku-Extreme, it trains on 1,000 base puzzles with 1,000 augmentations per puzzle.The Sudoku-Extreme schedule uses H = 3 outer loops and L = 6 inner updates across 12 deep-supervision segments.Source: §4.2 · pp. 5–6

4.3. Reasoning performance across puzzle benchmarks

  • Across four benchmarks, the complete 7M-parameter InfiLoop model leads the listed 7M-parameter configurations, including 97.9% exact accuracy on Sudoku-Extreme and 13.6% pass@2 on ARC-AGI-2.It also reaches 87.4% accuracy on Maze-Hard and 49.1% pass@2 on ARC-AGI-1.Source: Table 1 · p. 6, §4.3 · p. 6

4.4. Test-time depth and compute efficiency

  • At 24,960 layer evaluations, InfiLoop reaches 92.74% exact accuracy, versus 91.63% for FPRM and 83.65% for TRM.At 1,872 layers, InfiLoop’s 90.9% exceeds FPRM’s 90.1% at 3,744 layers, using half the layer evaluations.Source: Figure 3 · p. 7, §4.4 · p. 7

4.5. What the connection learns to carry

  • The learned step size averages approximately 0.40 during unsolved search, rises to 0.91 when a puzzle is solved, then returns to the 0.56 equal-weight baseline.For puzzles that remain unsolved, the step size stays suppressed across iterations.Source: §4.5 · p. 7, Figure 4 · p. 8
  • Content scores favor correct candidate boards: the median is 7.25 for fully correct boards, 2.75 at 20 wrong cells, and −0.58 at 33 wrong cells.The scorer receives no external hints or task labels during reasoning.Source: §4.5 · p. 8
  • Across perturbations from 10^-4 to 0.3 at update 36, InfiLoop retains all solved puzzles through update 72, while TRM loses 15–20% even without perturbation.TRM’s trajectory diverges to relative distance of order one within 12 updates after a perturbation of 10^-4; InfiLoop bounds displacement.Source: §4.5 · pp. 8–9, Figure 5 · p. 8

4.6. Connection placement and decay initialization

  • At H3 L6, dual-axis aggregation reaches 82.1% accuracy versus 72.6% for inner-only and 75.1% for outer-only; at H32 L6, it reaches 89.7%.Learning forward decay from β0 = 0.55 gives the best reported accuracy; reverse decay with β = 1.82 reaches 2.0% at both budgets.Source: §4.6 · p. 9, Table 2 · p. 9

5. Conclusion

The conclusion identifies state replacement under carry-last as a bottleneck in deep looped Transformers and presents InfiLoop as a way to selectively aggregate recurrent states. Across the reported reasoning benchmarks, it outperforms carry-last and fixed-point damping baselines, while its learned connection adapts updates to preserve useful states.

5. Conclusion

  • InfiLoop replaces carry-last state replacement with content-based pseudo-query scoring and exponential temporal decay over recurrent outputs.Its exact recurrence uses O(d) persistent memory per token, independent of loop count, and keeps the aggregated state within the convex hull of candidate representations.Source: §5 · pp. 9–10
  • On Sudoku-Extreme, Maze-Hard, ARC-AGI-1, and ARC-AGI-2, InfiLoop outperforms carry-last and fixed-point damping baselines and reaches their accuracy with fewer layers under test-time depth scaling.The conclusion frames selective aggregation of recurrent states as an effective mechanism for scaling test-time reasoning depth.Source: §5 · p. 10
  • The learned connection suppresses proposals with constraint violations, increases update magnitude upon reaching a valid solution, and resists state drift under perturbations.The paper identifies this behavior as an adaptive residual update.Source: §5 · p. 10

A. Compute and Persistent-State Accounting

The accounting separates persistent recurrent state from layer evaluations and counted computation. Persistent state does not depend on loop budget, while computation is reported under a specified FLOP convention.

A. Compute and Persistent-State Accounting

  • 582.8 KiB of persistent state is stored per puzzle, independent of the loop budget.At batch size one, this comprises 194.0 KiB for two BF16 carries and 388.8 KiB for InfiLoop’s FP32 numerator and denominator; the accounting excludes weights, inputs, and temporary workspaces.Source: A. Compute and Persistent-State… · p. 14
  • The depth study ranges from 156 to 24,960 Transformer layer evaluations by varying S from 1 to 160 with H=6 and L=12.Each segment executes 156 layers; the reported count omits the connection and prediction heads, and layer evaluations are not measured runtime.Source: A. Compute and Persistent-State… · p. 14
  • Training uses S=12, H=3, L=6, and K=2, yielding 504 layer evaluations.Each outer cycle applies the shared block L times to the inner state and once to the outer state; the layer-evaluation count scales with supervision segments, outer cycles, and block depth.Source: A. Compute and Persistent-State… · p. 14
  • At the depth-study setting, counted computation is 1.27 TFLOPs at 1,872 layers and 17.0 TFLOPs at 24,960 layers.The FLOP formula matches four profiled depths; the accounting counts two FLOPs per multiply-accumulate and excludes several operations, including normalization and data movement.Source: A. Compute and Persistent-State… · p. 14

B. Related Work

Related work places InfiLoop among recurrent reasoning architectures and approaches to stabilizing or learning information flow across depth. It distinguishes its iteration-level state aggregation from methods that preserve inputs, scale updates, or retain activation histories.

Reasoning through recurrent depth

  • Universal and looped Transformers reuse a shared block for reasoning, while HRM and TRM use nested latent updates and adaptive-computation methods learn how much computation to spend.InfiLoop modifies the connection between iterations of a nested TRM-style reasoner.Source: B. Related Work · pp. 14–15

Overthinking in recurrent networks

  • Input recall preserves access to the original problem instance, whereas InfiLoop aggregates candidate states produced during reasoning; both address behavior beyond the training iteration budget.The passage distinguishes the information acted on by the two approaches.Source: B. Related Work · p. 15

Stabilizing deep and looped Transformers

  • Prior stabilization approaches use normalization, residual scaling, or fixed-point formulations, while InfiLoop learns from task loss which earlier outputs the next state keeps and by how much.The cited approaches include ReZero, LayerScale, DeepNet, residual scaling for loops, and FPRM.Source: B. Related Work · p. 15

Learned residual connections

  • DenseFormer and Hyper-Connections learn cross-layer information flow, while InfiLoop specializes learned mixing to a shared recurrent block with content-dependent weights and exponential age decay.Its running weighted sum and normalization mass have size independent of iteration count.Source: B. Related Work · p. 15

Gated recurrent updates

  • InfiLoop’s interpolation resembles GRU and Highway Network gates, but its update ratio comes from the new content weight and decayed historical mass rather than a separate gating network.One shared content scorer and one decay parameter add d + 1 parameters per recurrence axis.Source: B. Related Work · p. 15

Relation to Attention Residuals

  • Unlike full Attention Residuals, which use layer-specific pseudo-queries over stored outputs, InfiLoop scores each iteration once with a shared scorer and retains only running aggregates.This avoids storing the full activation history in a model with no fixed number of iterations.Source: B. Related Work · p. 15

C. Qualitative analysis on ARC-AGI-2.

Figure 6 compares InfiLoop and TRM predictions on two held-out ARC-AGI-2 tasks. InfiLoop matches both targets exactly, while TRM’s predictions contain cell errors.

C. Qualitative analysis on ARC-AGI-2.

D. Removing the Content Score

On Sudoku-Extreme at 1,872 layers, removing InfiLoop’s content score lowers accuracy whether the score is removed during training or at test time. The results indicate that damping alone does not account for InfiLoop’s accuracy.

D. Removing the Content Score

  • 84.1% versus 90.9% exact accuracy shows a 6.8-percentage-point cost when the content score is removed during training.Both models use the same recipe, seed, and number of training steps.Source: D. Removing the Content Score · p. 16
  • Without retraining, setting q to zero lowers accuracy from 90.9% to 84.8%; setting β to zero reduces it to 1.5%.With β=0, the connection becomes carry-last; the authors attribute the drop to block weights trained for aggregated rather than raw outputs.Source: D. Removing the Content Score · p. 16
  • Both ways of removing the score lower accuracy by 6 to 7 percentage points, indicating that damping alone does not explain InfiLoop’s gain.A fixed exponential average damps updates but lacks the content score and is less accurate.Source: D. Removing the Content Score · p. 16

Related papers

Loading 2610.11570v1…