PaperlayerView exact revision on arXiv

Source-linked AI summary

Right Direction, Wrong Step: Geometric Analysis of Finite-Step Failure in Looped Transformers

Zhihao Guo, Zonghan Wu, Haizhou Du, Huan Huo, Yilei Shao, Athanasios V. Vasilakos, Qingsong Wen

arXiv:2609.16665v1cs.LGcs.AI

TL;DR

The paper asks whether recurrent updates reduce reference-answer support because their direction is unhelpful or because their full displacement goes too far. It analyzes reference utility along fixed update directions with pathwise curvature and a local quadratic scale model. Across two model families, shorter steps recover many selected failures, including under a fixed quarter step.

  • Problem

    It is unclear whether a harmful recurrent update has an unhelpful direction or moves too far along a locally improving one, limiting understanding of how recurrent computation affects reference-answer support.

    Source: §1 · p. 1

  • Method

    The paper analyzes reference utility along the model’s proposed update, using pathwise curvature, a local quadratic approximation, and fixed-direction interventions to evaluate shorter displacements.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    72.2–83.2% of selected finite-step failures recover under a fixed quarter step across four settings, with recovery observed in both model families.

    Source: §5 · p. 9

  • Takeaways & Limitations

    A harmful full update can retain a locally useful direction, and a shorter displacement along that direction can recover reference utility.

    Source: §1 · p. 2

    The experiments obtain the uniform curvature bound needed for sufficient-scale guarantees for Ouro’s affine readout, but not for Huginn.

    Source: §3.2 · p. 4

Abstract

from arXiv · show

Looped Transformers offer a parameter-efficient route to test-time scaling by reusing shared layers for iterative latent reasoning. However, additional iterations can reduce support for a reference answer, leaving unclear whether an update's direction is locally unhelpful or its full displacement moves too far. We study this distinction by analysing reference utility, which measures this support, along the model's own update direction, varying the fraction of the proposed displacement supplied to the readout. This reveals finite-step failures in which a locally improving direction produces a harmful full update. A pathwise curvature decomposition characterises how initial progress is lost, while a local quadratic model predicts full-step gains and useful step scales. Bounds based on accumulated curvature variation characterise the approximation error of these predictions. Experiments across two model families reveal this separation on mathematical and commonsense tasks. A fixed quarter step produces positive gains in reference utility for 72.2--83.2% of selected failures across four settings. These findings identify a mismatch between update direction and step scale as a mechanism of lost progress, explaining how some harmful updates retain useful computation.

1 INTRODUCTION

The paper asks whether harmful recurrent updates follow unhelpful directions or move too far along locally improving ones. It analyzes reference utility along proposed updates, using curvature to explain lost progress and step contraction to recover it.

1 INTRODUCTION

  • The paper distinguishes update direction from displacement magnitude by measuring reference utility along the model’s proposed update.This targets how added recurrent computation translates into task progress.Source: §1 · pp. 1–2
  • A finite-step failure occurs when an update initially improves reference utility but its full displacement lowers it.Accumulated curvature can outweigh the initial gain, while a shorter move along the same direction can outperform both endpoints.Source: §1 · p. 2
  • 72.2–83.2% of selected failures recover positive reference utility under a fixed quarter step across four settings.The interventions measure utility after proposal computation and before further recurrence.Source: §1 · p. 2
  • Pathwise curvature characterizes lost progress, while a local quadratic model predicts full-step gains and useful scales, with curvature variation bounding prediction error.The contributions also compare fixed and curvature-selected contractions with endpoint-oracle choices.Source: §1 · p. 2

2 RELATED WORK

Prior work studies recurrent computation, stability, adaptive depth, and latent-state refinement. This paper connects those themes by examining how update direction and displacement magnitude shape task progress.

Recurrent Computation and Overthinking

  • Studies of recurrent computation connect depth to generalisation, while overthinking work reports reversals from correct predictions to errors during deeper processing or latent iterations.The paper asks whether harmful updates follow an unhelpful direction or move too far along a locally improving one.Source: §2 · p. 2

Stability and Step Control

  • Prior approaches regulate recurrence through stability methods, stopping decisions, or controls on individual updates, including damping, gating, and conditioning.This paper instead analyzes utility along a proposed update while holding its direction and readout fixed.Source: §2 · p. 2

Task Feedback and Update Geometry

  • Task feedback and representation geometry guide internal-state refinement; related interpolation analysis derives reference-token loss derivatives, while this work examines curvature-driven reversals and shorter-step recovery.For affine logit readouts, the state interpolation induces the same distribution path.Source: §2 · p. 2

3 METHOD

The method holds the model’s proposed update direction and readout fixed while analyzing reference utility along the displacement. It uses pathwise curvature, local quadratic predictions, and fixed-direction interventions to study progress and useful step scales.

3 METHOD

  • A looped Transformer proposes a hidden-state update, and a fixed readout evaluates task utility against a fixed reference to measure progress.The method varies the fraction of the proposed displacement supplied to the readout while preserving its direction.Source: §3 · p. 3, §3.1 · p. 3

3.1 RECURRENT UPDATES AND TASK UTILITY

  • Finite-step failure means the full update harms utility despite a positive initial slope, distinguishing displacement-related failure from a harmful direction.The comparison uses the update’s initial utility slope and full-displacement gain.Source: §3.1 · p. 3

3.2 DIRECTION–STEP COMPATIBILITY

  • A pathwise curvature correction added to the initial linear gain decomposes utility change; a sufficiently negative correction can outweigh that gain.A curvature lower bound also yields a sufficient range of improving step scales.Source: §3.2 · p. 4

3.3 PREDICTING THE UTILITY-MAXIMISING SCALE

  • The local quadratic model uses initial slope and curvature to predict full-update gain and a maximizing scale; accumulated curvature deviation bounds prediction error and, for negative initial curvature, scale error and regret.The bounds apply to global maximizers without assuming concavity.Source: §3.3 · pp. 4–5

3.4 PREDICTION AND FIXED-DIRECTION INTERVENTION

  • Fixed-direction interventions evaluate a shared contracted step or update-specific scales, while endpoint and step oracles compare intermediate-scale utility with endpoint selection.Recovery is measured among finite-step failures; the oracle comparison measures additional utility available from intermediate scales.Source: §3.4 · p. 6

4 EXPERIMENTS

Across fixed-direction interventions, the experiments test whether local utility geometry predicts progress within recurrent updates and whether shorter or selected displacements recover it. Results show finite-step failures across conditions, accurate scale prediction, and recovery gains from adaptive step selection.

4.1 SETUP

  • The study evaluates Ouro-1.4B and 2.6B and Huginn-0125 across MATH-500, GSM8K, and HellaSwag, with nine fixed-transition conditions.Path and quarter-step analyses also include selected CommonsenseQA questions; the wider depth sweep covers 21 conditions.Source: §4 · p. 6, §4.1 · p. 6

4.2 RQ1: LOCAL DIRECTION VERSUS FINITE PROGRESS

  • Finite-step failures occur in all nine conditions and comprise 1.4–49.2% of harmful updates, including conditions with positive mean gain.These updates initially improve reference utility but lower it after the full displacement.Source: §4.2 · p. 7, Table 1 · p. 7
  • Adding local curvature Q raises endpoint sign accuracy to 97.2–99.9% and reduces MAE by factors of 5.9–29.8 across nine matched conditions.The largest sign-accuracy gains occur on Ouro-1.4B mathematical tasks, where harmful full updates often have positive initial slopes.Source: §4.2 · p. 7, Table 1 · p. 7

4.3 RQ2: PREDICTING THE SCALE OF PROGRESS

  • On Ouro/MATH-500 and Huginn/GSM8K, predicted and empirical scales correlate above 0.995, while quadratic selection cuts mean grid regret by over two orders of magnitude.The regret advantage also holds against the best fixed scales in hindsight: 0.5 for Ouro and 1 for Huginn.Source: §4.3 · pp. 7–8, Figure 2 · p. 8, Table 2 · p. 8
  • For interior optima, mean scale error is below 0.019 on 269 Ouro-1.4B/MATH-500 cases and 0.018 on 36 Huginn/GSM8K cases.Across four Ouro conditions, the integrated curvature-variation bound covers numerical scale-error intervals for all 2034 primary updates and is below the full search range on 99.8%.Source: §4.3 · p. 8

4.4 RQ3: RECOVERING PROGRESS BY STEP CONTRACTION

  • On Ouro-1.4B/GSM8K at 4→5, quadratic selection recovers 94.2% of 431 finite-step failures, versus 71.0% for the quarter step.Across all 1319 questions, it also yields positive mean utility gain and outperforms the quarter step.Source: §4.4 · p. 9, Table 3 · p. 9
  • On Ouro-1.4B/MATH-500, curvature-bound selection recovers all 179 finite-step failures, versus 132 for the quarter step, with a 1.44 × 10−4 mean paired advantage.Recovered states have higher reference utility than both endpoints.Source: §4.4 · p. 9, Table 3 · p. 9
  • Endpoint selection retains most aggregate gain, but intermediate scales provide additional reference utility that stopping would forgo on finite-step failures.Reported gains measure immediate reference utility after proposal computation and before further recurrence.Source: §4.4 · p. 9

5 CONCLUSION

An additional recurrent update can lower reference utility despite a locally improving direction. The conclusion attributes this mismatch to accumulated curvature and reports that shorter displacements can recover utility, with quadratic scale selection improving recovery in one setting.

5 CONCLUSION

  • On Ouro-1.4B/GSM8K test, quadratic scale selection raises recovery from 71.0% to 94.2% relative to the quarter step.A fixed quarter step recovers 72.2–83.2% of selected failures across four settings.Source: §5 · p. 9
  • For a frozen model and fixed readout, accumulated curvature explains how an additional update can lower reference utility despite a locally improving direction.The analysis evaluates shorter displacements along the same direction before further recurrence.Source: §5 · p. 9

A.1 DIRECTIONAL DERIVATIVES

  • Directional derivatives along the fixed displacement define initial slope A and curvature Q from the utility derivatives at points on the tested segment.The path is parameterized as ϕ(α) = U(H + αD) − U(H).Source: A PROOFS OF THE PATH ANALYSIS · p. 13, §A.1 · p. 13

A.2 FINITE PROGRESS AND BENEFICIAL SCALES

  • With positive initial slope and a uniform lower curvature bound, every scale below 2A/LD is sufficient for positive utility gain when LD > 0.When LD = 0, every positive scale in the interval improves utility.Source: §A.2 · p. 13

A.3 THE CONTINUOUS PROGRESS BOUNDARY

  • A finite-step failure has a positive initial-progress boundary below one, while two-sided curvature bounds can bracket that boundary and the quadratic root prediction.The experiments evaluate only the upper curvature envelope and compare the quadratic root with sampled crossings; they do not compute the stated common-interval bound.Source: §A.3 · pp. 13–14

A.4 CONSTRAINED QUADRATIC MAXIMISATION

  • The constrained quadratic maximizer is obtained by clipping its vertex to the allowed scale interval when Q < 0, with boundary cases handled separately.For Q ≥ 0, the smallest maximizer is either zero or one, depending on A + Q.Source: §A.4 · p. 14

A.5 QUADRATIC APPROXIMATION ERROR

  • The quadratic approximation error can be bounded by integrating a bound on derivative variation along the path.The proof uses derivative comparisons and the fundamental theorem of calculus.Source: §A.5 · p. 14

A.5.1 ENDPOINT PREDICTION ERROR

The appendix bounds endpoint, scale, and regret prediction errors using accumulated curvature deviation, and gives conditions for finite-partition estimates. It also identifies cases where small derivative error does not guarantee a close maximizing scale.

A.5.1 ENDPOINT PREDICTION ERROR

  • The endpoint gain differs from its quadratic prediction A + Q by at most C(1), which determines its sign when the prediction exceeds that error bound in magnitude.Specifically, A + Q > C(1) guarantees positive gain, and A + Q < −C(1) guarantees negative gain.Source: §A.5.1 · p. 15

A.5.2 INTEGRATED CURVATURE DEVIATION

  • Integrated curvature deviation bounds the distance between the true and quadratic maximizing scales, including when the true maximizer is at an endpoint or the path is not concave.The appendix also bounds regret using the same deviation control.Source: §A.5.2 · p. 15

A.5.4 FINITE-PARTITION ENCLOSURES

  • Finite-partition curvature estimates can replace C(1) in the location and regret bounds when valid cellwise Lipschitz constants control curvature variation.A finite sum of endpoint-based enclosures supplies the upper bound; sampled third derivatives alone do not establish the required cellwise condition.Source: §A.5.4 · pp. 16–17

A.5.5 ZERO DEVIATION AND NONNEGATIVE INITIAL CURVATURE

  • When initial quadratic curvature is nonnegative, an arbitrarily small derivative residual can still accompany a large maximizing-scale error.The appendix contrasts this location instability with a value bound for arbitrary quadratics, where derivative residual ε bounds regret by ε.Source: §A.5.5 · p. 17

A.5.6 UTILITY REGRET AND THE THIRD-DERIVATIVE BOUND

  • A global third-derivative bound yields quadratic step-scale approximation and regret bounds, including for boundary maxima without requiring path concavity.The bound assumes the utility path is C3 and its third derivative is bounded on the segment.Source: §A.5.6 · p. 18

B FINITE ACTION SETS

  • For a concave path with a unique maximizer, the best grid action lies within one grid spacing of that maximizer.With bounded second derivative and an interior maximizer, the appendix also supplies a grid-value bound.Source: B FINITE ACTION SETS · p. 19

C.1 DIRECTIONAL CURVATURE

  • For affine-logit reference-token utility, directional curvature is the negative mean probability-weighted variance of the logit displacement, which is strictly positive for a nonconstant displacement.Thus the path curvature is strictly negative when at least one displacement is nonconstant.Source: §C.1 · p. 20

C.2 ENDPOINT DECOMPOSITION AND SHAPE

For affine-logit utilities, the full-step utility change separates into initial directional gain and a nonnegative endpoint-distribution correction. Under nonconstant displacement, a finite-step failure has a unique interior maximizer and positive zero.

C.2 ENDPOINT DECOMPOSITION AND SHAPE

  • The endpoint identity gives ΔU = A − C with C ≥ 0, so a locally improving direction followed by a harmful full step is equivalent to 0 < A < C.The correction C is expressed as an integral of curvature along the path.Source: §C.2 · p. 20
  • With nonconstant logit displacement, a finite-step failure has a strictly concave utility path, a unique interior maximizer, and a unique positive zero.The result assumes A > 0 and ϕ(1) < 0.Source: §C.2 · p. 20

D.1 STATES AND READOUT BOUNDARIES

  • The analysis fixes readout boundaries separately for Ouro and Huginn, and differentiates through the readout while holding recurrent endpoints and displacement fixed.Ouro's boundary is before its linear language-model head; Huginn's is before its fixed nonlinear coda and output head.Source: §D.1 · p. 21

D.2 TASK UTILITIES

The task utilities score reference solutions or correct multiple-choice options under teacher forcing, while the scale interventions hold the computed displacement fixed. The depth sweep covers seven labelled splits across three models and reports task-dependent utility declines, including a case where utility and accuracy move differently.

D.2 TASK UTILITIES

  • Mathematical-task utility is mean reference-token log probability, while multiple-choice utility is the log-softmax of length-normalized option scores under teacher forcing.HellaSwag uses four options, and the joint option intervention applies the same scale to every option.Source: §D.2 · pp. 21–22

D.3 SCALES AND COMPARISONS

  • Scale prediction uses a 21-point grid, and the intervention set tests scales 0, 0.25, 0.5, 0.75, and 1, with quarter-step contraction prespecified.Recovery frequency and mean gain are measured on selected finite-step failures before further recurrence.Source: §D.3 · p. 22

E.1 COVERAGE AND DEPTH SELECTION

  • The depth sweep evaluates seven labelled splits for each of three models, totaling 58,086 model–question observations, with pointwise intervals unadjusted for selection or multiple comparisons.Evaluated depths include those following measured losses.Source: §E.1 · p. 22
  • On Ouro-1.4B/HellaSwag validation, the 4→5 utility change is −0.008006 [−0.010838, −0.004991] while option accuracy rises from 68.4% to 69.2%.The paper notes that smooth reference utility and selected-answer accuracy can move differently under the same update.Source: §E.1 · p. 22
  • Table 4 defines peak as the evaluated depth with highest mean reference utility and decline as the earliest running-best decrease whose pointwise 95% paired interval is below zero.Non-adjacent depth pairs denote cumulative changes.Source: Table 4 · p. 23

E.2 EVALUATION SETS AND NUMERICAL PROTOCOL

The protocol documents the provenance and numerical handling of the evaluation sets, then describes quarter-step recovery and the state, output, and task-geometry diagnostics. CommonsenseQA extensions show recoveries on exploratory training subsets, while the diagnostic section specifies how signals are compared with utility outcomes.

E.2 EVALUATION SETS AND NUMERICAL PROTOCOL

  • The evaluation sets have distinct provenance: the original Huginn/GSM8K set contains 256 training questions, while CommonsenseQA extensions use 1,024 training questions per model.The Huginn/GSM8K transition was selected on those same questions; earlier nested subsets are not independent replications.Source: §E.2 · p. 23
  • All nine conditions have zero neutral updates; bootstrap intervals recompute the finite-step-failure share over harmful updates, with a varying denominator.Failure shares and counts meeting both A > 10^-4 and −∆U > 10^-4 are reported in Table 5.Source: §E.2 · p. 23, Table 5 · p. 24
  • Quarter steps recover 13 of 22, 18 of 28, and 3 of 4 finite-step failures on exploratory CommonsenseQA training subsets for Ouro-1.4B, Ouro-2.6B, and Huginn, respectively.Figure 3 presents recovery fractions and mean gains, including unrecovered failures; its purple rows mark CommonsenseQA.Source: §E.2 · p. 24, Figure 3 · p. 24
  • For the affine reference-token readout, the endpoint correction is computed independently from endpoint logits; one near-zero Ouro-2.6B case leaves the strict failure inequality numerically unresolved.The nine-condition comparison instead differentiates each actual utility, including nonlinear coda and joint-option readouts.Source: §E.2 · p. 24, Table 6 · p. 25

E.3 STATE AND OUTPUT DIAGNOSTICS

  • The diagnostic signals include state-change norms, output KL and entropy changes, and task-utility derivatives; their correlations and harmful-update AUROC use fixed risk orientations.State and output diagnostics omit task-specific derivative information while using the same teacher-forced states.Source: §E.3 · p. 25, Table 7 · p. 25

E.4 SCALE PREDICTION AND CURVATURE MEASUREMENTS

The section evaluates quadratic scale predictions, observed progress boundaries, and directional curvature. Predictions track empirical grid optima closely on the primary mathematical sets, while the nonlinear-readout boundary interpretation has a stated limitation.

E.4 SCALE PREDICTION AND CURVATURE MEASUREMENTS

  • On exploratory CommonsenseQA training subsets, scale correlations are 0.873 and 0.837 for the Ouro models and 0.928 for Huginn; 268 of 275 Huginn grid optima are at one.These estimates use exploratory training subsets.Source: §E.4 · p. 25, Table 8 · p. 26
  • 0.006697 (Ouro/MATH-500) and 9.646×10^-5 (Huginn/GSM8K) are paired quadratic-minus-first-order utility gains, with respective intervals [0.005622, 0.007794] and [5.763×10^-5, 1.427×10^-4].The comparison evaluates the quadratic action against the first-order full-step choice on the original primary evaluation sets.Source: §E.4 · p. 26
  • Scale correlations and MAEs are 0.9932 and 0.0133 on Ouro/MATH-500, and 0.9983 and 0.0036 on Huginn/GSM8K.For Huginn, 93 of 129 primary questions have both predicted and empirical grid optima at one; the 35 interior optima have correlation 0.9924 and MAE 0.0262.Source: §E.4 · p. 26
  • The quadratic root falls inside the observed crossing interval for 121 of 179 Ouro and 13 of 18 Huginn questions with an observed crossing.The remaining 136 Ouro and 111 Huginn primary questions have no observed negative gain through scale one.Source: §E.4 · p. 26
  • For nonlinear readouts, negative excursions between earlier grid points are not ruled out, so an observed crossing interval need not locate the first continuous progress boundary.In the affine reference-token case, concavity connects the observed crossing to the end of the nonnegative prefix.Source: §E.4 · p. 26
  • Negative curvature is observed at every sampled point up to the empirical optimum in all 315 Ouro and 129 Huginn primary cases, but this does not establish concavity throughout the segment.The median maximum adjacent curvature differences, normalised by |ϕ′′(0)| + ε, are 0.0951 for Ouro and 0.0294 for Huginn.Source: §E.4 · pp. 26–27

E.5 FP64 PATH ANALYSIS AND CURVATURE-BASED SCALE SELECTION

The FP64 analysis refines scale optima and evaluates a curvature-bound scale choice for Ouro; a separate GSM8K test evaluation specifies an immediate quadratic intervention. The bound-selected Ouro interventions all exceed their computed sufficient gain bounds.

E.5 FP64 PATH ANALYSIS AND CURVATURE-BASED SCALE SELECTION

  • The FP64 path analysis retains recurrent endpoints and recomputes derivatives and utility, with the primary membership and harmful-update labels unchanged from the original analysis.The comparison reports maximum FP64–FP32 differences for A, Q, and ∆U.Source: §E.5 · p. 27
  • Ouro’s refined optimum uses concavity and derivative-root brackets, while Huginn’s estimate is the smallest maximiser among evaluated candidates and may miss the true continuous maximum.Neither model’s procedure provides a floating-point certificate; Huginn’s reported regret can increase if a maximum was missed.Source: §E.5 · p. 27
  • Ouro’s derivative envelope combines global moment bounds with fourth-derivative bounds on 256 intervals; Huginn assigns no continuous derivative bounds, global optimality certificate, or bound-selected scale.Huginn’s reported third-derivative checks use automatic differentiation and central differences.Source: §E.5 · pp. 28–29
  • All 315 Ouro primary gains exceed the computed sufficient lower bound for the curvature-selected scale; among 179 finite-step failures, the median selected scale is 0.32759.The selected scales range from 0.005902 to 0.821919, and 142 of the 179 positive gains exceed 10^-4.Source: §E.5 · p. 29, Table 11 · p. 29

E.6 IMMEDIATE QUADRATIC SCALE SELECTION ON THE COMPLETE GSM8K TEST SET

  • The complete GSM8K test evaluation applies the quadratic rule and a quarter step to the same H4, H5 pair, measuring iteration-5 gains over H4 across 1,319 questions.Recovery counts use the 431 finite-step failures of the original update; the intervention states are evaluated after casting to BF16.Source: §E.6 · p. 29

E.7 QUESTION-LEVEL INTERVENTIONS ACROSS HELLASWAG TRANSITIONS

Across HellaSwag transitions, quadratic selection improves recovery counts and mean utility relative to full and quarter steps, with distinct populations for recovery and mean-gain comparisons. A token-level GSM8K audit finds recoverable failures despite positive mean full-step gains, but does not evaluate gate discrimination against early exits.

E.7 QUESTION-LEVEL INTERVENTIONS ACROSS HELLASWAG TRANSITIONS

  • The HellaSwag intervention uses one common scale across four candidate options, with derivatives accounting for coupling through the option-score log-softmax.Each transition is evaluated independently before further recurrence.Source: §E.7 · p. 29
  • Across four HellaSwag transitions, quadratic selection recovers more finite-step failures than the quarter step and yields higher mean utility gain than either the full or quarter step.At 4 →5, it recovers all 88 recorded failures, including 62 with gains above 10^-4.Source: §E.7 · p. 30, Table 13 · p. 30
  • At 1 →2, A + Q has lower sign accuracy than A although quadratic selection recovers 297 of 322 failures; at 2 →3, its mean gain over failures is 0.009511 versus 0.009937 for the quarter step.Recovery counts and mean gains use separate populations and retain distinct interpretations.Source: §E.7 · p. 30

E.8 TOKEN-LEVEL AUDIT OF EARLY RECURRENT UPDATES

  • Because the audit has no population of early exits, it measures continuing-update geometry but cannot assess gate discrimination.The stored policy selects readout depth from a dense teacher-forced trajectory.Source: §E.8 · p. 31
  • The token-level audit evaluates reference-token log probability immediately after scaling, without cumulative intervention or further recurrence; its estimates are token-weighted rather than question-weighted.Bootstrap resampling retains all tokens from each sampled question together.Source: §E.8 · p. 31
  • Across three early GSM8K transitions, 8.64–9.92% of continuing prediction positions fail at the full step despite positive mean gains; quadratic selection recovers 97.77–99.11%.It also has positive paired mean advantages over full and quarter steps across the continuing population.Source: §E.8 · p. 31, Table 14 · p. 31

E.9 CURVATURE-BOUND EVALUATION ON THE COMPLETE MATHEMATICAL BENCHMARKS

Across the complete mathematical benchmarks, integrated and signed curvature bounds are tighter than the global bound, while the signed bound covers the numerical scale-error intervals after refining five cases. In the oracle comparison, intermediate scales add little mean gain over endpoint selection in three settings.

E.9 CURVATURE-BOUND EVALUATION ON THE COMPLETE MATHEMATICAL BENCHMARKS

  • The bound evaluation uses both Ouro models on complete MATH-500 and GSM8K test sets, covering 3638 model–question records and 2034 primary records.The four conditions use separately computed CUDA BF16 endpoints and FP64 readout analysis.Source: §E.9 · pp. 31–32
  • Integrated-bound medians are 0.0583–0.1136 and signed-bound medians 0.0485–0.1042, versus 2.84–4.61 for the global bound; integrated bounds cover all 2034 primary records.The signed bound directly covers 2029 records and covers the remaining five after numerical interval refinement.Source: §E.9 · pp. 32–33, Table 15 · p. 32, Table 17 · p. 33
  • On 315 primary Ouro-1.4B/MATH-500 records, the integrated scale bound has median 0.097 and 75th percentile 0.174; its regret bound has median 1.04 × 10^-4.The regret-bound distribution is compared with numerical regret distributions, not individual records.Source: §E.9 · p. 32, Table 16 · p. 32
  • The signed-residual envelope bounds approximation error uniformly and supports scale and regret bounds under Q < 0.The five refined numerical optimum error endpoints fall below the unchanged signed bounds, with no additional model evaluation.Source: §E.9 · p. 33

E.10 ORACLE GAINS

  • On Ouro/MATH-500, intermediate scales account for 5.65% of the five-scale oracle’s gain over the full step; their mean advantage is below 0.5% on Huginn/GSM8K and Ouro/HellaSwag.For Huginn/CommonsenseQA, six questions have strict interior advantage, but none exceed 10^-4.Source: §E.10 · p. 34, Table 18 · p. 34

F LIMITATIONS

The study measures immediate reference-utility effects of scaled updates in frozen looped Transformers under teacher forcing. Its findings do not establish how recoverable losses affect later generation states or accumulate over recurrent trajectories.

F LIMITATIONS

  • Measurements concern individual recurrent updates in frozen models, using reference utility under teacher forcing to measure task progress.Each scaled proposal is evaluated before further recurrence.Source: F LIMITATIONS · p. 34
  • The interventions isolate immediate update-magnitude effects, but generation use would require evaluating subsequent states, answer quality, and computation cost.Source: F LIMITATIONS · p. 34
  • How recoverable losses accumulate over a recurrent trajectory remains to be established.Source: F LIMITATIONS · p. 34

Related papers

Loading 2609.16665v1…