PaperlayerView exact revision on arXiv

Source-linked AI summary

Loop Dropout: Regularizing Shared Updates in Looped Language Models

Zirui Zhu, Hailun Xu, Xuanlei Zhao, Yong Liu, Yingxuan Ren, Kanchan Sarkar, Kun Xu, Yang You

arXiv:2609.34218v2cs.LGcs.CL

TL;DR

Standard LoRA adapts shared updates unevenly across loop positions, with limited adaptation at early loops. Loop Dropout combines stochastic masking with inverse-survival rescaling to preserve expected update strength. It improves mathematical adaptation and transfer across settings, with stronger early-loop adaptation and gains over LoRA at deeper recurrence.

  • Problem

    Standard LoRA fine-tuning exhibits pronounced late-loop bias, leaving its shared update less effective at early positions in the recurrent computation.

    Source: §1 · p. 1

  • Method

    Loop Dropout masks complete adapter applications stochastically and rescales retained updates by inverse survival probability to preserve expected update strength across loops.

    Source: §1 · p. 2

  • Results

    Loop Dropout improves mathematical adaptation and transfer across model sizes, adapter ranks, and training recipes; at eight loops it beats LoRA by 4.75 percentage points on GSM8K.

    Source: §1 · p. 2, §5 · p. 10

  • Takeaways & Limitations

    Effective adaptation across loops is a key consideration when fine-tuning looped language models, and Loop Dropout provides a simple way to train reusable shared updates.

    Source: §5 · p. 10

    Deployment requires application-specific reliability and safety evaluation beyond the benchmarks studied.

    Source: ETHICS STATEMENT · p. 10

Abstract

from arXiv · show

Looped language models separate computational depth from parameter count by repeatedly applying the same transformer block. Adapting these models requires a shared update that remains effective as hidden states evolve throughout the recurrent computation. Our empirical analysis reveals a pronounced late-loop bias in standard low-rank adaptation (LoRA): the shared update provides limited adaptation at early loop positions. This imbalance motivates training shared updates under varying combinations of their applications. Randomly omitting adapter applications alone, however, does not improve task performance; it reduces expected update strength during training while leaving inference unchanged. We introduce Loop Dropout, which couples stochastic masking of adapter applications with inverse-survival rescaling to preserve expected update strength and promote effective adaptation across loops. Extensive experiments demonstrate improved mathematical reasoning across model sizes, adapter ranks and training recipes, with benefits extending to general instruction tuning and code generation. Loop Dropout outperforms existing LoRA variants and adapter regularizers, while further analysis shows stronger early-loop adaptation. Every backbone loop remains active, and inference applies the adapter at all loops using standard LoRA without additional trainable parameters or inference computation. Code is available at https://github.com/NUS-HPC-AI-Lab/loop-dropout .

1 INTRODUCTION

Shared adaptation in looped language models must remain effective across recurrent positions, but standard LoRA is substantially weaker at early loops. Loop Dropout trains shared updates under varying application patterns while preserving expected update strength, and experiments report gains across mathematical reasoning and instruction tuning.

1 INTRODUCTION

  • Standard LoRA shows late-loop bias: on Ouro-1.4B fine-tuned on GSM8K, its update reduces loss by 12% at the first loop versus 32% at the fourth.The backbone runs all four loops, and reductions are relative to the same frozen model.Source: §1 · p. 1
  • Randomly omitting adapter applications alone does not improve task performance because it lowers expected training update strength while inference still applies the full update.Source: §1 · p. 2
  • Loop Dropout combines stochastic masking of adapter applications with inverse-survival rescaling, preserving expected update strength while keeping every backbone loop active.Inference uses standard LoRA at every loop, with no additional trainable parameters or inference computation.Source: §1 · p. 2
  • At eight loops, Loop Dropout outperforms LoRA by 4.75 percentage points on GSM8K after both adapters were trained with four loops.The deeper-loop evaluation requires no further fine-tuning.Source: §1 · p. 2
  • Loop Dropout improves mathematical reasoning across model sizes, adapter ranks, and training recipes, and also improves general instruction tuning while outperforming existing LoRA variants and adapter regularizers.The experiments also report stronger early-loop adaptation and transfer to deeper recurrence.Source: §1 · p. 2

2 PRELIMINARIES

A looped model repeatedly applies one transformer block with shared parameters, adding computational depth without another copy of the block. LoRA adapts frozen weights with low-rank updates, and the same update factors can be shared across all loops.

2 PRELIMINARIES

  • A looped language model embeds the input as h0, repeatedly applies the shared block F with parameters W for K loops, and produces output z from the final state hK.Source: §2 · p. 2
  • Figure 2 depicts Loop Dropout gates as gt = bt/q during training, while standard LoRA and inference use gt = 1 at every loop.A zero gate removes only the adapter update; all backbone loops remain active.Source: Figure 2 · p. 3
  • The base Ouro models use recurrence depth K = 4, and the study fixes this depth for all examples rather than using adaptive exit gates.Source: §2 · p. 3
  • LoRA adapts frozen pretrained weights with low-rank updates, and each adapted matrix uses one trainable factor pair shared across all K loops.Training separate factors per loop multiplies adapter parameter count by K at fixed rank; the paper compares equal-rank and equal-parameter alternatives.Source: §2 · p. 3

3 LOOP DROPOUT

Loop Dropout combines loop-level masking with inverse-survival rescaling to train shared updates across varying application patterns while preserving expected update strength. Its objective covers sparse and dense patterns, and its local expansion gives a regularization view of the gates.

3 LOOP DROPOUT

  • Loop Dropout couples loop-level masking with inverse-survival rescaling to train the shared update across varying application patterns while preserving expected strength.Masking varies which adapter applications are active; rescaling preserves the expected update strength at each loop.Source: §3 · p. 3, §3.3 · p. 4

3.1 LOOP-LEVEL MASKING

  • Each example samples independent Bernoulli(q) masks per loop, sharing each gate across adapted modules and tokens; retained updates are scaled by 1/q.The backbone remains frozen, and inference sets every loop’s gate to 1.Source: §3.1 · pp. 3–4

3.2 INVERSE-SURVIVAL RESCALING

  • Masking alone reduces each loop’s expected training update to q∆; inverse-survival scaling restores it, with p = 1/2 applying either 2∆ or no update.The mean update remains ∆, matching the inference update.Source: §3.2 · p. 4

3.3 TRAINING ACROSS APPLICATION PATTERNS

  • The weighted objective jointly optimizes single-loop and multi-loop applications, exposing shared parameters to sparse and dense application patterns.It averages losses across subsets of active loops.Source: §3.3 · p. 4
  • A local expansion around inference gates adds nonnegative curvature regularization along each application’s logit sensitivity, with strength p/(2q), half the gate variance.H is the positive-semidefinite cross-entropy Hessian with respect to logits; R collects higher-order terms and specified second-derivative contributions.Source: §3.3 · p. 5

4 EXPERIMENTS

Experiments compare Loop Dropout with LoRA and other adaptation methods across models, tasks, adapter configurations, and recurrence depths. They report gains in mathematical reasoning, instruction tuning, early-loop adaptation, and deeper-recurrence evaluation.

4.2 OVERALL PERFORMANCE

  • On Ouro-1.4B, MATH-500 accuracy rose from 38.87% to 47.53%; GSM8K/MATH-500 gains were 1.19/8.67 points on Ouro-1.4B and 1.11/4.33 on Ouro-2.6B.Both benchmarks also improved on LoopUS-Qwen3-4B and Huginn; instruction tuning raised the six-benchmark average to 61.51 versus 60.98 for LoRA.Source: §4.2 · pp. 5–6, Table 1 · p. 6, Table 2 · p. 6

4.3 COMPARISON WITH EXISTING ADAPTATION METHODS

  • On MATH-500, Loop Dropout exceeded CoTo-on-Ouro by 9.73 points and LoRA Dropout by 5.40; it also cut LoRA Dropout training time by 71% while improving both benchmarks.All methods used 15.1M adapter parameters in the Table 3 comparison.Source: Table 3 · p. 6, §4.3 · pp. 6–7

4.4 EARLY-LOOP ADAPTATION

  • Loop Dropout narrowed the first-to-final-loop loss-reduction gap from 20.5 to 4.9 points; its single-application generation also outperformed LoRA at loops two through four.At a higher learning rate, its loss reductions remained 33.8–38.4% across positions, while LoRA ranged from 11.3% to 37.0%.Source: §4.4 · p. 7, Table 4 · p. 7

4.5 ABLATION STUDIES

  • Ablations show highest accuracy with complete-application masks plus inverse-survival rescaling; rescaling preserves the mean first-order adaptation effect, unlike unscaled masking.Loop Dropout also outperformed dose, module-wise masking, and matched noise controls.Source: Table 5 · p. 8, §4.5 · p. 8
  • On MATH-500, masking gained 11.40 points with shared adapters versus 4.80 and 4.40 with independent rank-four and rank-sixteen adapters; on Ouro-2.6B, Loop Dropout exceeded those independent adapters by 3.60 and 5.80 points.The shared and independent configurations are compared in Table 6; the larger-model comparison is reported in Table 18.Source: §4.5 · p. 8, Table 6 · p. 9
  • No tested fixed-size mask, positional profile, schedule, or learned-gate variant exceeded uniform Bernoulli masking at both learning rates.Source: §4.5 · p. 9

4.6 ROBUSTNESS ACROSS ADAPTER RANKS AND RECURRENCE DEPTHS

  • With independent tuning budgets, MATH-500 gains over LoRA were 7.07, 8.67, 2.87, and 2.33 points at ranks 4, 16, 64, and 128.Mean accuracy improved on both mathematical benchmarks at all four ranks.Source: Figure 3 · p. 9, §4.6 · p. 9
  • For four-loop-trained adapters, Loop Dropout exceeded LoRA by 2.98 points at six-loop evaluation and 4.75 at eight loops; it led in all nine training–evaluation depth pairs.At eight loops, accuracy was 75.6% for Loop Dropout versus 70.8% for LoRA.Source: §4.6 · p. 10

5 CONCLUSION

Loop Dropout trains shared updates under varying application patterns while preserving expected update strength, and experiments show improved adaptation and transfer across looped models. The conclusion identifies effective adaptation across loops as an important consideration, while the related work distinguishes this training rule from approaches that alter recurrence, adapter parameterization, or regularization structure.

5 CONCLUSION

  • Loop Dropout masks shared adapter applications during training and inverse-survival rescales them, while keeping all backbone loops active and standard LoRA inference unchanged.The method adds no trainable parameters or inference computation.Source: §5 · p. 10
  • Experiments show improved mathematical adaptation and transfer across model sizes, ranks, and training recipes, with the largest gains on competition-mathematics transfer.Further analysis finds stronger early-loop adaptation and an advantage over LoRA beyond training depth.Source: §5 · p. 10

AI USE STATEMENT

  • The authors report using ChatGPT-6 Astra and Claude Fable 5.1 for writing assistance and identifying potentially relevant work, with details manually verified.The authors take responsibility for the paper’s content.Source: AI USE STATEMENT · p. 10

ETHICS STATEMENT

  • The study uses publicly released models and existing benchmark datasets, and notes that deployment needs application-specific reliability and safety evaluation beyond those benchmarks.The adapted models inherit risks from their pretrained weights and fine-tuning data.Source: ETHICS STATEMENT · p. 10

A.1 PARAMETER SHARING AND RECURRENT COMPUTATION

  • The study takes pretrained recurrence as given and trains a shared downstream update, unlike work that introduces or changes recurrent computation.Related approaches include adaptive-depth methods and recurrent-training objectives with auxiliary losses or teachers.Source: §A.1 · p. 17

A.2 PARAMETER-EFFICIENT ADAPTATION

  • The experiments keep LoRA’s parameterization fixed and study how its shared update is trained across repeated applications.This contrasts with methods that change adaptation parameters, memory budgets, optimization, or initialization.Source: §A.2 · pp. 17–18

A.3 STOCHASTIC REGULARIZATION

  • Loop Dropout masks complete loop-level adaptation branches while retaining the pretrained transformation, so model depth stays fixed.Its masks are sampled independently across loops, shared across adapted modules and token positions, and applied only to the fine-tuning update.Source: §A.3 · p. 18

A.4 SUPERVISION AND EVALUATION

  • The evaluation separates mathematical accuracy from code correctness, truthfulness, and instruction following across the study’s supervised recipes.The experiments vary the update-training rule within existing recipes.Source: §A.4 · p. 18

B IMPLEMENTATION DETAILS

The implementation applies shared LoRA factors to Ouro’s recurrent projections and uses loop-level gates to mask adapter contributions while retaining pretrained operations. The accompanying analysis describes expected-update preservation, the exact masked-loss mixture, and local second-order interpretations and control comparisons.

B IMPLEMENTATION DETAILS

  • The default Ouro adapter uses shared rank-16 LoRA factors on seven projections, with embeddings, normalization, and the output head frozen.The setup uses α = 32 and has 15,138,816 trainable parameters for Ouro-1.4B and twice that for Ouro-2.6B.Source: B IMPLEMENTATION DETAILS · p. 19
  • A loop-level gate removes all LoRA contributions from a dropped loop while retaining its frozen operations; evaluation sets every gate to one.The gate is shared across adapted modules and token positions within the loop.Source: B IMPLEMENTATION DETAILS · p. 19

C.1 FIRST-ORDER CONSISTENCY THROUGH THE RECURRENCE

  • Inverse-survival rescaling preserves the first-order adaptation response through the recurrence, with the nonlinear remainder bounded for fixed positive survival probability.The response at each loop includes propagation through the remaining backbone loops, whose state and propagation Jacobians can vary by position.Source: §C.1 · p. 20

C.2 EXACT RELATIONS

  • With p = 0.5 and four loops, the exact expected loss averages uniformly over 16 mask patterns, including the all-zero update pattern.The expected scaled number of applications equals the four applications used at inference, while the recurrence remains nonlinear in the gate vector.Source: §C.2 · p. 20

C.3 SECOND-ORDER APPROXIMATION

  • At p = 1/2, gate perturbations have unit magnitude, making Eq. (7) a local description of the objective, while Eq. (6) gives its exact form.Keeping only the Gauss–Newton part J^T H J is the gate-space analogue of local noise-regularization analyses and the adaptive-regularization view of dropout.Source: §C.3 · p. 21

C.4 WHAT THE CONTROLS MATCH

  • At unit strength, parallel noise matches Loop Dropout’s gate mean and covariance but includes negative and unbounded gates and lacks a point mass at zero.The dose and fixed-subset controls match different covariance components, whereas independent Bernoulli masks retain both.Source: §C.4 · p. 21

D.1 MODELS, DATA AND SPLITS

The experiments use Ouro base models and specified mathematical, instruction-tuning, and evaluation recipes, with hyperparameters selected through development-set procedures. Reporting includes compute and seed-variation details, and experiments on LoopUS-Qwen3-4B and Huginn assess performance on other looped transformers.

D.1 MODELS, DATA AND SPLITS

  • The experiments use Ouro-1.4B and Ouro-2.6B base models with four loops unless a depth experiment specifies otherwise.The mathematical recipes include MetaMath-GSM-100k and a smaller direct-GSM8K study; instruction tuning uses a filtered Tülu 2 mixture.Source: §D.1 · pp. 21–22, Table 7 · p. 22

D.2 HYPERPARAMETERS

  • Hyperparameters vary by study: MetaMath comparisons select learning rates using greedy-generation accuracy on a 500-example development split, while the controlled comparison fixes the rate at 10−4.Rank, masking, model, and training-recipe settings are specified separately across the experiments.Source: §D.2 · pp. 22–23

D.3 COMPUTE

  • The H100 cost comparison measures five methods on the same 100k examples, 3,125 optimizer steps, and rank-16 adapters, reporting mean training time and peak memory across three seeds.Training-time measurements exclude development search, generation, scoring, and queue time.Source: §D.3 · p. 23, Table 8 · p. 23

E.1 PROMPTS, DECODING AND SCORING

  • MetaMath evaluation uses zero-shot greedy generation with task-specific token budgets and answer extraction, while direct GSM8K uses a distinct exact-match protocol.Instruction-tuning evaluations use their specified benchmark metrics and remain separate from mathematical and base-model evaluations.Source: §E.1 · pp. 23–24

E.2 SEEDS AND UNCERTAINTY

  • Main configurations report means and sample standard deviations over three training seeds, with two GSM8K learning rates using five seeds.Paired same-rank methods share initial adapter weights and data order where specified.Source: §E.2 · p. 24

F.1 OTHER LOOPED TRANSFORMERS

  • Loop Dropout has higher accuracy than LoRA on both reported benchmarks for LoopUS-Qwen3-4B and Huginn.For LoopUS-Qwen3-4B, gains are 1.59 percentage points on GSM8K and 6.20 points on MATH-500.Source: §F.1 · p. 24, Table 9 · p. 24

F.2 PER-SEED RESULTS

The instruction-tuning results compare LoRA and Loop Dropout on Ouro-1.4B using matched seeds and report multiple task metrics and averages. Loop Dropout improves several averages and selected metrics, while remaining close to LoRA on the other six metrics.

F.3 INSTRUCTION TUNING ON TÜLU 2

  • The instruction-tuning comparison uses Ouro-1.4B checkpoints trained on a filtered 100k-example Tülu 2 draw, with assistant-only loss and rank-16 adapters.LoRA and Loop Dropout use learning rate 10−4 and three seeds; the reported metrics cover code, knowledge, reasoning, truthfulness, and instruction following.Source: §F.3 · pp. 24–25
  • Loop Dropout raises the nine-metric average from 62.62 to 63.20, the six-metric average from 60.98 to 61.51, and the four-code-metric average from 69.88 to 71.00.It improves HumanEval+, MBPP, and TruthfulQA MC2 by 2.24, 1.15, and 0.87 points, respectively; the MBPP gain holds in every seed.Source: §F.3 · p. 25, Table 11 · p. 25

G.1 DROPOUT VARIANTS AND STRUCTURAL CONTROLS

The section compares dropout choices and structural controls, then reports robustness of Loop Dropout across adapter ranks and answer availability. It also documents supporting per-seed measurements and the scope of additional baseline comparisons.

G.1 DROPOUT VARIANTS AND STRUCTURAL CONTROLS

  • 81.7% is the highest mean at p = 0.5 in the three-seed probability study, and rescaled dropout beats unscaled dropout at both tested learning rates.At 10−4, accuracies are 80.6% for LoRA, 77.0% for unscaled dropout with p = 0.5, and 81.7% with rescaling.Source: §G.1 · p. 25
  • Uniform masking is the default: no tested mask variant exceeds it at both learning rates, and learned per-loop gates add parameters while falling below it at 10−4.The five-seed follow-up favors uniform masking over exactly sized subsets and schedules that anneal dropout to zero at the higher rate.Source: §G.1 · p. 26, Table 12 · p. 26

H.1 RANK DEPENDENCE AT A COMMON LEARNING RATE

  • Loop Dropout’s MATH-500 and mean GSM8K differences are positive at every tested rank under the common learning-rate comparison.The nine rank-by-seed MATH-500 comparisons are positive; rank-16 results are reported at the same learning rate in Table 5.Source: §H.1 · p. 26, Table 13 · p. 26

H.2 ANSWER AVAILABILITY AND THE MATH-500 GAIN

  • 6.80, 4.80, and 6.00 points of the total gains across three ranks come from problems where both methods produce extractable answers.The advantage persists on problems for which both methods satisfy the answer-extraction requirement.Source: §H.2 · p. 27, Table 14 · p. 27

H.3 PER-SEED MEASUREMENTS

  • Table 15 reports per-seed results for the rank sweep and noise controls, including a second configuration of each control trained at 2 × 10−4.The additional suites use three-seed summaries with seeds ordered 101 / 102 / 103.Source: §H.3 · p. 27, Table 15 · p. 27

I ADDITIONAL BASELINE COMPARISONS

I.1 BASELINE CONFIGURATIONS AND PER-SEED RESULTS

This appendix specifies configurations for adapter-regularizer and adapter-sharing comparisons, with accompanying per-seed result tables. It also identifies the scope of additional robustness results.

I.1 BASELINE CONFIGURATIONS AND PER-SEED RESULTS

  • CoTo-on-Ouro follows the official physical-layer schedule, sharing each layer’s switch across recurrent uses and increasing keep probability from 0.1 to one over the first 75% of optimizer steps.The final checkpoint is evaluated with all adapters active; runs use 100k examples, 3,125 updates, and rank 16.Source: §I.1 · p. 27, Table 16 · p. 28
  • LoRA+ uses an A-factor rate of 5×10−5 on Ouro-1.4B, with a fourfold multiplier for B.The A-factor rate is selected from the halved five-candidate grid.Source: §I.1 · p. 27
  • LoRA Dropout uses probability 0.5 and four training masks, with deterministic inference using the expected masked update and no test-time ensemble.The three-seed Ouro-1.4B comparison uses learning rate 10−4; Table 17 reports per-seed results.Source: §I.1 · p. 28, Table 17 · p. 28

I.2 ADAPTER SHARING ON OURO-2.6B

  • Table 18 compares shared and independent adapters on Ouro-2.6B; independent rank-four adapters match the shared rank-16 adapter’s 30.3M parameters, while rank-sixteen adapters use 121.1M.The comparison uses development selection on seed 101 and three training seeds at the selected rate.Source: §I.2 · p. 28, Table 18 · p. 28, Table 19 · p. 29

J ADDITIONAL ROBUSTNESS RESULTS

  • The additional robustness section supplies full numerical results for Section 4.6, using MetaMath-GSM for rank studies and direct GSM8K for depth and shifted-test studies.Source: J ADDITIONAL ROBUSTNESS RESULTS · p. 29

J.1 ADAPTER RANK WITH INDEPENDENT TUNING

The rank study finds positive benchmark differences under independent tuning, and the appendix examines where transfer gains occur. It also reports depth generalization, learning-rate robustness, and transfer to shifted mathematical tests.

J.1 ADAPTER RANK WITH INDEPENDENT TUNING

  • Loop Dropout has positive mean differences on both benchmarks at every independently tuned adapter rank.Both methods use five learning-rate candidates per rank, with each configuration fixed before test evaluation; Table 20 reports three-seed means and standard deviations.Source: §J.1 · p. 29, Table 20 · p. 29, Table 21 · p. 29
  • The MATH-500 rank-16 gain spans all seven subjects and all five difficulty levels, with the largest level-wise difference at level 3.Most of the gain occurs on problems where both methods produce extractable answers; 7.87 of the 8.67 total points come from those problems.Source: §J.1 · pp. 29–30, Table 22 · p. 30

J.2 TRAINING AND EVALUATION DEPTH

  • Loop Dropout has higher mean accuracy than LoRA in all nine training-by-evaluation depth pairs, including evaluation beyond the training depth.Saved adapters are evaluated at K ∈ {4, 6, 8} without further fine-tuning; Figure 5 shows curves for adapters trained at four loops.Source: §J.2 · p. 30, Figure 5 · p. 30, Table 23 · p. 31

J.3 LEARNING RATES AND SHIFTED MATHEMATICAL TESTS

  • Loop Dropout improves mean accuracy at every tested learning rate and across matched recurrence depths from two to eight loops.Source: §J.3 · p. 31, Table 25 · p. 32, Table 27 · p. 32
  • GSM-Plus mini gains are 1.14 and 3.40 points at the two reported learning rates, and saved adapters retain positive margins on GSM8K-Platinum and GSM-Plus.The shifted tests use relabelled GSM8K-Platinum and perturbed GSM-Plus problem statements.Source: §J.3 · p. 31, Table 26 · p. 32

K.1 SINGLE-LOOP ACTIVATION

Single-loop diagnostics compare the loss reduction from activating one trained adapter application at a time while all four backbone loops run. After MetaMath fine-tuning, Loop Dropout also supports generation from individual applications and lowers readout loss throughout the recurrence.

K.1 SINGLE-LOOP ACTIVATION

  • At 10−4, single-loop loss reductions are 11.9/19.1/26.5/32.4% for LoRA and 30.8/35.2/35.9/35.7% for Loop Dropout.At the second learning rate, Loop Dropout again strengthens early applications and narrows the disparity between recurrent positions; all reductions are relative to the frozen model.Source: §K.1 · p. 33, Table 28 · p. 33
  • The single-loop loss diagnostic and all-on generation comparison use different update scales: Loop Dropout trains active applications at strength 1/q = 2, while the readout uses unit gates.At all-on inference, four unit-gate applications equal Loop Dropout’s expected total training update and twice the unscaled rule’s.Source: §K.1 · p. 33

K.2 GENERATION AND READOUTS AFTER METAMATH FINE-TUNING

  • Loop Dropout supports generation from individual applications at loops two through four and lowers MATH-500 loss at every readout, most strongly at the first.At the first readout, cross entropy decreases from 1.240 to 1.047; the generation and readout plots summarize three training seeds.Source: §K.2 · p. 33, Figure 6 · p. 34, Table 29 · p. 34

L LIMITATIONS

The experiments cover selected Ouro model sizes, adapter ranks, and tuning settings, with additional evaluations on LoopUS-Qwen3-4B and Huginn reported in Appendix F.1. The curvature analysis is local, while the finite-mask mixture gives the exact form; masking controls compare complete distributions.

L LIMITATIONS

  • Experiments cover mathematical fine-tuning of Ouro at two model sizes and four adapter ranks, plus rank-16 instruction tuning of Ouro-1.4B.These experiments use the learning rates and seeding protocols in Appendix D.2.Source: L LIMITATIONS · p. 34
  • Additional mathematical evaluations on LoopUS-Qwen3-4B and Huginn are reported in Appendix F.1.Source: L LIMITATIONS · p. 34
  • Curvature analysis describes the objective locally; the finite-mask mixture gives its exact form, and controls compare complete masking distributions, including gate support and higher moments.Source: L LIMITATIONS · p. 34

Related papers

Loading 2609.34218v2…