PaperlayerView exact revision on arXiv

Source-linked AI summary

One Model, Two Paths: Balancing Single-Pass and Looped Performance in Language Models

Jiarun Shen, Lei Zhao, Xiao Li

arXiv:2610.33517v1cs.CL

TL;DR

Because looped and single-pass paths share weights, improving looped predictions can change single-pass quality, making it important to assess both against continued training. The paper compares joint label losses and teacher guidance, finding that validation-selected Frozen-original KD preserves single-pass quality within a predefined tolerance while improving looped predictions in the main setting.

  • Problem

    Looped-path training can alter single-pass quality because both paths share trainable weights, and adaptation must be judged against the alternative of ordinary continued training.

    Source: §1 · p. 1

  • Method

    The study compares weighted next-token losses on both paths, with and without teacher guidance, including guidance from a frozen copy of the original model.

    Source: §1 · pp. 1–2

  • Results

    With validation-selected weights, Frozen-original KD keeps single-pass NLL within the predefined margin of ordinary continued training and improves looped NLL over that reference in the main setting.

    Source: §7 · p. 8

  • Takeaways & Limitations

    The results support comparing label strength and teacher source jointly, with the conclusion limited to the evaluated text-retention setting.

    Source: §7 · p. 8

    The main experiments use two fixed 133.8M-parameter pretrained models on English text and a finite set of weights, teacher configurations, and training budgets.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Repeating Transformer layers can improve language-model performance without adding parameters. A pretrained model can be adapted to support both single-pass and looped inference by reusing middle-layer weights for additional computation. Because both paths share weights, training to improve looped predictions can degrade single-pass quality. We study joint training strategies using weighted losses on both outputs, with guidance from the frozen original model. Joint supervision substantially reduces single-pass degradation compared to looped-only training, while preserving looped gains over ordinary continued training. Most of this benefit comes from adjusting the single-pass loss weight, with guidance from the frozen original model further improving the looped output. With validation-selected weights, our method preserves single-pass quality within a predefined tolerance of ordinary continued training while improving looped predictions.

1 Introduction

Shared-weight looping creates a training trade-off: improving looped predictions can alter single-pass quality, so adaptation must preserve both paths. Joint supervision and a frozen original-model teacher are evaluated as ways to manage that trade-off.

1 Introduction

  • The model retains a single-pass path and a looped path that reuses shared weights, allowing different computation paths to be used as available computation changes.Because all pretrained layers and the original output head remain trainable, updates that improve looping also change single-pass weights.Source: §1 · p. 1
  • Success requires comparing both outputs with ordinary continued training, not only checking whether adaptation preserves the starting checkpoint's quality.The same training budget could improve the original model further without looping.Source: §1 · p. 1
  • A small single-pass loss weight substantially reduces the single-pass degradation caused by looped-only training.In staged recovery, supervising both outputs preserves more looped quality than returning to single-pass training alone, but recovers less single-pass quality.Source: §1 · p. 2
  • Increasing the single-pass loss weight reproduces most of the observed benefit of adding a teacher signal.After validation-based weight selection, the teacher adds no further mean improvement over the selected label-only baseline at matched training time.Source: §1 · p. 2
  • With validation-selected weights, Frozen-original KD keeps single-pass quality within the predefined tolerance of ordinary continued training while improving looped predictions in the main setting.The joint result does not extend to the secondary WikiText corpus.Source: §1 · p. 2

2 Related Work

Prior work examines pretrained-model conversion to recurrent computation, supervision across computation depths, and predictor-based distillation. This paper distinguishes its setting by retaining the complete single-pass path alongside a shared-weight looped path.

Recurrent conversion

Supervision across depths

  • Other work supervises multiple computation depths through intermediate labels and distillation [Bae et al., 2025], trajectory conditioning [Jeddi et al., 2026], or on-policy cross-loop teachers [Wang et al., 2026].Here, both the complete single-pass path and looped path share all weights, with fixed-text next-token supervision and forward KL.Source: §2 · p. 2

Distillation and retention controls

  • This study compares single-pass label strength and teacher source for shared-weight paths, evaluating both outputs against ordinary continued training with teacher computation included.Its coefficient-matched label control tests how much teacher-signal improvement stronger label supervision can reproduce, but does not identify the residual difference with a particular mechanism.Source: §2 · p. 2

3 Training Strategies

The training setup defines single-pass and looped predictions from shared trainable weights, then compares weighted label supervision and teacher-guided objectives. Frozen-original KD uses the original predictor as a fixed guide while retaining label losses on both outputs.

3.1 Two Predictions from Shared Weights

  • The single-pass path applies all eight blocks once, while the looped path applies twelve blocks; both share all trainable parameters and branch after six shared block applications.The single-pass output at starting weights is exactly the original model.Source: §3.1 · pp. 2–3

3.2 Labels, Distillation, and a Matched Control

  • Joint CE and Online KD combine single-pass and looped cross-entropy losses, while Online KD additionally uses the current looped output as a detached teacher for the single-pass output.The looped label loss remains differentiable; Matched CE is a coefficient-matched label-only control, and the tested control is one label-weight setting rather than an optimized frontier.Source: §3.2 · p. 3
  • Training can begin from the ordinary parent or after 2,048 loop-only updates; the staged ordinary control follows the same switch state with ordinary-only updates.The methods and update counts are reported in Table 1.Source: §3.2 · p. 3

3.3 Our Method: Frozen-original KD

  • Frozen-original KD guides the current single-pass output with a frozen copy of the original single-pass predictor while retaining next-token label losses on both current outputs.The teacher uses the original single-pass prediction, is computed independently, and is not used at inference; validation selects α = 0.3 in the label-weight study.Source: §3.3 · p. 3
  • Additional controls vary teacher-target content and guidance timing, while label-weight selection tests m ∈ {0.2, 0.4, 0.8, 1.6}.The nominal coefficient m applies to the single-pass distribution in its logit gradient and does not ensure equal full gradients, clipping, or parameter updates.Source: §3.3 · p. 3

4 Experimental Setup

The experiments adapt two fixed pretrained models using looped and joint training schedules, then compare their outputs with ordinary continued training under compute budgets. Evaluation uses separate single-pass and looped NLL criteria, with predeclared retention margins and uncertainty intervals.

4.1 Models and Data

  • Six paired adaptation units reuse two fixed parent models, with methods matched within each unit on weights, optimizer state, random state, and data position.The training pool is English FineWeb-Edu; the supervision study uses a confirmation set, while later studies use new validation and confirmation documents.Source: §4.1 · pp. 3–4

4.2 Training Methods and Schedules

  • Loop→ schedules apply their named objective after 2,048 loop-only updates; validation selects label weights before evaluation on held-out continuation orders.Selection requires both single-pass gaps to meet their retention margins on each development order, then chooses the admissible weight with the lowest mean looped validation NLL.Source: §4.2 · p. 4

4.3 Training and Evaluation Budgets

  • Methods are compared under a 40,960-block cap and a 2,227.587-second compute cap, with teacher computation included in the time budget.Schedules can have different update counts and training exposure at the block cap; the time cap includes synchronized forward, loss, backward, clipping, and optimizer work.Source: §4.3 · p. 4

4.4 Retention and Uncertainty

  • Each path is evaluated separately using average next-token NLL in nats per target, and the criteria distinguish retention from useful looped computation.The fixed requirements include a 0.005 single-pass margin and require the looped output to beat both ordinary continuation and the adapted single-pass output.Source: §4.4 · pp. 4–5
  • The predeclared NLL margins are reporting scales, not deployment guarantees, and do not establish downstream capability preservation.Interval analyses separate text uncertainty conditional on fitted models from data-order uncertainty; selected-recipe means use four held-out continuation orders.Source: §4.4 · p. 5

5 Results

Joint supervision reduces the single-pass degradation seen with loop-only training, and validation-selected Frozen-original KD meets the predefined retention criteria while improving looped predictions in the main setting. Results also show that label weighting accounts for much of the observed benefit, while some comparisons depend on resource budget and continuation order.

5.1 Joint Supervision Retains More Single-Pass Quality

  • Loop-only training improves looped NLL by 0.009048 but worsens single-pass NLL by 0.171239; 5% single-pass label weight narrows the gap to 0.035882 while retaining a 0.006052 looped improvement.Both training approaches improved looped prediction over ordinary continuation in all six paired units.Source: §5.1 · p. 6

5.2 Stronger Labels Reproduce Most of the KD Increment

  • Online KD lowers single-pass NLL versus Joint CE by 0.010496 from the start and 0.012754 after loop-only training; Matched CE reproduces most of those increments.KD retains smaller benefits over Matched CE, but staged KD improves the looped output in five of six continuation orders, so a favorable mean need not hold in every order.Source: §5.2 · p. 6, Table 1 · p. 6

5.3 The Residual Depends on the Resource Budget

  • At the time cap, KD’s residual single-pass differences versus Matched CE are unresolved across continuation orders, and time-matched static KD has higher single-pass NLL on WikiText.The single-pass conditional order intervals span zero, as do the looped text intervals; the comparisons do not establish equivalence.Source: §5.3 · p. 7

5.4 A Frozen Original Teacher Primarily Improves the Looped Output

  • At fixed coefficients, Frozen-original KD lowers looped NLL by 0.009884 versus Matched CE at matched training time, while its single-pass difference is +0.000537.The frozen initial looped teacher harms the single pass while improving the looped output; the fixed Frozen-original KD recipe has a 0.012772 single-pass gap from validation-selected ordinary continuation.Source: §5.4 · p. 7

5.5 Label-Weight Selection and Joint Retention

  • All four criteria are supported for validation-selected Frozen-original KD on the four-order mean, with single-pass retention within the predefined margin and useful looped predictions.The single-pass interval supports retention, not superiority; the frozen-teacher recipe retains useful looped predictions unlike selected label-only CE.Source: §5.5 · p. 7, Table 2 · p. 8
  • Selected Frozen-original KD improves looped NLL by 0.0119 relative to selected CE, while selected Online KD shows no additional mean improvement over selected CE.The comparisons concern the tested weight grid, not a globally optimal label-only objective.Source: §5.5 · p. 7, Table 2 · p. 8

5.6 Receiving Path and Additional Configurations

  • With the same frozen teacher and total KL weight, guidance to any tested receiving path improves mean looped NLL over its label control, but these controls do not identify the best path.At fixed coefficients, the frozen teacher also improves looped NLL in deeper 8→16 and native Parcae configurations, but neither extension meets all joint criteria.Source: §5.6 · p. 7

5.7 Task Accuracy and Retained Correct Predictions

  • For selected recipes, Frozen-original KD reduces lost correct predictions on ARC-Easy examples by 3.53 percentage points for single-pass and 2.83 for looped output, but HellaSwag accuracy remains lower on average than ordinary continuation.The ARC-Easy reductions have paired item intervals supporting them, while conditional training-order intervals span zero; results do not show stable gains across tasks or orders.Source: §5.7 · p. 8

6 Analysis

The selected recipe meets the primary text criteria, but both outputs have higher NLL than ordinary continuation on WikiText. The analysis also cautions that retention depends on its reference and that inference-cost trade-offs remain unresolved.

6 Analysis

  • The selected recipe balances both outputs at an intermediate single-pass label weight, while increasing that weight can remove the looped advantage.Receiving-path controls show the looped benefit is not unique to guiding the single pass, but do not rank paths for the selected recipe because their controls and weights differ.Source: §6 · p. 8
  • Both outputs have higher NLL than ordinary continuation on WikiText, despite the selected recipe meeting the primary text criteria.This result does not isolate repeated computation as the cause or establish a noise-amplification mechanism.Source: §6 · p. 8
  • The looped-output gain over selected label-only training is measured at matched inference depth, but whether it justifies extra inference cost remains unresolved.Training-time comparisons do not establish the deployment value of using the looped rather than single-pass output.Source: §6 · p. 8
  • Retention within 0.005 is distinct from equal NLL or superiority, and comparisons across different references, texts, recipes, or averaging units are not controlled improvements.Ordinary continuation measures adaptation’s opportunity cost; updates can also change which examples are correct even when accuracy improves.Source: §6 · p. 8

7 Conclusion

The conclusion recommends evaluating both prediction paths against ordinary continued training when adapting pretrained models for repeated computation. In the main setting, validation-selected Frozen-original KD preserves single-pass NLL within a predefined margin and improves looped NLL over that reference.

7 Conclusion

  • With validation-selected weights, Frozen-original KD retains single-pass NLL within the predefined margin of ordinary continued training and improves looped NLL over that reference in the main setting.Its looped output also improves over selected label-only training at matched training time.Source: §7 · p. 8
  • Stronger single-pass labels reproduce most of the improvement from adding a teacher signal in the initial comparisons.Source: §7 · p. 8
  • The results support comparing label strength and teacher source jointly, while the conclusion is limited to the evaluated text-retention setting.Source: §7 · p. 8
  • Adapting a pretrained model for repeated computation requires evaluating both its original and looped predictions.Source: §7 · p. 8

Limitations

The study’s evidence is bounded by its model, language, and evaluation settings. The authors identify larger scales, other languages, and generation-time budgets as directions for further study.

Limitations

  • The main experiments study two fixed 133.8M-parameter pretrained models on English text.Source: Limitations · p. 9
  • Comparisons cover a finite set of label weights and teacher configurations under specified training budgets.Uncertainty is reported separately for evaluation data and continuation orders.Source: Limitations · p. 9
  • Larger model scales, other languages, and generation-time budgets remain directions for further study.Source: Limitations · p. 9

Ethical Considerations

The study uses existing English corpora and public benchmarks without new human participants or annotation. The authors note that likelihood evaluation and exact deduplication do not remove several risks associated with web text.

Ethical Considerations

  • The study uses existing English corpora and public benchmarks, with no new human participants or annotation.Source: Ethical Considerations · p. 9
  • Web text may contain biased, harmful, copyrighted, or personally identifying material, and likelihood evaluation plus exact deduplication do not remove those risks.Source: Ethical Considerations · p. 9
  • The authors conducted no separate personal-information or harmful-content audit, and did not measure energy use or emissions.Appendix A.1 describes data and software terms; Appendix F reports available compute records.Source: Ethical Considerations · p. 9

AI Statement

The authors disclose language and formatting assistance while retaining responsibility for the scientific content; the appendices document the study’s models, data, software, training, and additional configurations.

AI Statement

  • Generative AI assisted with language polishing, grammar checking, and manuscript formatting; the authors retain responsibility for the scientific content and final manuscript.Source: AI Statement · p. 9

A Reproducibility Details

  • Table 3 summarizes the main-study configuration; learning rates and optimizer states are inherited and retained across stage switches.It also points to method-specific continuation counts in Table 1 and GPU-hour reporting in Appendix F.Source: Table 3 · p. 12

A.1 Models, Data, and Licenses

  • The appendices specify model and corpus details, including tokenizer and decoder settings, FineWeb-Edu processing, and the distinction between WikiText packed-token NLL and word-level perplexity.Source: §A.1 · p. 12

A.2 Software and Training Details

  • Software versions, task-split counts, and parent-training schedules are documented alongside the software and training details.The listed setup includes zero-shot official templates and continuation through 32,768 updates.Source: §A.2 · pp. 12–13

A.3 Additional Configurations

  • Additional configurations include a deeper shared-middle-layer setup and a separate native Parcae architecture, rather than a tokenizer-only comparison.The native setup also specifies its initialization, adaptation seeds, and keyed noise shared across paired paths.Source: §A.3 · p. 13

B Direct Effects and Training-Order Variation

The section reports direct method comparisons and training-order variation, then describes secondary-corpus, validation-selection, and task-evaluation evidence. It keeps text uncertainty distinct from variation across adaptation orders and parents.

B Direct Effects and Training-Order Variation

  • Table 5 covers twelve confirmation-text effects across output-guidance, matched-control, recovery, and loop-utility comparisons, with adjusted intervals conditional on fitted checkpoints.The comparisons use static-from-parent or staged joint-suffix schedules, and the correction is applied within each comparison family.Source: Table 5 · p. 14, B Direct Effects and Training-Order… · p. 14
  • Training-order intervals summarize three paired differences per parent and are conditional on C/D, excluding text uncertainty and pretraining-seed variability.The interval uses a Welch–Satterthwaite approximation; three repetitions per parent provide limited precision.Source: Table 6 · p. 14, B Direct Effects and Training-Order… · p. 14

B.1 Secondary Corpus and Time-Budget Repetitions

  • At the forward-block cap, the staged looped mean favors KD over matched CE by 0.000753, but the D2 paired difference is positive.The reported text and data-order intervals are separate, and the conditional mean interval does not imply every adaptation order improves.Source: §B.1 · p. 16
  • At the time cap, looped KD-minus-matched-CE intervals span both signs, and the reported bounds are not a prespecified equivalence test.Historical guidance experiments do not add repetitions to the new means, and their evaluation sets remain separate evidence.Source: §B.1 · p. 16

C Selection and Corpus-Dependent Retention

  • All 120 validation selections are infeasible, and fixed and diagnostic confirmation results are 0/60 under both budgets.The confirmation groups reuse trajectories and may reuse checkpoints; their four intervals are pointwise, not a simultaneous confidence region.Source: C Selection and Corpus-Dependent… · p. 17

D Task Results and Correctness Transitions

  • The task-results table reports descriptive six-unit means for ARC-Easy, PIQA, and HellaSwag, with forward-block and training-time values.The table distinguishes accuracy from normalized accuracy and provides no pooled confidence intervals.Source: Table 13 · p. 18
  • The excess-loss table compares each candidate with ordinary continuation on the parent’s correct examples, using the same example set across methods and paths.Its values are descriptive six-unit means, expressed in percentage points.Source: Table 14 · p. 18

D.1 Evaluation and Per-Task Comparisons

This section evaluates task-level outcomes, the selected recipe’s confirmation-text criteria and finite-grid tradeoffs, and receiving-path and configuration extensions. It also considers diagnostics, validation procedures, and resource comparisons, with findings limited to the tested recipes, orders, and evaluation views.

D.1 Evaluation and Per-Task Comparisons

  • Staged KD’s single pass improves ARC-Easy accuracy over ordinary continuation by 1.333 pp, but changes PIQA and HellaSwag accuracy by −0.091 and −0.037 pp.Its looped output changes accuracy by +1.340, −0.299, and −0.053 pp on ARC-Easy, PIQA, and HellaSwag, respectively; the ARC-Easy mean gain is not uniform across tasks.Source: §D.1 · p. 18

E.3 Agreement and Prediction Quality

  • Static and staged forward KL are lower for KD than CE, but close KD and matched-CE diagnostic means do not establish equivalence.Static forward KL is 0.026262 for KD and 0.026378 for matched CE; KD fixes 1.452% and breaks 1.266% of tokens, versus 1.453% and 1.270% for matched CE. KD single-pass NLL is 3.636666 versus 3.625906 for ordinary continuation on this subset.Source: Table 15 · p. 20, §E.3 · p. 20

E.1 What the Label Control Matches

  • Preview updates have similar Muon group 0 parameter-update norms across CE, KD, and matched CE, but this diagnostic does not establish equal directions or trajectories.The norms are 0.871143, 0.871155, and 0.871159, respectively. Post-step gradient buffers are modified by Muon, so those buffers do not represent gradients immediately after clipping; component angles are not causal evidence.Source: §E.1 · p. 20, §E.2 · p. 20

F Computational Resources

  • The common weighted-CE timing control shows almost equal update times, whereas historical separate entries show an approximately 5.2% gap that is not attributed to one operation.The timing benchmark uses paired card/round measurements; these results do not replace historical update rates or change saved time-budget endpoints or quality results.Source: F Computational Resources · p. 22, Table 18 · p. 22

G.2 Data, Repetitions, and Intervals

  • Early-only guidance uses KD for 512 updates then matched CE for 2,413; delayed guidance reverses objectives, and eligible weights minimize mean looped validation NLL.A weight is admissible when both C1 and D1 satisfy Dstart ≤ 0.005 and x ≤ τ on validation at the fixed-time endpoint.Source: §G.1 · p. 23, §H.1 · p. 29

G.4 Teacher Quality and Retention References

  • The frozen initial looped teacher exceeds its single-pass student at every scheduled state, while the online looped teacher is above it initially and below it at updates 128, 256, and 512.At equal updates, the unpermuted frozen looped target lowers single-pass NLL by 0.463495 versus the permuted target, but has 0.028096 higher NLL than matched CE; the permutation diagnostic shows sensitivity to probability-to-token correspondence, not the source of the online residual. Prediction-permutation diagnostics have precedent in Furlanello et al. (2018).Source: §G.4 · p. 25

G.5 Task Comparisons

  • At the time cap, online KD versus validation-selected ordinary continuation changes ARC-Easy single-pass accuracy by +0.877 pp and excess loss E by −2.477 pp; PIQA and HellaSwag accuracy change by −0.462 and −0.003 pp.The ARC-Easy data-order 95% intervals are [−0.892, +2.646] for accuracy and [−4.472, −0.482] for E; PIQA and HellaSwag accuracy intervals are [−1.479, +0.554] and [−0.432, +0.425]. Task accuracy and retention remain distinct measures.Source: §G.5 · p. 25

G.6 Supplementary Threshold Sensitivity

  • Frozen-original-teacher single-pass intervals fit within ±0.002 but not ±0.001, while the mean gap to validation-selected ordinary continuation remains 0.012772.This supplementary reclassification uses archived intervals without retraining, retuning, or reselection; the margins describe NLL scales, not deployment requirements.Source: §G.6 · p. 26

H.1 Selection Rule and Complete Finite Grid

  • Across the tested grid, m = 0.4 met all six descriptive endpoint criteria; m = 0.2 retained a looped advantage with a larger single-pass gap, while higher weights weakened or reversed looped benefit.This post hoc analysis compares twelve trained points without interpolation or further selection; the measured-point relationships do not establish global optimality or dominance.Source: §H.1 · pp. 29–30

H.2 Uncertainty for the Selected Recipes

  • All four adjusted text upper bounds for Frozen-original KD meet their thresholds; x’s interval crosses zero, supporting retention within the single-pass margin rather than a strict improvement.The criteria use thresholds of 0.005 for Dstart and x, and strictly zero for y and d; separate order intervals are not jointly adjusted by the text-family procedure.Source: Table 30 · p. 31, §H.2 · p. 31
  • Frozen-original KD improved looped NLL over selected CE in all four orders, while its single-pass difference was positive only in D3; Online KD’s mean differences were positive on both outputs, but order intervals spanned zero.These observations do not establish the same ordering for future continuation orders.Source: §H.2 · p. 31

H.3 Individual Orders, Data, and Budget Identity

  • Confirmation text excluded recorded training and earlier evaluation documents; comparisons used each parent and order’s fixed-time ordinary reference under a cap including teacher computation, losses, backpropagation, updates, clipping, and synchronization.The old 0.012772 single-pass gap used a different recipe and scope. Existing aggregate intervals do not establish the four-held-out-order comparison against a stronger validation-selected ordinary reference.Source: §H.3 · p. 32

H.4 Receiving Path and Configuration Extensions

  • Loop-only guidance had lower looped NLL than guidance to both outputs in the adjusted text comparison, but its conditional order interval spanned zero; guidance need not target the single pass to benefit it.These interventions retained the original label coefficients and do not identify the best receiving path for the validation-selected recipe.Source: §H.4 · p. 33, Table 36 · p. 34
  • In the deeper configuration, Frozen-original KD had differences of +0.003189 and −0.010057; in native Parcae, the differences were −0.002250 and −0.002413, and neither extension met all four criteria.These extensions transferred the original fixed recipe rather than selecting m = 0.4; architecture, tokenizer, and training recipe change together in the native setting.Source: §H.4 · pp. 33–34, Table 35 · p. 33
  • On WikiText, the selected main recipe had d = −0.008728, while x and y remained positive versus ordinary continuation; its one-block scoring view cannot provide document-level bootstrap uncertainty.Selected CE and Online KD had looped gaps y = 0.028427 and 0.027315, respectively.Source: §H.4 · p. 33

H.5 Task Accuracy and Retention

The selected recipes do not rank uniformly across tasks: Frozen-original KD has higher mean ARC-Easy accuracy than ordinary continuation, but lower HellaSwag means. The favorable ARC-Easy retention direction is not a general task-retention result.

H.5 Task Accuracy and Retention

  • The three selected recipes have no uniform task ranking; Frozen-original KD has higher mean ARC-Easy accuracy but lower HellaSwag means than ordinary continuation.Task means are descriptive, with no pooled task interval.Source: Table 39 · p. 35, §H.5 · p. 35
  • −3.5337 pp ARC-Easy single-pass and −2.8279 pp looped excess loss were observed on the original-correct set for frozen m = 0.4.Both item intervals exclude zero, while both conditional order intervals span zero; PIQA and HellaSwag do not show a general task-retention result.Source: Table 40 · p. 35, §H.5 · p. 35
  • Original-correct-set excess loss subtracts ordinary continuation’s loss, so negative values indicate less loss than ordinary—not an accuracy gain or zero forgetting.Item intervals are paired and pointwise; order intervals are separate, and neither gives a cross-task simultaneous guarantee.Source: Table 40 · p. 35, §H.5 · p. 35

Related papers

Loading 2610.33517v1…