Why Constrained Updates Still Forget: A Functional-Path Analysis of Continual SFT

1 INTRODUCTION

• The study addresses forgetting during SFT and parameter-efficient adaptation, where methods protect different quantities and their relationship to retained behavior is unclear.Existing approaches restrict parameter changes, constrain gradients, or revisit old examples.Source: §1 · p. 1
• A functional-path account decomposes realized old-loss changes into a cached constraint term, probe mismatch, gradient staleness, and finite-step effects.The decomposition distinguishes what projection controls from changes accumulated during training.Source: §1 · p. 1
• Pre-Adam gradient projection preserved more old-behavior responses than post-Adam update projection in code-related experiments.This held regardless of the response evaluation metric.Source: §1 · p. 2
• Checkpointed generation distinguishes methods with similar final accuracy but different behavioral paths, while delayed replay raises old-task loss despite comparable final accuracy.Informative replay can counteract old-task loss increases; supervised replay preserved old responses in code-label adaptation.Source: §1 · p. 2
• In sequential tasks, subspace methods improved average retention but prioritized different early tasks, whereas supervised replay improved retention across all early tasks.The comparison motivates task-wise rather than aggregate-only evaluation.Source: §1 · p. 2

2 PROBLEM SETTING: WHAT FORGETS DURING SFT?

• The setup evaluates Old performance before and after New-task SFT using the same Old prompts and response interface, with replay drawn from a separate training split.The Old replay examples are separate from the evaluation prompts.Source: §2 · p. 2
• Positive Fm measures lost Old availability, while positive Bm measures mitigation separately from the absolute retention budget and New-task score.In the matched Mistral Code study, Legacy accuracy fell from 100% to 5.43% under plain SFT versus 97.83% with replay.Source: §2 · p. 2
• For sequentially learned tasks, backward transfer compares final scores with scores just after each task was learned; higher values are better.This task-wise view identifies which newly learned tasks subsequently regress.Source: §2 · p. 2
• A same-probe quadratic can satisfy a zero cached-gradient dot product while old loss increases, illustrating that local feasibility alone does not ensure functional retention.The figure also separates loss paths from generated-behavior audits.Source: Figure 1 · p. 3

3.1 THE RETAINED OBJECT AND THE REALIZED UPDATE

• The retained object is old behavior assessed on previously verified prompts through the same frozen interface, while the realized update includes optimizer state, clipping, weight decay, and post-step projection.In QLoRA, θ denotes adapter factors rather than the full effective weights.Source: §3.1 · p. 3

3.2 A CACHED HALFSPACE CONTROLS ONE TERM

• The functional-path decomposition separates the cached constraint term from probe mismatch, gradient staleness, and finite-step remainder; a conditional refresh-length budget bounds pathwise loss under stated assumptions.A same-probe construction shows geometric loss growth can attain the bound despite zero cached dot products; it tests the halfspace property, not experimental AdamW dynamics.Source: §3.2 · p. 4

3.3 WHY INFORMATIVE REPLAY CAN SUPPLY A RESTORING DIRECTION

• Informative replay can supply a restoring direction under a sufficient conditional descent condition, but increasing replay also changes the curvature penalty and new-task trade-off.Replay changes AdamW moments and future states, so the proposition motivates testing old-loss paths rather than guaranteeing AdamW improvement.Source: §3.3 · pp. 4–5

3.4 FROM LOSS TO GENERATION AND RETENTION

• Mean loss or margin and a single average-gradient halfspace do not ensure preservation of greedy generation; the paper therefore separates loss, scored content, and exact terminated responses at checkpoints and endpoints.Endpoint acceptance cannot certify a trajectory contract that constrains risk or integrated loss during training.Source: §3.4 · p. 5

4.2 CONSTRAINT PLACEMENT AND BEHAVIORAL RETENTION

• Post-minus-pre placement changes Legacy reference exact by −58.42 points, while the granularity–placement interaction is endpoint-dependent.Placement has the same direction on independent strict-EOS accuracy, but the interaction is near zero on that and independent reference exact endpoints.Source: §4.2 · p. 6
• Aggregate/pre and replay both finish at 97.83% legacy reference exact, but replay retains 73.96% versus 54.58% on independent strict-EOS audits.Aggregate/pre falls to 29.89% at update 16 and 36.96% at update 64, while replay records 91.30% and 95.11%.Source: §4.2 · p. 7, Figure 2 · p. 7
• Code replay improves independent strict-bank retention by 70.31 points over plain while its full-New content difference is +1.40 points, meeting the supporting −3-point NI rule.Its zero validation-NLL AUC+ accompanies 73.96% final independent strict accuracy.Source: §4.2 · p. 7
• Additional tasks separate content from exact-response retention: extraction replay has 99.58% content accuracy but 82.08% strict accuracy, versus plain’s 100% on both.All 42 replay records that were content-correct but strict-incorrect contain EOS; Bank B also loses content accuracy, so the effects are not uniformly response-form failures.Source: §4.2 · p. 7, Table 2 · p. 7

4.3 REPLAY TIMING: LOSS EXPOSURE BEYOND FINAL ACCURACY

• With identical Old counts, delaying replay targets raises Mistral exposure by .59443 AUC despite 98.37% versus 97.83% final Old generation for uniform and back schedules.All Qwen schedules finish at 100% Old generation, yet front-loading lowers AUC by .04865 and back-loading raises it by .25071.Source: §4.3 · pp. 8–9
• Back-minus-uniform positive-part AUC is .29160 on Mistral and .01067 on Qwen, showing above-baseline exposure that later improvement cannot cancel.Front and uniform both have zero positive-part area on the recorded grid.Source: §4.3 · p. 9
• Replay budget outcomes vary by model: Mistral reaches 98.37% Old generation at 10%, while Qwen retains 100% generation and supporting New NI at every tested budget.Replacing half the supervised objective mass with teacher KL raises Old-NLL AUC at both replay levels and on both models.Source: §4.3 · p. 9

4.4 SEQUENTIAL RETENTION: PROTECTED SUBSPACES VERSUS ACQUIRED TASKS

• CLoRA and OPLoRA improve mean final accuracy by 3.12 and 3.91 points but trade off sentiment against news, whereas replay improves both.OPLoRA’s verified initial-subspace protection coexists with regression on subsequently acquired task answers in this fixed two-seed sequence.Source: §4.4 · p. 9

• Functional regularization anchors predictions, while ARR combines KL with corpus replay and MSSR schedules memory strength.The paper’s matched-target interventions isolate timing effects beyond final accuracy.Source: §5 · p. 9
• GEM and A-GEM constrain episodic gradients, while I-GEM constrains LoRA gradients and PEGP protects feature subspaces.The paper distinguishes its nonlinear account by separating direction, staleness, mismatch, and finite-step residuals.Source: §5 · pp. 9–10
• EWC uses Fisher-weighted penalties, Adam-NSCL projects Adam updates into historical-feature null spaces, and CLoRA and OPLoRA protect fixed or weight singular subspaces.The paper compares EWC, CLoRA, and OPLoRA on acquired-task retention.Source: §5 · p. 10

6 CONCLUSION

• The functional-path account connects realized updates to retained behavior, while controlled interventions find placement effects and endpoint-dependent interactions.Checkpointed outputs and matched schedules distinguish endpoints from trajectories; sequential tasks show average subspace-protection gains alongside old-task trade-offs.Source: §6 · p. 10
• The authors recommend explicit functional anchors, trajectory evaluation, and joint old/new auditing within the scope of their findings.The study evaluates retention on declared, model-specific capabilities under controlled adaptation tasks; measured retention does not establish deployment safety or suitability.Source: §6 · p. 10

A.1 PROOF OF PROPOSITION 1

• The proof sums the decomposed gradient terms to obtain the loss-change accounting, bounding staleness and finite-step residuals under stated smoothness assumptions.Reporting increment sums alongside adjacent-evaluation gaps separates within-update accounting from continuity across the logged trajectory.Source: §A.1 · pp. 13–14

A.2 PROOF OF COROLLARY 1

• Uniform bounds on every partial block sum give a pathwise sufficient budget; bounding only final loss does not bound the largest intermediate loss.The same bound applies to every trajectory prefix after truncating its final refresh block.Source: §A.2 · p. 14

A.3 TIGHT QUADRATIC CONSTRUCTION

• Cached-orthogonal updates can multiply quadratic loss by (1 + r^2a^2)^K while attaining the refresh-bound increment exactly.For r = 1, the fresh dot is zero but curvature still raises loss; for r > 1, positive-age steps also have a positive fresh first-order term.Source: §A.3 · p. 14

A.4 A CROSS-ENTROPY AND GREEDY-OUTPUT CONSTRUCTION

• In the cross-entropy construction, loss rises at every step and the greedy answer becomes wrong when r^2a^2 > 2c − 1.For r = 8, a = 1/5, and c = 1, the margin changes from 1/2 to −39/50; this is an analytic example, not a measured language-model trajectory.Source: §A.4 · p. 15

0 Pg0 > 0. The smoothness inequality gives

• A sampled replay gradient is guaranteed a strictly beneficial first-order term when its P-norm error is smaller than the true restoring gradient’s P-norm.The uniform mismatch condition is not measured for a hidden audit, and AdamW, clipping, and interleaved updates make the conditional calculation distinct from empirical long-horizon effects.Source: §0 · p. 15

A.6 A SUFFICIENT CONDITION FOR GREEDY GENERATION RETENTION

• Per-prefix logit changes below half the baseline correct-token margin preserve the same greedy sequences and stopping positions.Mean margins do not establish this condition, and the experimental models’ pathwise Lipschitz constants were not certified.Source: §A.6 · pp. 15–16

B.1 SNAPSHOT AND STATISTICAL DESIGN

• Delayed replay increases positive-part loss exposure in both models, while front and uniform replay have zero positive-part area on the recorded grid.That grid result does not establish preservation at unobserved updates, within every stratum, or in generated behavior.Source: §B.1 · p. 17, Table 4 · p. 17

B.2 ORIGINAL REFRESH–COVERAGE ENDPOINTS AND PATHS

• For Mistral r1, n12, the diagnostic path sums were −.5213 cached term, approximately zero staleness, +.4634 probe mismatch, and +.3282 finite-step residual, totaling +.2703 loss change.At r1, n48, diagnostic change was near zero although final qualified generation was 4.35% and hidden-audit NLL increased by .3234.Source: §B.2 · pp. 19–20
• Replay’s Mistral path had a favorable −1.5641 net diagnostic sum despite positive staleness, mismatch, and residual terms.Replay need not shrink every diagnostic component; the different visited paths do not identify a causal mediation fraction.Source: §B.2 · p. 20
• Refreshing the cached gradient reduced immediate diagnostic-E change for Mistral by .01391 at n12 and .01272 at n48, while Qwen’s intervals included zero.These are age-seven, same-state local interventions, not full-path treatment effects.Source: §B.2 · p. 20, Figure 5 · p. 22

B.3 BASELINE SPECIFICATION AND CALIBRATION

• The matched-budget baselines distinguish constraint multiplicity and optimizer placement, while the hybrid target comparison replaces half the supervised objective mass rather than adding a KL penalty.The adapted settings do not reproduce source papers’ original benchmark settings, and the hybrid contrast cannot establish that an incremental KL term harms an otherwise identical objective.Source: §B.3 · pp. 20–21

B.4 NUMERICAL EXECUTION AND EMPIRICAL DECOMPOSITION

• All 8,192 actual displacements met the registered numerical tolerance, but tolerance feasibility is not the ideal real-arithmetic inequality.Across the 96 mechanism runs, decomposition closure error was at most 5.56×10−17; its finite-step residual is not a measured Hessian norm or causal fraction of forgetting.Source: §B.4 · p. 21, Table 9 · p. 22
• The original 256-run archive lacks per-example generated sequences and intermediate generation audits, so it supports derived loss-path analysis rather than intermediate-generation analysis.The separate structure–location study records generation at fixed intermediate checkpoints and retains adapter checkpoints.Source: §B.4 · p. 23

C.1 A SATISFIED CACHED CONSTRAINT CAN ACCOMPANY SEVERE FORGETTING

• The Mistral pure-Adam projection run retained none of 23 acquired old answers despite satisfying cached tolerances throughout 128 updates, while reaching 94.30% new-generation exact.At update 32, the cached dot after projection was 2.6702764 × 10−9, but the fresh-gradient dot with the same update was 0.1128072154.Source: §C.1 · p. 23

C.2 FIXED REPLAY PROTECTS BEHAVIOR WITHOUT FREEZING ITS IMPLEMENTATION

• Fixed replay improves old-token retention while measured adapter-factor displacement increases and final last-layer CKA decreases.The diagnostics concern trainable LoRA factors and a fixed probe readout, not the full effective weight or all representations; they are descriptive, not mediation evidence.Source: §C.2 · p. 23, Figure 7 · p. 24
• Replay shifts the current-gradient/actual-update dot downward in all three models, while raw-gradient-cosine intervals cross zero for every model.A negative method-minus-plain difference does not establish a negative absolute dot or a causal mediator.Source: §C.2 · p. 24

C.3 INFORMATIVE TARGETS AND THE LONG-HORIZON MECHANISM

• Moving old examples into the new-task phase improved old exact by 0.670 across six registered seeds, while source-normalized replay added an uncertain 0.021 over ordinary replay.The latter interval, [−0.129, 0.171], leaves objective normalization without a stable incremental effect in that comparison.Source: §C.3 · p. 24
• At 100 updates, old-loss AUC protection was 6.668 for source-normalized supervision and 7.067 for teacher KL, versus 0.510 for input-only replay.These target-information interventions support repeated functional supervision rather than old-input exposure as the useful payload in that setting; they do not establish a single universal source of forgetting.Source: §C.3 · pp. 24–25

D.1 THE STRONGEST GENERATION EFFECT SURVIVES COMPUTE MATCHING

• Fixed 10% supervision raises Mistral old generation over plain by 94.02 percentage points under both matched exposure and matched optimization MTE.These are matched proxies, not an exact hardware-FLOP comparison; Qwen and OLMo have positive point estimates that do not pass the registered correction.Source: §D.1 · p. 25

D.2 LONG-HORIZON PROTECTION WITH NEW-TASK NON-INFERIORITY

• Fixed replay raises Mistral old generation by 94.57 points and passes the registered new-generation and new-token non-inferiority criteria in all three long-horizon model comparisons.Qwen is at a generation ceiling in all arms; OLMo’s 2.88-point generation gain is not confirmed after correction.Source: §D.2 · p. 25, Table 10 · p. 26, A Absolute retention · p. 26

A Absolute retention

• On Mistral, Adam plus 2% replay reaches 96.20% old-generation accuracy, yet six of eight runs exceed the instruction-format margin budget, so high generation accuracy alone does not establish the full contract.Its new-generation NI lower bound is −3.0130649 points, below the −3-point boundary.Source: A Absolute retention · p. 26, Figure 9 · p. 26

D.3 TRAJECTORY AND BANK EVIDENCE

• Fixed replay improves long-horizon old-token accuracy by 30.12, 8.14, and 15.23 points for Mistral, Qwen, and OLMo, respectively, while lowering controller Old-NLL AUC in all three.Mistral fixed replay nevertheless has transient controller violations, so endpoint feasibility does not imply monotonic protection.Source: §D.3 · pp. 26–27
• Across four 7B phases, fixed replay has 104/104 feasible endpoints; Qwen bank variants show 5.7–8.0-point old-token benefits and all satisfy new-task NI.The bank study changes evaluation coverage along with bank composition, so it does not identify training-bank composition effects on a common broad hidden test.Source: §D.3 · p. 27
• The earlier 384-run matrix improves old-token endpoints in all nine model–adaptation cells, but does not establish new-generation NI; only Qwen has an eligible old-generation subset.These historical qualified-bank results support adaptation coverage, not a 7B generation claim.Source: §D.3 · p. 27

E MODEL-QUALIFIED BANKS AND MEASUREMENT

• The original 7B bank uses model-specific qualification rather than a Full/LoRA/QLoRA intersection, and zero-count domains are unmeasured rather than retained with zero loss.Teacher-forced and generation evaluations have different coverage, and different model banks preclude attributing effect-size differences solely to architecture.Source: E MODEL-QUALIFIED BANKS AND MEASUREMENT · p. 27, Figure 10 · p. 27

New non-inferiority

• All three long-phase fixed-replay comparisons pass the registered new-generation and new-token non-inferiority criteria, while standard-phase Mistral and equal-MTE Mistral/Qwen comparisons do not.The Mistral long-horizon Adam-plus-2% generation lower bound is −0.030130649384 and fails the criterion; its token lower bound is approximately −0.028920815 and passes.Source: F STATISTICAL FAMILIES AND COMPLETE… · p. 29

Other method contrasts

• Adaptive hybrid has 24/24 feasible endpoints without an established registered advantage over fixed replay; generic OpenWebText KL is not a substitute for task-relevant supervision.The KL inputs, masks, and target distributions differ, so this comparison alone does not isolate label information; dynamic masks likewise establish no registered generation advantage over static masks.Source: F STATISTICAL FAMILIES AND COMPLETE… · p. 29

Complete results

• The helper’s PARTIAL SUPPORT label is not projection-hypothesis confirmation: its generation confirmations are 0/3 for pure Adam in each phase and 0/3 for Adam-versus-raw contrasts.The paper relies on the actual endpoints and tests rather than interpreting that convenience label as confirmation.Source: F STATISTICAL FAMILIES AND COMPLETE… · p. 29
• Mistral’s Long horizon Old (%) is 97.28 for Fixed 10% versus 2.72 for Plain, while their F/8 values are 8/8 and 0/8, respectively.F counts runs passing the full retention and new-acquisition audit, not just generation.Source: Table 13 · p. 30, F STATISTICAL FAMILIES AND COMPLETE… · p. 31

G.1 SEPARATING RESIDUALS, STALENESS, AND OUTCOMES

• 11,110 of 11,143 Adam-projection interventions meet the frozen tolerance; 33 exceed it across 26 runs, so the audit records genuine deviations rather than an ideal projection guarantee.The largest positive residual is 6.3778134 × 10−7, and no threshold is relaxed or implicated seed removed.Source: §G.1 · p. 31
• The central Mistral seed-1101 witness has zero violations across 128 steps, while all 1,756 fresh-versus-cached diagnostics have gradient age seven.Refreshes occur at updates 1, 9, 17, and so on, whereas diagnostics occur at 16, 32, and so on; the sampled diagnostic counts are not estimates for uniformly sampled training steps.Source: §G.1 · p. 31

G.2 MEANING OF EACH DIAGNOSTIC

• The representation diagnostics cover a specified stratified probe and hidden-position readout, not the complete representation geometry; parameter snapshots measure trainable LoRA factors, not full effective-weight drift.Mean cosine and centered linear CKA therefore characterize that probe/readout rather than the complete model representation.Source: §G.2 · p. 31

G.3 COMPUTE AND ENVIRONMENT

• Shared-GPU execution was not treated as an isolated hardware-efficiency experiment, and long-horizon arms used 512, 569, and 523 optimizer updates for plain, fixed replay, and Adam plus 2%, respectively.Each arm consumed 8,192 new exposures; old-example exposures were 0, 910, and 167, respectively.Source: §G.3 · p. 32

G.4 ARTIFACT COMPLETENESS

• The package supports recomputation of logged statistics but not re-evaluation of the trained models or additional weight interventions; the local audit retains FAIL for the 33 numerical violations.The local analysis is a derived audit, not a rewritten remote completion receipt.Source: §G.4 · p. 32

H LONG-HORIZON FUNCTIONAL TRAJECTORIES

• Fixed replay lowers time-normalized Old-NLL AUC versus plain by 1.733, 0.490, and 0.679 for Mistral, Qwen, and OLMo, respectively, yet Mistral has 11 controller violations despite eight feasible final endpoints.Final generation can therefore coexist with different accumulated old-loss exposure; the phase contrast is not a same-state replay-versus-short-horizon counterfactual.Source: H LONG-HORIZON FUNCTIONAL TRAJECTORIES · p. 32, Figure 11 · p. 33

Controlled information experiment

• At 100 updates, final old exact is 0.792 under global supervision versus 0.167 under new-only and 0.142 under input-only.The input-only control tests prompt exposure without an old answer or teacher target, not every possible unlabeled learning objective.Source: I SUPPORTING CONTROLLED AND… · p. 34

Optimizer-state experiment

• The optimizer-state comparison leaves the transferable-moment hypothesis unresolved: replay-donor versus sequential-donor old-loss AUC is 0.956 [−0.351, 2.263], with exact p = .09375.The final old-exact difference is −0.052 [−0.129, 0.025].Source: I SUPPORTING CONTROLLED AND… · p. 34

Qualified Full/LoRA/QLoRA confirmation

• Fixed replay improves the registered old-token endpoint in all nine earlier model–adaptation cells, with eight positive paired effects per cell, while new-generation NI is not established in any cell.Only Qwen has an eligible old-generation subset, so the cross-model primary is token retention; effects span 1.5 to 44.6 token-accuracy points.Source: Figure 13 · p. 34, I SUPPORTING CONTROLLED AND… · p. 34

Complete matrix and controller-stress comparison

• Supervised and hybrid replay exceed input-only on final old token and old-loss AUC in each model, while adaptive hybrid uses less replay but retains fewer old tokens than fixed replay.These results motivate a fixed functional baseline while leaving adaptive scheduling as a separate trade-off.Source: I SUPPORTING CONTROLLED AND… · p. 34

J.1 DETAILED EXPERIMENTAL CONFIGURATION

• The paper uses separate 576-run scale, 256-run refresh–coverage, and 64-run structure–location designs, pairing results within protocols rather than pooling them.The studies cover different models and acquired-function settings; the structure–location study adds natural-language sentiment and structured extraction to the legacy code task.Source: §J.1 · p. 35

J.2 A RETENTION CONTRACT FOR SFT

• Endpoint acceptance requires New acquisition and hidden-audit R(θ) ≤ 1, whereas a trajectory contract additionally constrains training-time risk or integrated loss.Paired endpoint effects can show mitigation without satisfying the full contract; Old NLL, current-gradient dots, margins, and generation measure complementary aspects of retention.Source: §J.2 · pp. 36–37

K EXPANDED RELATED WORK

• The paper accounts for realized updates and old-loss changes, distinguishing its aggregate actual-update halfspace from PEGP’s factor-input subspaces and I-GEM’s task-gradient constraints.Its controlled interventions examine replay timing and objective allocation, and separately measure loss trajectories and final generated behavior; the comparisons are shared-budget adaptations, not official-code reproductions.Source: K EXPANDED RELATED WORK · p. 37

L.1 FROZEN PROTOCOL AND TASK CONSTRUCTION

• The Mistral structure–location study has 48 code runs and eight runs each for Natural sentiment and Extraction, with its EOS-supervised protocol frozen separately before formal runs.Code and Natural share SST-2 identities and splits, but Natural changes prompt and answer form; Extraction is a controlled task with new field names, not a broad natural benchmark.Source: §J.1 · p. 35, §L.1 · pp. 38–39

L.2 QUALIFIED BEHAVIORAL BANKS AND DECODING

• 100% is the baseline for both independent strict-EOS banks and Legacy reference exact, while Legacy strict-EOS baseline is 0%.Formal trajectories include 58/148,400 records with strict exact but not reference exact; in the targeted recheck, changing batch context altered prefixes for all 17 selected targets and no controls.Source: §L.2 · pp. 39–40

L.3 MATCHED PROJECTION IMPLEMENTATION

• Aggregate protection constrains the weighted mean gradient across four capability strata, while stratum-wise protection constrains all four rows using a common Euclidean solver.Pre-Adam projection changes the gradient before AdamW updates its moments; post-Adam projection corrects the resulting parameter displacement while retaining moments from the incoming gradient.Source: §L.3 · p. 41

L.4 COMPLETE OUTCOMES AND STATISTICAL FAMILIES

• Seven of 64 formal runs satisfy the full frozen retention contract; the other 57 fail that absolute criterion despite valid New acquisition and numerical checks.Code starts at zero on all three New-generation metrics; Extraction starts at zero on response-level measures despite 90.04% answer-token accuracy, and both methods reach 100% content accuracy after training.Source: §L.1 · p. 39, Table 17 · p. 41, §L.4 · p. 41
• The Legacy stratum-wise effect was −1.09 points before Adam and +63.59 after Adam; independent-bank reference and strict interactions were −1.67 and −2.19 points.The independent-bank interactions are post-hoc descriptive effects; the endpoint-dependent accuracy-scale interaction is not explained simply by requiring EOS. Bank content and prompt interface vary together, and near-ceiling pre-Adam accuracy limits observable granularity gains.Source: §L.4 · p. 44, Table 21 · p. 44
• Three of 1,104 final Legacy records differed in R versus P64 correctness, while the same-output sensitivity retained a 64.13-point interaction (CI [38.15, 90.11]).This post-hoc check did not change the registered endpoint or its Holm family.Source: §L.4 · p. 44, Table 22 · p. 44
• Code replay had positive independent strict-retention effects in all eight paired seeds; Natural and Extraction showed reverse strict-bank effects alongside Legacy reference-exact gains.The Natural and Extraction comparisons were descriptive four-seed results: Legacy reference exact improved by +52.17 and +21.74 points, respectively. Within banks A/B, strict replay-minus-plain effects were −8.75/−9.17 points for Natural and −17.92/−27.50 for Extraction; content effects were −.83/−7.08 and −.42/−10.00 points. Confirmatory non-inferiority was assessed only for Code.Source: §L.4 · p. 45, Table 23 · p. 45

L.5 BEHAVIORAL PATHS, POSITIVE-PART EXPOSURE, AND CONTRACT VIOLATIONS

• Zero validation-NLL AUC+ coexisted with independent strict-bank losses of 26.04 points for Code, 9.58 for Natural, and 24.79 for Extraction.The NLL and behavioral audits measured different objects. NLL was sampled every 16 updates through 128; generation at updates 0, 16, 32, 64, 96, and 128, with violations assessed only at observed checkpoints.Source: §L.5 · pp. 45–46, Table 24 · p. 45, Table 25 · p. 46

L.6 SAME-STATE INTERVENTIONS AND PROSPECTIVE DIAGNOSTICS

• At matched Code/plain branch states, the local pre stratum-minus-aggregate effect was −.005475 (CI [−.009880, −.001071]), while the local interaction was +.004515 (CI [−.001298, .010328]).Four branches shared pre-update parameters and optimizer state; five branch effects were averaged within each of eight seeds. Finite-step residuals include gradient, displacement, and evaluation numerics, and the local interaction does not establish an explanation or causal mediation of the long-trajectory generation interaction.Source: §L.6 · p. 48, Table 26 · p. 48, Table 27 · p. 48
• On the Extraction holdout, the prospective score’s mean correlation was +.4524 for Plain and −.3810 for Replay10, with no stable out-of-task ranking advantage established.Paired improvements over the aggregate-dot baseline had intervals [−.2654, .3726] for Plain and [−.7356, .1403] for Replay10; Replay’s difference from the raw-conflict baseline was −.6905 ([−.9855, −.3954]). These are descriptive associations with future validation NLL, not generated-behavior guarantees.Source: §L.6 · pp. 48–49, Table 28 · p. 49

L.7 SAVED OUTPUTS AND ERROR TAXONOMY

• The frozen taxonomy counts outputs across 120 fixed items and eight seeds, with priority-ordered categories including code intrusion, response mismatches, and length-cap errors.The post-hoc “Only A/B” count is within the intrusion category, not a new registered endpoint.Source: §L.7 · p. 49, Table 29 · p. 50
• In Extraction bank A, all 42 content-correct/strict-incorrect replay records contain EOS, distinguishing token-exact changes from missing termination or scored content loss.The comparisons do not isolate the contribution of the answer-only Old supervision and EOS-supervised New targets.Source: §L.7 · p. 50
• A fixed-order arithmetic replay example preserves integer content but fails token-exact format at three observations.The saved-output examples supplement the full error counts rather than defining a selected success rate.Source: §L.7 · p. 50, Table 30 · p. 50

L.8 REPRODUCIBLE ARTIFACTS AND ARCHIVE SCOPE

• The archive includes all 64 configurations and 8,192 update events, but the recheck does not restore missing intermediate outputs or deleted weights from the original studies.The materials can support an anonymized supplement; no public release is asserted.Source: §L.8 · p. 51

M SEQUENTIAL TASKS AND PROTECTION OBJECTS

• In the three-task sequence, replay improves retention of both preceding tasks relative to Plain, while projection and regularization show different task-retention patterns.Mean improvement, numerical constraint satisfaction, and retention of each learned task are measured separately.Source: M SEQUENTIAL TASKS AND PROTECTION… · p. 51

M.1 DATA, SEQUENCE, AND COMMON EXECUTION

• The sequential study uses one continuing QLoRA adapter across news, sentiment, and extraction, with 32 updates and 512 new-record exposures per task.Two paired seeds share new-record ordering and experienced-memory selection; evaluation reports the three completed task boundaries.Source: §M.1 · pp. 51–52

M.2 PROTECTION OBJECTS AND FIXED ADAPTATIONS

• Replay adds losses from 32 experienced records per preceding task; GEM-style projection constrains the pre-Adam descent gradient rather than Adam’s realized displacement.Replay keeps the new-data contribution, while the task-level projection uses one memory gradient per preceding task.Source: §M.2 · p. 52
• EWC anchors factor-space parameters with diagonal empirical Fisher estimates, while CLoRA and OPLoRA regularize or project selected adapter subspaces.OPLoRA protects selected initial-weight subspaces, not the historical-feature covariance null space used by Adam-NSCL.Source: §M.2 · p. 52

M.4 COMPLETE RESULTS AND TASK-DEPENDENT PROTECTION

• Replay reduced news and sentiment losses to 0.78 and 1.56 points, improved final old-task mean by 19.92 points over Plain, and raised BWT by 20.70 points.Final extraction accuracy was 99.22% with replay versus 98.44% with Plain; replay's 2.73-point forgetting differs from its −1.17-point BWT because news accuracy improved at an intermediate boundary.Source: §M.4 · p. 53
• CLoRA and OPLoRA improved mean ACC by 3.12 and 3.91 points and mean BWT by 4.30 and 4.69 points over Plain, but their task-level changes ran in opposite directions.CLoRA changed final news/sentiment by −5.47/+13.28 points, while OPLoRA changed them by +18.75/−7.81 points; Task-GEM improved mean BWT by 6.25 points and adapter EWC left it unchanged.Source: §M.4 · p. 53
• Across the task-boundary matrices, accuracy counts are out of 64, and pre-acquisition entries measure transfer rather than retention.Table 31 reports ACC and New as percentages and BWT and forgetting as percentage points; forgetting uses the maximum previously observed post-acquisition score, not simply −BWT.Source: Table 32 · p. 53, Table 31 · p. 57

M.5 NUMERICAL CHECKS, ARCHIVED CONTROLS, AND INTERPRETATION

• All declared protections passed their implementation checks, but protected weight subspaces do not represent all subsequently acquired task functions.Task-GEM constraints passed tolerance, CLoRA regularization was nonzero, and OPLoRA basis checks passed; these checks establish execution, not behavioral equivalence.Source: §M.5 · p. 57
• The comparison uses one model, one fixed task order, two paired seeds, and fixed QLoRA adaptations without method-specific hyperparameter search.Its scope is a descriptive comparison, not a universal method ranking or an equal-FLOPs test.Source: §M.5 · p. 57