When Can First-Order Models of Fine-Tuning Bound Forgetting?

Motivation and scope

• The study targets an upper bound on each protected fact’s forgetting probability, a guarantee not supplied by risk rankings alone.The paper also evaluates the bounds prospectively across models, CounterFact facts, and a doubled horizon.Source: §1 · pp. 1–2

Why full-run prediction fails

• First-order response predictions were locally accurate, yet predictions of forgetting failed when a run moved beyond the region in which the model was validated.In Qwen3-0.6B, forgetting required a parameter change about 18 times larger than the largest validated change.Source: §1 · pp. 1–2

Bounds from local probes

• Freedman and Azuma first-passage bounds use a coefficient-change term R, and their inputs are computed before the fine-tuning run.The complete Freedman bound certifies only facts with R < a; setting R = 0 gives the simplified bound.Source: §1 · p. 2

Main finding

• Across 112 conditions, the simplified Freedman bound was violated in 14, but every violated fact had R ≥a; the complete bound held in every condition.In 43 confirmatory conditions, the complete bound held everywhere and the simplified bound failed on only 3 facts, each with R ≥a.Source: §1 · p. 2

Screening alternative

• A screen based on the median standardized margin removed earlier failures but failed on new starting states, unlike the per-fact rules.The contrast is consistent with failures being confined to individual facts.Source: §1 · p. 2

Forgetting during fine-tuning

• Prior work measures or explains forgetting, forecasts examples at risk, and changes updates to mitigate forgetting; these bounds instead leave the update unchanged.The cited approaches include example-ranking forecasts from logit changes and task-example associations.Source: §2 · p. 2

Local models of training

• Influence functions, gradient tracing, and kernel linearizations describe training locally, and this paper asks what those local models support over a full run.The cited interpretability methods are usually validated on short horizons.Source: §2 · p. 3

Bounds and risk control

• Unlike population-risk bounds and exchangeability-based calibration, this work bounds a per-fact event during a run using design-selected conditions.Because the conditions rule out exchangeability, thresholds are fixed on earlier conditions and tested on new ones.Source: §2 · p. 3

Two-task fine-tuning

• Each condition specifies a model, data source, task-B learning rate, and seed; task B uses SGD after LoRA adaptation on the starting state.The study uses synthetic associations or CounterFact and 32 test runs per condition, except for two earlier conditions with 24.Source: §3 · p. 3

Margin and forgetting

• Protected facts are evaluated by their margin, the log-probability gap between the correct answer and strongest alternative; forgetting occurs if it falls below zero on the evaluation grid.The study uses a 40-step horizon, with one study at 80 steps, and claims concern facts whose reference-run margins remain positive.Source: §3 · p. 3

First-order response model

• The first-order model uses an eight-direction basis from starting-state margin gradients and finite-difference probes to estimate how perturbations change later margins.The probes also estimate projected clipped-gradient covariance, and all bound inputs are computed before the first test run.Source: §3 · p. 3

Probe cost

• The probes cost O(k |G| T) SGD steps, which grows with the square of the horizon.Each probe replays the run from a perturbation time to T.Source: §3 · p. 3

Bound procedure

• Figure 1 lays out how reference-run margins and probes determine per-fact bound inputs, including the coefficient change R and distance a to the boundary.The complete bound certifies only facts with R < a.Source: Figure 1 · p. 4

Response-model validation

• At learning rate 0.002, the response model passed 15/15 validation runs across 6 checkpoints and 3 model families, with ρ between 0.974 and 0.998.At learning rates 0.008 to 0.012, three further runs had ρ between 0.939 and 0.995.Source: §3 · p. 4

4.1 WHAT A LOCAL MODEL CAN DETERMINE ABOUT A RUN

• The coupling bound separates accumulated local transition error from the probability that training leaves the checked region.Theorem 2 shows that local data alone cannot identify event probabilities outside that region.Source: §4.1 · p. 4
• For Qwen3-0.6B, forgetting required a parameter change about 18 times larger than the largest change used to validate the response model.The authors therefore bound a surrogate margin's first crossing of a boundary near zero rather than infer forgetting from the local model over the whole run.Source: §4.1 · p. 4

4.2 TWO FIRST-PASSAGE BOUNDS

• The surrogate combines a reference-run margin with projected, centered update noise, while R bounds response-coefficient change over the grid.Here a is the reference margin's distance from boundary B0 = 0.01.Source: §4.2 · pp. 4–5
• The first-passage bounds apply maximal Azuma and Freedman inequalities to a surrogate margin whose discrepancy from the final-time-coefficient surrogate is bounded by R.They use a′ = a − R; Freedman additionally uses a bound V on predictable quadratic variation, while Azuma uses the increment bound c.Source: §4.2 · p. 5

4.3 ASSUMPTIONS OF EACH BOUND

• Applying the bounds to actual margins requires a uniform remainder assumption; both bounds need bounded increments, and Freedman also needs a valid variance estimate.The uniform remainder assumption remains unverified, and high response correlation alone does not establish it.Source: §4.3 · p. 5

5 RULES THAT CERTIFY FACTS

• Each rule computes a value before the run and certifies eligible facts when that value is below 0.05.A fact-level violation is determined by a one-sided binomial test with Bonferroni correction over the condition's 64 facts.Source: §5 · pp. 5–6
• The screened rule tests whether the median standardized margin among simplified-bound-certified facts is at least 30, then selects a bound for the whole condition.It uses the simplified Freedman bound if the screen passes and Azuma otherwise; the screen uses only pre-run quantities.Source: §5 · pp. 5–6
• Figure 2a plots the simplified-bound-certified facts by pre-run distance a and coefficient change R, with violations marked in the region R ≥ a.Panel (c) compares the fraction certified with the fraction of conditions having a violation.Source: Figure 2 · p. 6
• The per-fact rule requires a standardized margin of at least 56, while the margin-only baseline requires distance a_i of at least 0.66.Both thresholds were selected using earlier conditions.Source: §5 · p. 6

6 WHERE THE SIMPLIFIED BOUND FAILS

• 80.1% of eligible facts were certified by the simplified Freedman bound in 41 earlier conditions, but it was violated in 7 conditions.All 14 violated facts had R ≥ a; the complete bound certifies only facts with R < a and held in every condition.Source: §6 · p. 6
• Across 112 conditions, every one of the 14 facts violating the per-time bound had R ≥ a, and the margin-spread-to-prediction ratio on simplified-bound failures had a median of 14.6.The median ratio was 1.6 for facts with R < a; R ≥ a was necessary but not sufficient for failure, with a median ratio of 2.7 across all such facts.Source: §6 · p. 7
• The complete bound certified 31.2% of eligible facts, compared with 22.1% for Azuma, while excluding every fact with R ≥ a.Although 53.3% of facts certified by the simplified bound had R ≥ a, most survived every test run.Source: §6 · p. 7
• On earlier conditions, the screened bound held while certifying 64.2% of eligible facts, and the per-fact rule certified 37.1%.Only new starting states could test whether these rules generalized beyond the earlier conditions.Source: §6 · p. 7

7.1 DESIGN

• The prospective program covered new starting states, synthetic and CounterFact facts, horizons of 40 and 80 steps, and models up to Qwen3-14B.The main study used 33 conditions; four Qwen3-8B and Qwen3-14B conditions were stopped before bounds were computed after falling below the task-A accuracy threshold.Source: §7.1 · p. 7

7.2 RESULTS

• The screened bound failed its violation and assignment criteria, with 5 violating conditions among 31 evaluable conditions, above the allowed maximum of 4.Conditions with median z̃ between 32 and 56 were assigned to the simplified Freedman branch, where violations occurred.Source: §7.2 · pp. 7–8
• Across 47 conditions, all 19 facts violating the simplified bound had R ≥ a, while the complete bound, Azuma bound, per-fact rule, and margin threshold held in every condition.The complete bound and Azuma bound also had no violations in the 19-condition confirmatory test; both simplified-bound violations there had R ≥ a.Source: §7.2 · p. 8, Table 1 · p. 8
• Across studies, the per-fact rule certified 33.2% of eligible facts, versus 19.9% for the complete bound, 16.4% for Azuma, and 11.5% for the margin threshold.The per-fact rule depends on a threshold chosen on earlier data; task-B accuracy rose by 25.3 percentage points on average in the main study.Source: §7.2 · p. 8

7.3 SECOND CONFIRMATORY STUDY

• In the second confirmatory study, the complete bound had 0 violating conditions; the simplified bound was violated on 1 fact, with R ≥ a.The location criterion was undecided because it required at least five violated facts; the complete bound certified 17.0% of eligible facts, versus 28.1% for the restricted simplified bound and 9.6% for Azuma.Source: §7.3 · p. 8

7.4 POOLED FORGETTING COUNTS

• Across test runs pooled over all 112 conditions, facts certified by the complete bound had 0 forgetting events in 34,256 fact-test-run pairs, versus 130.8 allowed; eligible facts with R < a had 0 in 44,976 pairs.Source: §7.4 · pp. 8–9, Table 2 · p. 9

What the results support

• The complete bound and Azuma bound held in all 112 conditions, including 43 confirmatory ones, while the simplified bound failed only on facts with R ≥ a.The location pattern is mainly post hoc because the confirmatory studies contained only 3 violated facts; a screen designed on earlier conditions failed on new starting states.Source: §8 · p. 9

When the probes are worth their cost

• When one update is run repeatedly from a shared starting state, probes can be cheaper than simulation: median costs were 38 test runs at T = 40 and 68 at T = 80, versus 59 surviving runs for a 0.05 bound at 95% confidence.The passage gives repeated adaptation of a shared model for many users as an example where a probability matters.Source: §8 · p. 9

Limitations

• The bounds formally cover a linear surrogate, and the remainder needed to transfer them to actual margins is unquantified; results cover SGD at 40- and 80-step horizons, not Adam or full fine-tuning.The complete bound excludes every fact with R ≥ a, although most of those facts survived every test run.Source: §8 · p. 9

• A first-order model can be accurate yet fail to predict a full fine-tuning run, while still bounding the probability that a margin first crosses a boundary near zero.Source: §9 · p. 9
• Every observed simplified Freedman breakdown occurred on a fact with R ≥ a, a condition known before the run.Source: §9 · p. 9
• The complete bound, which excludes those facts, and the Azuma bound held in all 112 conditions, whereas a screen judging whole conditions failed on new starting states.Source: §9 · p. 9

ETHICS STATEMENT

• Bounds on forgetting could be over-trusted outside their assumptions or used to protect content that should be removed.Source: ETHICS STATEMENT · p. 10
• The work uses public checkpoints and synthetic or public factual data, with computational experiments only.Source: ETHICS STATEMENT · p. 10

A.1 THEOREM 1 (COUPLING BOUND)

• Stepwise maximal coupling bounds decoupling by the chance of leaving N plus the accumulated per-step kernel differences εt.Source: §A.1 · p. 13

A.2 THEOREM 2 (NON-IDENTIFIABILITY)

• Outside N, systems matching in value, derivatives, and noise can still differ in step-2 event probability by any p in (0, 1].A predictor using only N must incur error of at least p/2 on one of the systems.Source: §A.2 · p. 13

A.3 THEOREM 3 (FIRST-PASSAGE BOUNDS ON THE GRID)

• The grid-time first-passage proof decomposes the margin deviation into a martingale and residual, then bounds crossing using a′ = a − R.Azuma uses increment bound c; Freedman additionally uses a variance bound, while the final-time-coefficient surrogate has R = 0.Source: §A.3 · p. 13

A.4 ESTIMATED VARIANCE (SECTION 4)

• With variance bound bV holding except with probability δ, the estimated-variance Freedman crossing bound adds δ to the conditional exponential bound.An estimate bV depending on test-run outcomes needs a separate argument.Source: §A.4 · p. 14

A.5 SELECTION BEFORE THE RUN

• Choosing a bound branch and certified fact set from probe information before the run preserves the selected branch’s guarantee, even with dependence among facts or branches.The result does not establish that either branch is valid; numerical checks report a largest empirical-frequency-to-bound ratio of 0.354 and reimplementation discrepancy of 1.7 × 10−16.Source: §A.5 · p. 14

B SCOPE OF THE FIRST-ORDER RESPONSE MODEL

• Every validation run passed Table 3’s preregistered thresholds, while SGD probe responses scaled nearly linearly with perturbation size; forgetting required about 18× the validated radius on Qwen3-0.6B.At learning rate 0.002, the validation criterion required ρ ≥0.90 and slope in [0.8, 1.2]; the reported SGD scaling factor was 1.92 to 2.06 when perturbation size doubled.Source: Table 3 · p. 14, B SCOPE OF THE FIRST-ORDER RESPONSE… · p. 14

D HISTORY OF THE SCREENING THRESHOLD

• The threshold-30 screen passed every earlier condition but failed on new starting states, whereas the complete bound, Azuma bound, and per-fact rule passed there.The earlier data alone left the threshold indistinguishable from a fitted value, motivating prospective testing; uniform variance inflation still left violating conditions at κ = 128.Source: D HISTORY OF THE SCREENING THRESHOLD · p. 15

F EARLIER CONDITIONS

• In the earlier evaluable conditions, the reference margin ranked forgetting risk above the simplified Freedman bound by area under the ROC curve in five of six conditions.The reference margin provides a ranking rather than a probability.Source: F EARLIER CONDITIONS · p. 16

G SPREAD OF THE MARGINS ACROSS TEST RUNS

• The median ratio of realized margin spread to model-predicted spread was 14.6 on facts violating the simplified bound, and the complete bound held despite underestimating variance for many facts.For 99.0% of facts certified by the complete bound, a − R exceeded three standard deviations of the realized final margin.Source: G SPREAD OF THE MARGINS ACROSS TEST RUNS · p. 17

H FORGETTING RISK AND THE RATIO R/a

• 0 of 1,436 facts with R < a were forgotten in any test run; above R = a, the share forgotten rose to 63.1% for R/a ≥16.The remaining 61 eligible facts had a ≤0; every bound left them uncertified, and 59 were forgotten in at least one test run.Source: H FORGETTING RISK AND THE RATIO R/a · p. 17

I POWER OF THE TEST AND POOLED FORGETTING COUNTS

• The per-fact test had 0.8 power only when true forgetting probability was at least 0.304, so passing rules out only large excesses; pooled counts were descriptive because events across facts were dependent.For the margin threshold, per-fact rule, and restricted margin threshold, certified facts were forgotten 5, 1, and 0 times, against 911.6, 23.5, and 485.6 allowed events, respectively.Source: I POWER OF THE TEST AND POOLED… · p. 18

J VARIATION ACROSS SEEDS AND BUILDS

• For fixed configurations, forgetting events ranged from 0 to 210 for Llama-3.2-1B and 0 to 233 for Qwen3-0.6B across seeds; bounds therefore must be computed for each starting state.The second confirmatory study’s new seeds had fewer events than earlier seeds at the same settings; seed variation mixed starting-state and minibatch-order effects.Source: J VARIATION ACROSS SEEDS AND BUILDS · pp. 18–19

K PROSPECTIVE CONDITIONS; L PREREGISTRATION RECORD

• The family-wise screened-bound statements were slightly above nominal level: a set covering 60.9% of eligible facts had an event in 44 of 800 runs (5.5% versus 5%).The supplementary record includes preregistrations, frozen scripts, and verbatim outcomes; all reported numbers use corrected code after three defects were fixed.Source: K PROSPECTIVE CONDITIONS · p. 19, L PREREGISTRATION RECORD · p. 22