PaperlayerView exact revision on arXiv

Source-linked AI summary

The Sequential Price of Continual Learning

Zonghuan Xu, Xingjun Ma

arXiv:2609.29674v1cs.LGcs.AI

TL;DR

Sequential updates can leave a lasting cost because recent tasks influence the model more immediately. The paper analyzes this cost through forgetting and stationary population loss, finding a shared limit that separates intrinsic task-conflict loss from the sequential price.

  • Problem

    Because sequential updates favor recent tasks, the paper asks whether recency bias imposes a lasting cost on stationary population loss.

    Source: §1 · p. 1

  • Method

    The paper analyzes sequential task learning by relating forgetting to population loss and separating the stationary loss into intrinsic loss and a sequential price.

    Source: §1 · p. 2

  • Results

    Forgetting and population loss converge to the same stationary limit, which separates into intrinsic task-conflict loss and sequential price; on Jester, exact predictions match sampled task streams.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Separating intrinsic incompatibility from sequential price makes the additional cost of sequential access a distinct object of study.

    Source: §7 · p. 9

    The exact mean–variance expression, curvature-dependent bounds, and EWC price–rate law rely on quadratic task geometry and solvable random updates.

    Source: §7 · p. 8

Abstract

from arXiv · show

Sequential task updates are fundamental to continual learning, but their recency bias can impose a lasting performance cost. We study this cost in an overparameterized linear-regression model with i.i.d. task sampling. We prove that distribution-level forgetting and population loss converge to the same stationary limit. This common limit separates exactly into the intrinsic loss asymptotically attained by joint training and an additional sequential price, and in more homogeneous task geometries the two terms coincide, making the total loss twice that of joint training. We further analyze fixed-strength elastic weight consolidation (EWC) under general task curvatures and characterize its stationary sequential price at every regularization strength. Under strong regularization, the price decays inversely with EWC strength while convergence to stationarity slows at the same scale. On the Jester joke-rating dataset, the theory exactly quantifies both the sequential price generated by naturally conflicting user preferences and its reduction by EWC.

1 INTRODUCTION

Sequential updates give recent tasks more immediate influence, raising the question of whether recency bias leaves a lasting cost in stationary population loss. The paper connects forgetting to that loss, separates intrinsic task-conflict loss from the sequential price, and examines EWC and Jester preferences.

1 INTRODUCTION

  • Sequential learning gives recent tasks more immediate influence, while performance on past tasks can deteriorate over time.This recency bias motivates studying its effect on stationary population loss.Source: §1 · p. 1
  • Forgetting and population loss converge to the same stationary limit under i.i.d. task sampling, with their gap shrinking at O(T^-1).The result applies to conflicting tasks under the paper’s stability conditions.Source: §1 · p. 2
  • The stationary loss separates into intrinsic loss from task incompatibility and an additional sequential price; in more homogeneous geometries, the two terms are equal.The total stationary loss is then twice the loss asymptotically attained by joint training.Source: §1 · p. 2
  • Fixed-strength EWC under general task curvatures has a computable stationary sequential price, which falls as 1/λ under strong regularization while the horizon to stationarity grows as λ.The trade-off concerns stationary price and convergence speed.Source: §1 · p. 2
  • On Jester, the theory exactly predicts the sequential price from naturally conflicting user ratings and its reduction by EWC, matching sampled task streams.Source: §1 · p. 2

2 SETUP

The setup models sequential learning as exact fitting of individually realizable linear-regression tasks sampled i.i.d. It defines population loss and a stable stationary process, with joint training providing the benchmark for task incompatibility.

2 SETUP

  • Each task is an exact-fit linear regression problem, individually realizable even though different tasks need not share a solution.Source: §2 · p. 2
  • Sequential exact fitting chooses the minimum-change parameter satisfying the current task’s constraints, and incompatible tasks leave an affine update rather than a homogeneous projection.The update is based on projection onto the task’s constraint null space.Source: §2 · p. 3
  • Tasks arrive i.i.d., and collective coverage makes the update recursion contractive with a unique invariant distribution having finite second moment.The stationary parameter’s mean, covariance, and expected population loss are then defined from that distribution.Source: §2 · pp. 3–4
  • Population loss is expected task loss, and its minimum R⋆ is the intrinsic loss from incompatibility; joint training asymptotically attains R⋆.R⋆ is zero exactly when the task distribution admits a common solution almost surely.Source: §2 · p. 3

3 THE FORGETTING–LOSS EQUIVALENCE

Under the setup’s assumptions, distribution-level forgetting and expected population loss approach the same stationary limit. Their gap decays at O(T^-1), linking the forgetting plateau to stationary performance.

3 THE FORGETTING–LOSS EQUIVALENCE

  • Forgetting and expected population loss converge to the same stationary limit, with an O(T^-1) gap.Source: §3 · p. 4
  • Coupling the process to one initialized from the invariant distribution shows population loss converges to its stationary value through contraction of the recursion.Source: §3 · p. 4
  • To compare forgetting with population loss, the proof replaces a past task with an independent draw; the resulting trajectories contract together as later shared tasks accumulate.The contraction and second-moment bounds yield the rate for the averaged historical losses.Source: §3 · p. 4

4 INTRINSIC LOSS AND THE SEQUENTIAL PRICE

The stationary population loss separates into intrinsic incompatibility loss and an additional sequential price, defined by the stationary state. Under specified homogeneous curvature conditions, these terms are equal, so total loss is twice the joint-training loss.

4 INTRINSIC LOSS AND THE SEQUENTIAL PRICE

  • The stationary population loss equals R⋆ plus the sequential price, which captures the additional loss from sequential exact fitting.R⋆ is the minimum population loss for a single shared parameter and is asymptotically attained by joint training.Source: §4 · p. 5
  • The sequential price is the sum of squared stationary-mean bias and stationary fluctuations, measured in the population-loss geometry.The two contributions are C and tr(CΣ), respectively.Source: §4 · p. 5
  • With two equally likely one-dimensional tasks, sequential exact fitting yields stationary parameter values 0 and 1 equally often, so µ = 1/2; joint training gives w⋆ = 4/5.The tasks have different curvatures, so their sequential and joint-training solutions need not coincide.Source: §4 · p. 5
  • When task curvatures are proportional to their projection complements or share a common positive-definite curvature, the sequential price equals R⋆ and total loss is 2R⋆.The curvature-variation result allows task ranks and active subspaces to vary.Source: §4 · p. 6

5 EWC AND THE SEQUENTIAL PRICE

The section characterizes fixed-strength EWC's stationary sequential price and convergence to stationarity. Stronger regularization reduces the price but proportionally lengthens the task horizon needed for convergence.

5 EWC AND THE SEQUENTIAL PRICE

  • EWC uses a fixed average-task-curvature penalty to protect the preceding parameter in directions important across the task distribution.The analysis studies both stationary population loss and convergence to stationarity.Source: §5 · p. 6
  • For every λ > 0, fixed-strength EWC has a unique stationary distribution with finite second moment; finite-support task distributions permit exact loss calculation by linear equations.With diagonal task curvatures, the equations separate into independent scalar equations.Source: §5 · p. 6
  • At strong regularization, the stationary sequential price decreases as 1/λ while the task horizon required for stationarity grows linearly with λ.A constant-factor price reduction therefore requires a proportional increase in the task horizon.Source: §5 · p. 6
  • In the large-λ regime, EWC's stationary variance contribution is κΠ/λ + O(λ^-2), while squared stationary-mean bias is O(λ^-2).The price's leading-order term comes from variance.Source: §5 · p. 7

6 LEARNING FROM A STREAM OF USER PREFERENCES

On Jester user-rating tasks, sequential exact fitting produces a stationary loss above joint training, while EWC lowers that loss toward the joint-training value. The observed EWC reduction comes with a longer task horizon to the 1% criterion.

6 LEARNING FROM A STREAM OF USER PREFERENCES

  • Each Jester task is one user's observed ratings; the experiment independently samples users uniformly from the empirical user distribution.Dataset 1 contains 73,421 users and 100 jokes, and ratings are rescaled to [−1, 1].Source: §6 · p. 7
  • Jester's task ranks, observed subspaces, and curvature scales vary across users, while every joke has positive coverage and C ≻ 0.The task distribution falls within the curvature-variation regime of Corollary 4.2.Source: §6 · p. 8
  • Sequential exact fitting converges to G∞ = 0.5097 versus joint training's R⋆ = 0.2569, a sequential price of 0.2528, nearly half the stationary loss.Forgetting and population loss approach the same value, and sampled task streams closely follow the exact curves.Source: §6 · p. 8
  • At λ = 16, EWC lowers stationary population loss from 0.5097 to 0.2718, near R⋆ = 0.2569, while the 1% task horizon rises from 16 to 53.Exact finite-distribution curves agree with sampled streams; large-λ expressions capture the inverse price decrease and linear horizon increase.Source: §6 · p. 8
  • Appendix E reports a nonlinear Rotated-MNIST experiment measuring forgetting, population loss, sequential price, and consolidation beyond the exact linear model.Source: §6 · p. 8

7 DISCUSSION AND SCOPE

The paper studies sequential learning when tasks are independently sampled from a stable distribution, separating the cost of sequential access from loss intrinsic to task incompatibility. It specifies where its theory applies and highlights finite-horizon trade-offs and broader settings for future work.

7 DISCUSSION AND SCOPE

  • The framework models persistent task sources with i.i.d. arrivals, and does not cover streams whose tasks are selected by the learning history or a changing curriculum.The discussion gives user, client, robot, and generated-data streams as examples of the stable-source setting.Source: §7 · p. 8
  • The forgetting–loss equivalence relies on i.i.d. task generation and stable dynamics, while exact mean–variance and EWC price–rate results further rely on quadratic task geometry and solvable updates.Jester tasks fit the framework exactly; Rotated-MNIST illustrates measurable quantities under finite-step nonlinear training.Source: §7 · p. 8
  • The sequential price separates the loss remaining for a shared model from the additional loss associated with sequential updates and access.The population optimum is asymptotically attained by joint training.Source: §7 · p. 9
  • For finite streams, stationary price and time to stationarity must be considered together; extending the analysis to nonlinear training, replay, and history-dependent streams remains future work.The paper notes that lower stationary loss is useful only if reached within the available horizon.Source: §7 · p. 9

8 RELATED WORK

Prior work studies forgetting across continual-learning settings, task orders, and mitigation methods, including distributional analyses for compatible tasks. This paper extends the distributional viewpoint to conflicting optima and distinguishes intrinsic loss from the additional cost of sequential access.

8 RELATED WORK

  • Prior distributional analysis characterizes decay of forgetting for tasks with a shared solution; this work allows conflicting optima, yielding a generally nonzero limiting value.The shared-solution result is the zero-loss case.Source: §8 · p. 9

A ADDITIONAL RELATED WORK

  • Existing work examines continual-learning scenarios, forgetting metrics, replay, regularization, architectures, and how training regimes and evaluation assumptions affect observed forgetting.The paper positions its focus as quantifying additional stationary population loss for a specified sequential learner.Source: A ADDITIONAL RELATED WORK · p. 14
  • The related theory spans fixed-order and distributional analyses, task geometry, generalization, replay, regularization, and complementary neural, statistical-physics, and information-theoretic approaches.From Order to Distribution studies i.i.d. task draws under a shared solution (Xu & Ma, 2026); here conflicting optima produce a driven affine recursion.Source: A ADDITIONAL RELATED WORK · p. 14
  • The paper distinguishes intrinsic shared-model loss from the extra loss of sequential access and updates, and analyzes fixed-strength EWC under conflicting tasks.The EWC analysis characterizes stationary sequential price and convergence to stationarity under a fixed population-curvature penalty.Source: A ADDITIONAL RELATED WORK · pp. 14–15

B AUXILIARY RESULTS FOR THE SETUP

The auxiliary results establish the population benchmark and conditions for contraction and stationarity. They support the convergence and decomposition results, including bounds on the sequential price and cases where it equals intrinsic loss.

B AUXILIARY RESULTS FOR THE SETUP

  • R⋆ is the minimum population loss attainable by a shared parameter, and it is zero exactly when a common solution fits almost every task.An independent evaluation task makes expected task loss equal to expected population loss; equality with R⋆ requires the learner's output to equal w⋆ almost surely.Source: B AUXILIARY RESULTS FOR THE SETUP · p. 15
  • Positive-definite average curvature yields contraction from collective task coverage and a unique invariant law with finite second moment for the affine recursion.The auxiliary lemmas also restrict singular-curvature analysis to the effective parameter space without changing task losses.Source: B AUXILIARY RESULTS FOR THE SETUP · pp. 16–17

C.1 PROOF OF THEOREM 3.1

  • The proof couples the trajectory to a stationary one and uses contraction to show population loss approaches its stationary value, while task-age dependence decays.The coupling bound contracts geometrically in the mean-square analysis.Source: §C.1 · pp. 17–18

C.2 PROOF OF THEOREM 4.1

  • Centering the stationary state at w⋆ makes the cross term vanish, giving stationary loss G∞ = R⋆ + Δseq.The two terms are intrinsic loss and the sequential price.Source: §C.2 · pp. 18–19

C.3 PROOF OF COROLLARY 4.2

  • Curvature bounds place the sequential price between m E∥rτ∥2 and L E∥rτ∥2; under specified homogeneous geometries, Δseq = R⋆ and total loss is 2R⋆.In those cases, forgetting and population loss converge to 2R⋆.Source: §C.3 · p. 19

C.4 PROOFS FOR FIXED-STRENGTH EWC

The fixed-strength EWC analysis characterizes stationary moments and convergence, then applies the theory to exact calculations for sparse Jester user-rating tasks. Strong regularization lowers the stationary sequential price while slowing convergence on the same scale.

C.4 PROOFS FOR FIXED-STRENGTH EWC

  • For fixed-strength EWC, stationary mean and covariance satisfy finite linear systems, and diagonal task curvatures separate the moment equations coordinate by coordinate.The proof establishes contraction and a unique invariant distribution with finite second moment.Source: §C.4 · pp. 20–21
  • In the reported asymptotic regime, the contraction factor is ρλ = 1 − 2/λ + O(λ⁻²), so its leading departure from 1 is a decrease of 2/λ.Source: §C.4 · p. 21

D.1 DATA AND TASK DISTRIBUTION

  • Jester Dataset 1 removes the first, rating-count column and retains all 73,421 users and 4,136,360 observed ratings; tasks are independent uniform draws with replacement over users.The value 99 denotes a missing rating; dividing observed scores by ten maps the scale from [−10, 10] to [−1, 1].Source: §D.1 · p. 22

D.2 MODEL AND EXACT THEORETICAL QUANTITIES

  • Exact fitting overwrites rated coordinates and retains unrated ones; at stationarity each coordinate equals the rating from its most recent observing task.The stationary state has coordinate-wise mean and variance, whose components contribute to G∞.Source: §D.2 · p. 22

D.3 SAMPLED TASK STREAMS

The sampled streams estimate the stationary population loss and show that empirical stationary-loss estimates closely match the exact heterogeneous-curvature predictions. The Rotated-MNIST stress test specifies a nonlinear setting and measures forgetting, population loss, the sequential price, and time to stationarity.

D.3 SAMPLED TASK STREAMS

  • At task 1,000, sampled population loss was 0.50899 ± 0.00258 and forgetting was 0.50854 ± 0.00258; 20,000 streams estimated stationary loss at 0.50947 ± 0.00081.The first pair averages 2,000 streams; the stationary estimate uses streams of length 80.Source: §D.3 · p. 23

D.4 EWC CALCULATIONS

  • Across λ = 0, 1, 4, 16, and 64, every sampled stationary population-loss estimate was within 1.65 standard errors of its exact heterogeneous-curvature value.The sampled estimates average 1,000 task streams of length 1,000 at each listed strength.Source: §D.4 · p. 24

E BEYOND THE EXACT MODEL: ROTATED MNIST

  • The Rotated-MNIST experiment tests whether forgetting, population loss, sequential price, and their price–rate trade-off remain informative beyond the exact theoretical setting.It uses a neural network, finite-step task optimization, and consolidation acting on network outputs.Source: E BEYOND THE EXACT MODEL: ROTATED MNIST · p. 24

E.1 TASK DISTRIBUTION AND MODEL

  • Four rotations are sampled uniformly and independently, with fresh image subsets; the predictor is a two-hidden-layer, width-128 ReLU network trained for three epochs per task.The rotations are 0°, 45°, 90°, and 135°; each task draws 5,000 examples from 50,000 training images.Source: §E.1 · p. 24
  • Functional EWC applies a function-space consolidation objective aligned with the fixed population-curvature penalty analyzed in Section 5.For linear predictors, the function-space penalty equals the multi-output population-curvature penalty.Source: §E.1 · p. 24

E.2 METRICS AND STATIONARY ESTIMATES

  • Forgetting averages rotation-specific losses using their frequencies in earlier tasks, while population loss averages current test loss uniformly across rotations.Stationary loss is estimated by averaging population loss from tasks 2,000 through 3,000; the difference from joint-training loss defines the empirical sequential price.Source: §E.2 · p. 24
  • The relaxation time is the first task followed by ten consecutive recorded evaluations within a band set at 1% of the task-1-to-stationary loss gap.All 50 trajectories met this criterion; the latest crossing was task 2,060.Source: §E.2 · p. 25, Figure 3 · p. 25

E.3 RESULTS BEYOND THE LINEAR DYNAMICS

In nonlinear Rotated-MNIST, moderate functional EWC sharply reduces the sequential price and reaches stationarity fastest. At very high strength, stationary loss and relaxation time rise, while forgetting still approaches population loss.

E.3 RESULTS BEYOND THE LINEAR DYNAMICS

  • Over the final 1,000 tasks, forgetting and population loss differed by a mean absolute 7.95 × 10^-5, preserving their observed connection in the nonlinear stream.Source: §E.3 · p. 25
  • At λ = 4, stationary loss was 0.00643 ± 0.00001 versus joint-training loss of 0.00600 ± 0.00006; the sequential price fell 95.5%, from 0.00947 to 0.00043.Without consolidation, stationary population loss was 0.01547 ± 0.00029.Source: §E.3 · p. 25
  • Moderate consolidation reached stationarity fastest: mean relaxation times were 424 ± 33 tasks at λ = 2 and 468 ± 17 tasks at λ = 4.At λ = 64, stationary loss rose to 0.01295 ± 0.00025 and relaxation time to 1,392 ± 245 tasks.Source: §E.3 · p. 25
Loading 2609.29674v1…