PaperlayerView exact revision on arXiv

Source-linked AI summary

A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix

Thang Doan, Mehdi Bennani, Bogdan Mazoure, Guillaume Rabusseau, Pierre Alquier

arXiv:2010.04003v2cs.LGcs.AIstat.ML

TL;DR

Catastrophic forgetting remains a central challenge, and relatively little theoretical work has examined it. This paper analyzes forgetting and projection methods under the NTK framework, relating forgetting to task similarity and proposing PCA-OGD. Experiments support the analysis and report that PCA-OGD efficiently mitigates forgetting, subject to the analysis’s assumptions.

  • Problem

    Catastrophic forgetting remains an unsolved challenge, and theoretical study of the phenomenon has been limited.

    Source: §1 · p. 1

  • Method

    The paper analyzes forgetting for projection methods under the NTK framework and proposes PCA-OGD, which compresses relevant information into fewer principal components.

    Source: §1 · pp. 1–2, §4.3 · p. 5

  • Results

    Experiments support the findings that the NTK overlap matrix is crucial to forgetting and that PCA-OGD efficiently mitigates it.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The analysis links forgetting to task similarity and shows PCA-OGD can leverage data structure to mitigate it.

    Source: §1 · p. 2, §6 · p. 8

    The analysis assumes overparameterization and a constant NTK, so changes in the NTK over time remain unaccounted for.

    Source: §6 · p. 8, §8.6 · p. 16

Abstract

from arXiv · show

Continual learning (CL) is a setting in which an agent has to learn from an incoming stream of data during its entire lifetime. Although major advances have been made in the field, one recurring problem which remains unsolved is that of Catastrophic Forgetting (CF). While the issue has been extensively studied empirically, little attention has been paid from a theoretical angle. In this paper, we show that the impact of CF increases as two tasks increasingly align. We introduce a measure of task similarity called the NTK overlap matrix which is at the core of CF. We analyze common projected gradient algorithms and demonstrate how they mitigate forgetting. Then, we propose a variant of Orthogonal Gradient Descent (OGD) which leverages structure of the data through Principal Component Analysis (PCA). Experiments support our theoretical findings and show how our method can help reduce CF on classical CL datasets.

1 Introduction

Continual learning learns from incoming data while reusing prior knowledge, but catastrophic forgetting remains unresolved. The paper develops an NTK-based analysis linking forgetting to task similarity and examines projection methods, including PCA-OGD.

1 Introduction

  • Catastrophic forgetting—past solutions degrading as new tasks arrive—remains a central unresolved challenge in continual learning.The introduction notes that prior work has studied forgetting empirically or proposed quantitative metrics, with relatively little theoretical study.Source: §1 · p. 1
  • The paper analyzes catastrophic forgetting theoretically for projection algorithms, building on NTK-based continual-learning work.It considers a family including OGD and GEM.Source: §1 · p. 1
  • The paper defines forgetting using source–target task similarity and examines its special case in the NTK regime.Source: §1 · p. 1
  • Unlike SGD, projection methods reduce forgetting by projecting source and target tasks onto a residual subspace.Source: Figure 1 · p. 2
  • The NTK overlap matrix captures source–target task alignment, which the paper identifies as a critical component of forgetting.The introduction says the matrix reduces to the angle between the tasks.Source: §1 · pp. 1–2
  • PCA-OGD extends OGD by compressing relevant information into fewer principal components and is advantageous when datasets have dependence patterns between tasks.Source: §1 · p. 2

2 Related Works

Prior work has developed diverse methods and empirical analyses of catastrophic forgetting, but its underlying theory remains limited. This paper situates its analysis within NTK-based continual-learning theory and extends the setting to finite memory.

2 Related Works

  • Existing approaches include regularization, memory and projection, and parameter-isolation methods, but their underlying theory remains unclear.The related-work discussion also notes methods for identifying memorable examples and surveys of the field.Source: §2 · p. 2
  • Prior studies examined task similarity, network representations, neural variability, training regimes, and evaluation measures, with few theoretical works confirming empirical evidence of forgetting.Source: §2 · p. 2
  • The paper builds on Bennani et al.’s NTK framework for infinite-memory continual learning and relaxes that constraint to finite memory.Source: §2 · p. 2

3 Preliminaries

The preliminaries establish notation, describe continual learning as sequential supervised tasks with restricted access to prior samples, and introduce an NTK framework for analyzing forgetting and task similarity.

3.1 Notations

  • The notation section establishes symbols for dot products, task indices, learnable parameters, and end-of-task weights.It also identifies N as the natural numbers and R as a space.Source: §3.1 · p. 2

3.2 Continual Learning

  • Continual learning is framed as a stream of supervised tasks for which a predictor should make accurate predictions, with prior samples unavailable unless retained in a memory buffer.Source: §3.2 · p. 2

3.3 NTK framework for Continual Learning

  • Under the NTK framework, constant feature maps support a recursive kernel-regression view; the paper uses this framework to study forgetting and quantify task similarity through eigenvalues and singular values.The cited prior framework gives OGD guarantees in the infinite-memory setting.Source: §3.3 · p. 3

4 Analysis of Catastrophic Forgetting in finite memory

The paper analyzes catastrophic forgetting (CF) in finite-memory continual learning under the NTK framework, where task overlap is a central factor. It defines forgetting, studies SGD and projection methods, and introduces PCA-OGD.

4.1 A definition of Catastrophic Forgetting under the NTK regime

  • CF measures squared prediction change on a source task’s test set after training on subsequent tasks.Under the NTK regime, the expression is linear in the feature map and the change in weights from the source task to the target task.Source: §4.1 · p. 3

4.2 High correlations across tasks induce forgetting for vanilla SGD

  • Vanilla SGD’s forgetting depends on source-task and later-task subspace overlap, with aligned tasks carrying greater forgetting risk.The overlap matrix’s diagonal elements are cosines of principal angles between task subspaces.Source: §4.2 · p. 4
  • High overlap between source and later-task subspaces increases the NTK overlap matrix norm and implies greater forgetting, which can erase prior knowledge along shared components.The overlap is characterized by cosines of principal angles between task subspaces.Source: §4.2 · p. 4

4.3 The effectiveness of the orthogonal projection against Catastrophic Forgetting

  • OGD projects updates orthogonally to previous-task feature maps, while GEM projects to previous-task loss gradients to constrain their training losses.OGD and GEM use distinct projection spaces.Source: §4.3 · p. 5

(Differences with the vanilla case SGD are highlighted

  • GEM-NT projects onto previous-task gradients at every step, unlike GEM’s conditional projection, to decouple forgetting from transfer.Its projection applies regardless of the sign of the gradient dot product.Source: §4.3 · p. 5
  • GEM-NT weights projections by residuals, so well-learned tasks can contribute less and may be unlearned when their residuals are zero.This contrasts with projecting orthogonally to the previous model’s feature maps.Source: (Differences with the vanilla case SGD… · pp. 5–6

4.4 PCA-OGD: leveraging structure by projecting orthogonally to the top d principal directions

  • PCA-OGD stores top principal directions and projects updates orthogonally to them, with the forgetting bound including the (d + 1)-th singular value.PCA is helpful when eigenvalues decrease exponentially, but uniformly spread eigenvalues require all components and more memory; the stated worst case has OGD outperforming PCA-OGD.Source: §4.4 · pp. 6–7

5 Experiments

Experiments relate lower NTK overlap eigenvalues to less forgetting and find that PCA-OGD can use memory more efficiently than OGD on Rotated MNIST. Across baselines, PCA-OGD obtains results comparable to A-GEM, while its efficiency is weaker on unstructured Permuted MNIST.

5.1 Low eigenvalue of the NTK overlap matrix induces smaller drop in performance

  • 7% to 10% less drop in performance: PCA-OGD improves on OGD as the maximum NTK overlap eigenvalue varies on Rotated MNIST.Lower eigenvalues correspond to a smaller accuracy drop between tasks 1 and 15.Source: §5.1 · p. 7, Figure 2 · p. 7

5.2 PCA-OGD reduces forgetting by efficiently leveraging structure in the data

  • PCA-OGD has lower NTK overlap eigenvalues than OGD and GEM-NT, with the gap increasing at memory size 200; 200 components explain 90.71% of variance.SGD has higher eigenvalues than the projection methods because it does not project source or target tasks.Source: §5.2 · p. 7
  • On Rotated MNIST, PCA-OGD has comparable final accuracy to OGD with half the memory: 100 components versus 200.PCA-OGD’s marginal gains are roughly constant from memory 25 to 50 and 50 to 100, while OGD’s largest gain is from 100 to 200.Source: §5.2 · p. 8, Figure 4 · p. 8
  • PCA-OGD is less efficient on unstructured Permuted MNIST, where it must retain more principal components than on structured data.The experiment uses Permuted MNIST as a counterexample with no dataset structure.Source: §5.2 · p. 8

5.3 General performance of PCA-OGD against baselines

  • PCA-OGD obtains comparable results to A-GEM across the reported baselines, while A-GEM updates the NTK and PCA-OGD and OGD store past gradients.The reported Split CIFAR pattern is that the NTK stops varying after one task, unlike its linear increase for MNIST datasets.Source: §5.3 · p. 8

6 Conclusion

The paper analyzes catastrophic forgetting under the NTK regime, relating it to task similarity through the NTK overlap matrix. Its experiments support the analysis and show PCA-OGD mitigating forgetting, but the analysis assumes overparameterization and leaves time-varying NTKs for future work.

6 Conclusion

  • The analysis covers SGD and projection methods OGD, GEM-NT, and PCA-OGD, and uses the NTK overlap matrix to quantify how task similarity relates to forgetting.The conclusion reports experimental support for the findings and says PCA-OGD efficiently mitigates forgetting.Source: §6 · p. 8
  • The analysis relies on overparameterization; accounting for NTK change over time remains a next step.Source: §6 · p. 8

8 Appendix

The appendix sketches proofs using the NTK feature map, a telescopic decomposition with SVD, and principal angles between task subspaces.

8.1 Proof of Lemma 1

  • The proof of Lemma 1 uses a result from Bennani et al. and the NTK kernel identity relating the kernel to the gradient feature map.Source: §8.1 · p. 11

8.2 Proof of Theorem 1

  • Theorem 1’s proof decomposes drift from source to target tasks into a telescopic sum and uses SVD to factorize an expression.Source: §8.2 · p. 11

8.3 Proof of Corolary 1

  • The Corollary 1 proof bounds forgetting using principal angles between source and target subspaces; singular values encode cosines of those angles.Source: §8.3 · p. 12

8.4 Proof of Corollary 2

The appendix gives proof sketches for GEM-NT and PCA-OGD results, including a finite-memory convergence statement for PCA-OGD.

8.4 Proof of Corollary 2

  • The Corollary 2 proof begins from a corresponding result under Corollary 4, with the kernel definition as the stated difference.Source: §8.4 · p. 12

8.5 Forgetting for PCA-OGD

  • The finite-memory convergence result assumes an invertible kernel matrix with weight decay λ > 0 and a learning-rate bound for each task.Source: §8.5 · p. 13
  • The PCA-OGD proof proceeds by induction, and its optimization objective is stated to be quadratic with a positive-definite Hessian and a unique minimum.Source: §8.5 · p. 14
  • PCA-OGD adds the first d eigenvectors of each task’s feature-map Gram matrix to its retained subspace, then projects the next task’s update orthogonally to those components.Source: §8.5 · p. 15

8.6 Algorithms summary

The algorithms differ in the matrix and stored information used for projection. PCA-OGD compresses information with principal components, while OGD and PCA-OGD assume a constant NTK that may change in practice.

8.6 Algorithms summary

  • Table 2 summarizes the overlap-matrix property associated with each method's mitigation of catastrophic forgetting.Source: Table 2 · p. 16
  • GEM-NT, OGD, and PCA-OGD differ from SGD in the matrix X, which contains feature maps or gradient-loss functions.Source: §8.6 · p. 16
  • GEM-NT and OGD store randomly sampled elements at each task's end, while PCA-OGD stores principal components of the feature maps through PCA.Source: §8.6 · p. 16
  • OGD and PCA-OGD assume a constant NTK, so their projections may use outdated vectors when the NTK changes.Source: §8.6 · p. 16

8.7 The counter-example of Permuted MNIST: no structure

Permuted MNIST removes extra-task correlations and patterns, limiting the advantage of PCA-OGD. In this setting, PCA-OGD and OGD have comparable final performance except on the first task.

8.7 The counter-example of Permuted MNIST: no structure

  • Permuted MNIST applies a different uniform pixel permutation to each task, removing extra-task correlations and patterns.Source: §8.7 · p. 17
  • As memory increases, the overlap-matrix eigenvalues remain small and change little; PCA-OGD therefore offers little advantage over OGD in this setting.The passage attributes this to weak inter-task correlations and eigenvalues spread more uniformly, requiring more components to explain variance.Source: §8.7 · p. 17
  • PCA-OGD has comparable final accuracy to OGD on Permuted MNIST, except on the first task.The account attributes this to needing many components to explain variance, making random selection like OGD comparable; 50 components explain 50% of variance, versus 81% for Rotated MNIST and 72% for Split CIFAR.Source: §8.7 · p. 17

8.8 Comparison PCA-OGD versus OGD

PCA-OGD's memory advantage is most evident on structured data: it matches OGD's performance with half the memory on Rotated MNIST, while Permuted MNIST shows comparable results except on the first task. The experiments also examine dataset structure and NTK variation.

8.8 Comparison PCA-OGD versus OGD

8.9 Structure in the data

  • With total memory 200, 14 kept components explain 38.84% of Permuted MNIST variance and 71.06% of Rotated MNIST variance; the text attributes the gap to random permutations breaking data structure.Source: §8.9 · p. 19

8.10 NTK changes

  • For the tested architecture, the NTK increases linearly on MNIST datasets but is approximately constant after the first task on Split CIFAR.The authors say the Split CIFAR observation validates the constant-NTK assumption and explains PCA-OGD's good result on that dataset.Source: §8.10 · p. 20

8.11 Experimental setup and general performance

  • Experiments use Permuted MNIST, Rotated MNIST, Split MNIST, and Split CIFAR, comparing PCA-OGD with SGD, A-GEM, and OGD using average accuracy and forgetting measures.Source: §8.11 · pp. 21–22

8.12 Worst-case scenario for PCA-OGD: data spread uniformly along all directions

When data variance is spread across directions, PCA-OGD can perform worse than OGD; the appendix also describes GEM-NT pseudocode, computational overhead, and overlap-matrix eigenvalue trends.

8.12 Worst-case scenario for PCA-OGD: data spread uniformly along all directions

  • In a uniform-direction toy setting, the first 25 components explain only 26% of variance, and PCA-OGD has drastic testing-loss variation while OGD shows practically no forgetting.The experiment trains on 15 tasks with total memory 25 per task and reports the first nine tasks.Source: §8.12 · p. 24

8.13 Pseudo-code for GEM-NT

  • GEM-NT pseudocode takes a task sequence, learning rate, and number of components to keep as inputs.Source: §8.13 · p. 25

8.14 Computational overhead of PCA-OGD

  • PCA is the only additional step over OGD; with n = M/T, the authors state that both methods have the same asymptotic complexity.The stated PCA complexity is O(n^2p), with n ≪ p; the Gram-Schmidt step is O(M^2p).Source: §8.14 · p. 25

8.15 Eigenvalues evolution of the NTK overlap matrix between the source and target task

Loading 2010.04003v2…