PaperlayerView exact revision on arXiv

Source-linked AI summary

How do Quadratic Regularizers Prevent Catastrophic Forgetting: The Role of Interpolation

Ekdeep Singh Lubana, Puja Trivedi, Danai Koutra, Robert P. Dick

arXiv:2102.02805v5cs.LG

TL;DR

Catastrophic forgetting limits DNN adaptation in continual and lifelong learning, yet the reasons quadratic regularization works remain insufficiently explained. The paper analyzes its parameter updates, showing that interpolation protects important parameters, then introduces explicit interpolation to address instability and deeper-layer limitations. The modification yields higher average accuracy and lower average forgetting.

  • Problem

    Catastrophic forgetting undermines DNN performance in continual and lifelong learning, while limited work explains why quadratic regularization prevents it.

  • Method

    The paper analyzes quadratic-regularization updates as parameter interpolation and introduces an explicit interpolation variant with relative importance scores.

  • Results

    6.2% higher average accuracy and 4.5% lower average forgetting are achieved with the explicit interpolation modifications.

  • Takeaways & Limitations

    Quadratic regularization protects past tasks by reducing effective learning rates for important parameters, while explicit interpolation avoids identified pitfalls and consistently improves performance.

  • Takeaways & Limitations

    A related comparison stores importance scores after every task, producing memory complexity proportional to model size times the number of tasks and violating sub-linear memory growth.

Abstract

from arXiv · show

Catastrophic forgetting undermines the effectiveness of deep neural networks (DNNs) in scenarios such as continual learning and lifelong learning. While several methods have been proposed to tackle this problem, there is limited work explaining why these methods work well. This paper has the goal of better explaining a popularly used technique for avoiding catastrophic forgetting: quadratic regularization. We show that quadratic regularizers prevent forgetting of past tasks by interpolating current and previous values of model parameters at every training iteration. Over multiple training iterations, this interpolation operation reduces the learning rates of more important model parameters, thereby minimizing their movement. Our analysis also reveals two drawbacks of quadratic regularization: (a) dependence of parameter interpolation on training hyperparameters, which often leads to training instability and (b) assignment of lower importance to deeper layers, which are generally the place forgetting occurs in DNNs. Via a simple modification to the order of operations, we show these drawbacks can be easily avoided, resulting in 6.2\% higher average accuracy at 4.5\% lower average forgetting. We confirm the robustness of our results by training over 2000 models in different settings. Code available at \url{https://github.com/EkdeepSLubana/QRforgetting}

1 INTRODUCTION

This paper analyzes why quadratic regularization mitigates catastrophic forgetting and identifies limitations that can undermine its effectiveness. It shows that explicit interpolation avoids these limitations, improving average accuracy and reducing average forgetting.

  • 1 INTRODUCTION: Quadratic regularization penalizes changes to parameters important for preserving performance on previously learned tasks.Its loss combines the current task’s loss with a regularization term involving previous parameter values and importance scores.
  • 1 INTRODUCTION: Quadratic regularizers prevent forgetting by interpolating current parameters with their values after the previous task, reducing effective learning rates for more important parameters.Less-important parameters remain freer to change, allowing adaptation to new tasks.
  • 1 INTRODUCTION: Improper hyperparameters can turn interpolation into extrapolation and cause training instability.The interpolation operation therefore depends on training hyperparameters.
  • 1 INTRODUCTION: Standard importance definitions may assign lower importance to deeper layers, even though changes to deeper layers are a primary source of forgetting in DNNs.This reduces the effectiveness of quadratic regularizers against catastrophic forgetting.
  • 1 INTRODUCTION: 6.2% higher average accuracy and 4.5% lower average forgetting result from explicit interpolation modifications to quadratic regularization.The analysis was verified by training more than 2000 models in different settings.

2 RELATED WORK

Prior work proposes quadratic regularizers and other approaches to mitigate catastrophic forgetting, while a smaller body of research studies why these methods work. Relevant analyses indicate that forgetting primarily involves adaptation of deeper layers.

  • 2 RELATED WORK: The paper focuses on quadratic regularization methods, while discussion of other catastrophic-forgetting techniques is provided in the appendix.This related-work scope prioritizes approaches most relevant to the paper’s analysis.
  • 2 RELATED WORK: Quadratic regularization methods differ in how they assign parameter importance, using Fisher information, loss-contribution measures, output sensitivity, or Hessian approximations.Examples include EWC, SI, MAS, RWalk, and block-diagonal Hessian regularization.
  • 2 RELATED WORK: Studies of catastrophic forgetting have analyzed its causes and mitigation, including representational similarity across models trained on different numbers of tasks.Ramasesh et al. (2021) show that catastrophic forgetting primarily arises from adaptation of deeper layers to new tasks.

3 THE ROLE OF INTERPOLATION IN QUADRATIC REGULARIZATION

Quadratic regularization prevents forgetting by interpolating current parameters with their previous-task values, which reduces the effective learning rates of important parameters over repeated updates. The analysis identifies instability from extrapolation and weaker protection in deeper layers, motivating balanced importance assignments.

  • 3 THE ROLE OF INTERPOLATION IN QUADRATIC REGULARIZATION: Quadratic regularization updates parameters using task-specific derivatives while interpolating them with their previous-task values, limiting drift in important parameters.This short-term interpolation preserves prior-task performance while allowing task-specific learning.
  • 3 THE ROLE OF INTERPOLATION IN QUADRATIC REGULARIZATION: Repeated interpolation reduces a parameter’s effective learning rate according to its importance, while low-importance parameters remain free to adapt to new tasks.The effective learning rate multiplies η by powers of 1 − ηλα(k) n−1.
  • 3.1 LIMITATIONS IN QUADRATIC REGULARIZATION: Quadratic regularizers can become extrapolators when ηλα(k) n−1 is negative or exceeds 1, producing unstable training and severe performance degradation.Negative importance can cause exponentially large parameter updates, while unsuitable hyperparameters can place parameters outside the interpolation regime.
  • 3.1.1 EXTRAPOLATION AND TRAINING INSTABILITY: A few violating parameters can destabilize training: 5 for EWC, 26 for MAS, 12 for SI, and 31 for RWalk among 1.2 million parameters, whereas clamping avoids instability.Clamping reassigns violating parameters an importance score of 1/ηλ and produces good performance.
  • 3.1.1 EXTRAPOLATION AND TRAINING INSTABILITY: Using a small λ can avoid instability but weakens interpolation, yielding performance similar to plain fine-tuning; an overly large λ can instead freeze substantial parts of the model.Thus, stable training requires a hyperparameter range that preserves interpolation without preventing adaptation.
  • 3.1.2 DISPARATE IMPORTANCE ASSIGNMENT: Popular importance definitions often assign lower importance to deeper layers, allowing them to change more readily even though deeper-layer adaptation is identified as a primary source of forgetting.Balanced assignments rescale layerwise importance so each layer has the same average importance, and are evaluated using CKA across layers.

4 ADDRESSING LIMITATIONS IN QUADRATIC REGULARIZATION

The proposed modification separates task-specific updates from parameter interpolation, ensuring interpolation remains stable and importance remains comparable across layers. Across three datasets, explicit-interpolation variants outperform plain quadratic regularizers, especially when tasks are short, unbalanced, or numerous.

  • 4 ADDRESSING LIMITATIONS IN QUADRATIC REGULARIZATION: Explicit interpolation splits each iteration into a task-specific gradient update followed by interpolation toward the previous task’s parameters.This changes the order of operations and makes the previously implicit interpolation explicit.
  • 4 ADDRESSING LIMITATIONS IN QUADRATIC REGULARIZATION: The interpolation amount depends only on relative-importance vector R_j, removing direct dependence on training hyperparameters and helping avoid instability from gradient-descent dynamics.The proposed relative importance compares importance for previous tasks with importance across all tasks.
  • 4 ADDRESSING LIMITATIONS IN QUADRATIC REGULARIZATION: Because R_j ranges from 0 to 1, more important parameters remain closer to their previous values while less important parameters adapt more strongly to the current task.Relative importance also avoids problems caused by assigning lower absolute importance to deeper layers.
  • 4.1 EXPERIMENTAL EVALUATION: 6.2% average accuracy improvement, reaching up to 23%, is obtained by explicit-interpolation variants over plain quadratic regularizers in the 1-epoch setting.The comparison covers EWC, MAS, SI, and RWalk across CIFAR-100, Oxford-Flowers, and Caltech-256.
  • 4.1 EXPERIMENTAL EVALUATION: Explicit-interpolation variants are most beneficial on unbalanced datasets and longer task sequences, whereas plain regularizers can remain competitive on CIFAR-100.On Oxford-Flowers and Caltech-256, short tasks provide fewer updates for effective hyperparameter search, and plain regularizers can produce up to 10% forgetting.
  • 4.1 EXPERIMENTAL EVALUATION: The modified regularizers achieve lower forgetting than replay methods but still underperform them in average accuracy.The comparison includes A-GEM and ER-Reservoir, alongside plain fine-tuning and the quadratic-regularization variants.

5 CONCLUSION

The paper explains quadratic regularization through short-term parameter interpolation and long-term learning-rate reduction for important parameters. It identifies hyperparameter-driven extrapolation and low deeper-layer importance as pitfalls, then proposes explicit interpolation to address them.

  • 5 CONCLUSION: Quadratic regularizers interpolate current and past parameter values in the short term, then reduce the effective learning rates of important parameters over multiple iterations.This limits movement of parameters that preserve performance on earlier tasks.
  • 5 CONCLUSION: The analysis identifies two pitfalls: inappropriate hyperparameters can cause extrapolation, and common importance definitions assign lower importance to deeper layers.The first problem can destabilize training, while the second concerns layers where forgetting commonly occurs.
  • 5 CONCLUSION: Explicit-interpolation quadratic regularizers are proposed to circumvent these pitfalls and obtain consistently better performance.The paper reports validation across more than 2000 trained models.
  • 5 CONCLUSION: The paper focuses specifically on quadratic regularization, while identifying replay-based mechanisms as a direction for future analysis.Related approaches discussed include knowledge distillation, Bayesian modeling, experience replay, and model expansion.

C DERIVATION OF RELATIONSHIP BETWEEN PARAMETERS OF nTH AND (n −1)TH TASK

The derivation unrolls quadratic-regularization updates across training iterations to establish the relationship between parameters for consecutive tasks. It relies on fixed previous-task importance scores and verifies the relationship inductively.

  • C DERIVATION OF RELATIONSHIP BETWEEN PARAMETERS OF nTH AND (n −1)TH TASK: The analysis treats η as the learning rate, λ as the regularization constant, and α_n−1 as importance scores computed from tasks 0 through n−1.Those importance scores remain constant while task n is trained.
  • C DERIVATION OF RELATIONSHIP BETWEEN PARAMETERS OF nTH AND (n −1)TH TASK: The derivation starts from the quadratic-regularization update and unrolls it across iterations to relate current parameters to the previous task’s final parameters.The nth-task model is initialized with the parameterization obtained at the end of task n−1.
  • C DERIVATION OF RELATIONSHIP BETWEEN PARAMETERS OF nTH AND (n −1)TH TASK: The induction establishes that the parameter relationship holds at every training iteration after being verified for the initial iterations.The argument assumes the relationship at iteration i and shows it also holds at iteration i+1.

D EXPERIMENTAL SETUP

The experiments use reported hyperparameter settings for multiple methods and datasets across two training regimes.

  • D EXPERIMENTAL SETUP: Hyperparameters are reported for different methods and datasets in both 1-epoch, batch-size-10 and 30-epoch, batch-size-256 settings.The supplied tables summarize the selected configurations for each regime.
  • D EXPERIMENTAL SETUP: The experimental setup includes evaluations of continual-learning methods such as EWC, MAS, SI, RWalk, Van., Rand, A-GEM, and ER.

D.1 DATASETS

The study evaluates continual learning across CIFAR-100, Oxford-Flowers, and Caltech-256, partitioning each dataset into sequential class-incremental tasks.

  • D.1 DATASETS: The experiments use CIFAR-100, Oxford-Flowers, and Caltech-256 as continual-learning datasets.
  • D.1 DATASETS: CIFAR-100 is divided into 10 tasks with 10 sequential classes each, using 500 training and 100 test samples per class.
  • D.1 DATASETS: Oxford-Flowers is divided into 17 tasks with 6 sequential classes per task and has few, imbalanced samples per class.
  • D.1 DATASETS: Caltech-256 is divided into 32 tasks with 8 sequential classes per task and has imbalanced class sizes averaging 95 training and 27 test samples.

D.2 TRAINING SETUP

The study trains a six-layer CNN in a multi-head, streaming-style protocol, using pretraining, SGD, grid-searched hyperparameters, and deliberately varied datasets.

  • D.2 TRAINING SETUP: Experiments use a six-layer VGG-16-like CNN with convolutional layers, max-pooling, and ReLU activations.
  • D.2 TRAINING SETUP: Models are pretrained on CIFAR-10 because quadratic regularization requires an initially discriminative parameterization.
  • D.2 TRAINING SETUP: All experiments use a multi-head setting consistent with prior quadratic-regularization and continual-learning evaluations.
  • D.2 TRAINING SETUP: The setup varies dataset complexity, class balance, samples per class, and classes per task so random performance remains low at 10–16%.
  • D.2 TRAINING SETUP: Training uses SGD with momentum 0.9, one epoch per task, batch size 10, and final-model evaluation after hyperparameter search on the first three tasks.
  • D.2 TRAINING SETUP: The regularization constant λ is selected from a broad grid spanning 10^-5 to 10^4, reflecting strong hyperparameter sensitivity.

E MORE RESULTS ON EXPERIMENTS FROM SECTION 3.1

Additional experiments show that instability from negative importance scores and interpolation-condition violations persists across datasets, even when only a small fraction of parameters is affected.

  • E MORE RESULTS ON EXPERIMENTS FROM SECTION 3.1: Negative importance scores trigger extrapolation and unstable training across all datasets considered.
  • E MORE RESULTS ON EXPERIMENTS FROM SECTION 3.1: For all-positive importance scores, training is always stable, whereas even a few negative scores often destabilize training.
  • E MORE RESULTS ON EXPERIMENTS FROM SECTION 3.1: Violations of the interpolation inequality produce instability across CIFAR-100, Oxford-Flowers, and Caltech-256.
  • E MORE RESULTS ON EXPERIMENTS FROM SECTION 3.1: Across 1.2 million parameters, instability first appears after only 35 EWC, 4 MAS, 8 SI, or 17 RWalk violations in one reported setting.
  • E MORE RESULTS ON EXPERIMENTS FROM SECTION 3.1: In another reported setting, instability first appears after 1 EWC, 4 MAS, 3 SI, or 6 RWalk violations across 1.2 million parameters.

E.3 DISPARATE IMPORTANCE ASSIGNMENT

The experiments show that commonly used importance definitions assign lower importance to deeper layers, while balanced importance regularizers preserve representational similarity across layers and reduce forgetting.

  • DISPARATE IMPORTANCE ASSIGNMENT: Lower importance assigned to deeper layers is observed across CIFAR-100, Oxford-Flowers, and Caltech-256, leaving those layers more able to adapt to recent tasks.The paper links this imbalance to forgetting because changes in deeper layers majorly cause forgetting of previously learned tasks.
  • DISPARATE IMPORTANCE ASSIGNMENT: Average parameter importance is reported by layer for Oxford-Flowers and Caltech-256, documenting the layerwise importance patterns examined in this section.These datasets correspond to Figures 8 and 9.
  • BALANCED IMPORTANCE SCORES: Vanilla and Random assign comparable importance scales across layers, addressing the biased assignment found in popular quadratic regularizers.Vanilla uses unit importance scores, whereas Random uses uniformly picked random scores.
  • BALANCED IMPORTANCE SCORES: CKA shows high representational similarity across all layers for Vanilla and Random between models trained on the first task and on all tasks.The figure also reports average accuracy and average forgetting in its legend.

F MORE RESULTS ON EXPERIMENTS FROM SECTION 4.1

Additional experiments compare plain quadratic regularizers with explicit interpolation variants across three datasets and also extend the comparison to ResNet-18. The reported tables state that explicit interpolation variants consistently outperform their plain counterparts.

  • RESNET-18 RESULTS: The same comparison is provided for ResNet-18, using one-third the number of filters in each layer and hyperparameters selected by grid search.The search uses the first three tasks, with other training configurations unchanged from the earlier setup.
  • COMPARISON OF VARIANTS: Explicit interpolation variants consistently outperform plain quadratic regularizers across CIFAR-100, Oxford-Flowers, and Caltech-256.Table 5 compares average accuracy and average forgetting for EWC, MAS, SI, and RWalk over 10, 17, and 32 tasks, respectively.
  • RESNET-18 RESULTS: Table 6 reports average accuracy and average forgetting for plain and explicit interpolation variants across the three datasets on ResNet-18.For each regularizer, the better-performing variant is marked in bold.
  • DETAILED RESULTS: The detailed results include standard deviations alongside average accuracy and average forgetting, supplementing the main-paper comparison.These statistics are provided because the main paper omitted standard deviations due to space constraints.
Loading 2102.02805v5…