PaperlayerView exact revision on arXiv

Source-linked AI summary

Flattening Sharpness for Dynamic Gradient Projection Memory Benefits Continual Learning

Danruo Deng, Guangyong Chen, Jianye Hao, Qiong Wang, Pheng-Ann Heng

arXiv:2110.04593v1cs.LG

TL;DR

Continual learning in DNNs is challenged by forgetting old tasks as new ones are learned, while prior work has rarely explored the weight-loss landscape. FS-DGPM dynamically adjusts the importance of past-task bases and regulates landscape flatness; the authors report consistent outperformance of state-of-the-art methods across benchmarks while alleviating forgetting.

  • Problem

    DNNs can abruptly lose performance on earlier tasks when learning new ones, and prior approaches have rarely explored weight-loss landscapes in addressing the sensitivity-stability dilemma.

    Source: §1 · pp. 1–2

  • Method

    FS-DGPM adaptively weights past-task bases so less important ones can be released, and uses Flattening Sharpness to regulate the weight-loss landscape across seen tasks.

    Source: §1 · p. 2, §6 · p. 9

  • Results

    FS-DGPM consistently outperforms state-of-the-art methods across benchmark datasets; on visual benchmarks it achieves the highest accuracy and second-lowest forgetting after GPM.

    Source: §1 · p. 2, §5.2 · p. 9

  • Takeaways & Limitations

    The authors report that lower loss with a flatter neighborhood often accompanies better continual-learning performance, and that FS-DGPM achieves high classification performance while alleviating forgetting.

    Source: §6 · pp. 9–10

    Whether a closer upper bound exists for continual-learning performance remains unresolved.

    Source: §6 · p. 10

Abstract

from arXiv · show

The backpropagation networks are notably susceptible to catastrophic forgetting, where networks tend to forget previously learned skills upon learning new ones. To address such the 'sensitivity-stability' dilemma, most previous efforts have been contributed to minimizing the empirical risk with different parameter regularization terms and episodic memory, but rarely exploring the usages of the weight loss landscape. In this paper, we investigate the relationship between the weight loss landscape and sensitivity-stability in the continual learning scenario, based on which, we propose a novel method, Flattening Sharpness for Dynamic Gradient Projection Memory (FS-DGPM). In particular, we introduce a soft weight to represent the importance of each basis representing past tasks in GPM, which can be adaptively learned during the learning process, so that less important bases can be dynamically released to improve the sensitivity of new skill learning. We further introduce Flattening Sharpness (FS) to reduce the generalization gap by explicitly regulating the flatness of the weight loss landscape of all seen tasks. As demonstrated empirically, our proposed method consistently outperforms baselines with the superior ability to learn new skills while alleviating forgetting effectively.

1 Introduction

Continual learning is difficult for DNNs because learning new tasks can sharply reduce performance on earlier ones, while retraining on all task data is inefficient. The paper links flatter weight-loss landscapes to better continual-learning performance and proposes FS-DGPM to balance retaining past knowledge with learning new tasks.

1 Introduction

  • DNNs can abruptly lose performance on earlier tasks as they learn new ones, while retraining an oracle on all task data is too inefficient for real-time learning.The introduction frames this as catastrophic forgetting in dynamically changing environments.Source: §1 · p. 1
  • Methods using small memories of old-task samples, gradients, or feature subspaces can still overfit their limited stored information.Separate task-specific branches avoid some interference but make the network redundant as the number of tasks grows.Source: §1 · p. 2
  • A flatter weight-loss landscape with lower loss is reported to often accompany better continual-learning performance, motivating its study for the sensitivity-stability dilemma.The authors describe this as their first contribution and point to Figures 1 and 3.Source: §1 · p. 2
  • FS-DGPM adaptively weights past-task bases so less important ones can be released, while Flattening Sharpness regulates the loss landscape across seen tasks.The authors report that the method consistently outperforms state-of-the-art methods across widely used benchmark datasets.Source: §1 · p. 2

2 Related Work

The related-work section groups continual-learning approaches into parameter-isolation, regularization-based, and memory-based methods. It contrasts how they allocate or protect knowledge from previous tasks and describes GPM as constraining gradient updates to reduce interference.

2 Related Work

  • Parameter-isolation methods assign separate parameter subsets to tasks, either by expanding the network or masking previously allocated parameters.Examples include Progressive Neural Network [33], Dynamically Expandable Networks [45], PackNet, and HAT.Source: §2 · p. 2
  • Regularization-based methods consolidate previous knowledge without replay, using distillation or penalties on changes to weights deemed important.Importance may be estimated through Fisher information, learning trajectories, or other approaches, including EWC, SI, and MAS.Source: §2 · pp. 2–3
  • Memory-based methods use stored information for replay or to constrain new-task optimization; GPM [34] restricts updates to directions orthogonal to previous-task inputs.The section notes that replay can overfit a stored subset and describes GEM [24] and A-GEM [4] as projecting gradients using memory samples.Source: §2 · p. 3

3 The Weight Loss Landscape of Continual Learning

The section formulates sequential task learning with limited replay memory, examines how weight-loss landscapes relate to stability and sensitivity, and tests flattening sharpness in vanilla ER.

3.1 Problem Formulation

  • Tasks arrive sequentially from task-specific datasets, and training minimizes empirical risk over seen tasks using limited memory for earlier-task data.The overall objective is to minimize risk across all tasks.Source: §3.1 · p. 3

3.2 Connection of Weight Loss Landscape and Continual Learning

  • The landscape is measured by plotting task training loss as weights move along a random direction with magnitude α, normalized filter-wise to remove scaling invariance.The visualization is repeated with ten different directions.Source: §3.2 · pp. 3–4
  • After new tasks, ER, La-MAML, and GPM all lose first-task stability as its landscape sharpens and loss rises; ER learns task five with the lowest loss and flattest landscape.The study uses a two-hidden-layer MLP on ten Permuted MNIST tasks and compares with an Oracle network retrained on all tasks.Source: Figure 1 · p. 4, §3.2 · p. 4

3.3 A Case Study of Flattening Sharpness for Vanilla ER

  • 90.44% average testing accuracy for FS-ER versus 86.16% for ER; FS-ER also yields lower loss and flatter landscapes after task five and all ten tasks.The authors report this as evidence that flattening sharpness benefits continual learning.Source: §3.3 · p. 4, Figure 3 · p. 4

4 Flattening Sharpness for Dynamic Gradient Projection Memory

FS-DGPM evaluates sharpness in the old-task subspace and dynamically adjusts the importance of its bases while updating weights. Its two-task theoretical analysis connects flatness and lower loss to continual-learning performance.

4 Flattening Sharpness for Dynamic Gradient Projection Memory

  • FS-DGPM combines flatness regulation with dynamically adjusted importance for bases spanning past-task gradients, aiming to improve new-task sensitivity while maintaining old-task stability.Each basis importance λ_i lies between 0 and 1.Source: §4 · p. 5

4.1 Sharpness Evaluation

  • Sharpness is evaluated by maximizing new-task training loss within the important old-task subspace, using replay-buffer examples to estimate that subspace.The authors describe this value as reflecting both difficulty learning the new task and interference with past tasks.Source: §4.1 · p. 5

4.2 Dynamic Gradient Projection Memory

  • Basis importance starts at 1 and adapts from task two: aligned gradients lower λ_i and release update limits, while interference raises λ_i and tightens them.The importance update considers current-task and replay-buffer batches.Source: §4.2 · p. 5

4.3 Weight Updating

  • Weights are updated by minimizing worst-case performance under perturbation, with update magnitude on each basis adjusted by its importance; replay data are reservoir-sampled and GPM bases recalculated by SVD.Setting η1 and η2 to zero reduces the method to GPM.Source: §4.3 · p. 6

4.4 Theoretical Understanding

  • In a two-task PAC-Bayes framing, a bound combines perturbation-averaged old-task loss with new-task loss and sharpness, motivating complementary updates and flatness control.The authors argue that optimizing worst-case flatness controls the bound and theoretically justifies lower loss and flatter landscapes for continual learning.Source: §4.4 · pp. 6–7

5 Experiments

The experiments evaluate FS-DGPM against continual-learning baselines on four image-classification datasets, using average accuracy and backward transfer. FS-DGPM achieves the highest average accuracy across the reported benchmark comparisons, while ablations examine flattening sharpness and dynamic basis weighting.

5 Experiments

  • The experiments compare FS-DGPM with state-of-the-art continual-learning methods on widely used benchmark datasets.Additional dataset, setup, baseline, and architecture details are provided in Appendices D and E.Source: §5 · p. 7

5.1 Experimental Setup

  • Experiments cover PMNIST, CIFAR-100 Split, CIFAR-100 Superclass, and TinyImageNet, with benchmark-specific task counts and network architectures.PMNIST uses a shared classifier; the other experiments use separate task classifiers.Source: §5.1 · p. 7
  • The comparisons include regularization, replay, gradient-constraint, and oracle multitask baselines; training uses SGD and dataset-specific epoch, batch-size, and replay-buffer settings.Average accuracy and backward transfer measure performance; negative backward transfer indicates forgetting.Source: §5.1 · pp. 7–8

5.2 Results and Discussion

  • 76.96% ± 0.77 average accuracy on 20 sequential PMNIST tasks is FS-DGPM’s best result, with least forgetting except GPM.FS-DGPM uses about 31% of GPM’s final memory size and achieves approximately 2.5% better accuracy.Source: §5.2 · p. 8, Table 2 · p. 8
  • 74.33%, 58.81%, and 70.41% are FS-DGPM’s best average accuracies across the three multi-head visual-classification benchmarks, outperforming all baselines.It also has the highest accuracy on each dataset and the second-lowest forgetting after GPM.Source: Table 1 · p. 8, §5.2 · p. 9

5.3 Ablation studies on FS-DGPM

  • Flattening sharpness improves GPM by approximately 1.0% on both CIFAR-100 datasets, while FS-DGPM has lower-loss, flatter landscapes than DGPM and La-DGPM.DGPM learns new tasks well but also leads to forgetting previous tasks; flattening sharpness alleviates this in the reported ablation.Source: §5.3 · p. 9, Figure 6 · p. 9, Table 3 · p. 9

6 Conclusion

The conclusion links better continual-learning performance to lower loss and flatter neighboring landscapes, and presents FS-DGPM as a method for adapting GPM and flattening the landscapes of seen tasks. Its PAC-Bayes analysis frames these goals in terms of old-task stability, new-task sensitivity, sharpness, and divergence from a prior.

6 Conclusion

  • Lower loss with a flatter neighboring landscape often accompanies better continual-learning performance, motivating the paper’s weight-landscape analysis.FS-DGPM adds soft importance weights that can release less-important GPM bases and applies Flattening Sharpness to regulate the landscapes of seen tasks.Source: §6 · p. 9
  • FS-DGPM is reported to achieve high classification performance while alleviating forgetting across image-classification tasks and network architectures.The paper leaves a closer upper bound on continual-learning performance and better dynamic adjustment for future work.Source: §6 · p. 10

A.1 PAC Bayesian Bound

  • The continual-learning PAC-Bayes bound depends on old-task stability, new-task sensitivity, expected sharpness on the new-task training set, and KL divergence to the prior.The stated bound applies to any prior distribution, perturbation distribution, and training-set-dependent choice of Δw.Source: §A.1 · p. 13
  • To protect old-task stability, the analysis constrains parameter updates to the complement of the important old-task subspace, following GPM.The resulting bound balances the old-task loss under perturbation against the new-task loss.Source: §A.1 · p. 13
  • FS-DGPM optimizes worst-case weight-landscape flatness to control the PAC-Bayes bound, providing the paper’s theoretical justification for its approach.The analysis connects lower loss and flatter landscapes with better continual-learning performance.Source: §A.1 · p. 13

A.2 Derivation for DGPM

The appendix derives how DGPM updates basis-importance values and describes updating GPM bases from replay representations. It also gives a procedure for visualizing task-specific weight-loss landscapes.

A.2 Derivation for DGPM

  • DGPM derives each basis-importance gradient by minimizing worst-case performance on a batch from the current task and a replay-buffer batch.The derivation uses a first-order Taylor approximation and treats first-order gradient terms as constants to ignore second-order derivatives.Source: §A.2 · p. 14
  • Each basis importance lies between 0 and 1; zero allows its basis to be fully released for new-task learning, and values are initialized at 1.A sigmoid with temperature 10 is applied after the gradient update.Source: §A.2 · p. 14

A.3 Pseudo-code for updating GPM

  • GPM bases are computed layer-wise by applying SVD to representations after each task, with replay examples used to form the representation matrices.The described update performs SVD in the entire gradient space and replaces the previously calculated bases, unlike GPM’s SVD in the subspace orthogonal to existing bases.Source: §A.3 · pp. 14–15
  • For fully connected layers, the representation matrix concatenates sampled layer inputs as columns; convolutional inputs are reshaped to form the matrix.The appendix states that GPM is a special case when η1 and η2 are set to 0.Source: §A.3 · p. 15

B.1 Pseudo-code for Visualization

  • The visualization procedure samples a Gaussian direction, applies filter-wise normalization, and plots each seen task’s training loss across perturbed weights.For each task descriptor, it evaluates the model at weights w + αd over a range of α values.Source: §B.1 · p. 15

B.2 More Results for Section 3.2: Connection of Weight Loss Landscape and Continual Learning

The appendix describes additional loss-landscape visualizations for continual-learning methods and provides the FS-ER training procedure. It specifies the visualization setup but does not report comparative outcomes in these passages.

B.2 More Results for Section 3.2: Connection of Weight Loss Landscape and Continual Learning

  • The additional landscape visualizations train GPM, La-MAML, ER, Oracle, and Finetune on 10-task Permuted MNIST using an MLP with two hidden layers.Each task uses 60,000 training samples, the replay buffer holds 1,000 samples, and visualizations are repeated across 10 random directions.Source: §B.2 · p. 15
  • In the landscape figures, rows track each task’s loss during model evolution, while columns indicate how many tasks the model has learned.The passage specifies the figure layout but does not state which method has the better landscape.Source: §B.2 · p. 16

C.1 Pseudo-code for FS-ER

  • FS-ER adds adversarial weight perturbation to Vanilla ER while training on current-task and replay-buffer batches.The pseudocode iteratively updates the perturbation, updates network weights using the perturbed weights, and adds current-task data to memory with reservoir sampling.Source: §C.1 · p. 16

C.2 More Results for Section 3.3: A Case Study of Flattening Sharpness for Vanilla ER

The section presents weight-loss landscapes for ER and FS-ER during new-task training.

C.2 More Results for Section 3.3: A Case Study of Flattening Sharpness for Vanilla ER

  • Figure 14 shows ER landscapes and Figure 15 shows FS-ER landscapes as new tasks are trained; no comparative outcome is stated here.The figures are identified as per-task landscapes.Source: §C.2 · p. 16

D.1 Datasets

The appendix reports dataset statistics in Table 4.

D.1 Datasets

D.2 Architecture

The appendix specifies the network architectures, baselines, and experimental settings used for the comparisons.

D.2 Architecture

  • The experiments use dataset-specific AlexNet-like, modified LeNet-5, or CNN architectures, with ReLU hidden units and multi-head outputs.The first two architectures are used for CIFAR-100 Split and Superclass, respectively; the CNN is used for TinyImageNet.Source: §D.2 · p. 17

D.3 Baselines

  • Baselines include regularization, episodic-memory, replay, and gradient-projection methods; Multitask jointly trains on all tasks as an average-accuracy upper bound.The listed methods include EWC, ICARL, GEM, ER, La-MAML, and GPM.Source: §D.3 · p. 17

D.4 GPU Device

  • Average training time per task is measured on an NVIDIA GeForce RTX 2080 Ti for the 20-Split CIFAR-100 Superclass experiment.Source: §D.4 · p. 17

D.5 Threshold Hyperparameter

  • The GPM threshold ϵ varies by dataset and architecture, with specified initial values and per-task increases.The settings range from ϵ = 0.9 for TinyImageNet to ϵ = 0.99 for PMNIST.Source: §D.5 · p. 18

D.6 List of Hyperparameters

  • Hyperparameters are selected by grid search; PMNIST uses five glances per incoming sample, and Multitask trains five epochs per task.Other methods use one epoch per PMNIST task; Table 5 lists hyperparameters for the baselines and proposed approach.Source: §D.6 · p. 18, Table 5 · p. 18

E Additional Experimental Results

Additional experiments examine perturbation-step and memory-size settings, alongside task-loss landscape visualizations and checklist disclosures.

E Additional Experimental Results

Checklist

  • The checklist reports code and reproducibility materials, training details, error bars, and compute-resource information, with further details in the appendices.Source: Checklist · p. 26

Related papers

Loading 2110.04593v1…