PaperlayerView exact revision on arXiv

Source-linked AI summary

Toward Understanding Catastrophic Forgetting in Continual Learning

Cuong V. Nguyen, Alessandro Achille, Michael Lam, Tal Hassner, Vijay Mahadevan, Stefano Soatto

arXiv:1908.01091v1cs.LGcs.CVstat.ML

TL;DR

Continual learning must adapt to new tasks without revisiting old data, but deep models can forget previously learned tasks. This paper uses task-space modeling and correlation analysis to examine which task-sequence properties relate to error. Total complexity is strongly and positively correlated with error for some algorithms, while sequential heterogeneity has little or no correlation and can be negatively correlated in some cases.

  • Problem

    The paper asks which properties of a task sequence influence continual-learning error, given the challenge of catastrophic forgetting in deep models.

    Source: §1 · p. 1

  • Method

    The authors use task-space modeling to specify task-sequence properties and correlation analysis to relate them to measured forgetting, applying the procedure to total complexity and sequential heterogeneity.

    Source: §1 · p. 2, §4 · p. 3

  • Results

    Total complexity strongly and positively correlates with actual error for some algorithms, while sequential heterogeneity has little or no correlation and can be negatively correlated when task complexity is factored out.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The findings suggest considering task complexity in continual-learning benchmark and algorithm design, and customizing algorithms for specific task-pair transfers.

    Source: §1 · p. 2, §8 · p. 8

    The total-complexity measure is crude and captures only some aspects of task-sequence complexity.

    Source: §4.2 · p. 4

Abstract

from arXiv · show

We study the relationship between catastrophic forgetting and properties of task sequences. In particular, given a sequence of tasks, we would like to understand which properties of this sequence influence the error rates of continual learning algorithms trained on the sequence. To this end, we propose a new procedure that makes use of recent developments in task space modeling as well as correlation analysis to specify and analyze the properties we are interested in. As an application, we apply our procedure to study two properties of a task sequence: (1) total complexity and (2) sequential heterogeneity. We show that error rates are strongly and positively correlated to a task sequence's total complexity for some state-of-the-art algorithms. We also show that, surprisingly, the error rates have no or even negative correlations in some cases to sequential heterogeneity. Our findings suggest directions for improving continual learning benchmarks and methods.

1. Introduction

The paper asks which task-sequence properties influence continual-learning error and proposes a procedure to study their relationship to catastrophic forgetting. It finds strong positive correlations between total complexity and error, while sequential heterogeneity has little or no correlation and can be negatively correlated after accounting for task complexity.

1. Introduction

  • The paper asks which properties of a task sequence influence its hardness, measured by the final error rate after sequential training.The motivation is to understand catastrophic forgetting more fundamentally and inform benchmark design, estimates of computational cost and limits, and future mitigation methods.Source: §1 · pp. 1–2
  • The proposed general procedure uses task-space modeling to specify sequence properties, measures forgetting in experiments, and applies correlation analysis to relate them.The authors describe this as the first attempt to answer the stated question.Source: §1 · p. 2
  • The application studies total complexity, the combined hardness of individual tasks, and sequential heterogeneity, the total dissimilarity between consecutive tasks.Task2Vec maps datasets or tasks to vectors; the authors chose these properties because task hardness and dissimilarity intuitively relate to knowledge transfer.Source: §1 · p. 2
  • Total complexity strongly correlates with task-sequence hardness measured by actual error, whereas sequential heterogeneity has little or no correlation and can be negatively correlated when task complexity is factored out.The findings support considering task complexity in benchmark and algorithm design; they also support the view that task dissimilarity could help continual learning.Source: §1 · p. 2
  • The deeper analysis suggests sequence hardness also depends on backward transfer, and that algorithms should be customized for specific task pairs.The supplied discussion identifies these as further findings of the analysis.Source: §1 · p. 2

2. Continual learning algorithms and existing benchmarks

This section surveys continual-learning algorithms for limiting forgetting and benchmarks that evaluate learning across task sequences. It covers regularization, memory and other approaches, alongside image- and game-based benchmark examples.

2. Continual learning algorithms and existing benchmarks

  • The section overviews modern continual-learning algorithms and existing benchmarks used to evaluate them.It directs readers to Chen et al. and Parisi et al. [30] for more comprehensive reviews.Source: §2 · p. 2

2.1. Continual learning algorithms

  • Continual-learning algorithms include weight regularization, Bayesian methods, episodic memory, moment matching, learning without forgetting, generative replay, and structure sharing.EWC scales regularization using the previous task's diagonal Fisher information; SI uses the gradient field along the optimization path, while VCL uses a sequence of variational approximations.Source: §2.1 · pp. 2–3

2.2. Existing benchmarks

  • MNIST-based benchmarks construct sequences through pixel permutations, digit splits, or rotations; related variants use not-MNIST, fashion MNIST, and CIFAR.Other examples include Atari evaluation for EWC and a StarCraft II benchmark requiring an agent to master skills without forgetting earlier ones.Source: §2.2 · p. 3

3. Analysis of catastrophic forgetting

The paper presents a general procedure for studying catastrophic forgetting through task-space modeling and correlation analysis. It demonstrates the procedure on properties of task sequences and notes that it can also be used to study transfer or multi-task learning.

3. Analysis of catastrophic forgetting

  • The procedure specifies task-sequence properties with a suitable task-space modeling method, measures forgetting in experiments, and uses correlation analysis to relate the two.In this paper, forgetting is measured as task-sequence hardness, defined by final error after sequential training.Source: §3 · p. 3
  • The analysis can also be applied to transfer or multi-task learning to study properties of new algorithms.The paper demonstrates the procedure by analyzing two task-sequence properties and their effects on continual-learning algorithms.Source: §3 · p. 3

4. Total complexity and sequential heterogeneity of task sequences

The section defines total complexity and sequential heterogeneity using Task2Vec embeddings, which represent visual tasks as vectors. It uses embedding norms to estimate task difficulty and embedding distances to estimate task dissimilarity.

4.1. Preliminaries: Task2Vec

  • Task2Vec embeds visual classification tasks as vectors; embedding norms encode task difficulty, while distances can approximate task similarity.The paper notes that uncertain examples contribute more to Fisher information, helping explain why difficult tasks have larger embeddings.Source: §4.1 · p. 3

4.2. Total complexity

  • Total complexity sums individual task complexities, estimated as each task embedding’s distance from the trivial-task embedding; the measure is crude and assumes distinct tasks observed once.The paper restricts its analysis to sequences of distinct tasks and leaves repeated observations to future work.Source: §4.2 · p. 4

4.3. Sequential heterogeneity

  • Sequential heterogeneity sums the embedding-based dissimilarities of consecutive task pairs, assuming these pairs mostly influence catastrophic forgetting.The paper leaves measures using all task pairs for future work.Source: §4.3 · p. 4
  • Task2Vec’s setup fits multi-head models, which share weights across tasks but train a separate final layer for each task.Tasks with reversed labels can be considered similar and transferred between by changing only the head.Source: §4.3 · pp. 4–5

5. Correlation analysis

The paper measures a task sequence’s hardness by its final error rate after sequential training, then uses Pearson correlations to examine its relationship with sequence properties. It notes normalization choices intended to reduce confounding by sequence length or task complexity.

5. Correlation analysis

  • Hardness is the final error rate after sequentially training algorithm A on the task sequence, selected as a commonly used continual-learning metric.The paper identifies forgetting rate as another metric for future study.Source: §5 · p. 5
  • Pearson coefficients and p-values quantify the association and statistical significance between sampled sequence-hardness measures and complexity or heterogeneity measures.The analysis samples M task sequences and computes their hardness measures before calculating correlations.Source: §5 · p. 5
  • For complexity correlations, sequences should have equal length because longer sequences tend to have larger total complexity; for heterogeneity, the paper fixes the task set through permutations.Using permutations constrains total complexity so individual task complexities do not affect the heterogeneity correlations.Source: §5 · p. 5

6. Experiments

Experiments on MNIST and CIFAR-10 relate error rates to total complexity and sequential heterogeneity across several continual-learning algorithms and settings. Total complexity is positively associated with error, while heterogeneity has weak, absent, or sometimes negative associations depending on the setting and measure.

6.1. Settings

  • The experiments use 120 length-five sequences sampled from 45 binary tasks on each of MNIST and CIFAR-10, plus 120 permutations of a fixed task set for normalized heterogeneity.They evaluate SI and VCL, and also coreset VCL on MNIST.Source: §6.1 · p. 5
  • The study compares SI and VCL, with coreset VCL on MNIST, across CIFAR-10 and MNIST settings using different network architectures and capacities.The authors average errors over ten random-seed runs per setting.Source: §6.1 · pp. 5–6

6.2. Results

  • 0.86 (SI, CIFAR-10) and 0.69 (VCL, CIFAR-10) are significant positive total-complexity correlations (both p<0.01); MNIST-2562 correlations are 0.24 (SI) and 0.28 (coreset VCL), both p<0.01.On MNIST-50 and MNIST-20, correlations strengthen as network capacity is reduced; all three algorithms show statistically significant positive correlations in MNIST-20.Source: §6.2 · p. 6
  • Sequential heterogeneity has weak positive correlations with error on CIFAR-10, but no significant correlations in the MNIST settings.On CIFAR-10, coefficients are 0.30 for SI and 0.21 for VCL, both statistically significant.Source: §6.2 · p. 6
  • -0.25 (SI, CIFAR-10; p<0.01), -0.20 (VCL, MNIST-50; p<0.05), -0.21 (VCL, MNIST-20; p<0.05), and -0.26 (coreset VCL, MNIST-50; p<0.01) correlate negatively with normalized sequential heterogeneity.These results suggest that task dissimilarity may help some algorithms in some cases.Source: §6.2 · p. 6

7. Discussions

The discussion argues that total task complexity should inform continual-learning benchmarks and transfer choices, while sequential heterogeneity has weak or sometimes negative associations with hardness. A split-MNIST comparison illustrates how varying complexity levels can change algorithm rankings.

On total complexity

  • Across three MNIST sequences, SI is slightly better than coreset VCL on the two varied-complexity sequences, although the standard sequence suggests equal performance.The sequences have high, low, and standard total complexities of 0.48, 0.35, and 0.41, respectively.Source: §7 · p. 7, Figure 3 · p. 8
  • Using benchmarks with different complexity levels can reveal algorithm-performance differences that a standard sequence alone may hide.The authors recommend varied-complexity benchmarks for more informative algorithm comparisons.Source: §7 · p. 7
  • The proposed total-complexity measure is the first for explaining catastrophic forgetting, and its strong positive correlation makes task complexity important to continual-learning effectiveness.The authors suggest accounting for complexity in algorithm and benchmark design, including transfer choices across easy-to-hard versus hard-to-easy tasks.Source: §7 · pp. 6–7

On sequential heterogeneity

  • Weak or negative correlations between error rate and sequential heterogeneity support the view that task dissimilarity can help continual learning [13] on split MNIST and split CIFAR-10.The authors contrast this with prior work associating task similarity with better transfer [2].Source: §7 · p. 7
  • Two equal-complexity CIFAR-10 sequences have higher final average errors on the less heterogeneous sequence; VCL can also forget an immediate predecessor while helping an earlier task.These observations motivate considering dissimilarities between a task and all previously learned tasks, not only consecutive tasks.Source: Figure 2 · p. 8, §7 · p. 8

Future directions

  • The two studied measures explain only some aspects of catastrophic forgetting, and their correlation coefficients are not near 1 or -1.The authors identify improving these measures and studying properties such as intransigence as future directions.Source: §7 · p. 8

8. Conclusion

The paper develops an analysis for studying relationships between catastrophic forgetting and task-sequence properties. Applying it to two properties suggests considering task complexity in continual-learning algorithms and benchmarks, and customizing algorithms for specific transfers.

8. Conclusion

  • The analysis suggests considering task complexity when designing continual-learning algorithms or benchmarks, and customizing algorithms for specific transfers.Source: §8 · p. 8
  • Figure 4 plots average error against total complexity for each algorithm and test, with linear fits, 95% confidence intervals, and colors marking significant positive or negligible correlations.Source: Figure 4 · p. 9
  • Figures 5 and 6 plot average error against sequential and normalized sequential heterogeneity, marking significant positive or negative correlations and negligible correlations, respectively.Source: Figure 5 · p. 9, Figure 6 · p. 10

Related papers

Loading 1908.01091v1…