Source-linked AI summary
Measuring Catastrophic Forgetting in Neural Networks
Ronald Kemker, Marc McClure, Angelina Abitino, Tyler Hayes, Christopher Kanan
TL;DR
The paper addresses catastrophic forgetting, which makes incrementally learning new tasks difficult while preserving earlier knowledge and has been evaluated inconsistently on mostly small datasets. It introduces metrics and large-scale benchmarks to compare five mitigation mechanisms across incremental-learning settings. The results show that preferred mechanisms vary by training paradigm and data type, but none of the tested methods solves catastrophic forgetting.
Problem
Catastrophic forgetting makes neural networks lose previously learned tasks during incremental learning, while existing methods lack direct comparisons and large-scale evaluation.
Method
The paper introduces metrics and benchmarks to compare regularization, ensembling, rehearsal, dual-memory, and sparse-coding mechanisms across incremental classification experiments.
Results
No tested method solves catastrophic forgetting; performance depends on the incremental-learning paradigm and data, with rehearsal or dual-memory favoring new-class learning and regularization or ensembling separating dissimilar sessions.
Takeaways & Limitations
Mechanisms important for incremental performance vary across tasks and data, so catastrophic forgetting remains an open problem rather than a solved capability.
Takeaways & Limitations
Rehearsal can require retaining all training data, while sparsity can increase model memory footprint by 40x, limiting deployment scalability.
Abstract
from arXiv · showhide
Deep neural networks are used in many state-of-the-art systems for machine perception. Once a network is trained to do a specific task, e.g., bird classification, it cannot easily be trained to do new tasks, e.g., incrementally learning to recognize additional bird species or learning an entirely different task such as flower recognition. When new tasks are added, typical deep neural networks are prone to catastrophically forgetting previous tasks. Networks that are capable of assimilating new information incrementally, much like how humans form new memories over time, will be more efficient than re-training the model from scratch each time a new task needs to be learned. There have been multiple attempts to develop schemes that mitigate catastrophic forgetting, but these methods have not been directly compared, the tests used to evaluate them vary considerably, and these methods have only been evaluated on small-scale problems (e.g., MNIST). In this paper, we introduce new metrics and benchmarks for directly comparing five different mechanisms designed to mitigate catastrophic forgetting in neural networks: regularization, ensembling, rehearsal, dual-memory, and sparse-coding. Our experiments on real-world images and sounds show that the mechanism(s) that are critical for optimal performance vary based on the incremental training paradigm and type of data being used, but they all demonstrate that the catastrophic forgetting problem has yet to be solved.
Introduction
Catastrophic forgetting makes incremental learning difficult because networks lose previously learned knowledge as new sessions arrive. The paper introduces benchmarks and metrics to compare mitigation mechanisms on larger, real-world classification settings.
- Retraining with old and new examples can address forgetting but becomes slow for datasets with millions or billions of instances.
- Existing mitigation methods use varied training and evaluation procedures and have mainly been tested on small datasets such as MNIST.
- The paper establishes new benchmarks and metrics for comparing catastrophic-forgetting mechanisms on real-world image and audio datasets with 100–200 classes.
- The paper identifies regularization, ensembling, rehearsal, dual-memory, and sparse-coding as five mechanisms for mitigating catastrophic forgetting.
- The incremental-learning setup organizes labeled data into sequential study sessions, permits auxiliary memory only when reported, and does not assume sessions are iid.
Why Does Catastrophic Forgetting Occur?
Catastrophic forgetting reflects a stability-plasticity dilemma: networks must change enough to learn new tasks without disrupting earlier representations. Existing approaches separate new and old representations or rehearse prior tasks, but rehearsal requires storage and costly relearning.
- Learning new tasks requires plasticity, while large weight changes can disrupt previously learned representations and cause forgetting.
- One approach separates new and old representations through distributed models, regularization, and ensembling.
- Another approach trains on old tasks or their approximations alongside new tasks to preserve prior knowledge.
- Rehearsal requires additional storage and costly relearning, and remains less effective than combining old and new data through full retraining.
Previous Surveys
Prior surveys reviewed forgetting mechanisms and plasticity-adapting networks, but the literature lacked standardized large-scale benchmarks and direct experimental evaluation for measuring catastrophic forgetting.
- Earlier work reviewed mechanisms explored in the 1980s and 1990s, while later comparisons examined activation functions and learning algorithms rather than dedicated mitigation methods.
- A review of plasticity-adapting neural networks identified a lack of appropriate benchmarks but did not establish benchmarks or conduct experiments.
Mitigating Catastrophic Forgetting
Mitigation mechanisms protect prior knowledge by constraining updates, separating task pathways, rehearsing earlier data, consolidating memories, or sparsifying representations. These strategies differ in memory demands, capacity, and effects on learning new tasks.
- Regularization: Regularization constrains weight updates so new sessions are learned with less interference to prior memories.EWC directs plasticity away from weights contributing most to previous tasks.
- Ensembling: Ensemble methods combine multiple classifiers, often adding a new sub-network for each session at increasing memory cost.
- Ensembling: PathNet uses a genetic algorithm to select a trainable path through a fixed-size network, then freezes that path to preserve learned knowledge.Its fixed base-network size permits representation reuse, but capacity can become limiting as paths are frozen.
- Rehearsal: Rehearsal mixes earlier-session data with the current session, whereas pseudorehearsal generates pseudopatterns to stabilize older memories without storing all prior training data.
- Dual-memory: Dual-memory models separate fast and slow learning systems and consolidate recent memories into the broader network during a later phase.GeppNet+STM stores uncertain new inputs in a short-term buffer before consolidation during sleep.
- Sparse-coding: Sparse representations can reduce representational interference, but sparsity may impair generalization and new-task learning.FEL uses fixed excitatory and inhibitory weights to sparsify inputs and gate network-wide weight updates.
Experimental Setup
The experiments use hard, real-world image and audio classification datasets with many classes and few samples per class, alongside MNIST for comparison.
- Real-world datasets: The benchmarks target fine-grained image and audio classification with different modalities, many classes, and few samples per class.These datasets are presented as more practical and meaningful than MNIST.
- MNIST comparison: MNIST contains 10 digit classes with grayscale images measuring 28×28 pixels.
- Image dataset: CUB-200 contains 200 bird species, represented as 2048-dimensional vectors extracted from pre-trained ResNet-50 features.
- Audio dataset: AudioSet contains over 2 million labeled 10-second sound bytes from 632 classes; the study uses 100 single-label classes.
Models Evaluated
The study evaluates five continual-learning mechanisms using matched-capacity models and an offline-trained MLP baseline, with model-specific training procedures and stopping criteria.
- Compared mechanisms: Five evaluated models represent regularization, ensembling, rehearsal, dual-memory, and sparse-coding mechanisms: EWC, PathNet, GeppNet, GeppNet+STM, and FEL.
- Baseline: The baseline is a standard MLP optimized offline on the full training sets, achieving 62.1% on CUB-200 and 46.1% on AudioSet.
- Regularization: EWC adds a Fisher-information constraint that preserves weights important to earlier tasks while directing plasticity toward less important parameters.
- Ensembling: PathNet uses a genetic algorithm to select a trainable path, creates an independent output layer per task, and sequentially freezes network portions.
- Rehearsal and dual-memory: GeppNet uses rehearsal, while GeppNet+STM adds a fixed-size short-term memory buffer that replays novel examples during a sleep phase.
- Sparse coding: FEL uses fixed sparse connections and selective outputs so that only subsets of earlier-layer units are updated during training.
Experiments and Results
The paper establishes three benchmark experiments for measuring catastrophic forgetting.
- The study defines three benchmark experiments to measure catastrophic forgetting.
1. Data Permutation Experiment
In the data permutation experiment, feature dimensions are permuted within each session but differently across sessions, testing recall of earlier sessions.
- Each feature vector is randomly permuted using a session-specific permutation that remains constant within that session.
- The model is evaluated on its ability to recall data learned in prior study sessions.
- Every session contains the same number of examples.
2. Incremental Class Learning
The paper evaluates incremental learning across datasets and paradigms, finding that models differ in their ability to retain prior knowledge while acquiring new classes or modalities.
- Evaluation setup: The evaluation uses new metrics to assess retention of prior sessions while learning new knowledge.Ωbase measures retention of the first session, Ωnew measures immediate recall of new tasks, and Ωall combines retention with acquisition.
- Evaluation setup: Data permutation experiments test whether models retain multiple representations learned sequentially from randomly permuted input features.Each session uses a different random permutation, with an identically permuted test set for evaluation.
- Data permutation: On MNIST, CUB-200, and AudioSet, Ωall is generally higher for MNIST, showing that real-world datasets provide more demanding incremental-learning benchmarks.PathNet and EWC retain base knowledge while learning new information, with PathNet stronger on AudioSet and MNIST and EWC stronger on CUB-200.
- Incremental class learning: Incremental class learning begins with half of each dataset’s classes, followed by sessions containing one new class at a time.The study measures base-set mean-per-class accuracy after each new class and also evaluates newly trained and previously learned classes.
- Incremental class learning: GeppNet, GeppNet+STM, and FEL retain base knowledge while learning new classes, with GeppNet performing best overall.GeppNet+STM struggles to learn new classes on CUB-200 and AudioSet, whereas FEL learns new classes well but suffers base-set forgetting.
- Multi-modal learning: In the multi-modal experiment, EWC is the only model that preserves the first modality while learning the second in both training orders.The experiment alternates CUB-200 image classification and AudioSet audio classification, then evaluates retention of the modality learned first.
Discussion
The experiments show that no tested method solves catastrophic forgetting, while the strongest mechanism depends on the incremental learning paradigm and data type. Performance and deployment trade-offs also involve training time, memory usage, and data characteristics.
- No tested method solves catastrophic forgetting, although some methods perform better than others.
- GeppNet variants performed best at incremental class learning, whereas PathNet performed best overall on data permutation experiments except CUB-200.
- EWC performed best on multi-modal experiments and may suit non-redundant modalities, while PathNet may suit representations that differ without being entirely dissimilar.
- Models differ in memory demands: PathNet adds an output layer per session, GeppNet stores all training data, and EWC can use prior validation data.
- Both GeppNet variants were orders of magnitude slower because they trained one sample at a time, while fixed-size models were noticeably faster.
- Increasing FEL capacity can improve performance but may increase the model memory footprint 40x, limiting its appeal for deployment.
Conclusion
The study finds that catastrophic forgetting remains unsolved: different mechanisms work best under different incremental-learning scenarios, and no single method consistently succeeds.
- New metrics show rehearsal or pseudo-rehearsal combined with dual-memory is optimal for learning new classes incrementally.Rehearsal performed reasonably well but required retaining all training data for replay.
- Regularization and ensembling are best at separating multiple dissimilar sessions within a common DNN framework.
- Performance is significantly better on MNIST than on the larger datasets evaluated.
- No single method solves catastrophic forgetting across the evaluated scenarios.The authors suggest future lifelong-learning systems should combine mechanisms.
Supplemental Material
The supplemental material documents training settings, reproduction checks, and comparative results across permutation, incremental-class, multimodal, and feature-redundancy analyses.
- The supplemental experiments reproduce or verify EWC, GeppNet, GeppNet+STM, FEL, and PathNet-related procedures using documented training parameters.PathNet was obtained directly from its original authors, while other models were reproduced or verified against prior work.
- Data Permutation Experiment: Data-permutation results compare retention of the original task with performance on the current permutation across MNIST, CUB-200, and AudioSet.
- Data Permutation Experiment: Performance is worse on CUB-200 and AudioSet than on MNIST; GeppNet variants retain prior data but struggle to learn new information, while FEL tends to forget original data.
- Data Permutation Experiment: EWC and PathNet have the best overall performance, but both degrade on CUB-200 and AudioSet relative to MNIST.
- Data Permutation Experiment: EWC performs best on the CUB-200 permutation task and on the multimodal experiment, underscoring the need to consider scalability before selecting a model.
- Incremental Class Learning Experiment: For incremental class learning, GeppNet, GeppNet+STM, and FEL perform best across datasets, although results remain substantially better on MNIST.
- Feature Redundancy Analysis: FCBF analysis finds MNIST and AudioSet features more redundant than CUB-200 features.The symmetric-uncertainty matrix shows strong MNIST overlap, repeated correlated AudioSet sub-vectors, and comparatively weak CUB-200 correlations.
- Unconstrained model-capacity experiments produce somewhat higher base results than capacity-constrained experiments.