PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Fast, Learning Slow: A General Continual Learning Method based on Complementary Learning System

Elahe Arani, Fahad Sarfraz, Bahram Zonooz

arXiv:2201.12604v2cs.LGcs.AIcs.CV

TL;DR

Deep neural networks struggle to learn continually because new information can cause catastrophic forgetting. CLS-ER combines episodic replay with short-term and long-term semantic memories, and reports state-of-the-art performance across benchmark and general continual-learning settings.

  • Problem

    Deep neural networks experience catastrophic forgetting when learning incrementally from non-stationary data, hindering retention of previously learned tasks.

  • Method

    CLS-ER combines episodic replay with short-term and long-term semantic memories that constrain working-model updates and align decision boundaries.

  • Results

    CLS-ER achieves new state-of-the-art performance in the vast majority of continual-learning settings, including benchmark and general incremental-learning scenarios.

  • Takeaways & Limitations

    CLS-ER provides a general continual-learning method without task boundaries or strong assumptions about data and task distributions.

Abstract

from arXiv · show

Humans excel at continually learning from an ever-changing environment whereas it remains a challenge for deep neural networks which exhibit catastrophic forgetting. The complementary learning system (CLS) theory suggests that the interplay between rapid instance-based learning and slow structured learning in the brain is crucial for accumulating and retaining knowledge. Here, we propose CLS-ER, a novel dual memory experience replay (ER) method which maintains short-term and long-term semantic memories that interact with the episodic memory. Our method employs an effective replay mechanism whereby new knowledge is acquired while aligning the decision boundaries with the semantic memories. CLS-ER does not utilize the task boundaries or make any assumption about the distribution of the data which makes it versatile and suited for "general continual learning". Our approach achieves state-of-the-art performance on standard benchmarks as well as more realistic general continual learning settings.

1 INTRODUCTION

Continual learning requires acquiring and retaining knowledge from changing data, but deep networks suffer catastrophic forgetting. CLS-ER addresses this by combining rapid and slow semantic memories with episodic replay, without task-boundary or data-distribution assumptions.

  • Continual learning requires acquiring new knowledge while consolidating and retaining earlier knowledge from dynamic, non-stationary data.
  • Existing continual-learning methods include regularization, network expansion, and rehearsal approaches for mitigating catastrophic forgetting.
  • CLS theory motivates complementary rapid episodic learning and slow structured consolidation between hippocampal and neocortical systems.
  • CLS-ER uses short-term and long-term semantic memories alongside episodic memory to consolidate representations and align the working model’s decision boundary.
  • CLS-ER is designed for general continual learning without task boundaries or strong assumptions about data and task distributions, and is evaluated on benchmarks and realistic scenarios.

2 RELATED WORK

Prior continual-learning work includes rehearsal methods and earlier CLS-inspired dual-memory systems. CLS-ER extends this direction while targeting scalability and settings where task boundaries are unavailable.

  • Experience Replay and related methods use stored samples to transfer knowledge from previous tasks and reduce interference.
  • Earlier CLS-inspired dual-memory approaches used generative models or other architectures but had not demonstrated scalability to current computer-vision tasks.
  • Figure 2 compares task-wise performance across training progress and indicates less forgetting in semantic memories than in the working model.
  • Some prior dual-memory methods rely on task boundaries, while generative replay faces difficulty learning faithful distributions and can underperform instance-based replay.

3 METHOD

CLS-ER combines episodic replay with short-term and long-term semantic memories maintained through exponential moving averages. Their replayed information constrains the working model to acquire new knowledge while preserving prior structure.

  • CLS-ER mirrors CLS theory by coupling rapid episodic encoding with slower semantic consolidation across tasks.
  • Semantic Memories: The method maintains short-term and long-term semantic memories that accumulate and consolidate information over different timescales.
  • Semantic Memories: Exponential moving averages of working-model weights provide a computationally and memory-efficient way to accumulate task knowledge into semantic memories.
  • Episodic Memory: Reservoir sampling maintains a fixed episodic buffer by giving each stream sample equal probability of representation, without requiring task boundaries or distribution assumptions.
  • Consolidation of Information: Semantic and episodic memories produce consolidated activations that constrain working-model updates, limiting parameter changes and smoothing decision-boundary adaptation.
  • Formulation: The training objective combines cross-entropy on stream and memory samples with a consistency loss on episodic exemplars.
  • Formulation: The plastic and stable semantic models are updated stochastically, with the plastic model adapting more rapidly and the stable model acquiring structured knowledge more slowly.
  • Formulation: Inference uses the stable model because it retains long-term information and consolidated structural knowledge across tasks.

4 EXPERIMENTAL SETUP

The experiments use uniform training settings across continual-learning evaluations and tune CLS-ER’s main hyperparameters on a small validation set. Architectures and augmentation procedures vary by dataset family.

  • CLS-ER is evaluated under training settings matched to the Mammoth framework for fair comparison across continual-learning methods.
  • The main hyperparameters are selected through grid search over consistency weight, decay factors, and semantic-memory update rates.
  • Experiments use a two-hidden-layer fully connected network for MNIST variants and an unpretrained ResNet-18 for other datasets, with SGD throughout.
  • Table 1 compares prior methods on Class-IL and Domain-IL settings, using baseline results from Buzzega et al. (2020a).

5 EMPIRICAL EVALUATION

CLS-ER is evaluated across diverse continual-learning protocols, memory budgets, and task complexities, including Class-IL, Domain-IL, General Incremental Learning, and online settings. It achieves the strongest reported performance across these challenging scenarios while improving knowledge retention, transfer, and robustness to distributional and class-frequency changes.

  • Evaluation protocol: The evaluation compares rehearsal-based methods across varied continual-learning settings and memory budgets under uniform experimental conditions.SGD is used as standard training and JOINT as an upper bound from joint training across all tasks.
  • Class-IL: CLS-ER provides the highest performance across the evaluated Class-IL scenarios, with gains increasing as dataset complexity and task length rise.On S-Tiny-ImageNet with a 200-sample buffer, it gains 176% over ER and 114% over DER++.
  • Class-IL: 176% and 114% gains on S-Tiny-ImageNet with a 200-sample buffer over ER and DER++, respectively, demonstrate stronger retention under complex, memory-restricted sequences.The reported gains are specifically relative to ER and DER++ in the S-Tiny-ImageNet, 200-buffer setting.
  • Domain-IL: CLS-ER provides generalization gains in Domain-IL, particularly with lower memory budgets, and performs on par with DER++ at a 5120-sample buffer.The authors attribute this to consolidated soft targets that preserve class similarity structure across sequences.
  • General Incremental Learning: CLS-ER provides considerable gains on MNIST-360 and is effective on GCIL-CIFAR-100 under both uniform and imbalanced class samples.These settings include recurring classes, requiring transfer from earlier occurrences; the longtail results indicate greater robustness to class imbalance.
  • Online continual learning: CLS-ER demonstrates potential for online continual learning because the MNIST-based settings expose the model to each task's data only once.The paper characterizes these settings as online CL and uses the observed performance to demonstrate the method's potential in that setting.

6 MODEL CHARACTERISTICS

CLS-ER is analyzed through flatness, task-prediction bias, and calibration. These characteristics jointly support more uniform performance and reliable predictions across tasks.

  • CLS-ER is analyzed against ER and DER++ to identify characteristics enabling effective continual learning.
  • Flatter minima can preserve proximity to optimal parameters for previous tasks during adaptation to new tasks.
  • CLS-ER produces better-calibrated models and therefore more reliable predictions on S-CIFAR-10 with a 500-example buffer.
  • CLS-ER maintains more uniform prediction probabilities across tasks over long sequences.
  • Flatter minima, uniform task performance, reduced recent-task bias, and calibration are described as complementary characteristics of CLS-ER.

7 CONCLUSION

The conclusion presents CLS-ER as a dual-memory replay method evaluated across increasingly realistic continual-learning settings. It reports broad empirical effectiveness while emphasizing robust evaluation and online-learning considerations.

  • CLS-ER maintains long-term and short-term semantic memories that replay episodic neural activities and align the working model’s decision boundary.
  • The method achieves new state-of-the-art performance in the vast majority of continual-learning settings and provides flatter, less recent-task-biased, calibrated models.
  • Class-IL: Class-IL adds new classes over tasks and evaluates discrimination across both current and previously seen classes.
  • Domain-IL: Domain-IL keeps classes fixed while changing the input distribution through rotations or pixel permutations.
  • General continual learning: General continual-learning scenarios include blurry task boundaries, recurring classes, varying distributions, sample-efficiency demands, and class imbalance.
  • Online continual learning: Online continual learning presents a non-iid stream whose samples are seen once, with epoch counts adjusted for dataset complexity in the reported experiments.
  • Online continual learning: MNIST-based experiments use one epoch per task and are treated as online continual-learning settings for CLS-ER.
  • Episodic memory: Reservoir sampling maintains a fixed-size episodic buffer by giving each stream sample equal representation probability without priority.

C ADDITIONAL RESULTS

Additional analyses examine the separate CLS-ER components and their task-wise behavior. The stable model consolidates knowledge across tasks, while the plastic model adapts more effectively to recent tasks.

  • The additional-results section analyzes CLS-ER behavior and component performance across experimental settings.
  • The stable model provides the highest mean performance for the vast majority of settings by consolidating knowledge across tasks.
  • The stable model retains performance on previous tasks, whereas the plastic model adapts better to the recent task.
  • Feedback from both semantic memories improves the working model and, in turn, the plastic and stable models.

C.2 TASK PROBABILITIES

CLS-ER is evaluated for task-probability uniformity and calibration under varying buffer sizes. It consistently reduces recent-task bias and improves calibration relative to ER and DER++.

  • Task probabilities: CLS-ER consistently achieves more uniform task probabilities than ER and DER++ across buffer sizes and datasets.
  • Model calibration: CLS-ER consistently provides better-calibrated models than ER and DER++ on S-CIFAR-10 and S-TinyImageNet.
  • Model calibration: On lower buffer sizes, the difference in Expected Calibration Error is considerable for both datasets.
  • Model calibration: The calibration results demonstrate CLS-ER’s ability to train high-performance and reliable models under challenging conditions.

C.4 EFFECT OF HYPERPARAMETERS

CLS-ER’s complementary semantic memories improve learning across tasks and reduce the trade-off between adapting to recent tasks and retaining earlier ones. Its performance is relatively robust to hyperparameter choices, simplifying tuning.

  • Component interaction: Both semantic memories contribute to CLS-ER’s performance, with the best results occurring when the plastic and stable models both perform well.The semantic memories can emphasize different tasks while jointly supporting the working model.
  • Hyperparameter sensitivity: CLS-ER is not highly sensitive to particular hyperparameter settings because different configurations can achieve similar performance.This robustness allows some parameters to remain fixed while only the remaining parameters are tuned.
  • Comparison with Mean-ER: Dual semantic memories provide additional gains over Mean-ER, especially on complex datasets with smaller memory buffers, while producing lower variance.Mean-ER uses a single semantic memory and struggles to maintain recent and earlier task performance simultaneously.
  • Comparison with Mean-ER: Mean-ER exposes a trade-off between adapting to recent changes and preserving earlier-task performance, which CLS-ER addresses with specialized memories.The short-term and long-term memories separately support rapid adaptation and consolidated information.

E TRAINING AND IMPLEMENTATION DETAILS

The experiments use settings designed for comparison with prior continual-learning work while evaluating CLS-ER across varying memory budgets and more realistic general continual-learning conditions. Implementation choices include reduced training for S-Tiny-ImageNet and uniform settings across many budgets.

  • Experimental setup: Experiments largely match DER++ in optimizer, epochs, batch size, and memory batch size to reduce effects from the training schedule.For S-Tiny-ImageNet, CLS-ER uses 50 rather than 100 epochs because it learns efficiently with fewer epochs.
  • Supplementary analyses: The evaluation includes reliability plots and task-probability analyses on S-CIFAR-10 and S-Tiny-ImageNet under varying memory budgets.The supplementary figures report reliability for both datasets and task probabilities for both datasets.
  • Hyperparameter settings: CLS-ER requires only slight hyperparameter modifications across memory budgets for most datasets, with uniform learning rates, epochs, batch sizes, and lambda values.This setup is reported in the hyperparameter configuration for the experimental settings.
  • General continual learning: The general continual-learning evaluation uses 20 phases, 1,000 samples per phase, and up to 50 classes per phase under uniform and longtail distributions.These settings are intended to simulate more realistic continual-learning challenges.

E.2 PERTURBATION ANALYSIS

The perturbation analysis relies on released DER++ and ER code and checkpoints and uses supplementary tables to document experimental hyperparameters and comparison settings.

  • Perturbation analysis: The perturbation analysis uses code and checkpoints provided for DER++ and ER within the Mammoth framework.The framework supports comparisons under uniform experimental conditions.
  • Perturbation analysis: Table S3 evaluates how hyperparameter settings affect CLS-ER components on S-CIFAR-10 with a 500-sample buffer.The results average performance over three runs, while αS and αP are fixed at 0.999.
  • Implementation documentation: Tables S4, S5, and S6 document the hyperparameters used for CLS-ER, DER++, and Mean-ER across the experimental settings.DER++ parameters are specified for GCIL-CIFAR-100, while CLS-ER uses the same settings for uniform and longtail conditions.
Loading 2201.12604v2…