PaperlayerView exact revision on arXiv

Source-linked AI summary

Online Coreset Selection for Rehearsal-based Continual Learning

Jaehong Yoon, Divyam Madaan, Eunho Yang, Sung Ju Hwang

arXiv:2106.01085v4cs.LGcs.CV

TL;DR

Rehearsal-based continual learning needs to choose useful replay examples because data points differ in representativeness, especially under imbalanced or noisy streams. OCS selects representative and diverse samples online using gradient-based criteria and affinity to previous tasks. Across continual-learning benchmarks, it improves performance while supporting current-task adaptation and mitigating catastrophic forgetting.

  • Problem

    Rehearsal-based continual learning requires effective coreset selection because data points differ in usefulness, particularly in imbalanced or noisy settings.

  • Method

    OCS selects representative and diverse samples online using gradient-based criteria that also promote affinity with previous-task knowledge.

  • Results

    OCS outperforms state-of-the-art rehearsal-based techniques across balanced, imbalanced, and noisy continual-learning benchmarks.

  • Takeaways & Limitations

    OCS is generally applicable to rehearsal-based continual-learning methods and improves task adaptation while inhibiting catastrophic forgetting.

  • Takeaways & Limitations

    The formulation assumes each task's datapoints and labels are independently and identically distributed, with the task distribution treated as unknown.

Abstract

from arXiv · show

A dataset is a shred of crucial evidence to describe a task. However, each data point in the dataset does not have the same potential, as some of the data points can be more representative or informative than others. This unequal importance among the data points may have a large impact in rehearsal-based continual learning, where we store a subset of the training examples (coreset) to be replayed later to alleviate catastrophic forgetting. In continual learning, the quality of the samples stored in the coreset directly affects the model's effectiveness and efficiency. The coreset selection problem becomes even more important under realistic settings, such as imbalanced continual learning or noisy data scenarios. To tackle this problem, we propose Online Coreset Selection (OCS), a simple yet effective method that selects the most representative and informative coreset at each iteration and trains them in an online manner. Our proposed method maximizes the model's adaptation to a current dataset while selecting high-affinity samples to past tasks, which directly inhibits catastrophic forgetting. We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.

1 INTRODUCTION

Continual learning must preserve knowledge across sequential tasks, but rehearsal methods can select unrepresentative or harmful examples, especially with imbalanced or noisy data. OCS addresses this by selecting online coresets that support current-task adaptation while reducing interference with previous tasks.

  • Continual learning models continuously learn across tasks but risk catastrophic forgetting of previously learned knowledge.
  • Random replay-buffer sampling can be inadequate because examples differ in representativeness and informativeness, particularly in imbalanced, streaming, or noisy settings.
  • OCS selects representative current-task samples, encourages diversity, and favors affinity with previous-task knowledge through three online strategies.
  • OCS is applicable to rehearsal-based continual learning and outperforms state-of-the-art rehearsal techniques across balanced, imbalanced, and noisy benchmarks.
  • The method obtains representative, diverse, high-affinity subsets from each minibatch using gradient-based selection criteria.

2 RELATED WORK

Related work addresses catastrophic forgetting through regularization, architectural changes, and replay-buffer selection. Existing coreset methods use importance, sampling, herding, or loss-based criteria, but rehearsal-based approaches remain concerned with selecting useful replay instances.

  • Regularization methods constrain model parameters, while architecture methods isolate or expand networks to alleviate catastrophic forgetting.
  • GSS selects replay examples by maximizing gradient-direction variance, whereas ER-MIR selects instances that increase loss between successive iterations.
  • Coreset-selection research includes importance sampling, Gumbel-top-k sampling, beam search, and herding-based strategies.

3 REHEARSAL-BASED CONTINUAL LEARNING

Rehearsal-based continual learning replays a small subset of earlier data, making the quality of selected instances central to retaining prior knowledge. The paper motivates gradient-based selection by showing that individual incoming examples can differ substantially in their effects on adaptation and forgetting.

  • The task stream comprises sequential tasks with independently and identically distributed datapoints and labels, with each task available only at its own step.
  • The model uses a standard loss objective, while naive sequential training without forgetting mitigation cannot retain previous-task knowledge.
  • Rehearsal methods replay small randomly sampled buffers, so selected-instance quality is essential because examples can either improve performance or degrade past-task memorization.
  • Individual CIFAR-10 examples produce widely varying effects on corresponding-class accuracy and previous-task forgetting after a single update.
  • The proposed selection objective maximizes gradient similarity between representative instances and the current-task dataset.
  • Minibatch approximation makes representative-subset selection tractable online, while diversity and coreset affinity address limitations of similarity alone.

4 ONLINE CORESET SELECTION

OCS selects online coresets that balance representative current-task training, sample diversity, and compatibility with previously learned knowledge. It then uses these selected instances for current-task adaptation and bounded replay-buffer construction.

  • Selection criteria: OCS selects useful instances online by combining minibatch similarity, sample diversity, and coreset affinity.The criteria target representative current-task samples, low redundancy, and minimal interference with previous-task knowledge.
  • Selection criteria: Minibatch similarity measures how well an instance describes the current task, while high similarity can increase redundancy when task-instance variance is high.The minibatch is treated as an approximation of the current task dataset.
  • Selection criteria: Sample diversity is formulated as averaged dissimilarity from other instances in the same minibatch, producing values in [−1, 0].The criterion uses negative similarities rather than average similarity to favor less redundant examples.
  • Online current-task adaptation: OCS combines similarity and diversity to select helpful current-task instances despite noisy or redundant incoming data.The selected top-κ instances are obtained online at each iteration.
  • Continual-learning adaptation: A selected coreset can become a replay-buffer candidate, after which candidates are retained or updated subject to a bounded buffer size.The buffer may be finalized after task training or updated iteratively.
  • Continual-learning adaptation: Coreset affinity compares current samples with the previous-task replay coreset and favors examples that minimize gradient-angle interference.The replay buffer serves as a proxy for inaccessible past-task data.
  • Optimization: The training objective balances adaptation to the current task and the past-task coreset through λ, while τ controls model plasticity and stability.The first task omits previous-task interference and selects using minibatch similarity and sample diversity.

5 EXPERIMENTS

Experiments evaluate OCS across balanced, imbalanced, noisy, and collaborative continual-learning settings, including ablations of gradient selection and the S, V, and A criteria. OCS generally improves accuracy and forgetting relative to rehearsal-based baselines while remaining computationally efficient.

  • Experimental setup: OCS is evaluated on domain-, task-, and class-incremental benchmarks, including balanced and imbalanced Rotated MNIST, Split CIFAR-100, Multiple Datasets, and noisy Rotated MNIST.Experiments use five independent runs and report average accuracy and average forgetting.
  • Balanced continual learning: 19% relative gain in average accuracy and 62% and 79% reductions in forgetting over the strongest random-replay baselines on Rotated MNIST and Split CIFAR-100, respectively.Against coreset-based techniques, OCS reduces forgetting by 38% on Rotated MNIST and 57% on Multiple Datasets.
  • Imbalanced continual learning: Approximately 7% and 9% relative accuracy improvements, with approximately 11% and 40% forgetting reductions, over the best baselines on imbalanced Rotated MNIST and Multiple Datasets, respectively.OCS also outperforms across larger episodic-memory sizes in imbalanced continual learning.
  • Noisy continual learning: 43% relative accuracy gain and 20% and 35% forgetting reductions on noisy Rotated MNIST with 40% and 60% noisy data, respectively.The performance gap becomes more significant as the proportion of noisy examples increases.
  • Ablation studies: Grad-OCS significantly outperforms Input-OCS and Feat-OCS on balanced and imbalanced continual learning, supporting gradients as a better dataset-approximation metric.The ablation compares gradient selection with raw inputs and feature representations.
  • Ablation studies: S selects valuable samples, V reduces redundancy, and A improves forgetting through affinity to previous-task coresets; all three contribute on complicated continual-learning problems.For Multiple Datasets, A alone achieves 58.1% average accuracy versus 61.5% for S + V + A; S + A and V + A are lower than full OCS by 2.1%p and 5.1%p, respectively.
  • Coreset visualization: OCS selects beneficial class examples and discards uninformative noisy instances, while its selected samples are diverse and form representative clusters.The visual analyses attribute these properties to the S and V criteria.
  • Collaborative learning: OCS improves MC-SGD accuracy by 1.2% to 3.4% across Permuted MNIST, Rotated MNIST, and Split CIFAR-100 and produces considerably lower forgetting.This demonstrates collaborative learning with an existing rehearsal-based method.

6 CONCLUSION

OCS is a gradient-based online coreset-selection method for online continual learning. It selects representative and diverse instances that help preserve previous-task knowledge while supporting current-task adaptation, with promising results across balanced, imbalanced, and noisy benchmarks.

  • OCS selects representative and diverse instances that help preserve previous-task knowledge during online continual learning.The method uses gradient-based selection to construct coresets at each iteration.
  • OCS improves current-task adaptation while mitigating catastrophic forgetting on previous tasks.
  • OCS outperforms state-of-the-art rehearsal-based techniques on standard balanced continual-learning datasets.
  • OCS also shows promising results on imbalanced and noisy continual-learning datasets.
  • OCS extends naturally to existing rehearsal-based continual learning using a random-replay buffer.

A EXPERIMENTAL DETAILS

The experiments cover balanced, imbalanced, and noisy continual-learning settings across multiple datasets, architectures, and rehearsal baselines. They specify shared memory, implementation, hyperparameter, and reporting conventions for fair comparisons.

  • Datasets: The benchmark suite includes balanced and imbalanced Rotated MNIST and Split CIFAR-100 datasets.Rotated MNIST contains 20 rotation tasks, while Split CIFAR-100 tasks contain five random classes.
  • Network Architectures: Rotated MNIST uses a single-head MLP, whereas CIFAR-100 and Multiple Datasets use multi-head ResNet-18 architectures.
  • Implementations: The evaluation compares OCS with implementations of Finetune, EWC, Stable SGD, A-GEM, ER-Reservoir, MC-SGD, Uniform Sampling, iCaRL, and k-means methods.
  • Memory Capacity: Rehearsal methods either allocate a fixed number of examples per task or use the memory capacity while randomly discarding stored data.
  • Reporting: Tables report performance comparisons using mean and standard deviation across five independent runs.
  • Hyperparameter configurations: OCS uses batch sizes of 100 for Rotated MNIST and 20 for Split CIFAR-100 and Multiple Dataset experiments.
  • Hyperparameter configurations: The same κ = 10 and τ = 1k values are used across balanced, imbalanced, and noisy experiments.The paper notes that careful tuning could further improve OCS performance.

B ADDITIONAL EXPERIMENTS

Additional experiments examine OCS under current-task adaptation, class-incremental learning, uniform current-task training, partial-gradient selection, and minibatch-gradient approximation. Across these analyses, OCS maintains advantages while partial gradients can reduce computation.

  • Current task adaptation with the baselines: OCS improves current-task performance in both balanced and imbalanced scenarios, unlike most baseline coreset-selection criteria.On Balanced Rotated MNIST, most baselines decrease performance by 1.0–9.8%p, while Grad Matching increases it by 0.5%p.
  • Class-incremental continual learning: OCS outperforms the strongest baseline by 10.61% in accuracy and 63.6% in forgetting on balanced CIFAR-100 class-incremental learning.
  • Uniform training with OCS coreset: Uniform current-task training with an OCS coreset decreases performance by 2.6% on Rotated MNIST and 9.2% on Multiple Datasets.It also deteriorates catastrophic forgetting across datasets.
  • OCS with partial gradients: Using earlier ResNet blocks for OCS yields 0.36%p higher performance while computing only 6.3% of the gradients.The analysis attributes this potential benefit to earlier blocks capturing more generic representations.
  • OCS with partial gradients: The partial-gradient observation remains an area for further investigation and may support lower-cost OCS selection.
  • Distance between the whole dataset and its minibatch: Larger minibatches better approximate the entire-dataset gradient for both ℓ2 distance and cosine similarity.The experiment compares MNIST and CIFAR-10 with MLP and ResNet-18 architectures.
Loading 2106.01085v4…