PaperlayerView exact revision on arXiv

Source-linked AI summary

Online Continual Learning in Image Classification: An Empirical Survey

Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, Scott Sanner

arXiv:2101.10423v4cs.LGcs.CV

TL;DR

Online continual learning must classify a single-pass stream while retaining knowledge despite changing classes or domains and limited access to prior data. This survey systematically compares methods and practical tricks across these settings. It finds that MIR is broadly versatile, while review and NCM enhancements bring online continual learning closer to offline performance.

  • Problem

    Many continual-learning methods and tricks had not been fairly and systematically compared across realistic memory and data settings.

  • Method

    The survey compares online continual-learning methods across class and domain incremental image-classification settings and evaluates seven practical tricks.

  • Results

    MIR consistently surpasses other methods on the larger CORe50-NC dataset and remains strongly competitive across online class and domain incremental settings.

  • Takeaways & Limitations

    Combining the best methods and tricks brings online continual learning with very small mini-batches closer to matching offline training.

  • Takeaways & Limitations

    EWC is impractical for online continual learning because it requires storing task-specific Fisher information and an extra pass over each task.

Abstract

from arXiv · show

Online continual learning for image classification studies the problem of learning to classify images from an online stream of data and tasks, where tasks may include new classes (class incremental) or data nonstationarity (domain incremental). One of the key challenges of continual learning is to avoid catastrophic forgetting (CF), i.e., forgetting old tasks in the presence of more recent tasks. Over the past few years, many methods and tricks have been introduced to address this problem, but many have not been fairly and systematically compared under a variety of realistic and practical settings. To better understand the relative advantages of various approaches and the settings where they work best, this survey aims to (1) compare state-of-the-art methods such as MIR, iCARL, and GDumb and determine which works best at different experimental settings; (2) determine if the best class incremental methods are also competitive in domain incremental setting; (3) evaluate the performance of 7 simple but effective trick such as "review" trick and nearest class mean (NCM) classifier to assess their relative impact. Regarding (1), we observe iCaRL remains competitive when the memory buffer is small; GDumb outperforms many recently proposed methods in medium-size datasets and MIR performs the best in larger-scale datasets. For (2), we note that GDumb performs quite poorly while MIR -- already competitive for (1) -- is also strongly competitive in this very different but important setting. Overall, this allows us to conclude that MIR is overall a strong and versatile method across a wide variety of settings. For (3), we find that all 7 tricks are beneficial, and when augmented with the "review" trick and NCM classifier, MIR produces performance levels that bring online continual learning much closer to its ultimate goal of matching offline training.

1. Introduction

Online continual learning addresses learning from a single-pass, non-i.i.d. image stream while preserving earlier knowledge under memory, privacy, and computation constraints. This survey compares methods across online class and domain incremental settings and evaluates practical tricks.

  • Motivation and scope: Incremental updates can cause catastrophic forgetting, making offline retraining inefficient or infeasible when prior data are unavailable.The motivating constraints include privacy, storage limits, communication bandwidth, and real-time performance on edge devices.
  • Motivation and scope: Online continual learning processes one tiny batch at a time without access to previously seen batches, covering new classes and changing data domains.The supervised image-classification scope includes online class incremental and online domain incremental settings.
  • Survey goals: The survey fairly compares state-of-the-art methods to identify which approaches work best under different memory and data settings.The comparison targets relative advantages across realistic online continual-learning conditions.
  • Main findings: iCaRL remains competitive with small memory, GDumb is strong on medium-size datasets, and MIR performs best on larger-scale data.These findings are reported for online class incremental learning.
  • Main findings: GDumb performs poorly in domain incremental learning, whereas MIR remains strongly competitive across both class and domain incremental settings.The survey concludes that MIR is a strong and versatile method across a wide variety of settings.
  • Main findings: All seven evaluated tricks are beneficial, and combining MIR with review and an NCM classifier brings online continual learning closer to offline training.The result concerns the relative impact of simple practical enhancements.

2. Related Work

Prior continual-learning surveys include conceptual reviews and empirical comparisons, but empirical analyses have often been limited in dataset scale. This survey builds on that literature while emphasizing broader empirical comparison.

  • Existing surveys: Non-empirical surveys cover biological perspectives, formal problem definitions, benchmarks, metrics, approaches, robotics applications, and online continual learning.These works summarize methods and recommend desiderata or guidelines rather than primarily reporting broad empirical comparisons.
  • Existing surveys: Empirical surveys compare methods across task, class, and domain incremental scenarios and critique experimental settings such as multi-head outputs and permuted datasets.The cited analyses examine common continual-learning assumptions and evaluation choices.
  • Gap addressed: Earlier empirical analyses were limited to small datasets such as MNIST and Fashion-MNIST.This limitation motivates broader comparisons across more realistic dataset scales.
  • Scenario taxonomy: Table 1 organizes continual-learning scenarios according to differences between consecutive input and target-label distributions.It distinguishes scenarios using P(X), P(Y), and whether output spaces are disjoint and task-ID separated.

3. Problem Definition and Evaluation Metrics

The paper formalizes online continual learning as single-pass supervised classification from a potentially infinite non-i.i.d. stream, with limited access to current data and external memory. It evaluates accuracy, forgetting, transfer, and runtime across task settings.

  • Problem definition: The classifier receives each mini-batch once and can access current data plus an external memory containing selected samples or other useful data.The memory may be reused multiple times, unlike the one-pass stream samples.
  • Problem settings: The framework assumes each task distribution is locally i.i.d. and stationary, while also accommodating non-i.i.d. sampling and concept drift within a task.This is an explicit simplifying assumption rather than a restriction of the broader framework.
  • Problem definition: The learning objective is to acquire new samples without interfering with performance on previously observed samples.At each step, the algorithm uses the current mini-batch and the preceding memory contents.
  • Problem settings: Class incremental learning uses disjoint new classes with one shared output space, while domain incremental learning changes inputs but retains the same outputs.Neither class nor domain incremental learning provides task IDs at inference.
  • Evaluation metrics: Five metrics assess average accuracy, forgetting, forward transfer, backward transfer, and total running time.The metrics cover final performance, retention, transfer, and computational cost.
  • Evaluation metrics: Average accuracy is computed from accuracies a_i,j on held-out task test sets after training through task i.At i = T, A_T is the average accuracy after the full sequence.
  • Evaluation metrics: Average forgetting compares each task’s best earlier test accuracy with its accuracy after later training.The definition uses the maximum prior accuracy and the post-training accuracy for that task.
  • Evaluation metrics: Positive backward transfer measures beneficial effects of later tasks on earlier-task performance, while forward transfer measures effects on future-task performance.The two metrics isolate transfer toward preceding and future tasks.

4. Overview of Continual Learning Techniques

Continual-learning methods are grouped by regularization, memory, and parameter isolation, but their assumptions and required supervisory signals differ across settings. The survey highlights online feasibility and practical limitations of these technique families.

  • Overview: Different continual-learning methods make inconsistent assumptions, so supervisory-signal requirements help assess generalization capacity and enable fair comparison.The taxonomy includes regularization, memory-based, and parameter-isolation techniques, often combined in recent methods.
  • Supervisory signals: Task IDs are available during training and inference in task incremental learning, inferred during training for class incremental learning, and unavailable in domain incremental learning.Other supervisory signals can include natural-language task descriptions or attribute matrices.
  • Online feasibility: A method is online-able only if it learns in one pass without revisiting processed samples.iCaRL’s herding-based memory update requires all samples from a class, so methods using it are not online-able.
  • Regularization: Regularization methods mitigate forgetting by adding loss penalties or modifying parameter gradients.Knowledge distillation is treated separately because of its widespread use in continual-learning methods.
  • Regularization: Regularization and knowledge distillation make it difficult to balance retaining prior knowledge with learning current data over long streams.This is identified as a shortcoming of regularization-based techniques including KD.
  • Memory-based methods: Memory-based methods replay or regularize using stored subsets of previous samples, but raw-sample storage can be infeasible under privacy or storage constraints.Generative replay avoids raw samples but requires training a generative model and is currently unsuitable for more complex datasets.
  • Parameter isolation: Parameter isolation avoids interference by allocating separate parameters, either activating task-relevant parameters or adding parameters for new tasks.Most prior parameter-isolation methods require task IDs at inference.

5. Compared Methods

The survey compares regularization- and memory-based continual-learning methods, including EWC++, LwF.MC, GEM variants, and generic online memory procedures. These methods differ in how they preserve past knowledge, constrain updates, and manage memory.

  • Regularization-based methods: EWC++ regularizes parameter updates using a Fisher Information Matrix while maintaining a single moving-average matrix for online learning.EWC penalizes changes to parameters important for past tasks; EWC++ uses a more efficient online variant.
  • Regularization-based methods: LwF uses knowledge distillation to preserve old-task outputs while learning the current task, but depends heavily on relatedness between old and new tasks.LwF.MC adapts this approach to a single-head setting.
  • Memory-based methods: Generic online memory methods retrieve a memory mini-batch, jointly update on incoming and memory data, then update the buffer.Methods differ mainly in retrieval, model-update, and memory-update strategies.
  • Memory-based methods: A-GEM constrains updates using average memory loss, projecting the incoming gradient when it conflicts with the reference gradient.The compared implementation uses reservoir sampling for memory updates and random sampling for retrieval.
  • Method comparison: Table 4 summarizes continual-learning methods according to required supervisory signals and the techniques they use.The table defines task-ID-free status separately for testing and training.

Incremental Classifier and Representation Learning

The compared methods span replay strategies, classifier designs, gradient-based memory selection, and dynamic architectures. Their main differences concern how samples are stored or retrieved, how representations and classifiers are learned, and whether models are expanded across tasks.

  • Replay-based methods: iCaRL mixes memory-buffer samples with current-task samples for representation learning and uses binary cross entropy to address class imbalance.Its training objective also combines classification and knowledge-distillation losses for new and old classes.
  • Replay-based methods: Experience Replay uses reservoir sampling for memory updates, random retrieval, and cross-entropy training on incoming and memory mini-batches.Despite its simplicity, ER has been reported to outperform many specialized continual-learning approaches.
  • Replay-based methods: MIR retrieves samples whose losses increase most after a virtual update estimated from the incoming mini-batch.This targets the memory samples most interfered with by the anticipated update.
  • Memory selection: GSS diversifies memory contents by comparing samples’ gradient directions, whereas GDumb maintains a balanced class buffer and trains from scratch on it at inference.GDumb was not specifically designed for continual learning but performs competitively.
  • Dynamic architectures: CN-DPM expands a group of experts using a Dirichlet Process Mixture, with each expert combining a classifier and generative model.The method models the overall conditional distribution as a mixture of task-wise conditional distributions.

6. Tricks for Memory-Based Methods in the OCI Setting

The survey attributes catastrophic forgetting in class-incremental learning largely to class imbalance, which biases classifiers toward new classes and under-represents minority classes. It evaluates simple remedies that modify losses, classifiers, training schedules, or replay usage.

  • Recency bias: Class imbalance penalizes old-class logits during cross-entropy training because new-class samples substantially outnumber stored old-class samples.The resulting logits become smaller for old classes, biasing predictions toward new classes.
  • Recency bias: Class imbalance also under-represents minority classes in feature space, making classification harder for NCM and cosine-similarity classifiers.Minority classes may lack discernible latent patterns.
  • Compared tricks: The survey evaluates seven simple tricks, including Labels Trick, KDC, multiple iterations, NCM, separated Softmax, Review Trick, and related loss modifications.These tricks are presented as building blocks for mitigating recency bias.
  • Loss modifications: Labels Trick masks outputs for classes absent from the current mini-batch, preventing their logits from being overly penalized during training.Its loss considers only outputs corresponding to current-mini-batch classes.
  • Knowledge distillation: The evaluated KDC formulation can fail with a large memory buffer because a small λ makes distillation dominate and drives new-class accuracy almost to zero.Under this condition, the model cannot learn new knowledge effectively.
  • Classifier modifications: NCM replaces the fully connected Softmax classifier with prototype-based prediction using the most similar class mean.In class-incremental learning, prototypes are approximated from memory-buffer samples.

7. Online Continual Hyperparameter Tuning

Online continual-learning hyperparameter tuning must avoid selecting settings by repeatedly sweeping the full data sequence. The survey discusses a stream-based protocol that separates cross-validation from final single-pass training and evaluation.

  • Motivation: Offline hyperparameter sweeps over the whole data sequence violate the online continual-learning setting and may produce results that are difficult to reproduce in real applications.The protocol assumes a classifier can make only one pass over the data.
  • Online protocol: The online protocol divides the stream into DCV for cross-validation and DEV for final training and evaluation.Multiple passes over DCV are allowed, while training on DEV remains single-pass; metrics are reported on DEV test sets.

8. Experiments

Experiments compare continual-learning methods across online class- and domain-incremental settings, memory sizes, datasets, metrics, and practical tricks. MIR is broadly robust, while the strongest alternative depends on memory size and setting.

  • OCI results: iCaRL shows the best performance with small memory buffers in Split CIFAR-100 and Mini-ImageNet.
  • OCI results: GDumb outperforms other methods by a large margin with 5k and 10k memory buffers, but is less effective on larger CORe50-NC.GDumb trains only with the memory buffer, making its proportional memory disadvantage more consequential on larger datasets.
  • Metrics and forgetting: KD-based and dynamic-architecture methods can show lower forgetting alongside lower accuracy, reflecting intransigence rather than uniformly better retention.The experiments report that iCaRL, LwF, and CN-DPM have lower forgetting, while strong regularization can reduce learning on new tasks.
  • Catastrophic forgetting: New-class bias is widespread: A-GEM classifies all old samples as new, while ER and MIR misclassify 72% and 66% of old samples, respectively.Higher new-class logits, weights, and bias terms in the fully connected layer explain the observed prediction bias.
  • Tricks: All seven tricks benefit the base methods; NCM is robust across memory sizes, while LB and RV are more advantageous with smaller and larger memory, respectively.For M=10k, RV improves ER from 18.4% to 32.0% and MIR from 19.3% to 35.2%; NCM and RV also incur memory-dependent runtime increases.
  • ODI results: MIR remains robust across memory sizes and is strongly competitive in ODI, whereas GDumb performs poorly and fails completely on CORe50-NI.In ODI, replay-based methods outperform other methods because class imbalance is absent and sequential replay better supports gradually changing nonstationarity.

9. Trendy Directions in Online CL

The survey identifies raw-data-free learning, feature replay, meta-learning, deployment, and applications beyond image classification as active directions. Privacy constraints and the difficulty of generating high-quality complex images remain important boundaries.

  • Raw-data-free methods: Raw-data-free continual learning is motivated by privacy and security concerns, but regularization alone cannot reach decent class-incremental performance.
  • Raw-data-free methods: Generative replay remains impractical for complex datasets because current deep generative models cannot generate satisfactory images and require lengthy training.
  • Raw-data-free methods: Feature replay stores latent features instead of raw samples, using layer freezing or feature adaptation to keep replayed representations valid as the model changes.
  • Future directions: Raw-data-free methods are expected to attract more research as data privacy and security concerns increase.
  • Meta-learning: Meta-learning-based continual learning uses related learning episodes to generalize across tasks, with online methods such as MER and Continual-MAML among emerging approaches.
  • Other areas and deployment: Online continual learning is extending to object detection, recommender systems, recurrent networks, and practical on-the-job deployment.

10. Conclusion

The survey finds that online continual-learning performance depends strongly on memory size, dataset scale, and setting. MIR is broadly versatile, while several tricks substantially improve OCI performance and bring online learning closer to offline training.

  • Overall evaluation: Extensive experiments compare nine methods and seven tricks across online class-incremental and domain-incremental settings.The evaluation targets the settings where competing approaches work best.
  • OCI methods: When memory is small, iCaRL performs best on CIFAR100 and Mini-ImageNet, followed by CN-DPM.The margin is small on CIFAR100 and large on Mini-ImageNet.
  • OCI methods: With larger buffers, GDumb outperforms CL-specific methods on CIFAR100 and Mini-ImageNet but requires much longer training times.Its advantage is reported for the larger-memory OCI setting.
  • OCI methods: On the larger CORe50-NC dataset, MIR consistently surpasses all other methods across memory sizes.This supports MIR’s robustness as dataset scale increases.
  • Forgetting and transfer: Class imbalance biases the last fully connected layer toward new classes, and none of the methods achieves positive forward or backward transfer.The survey experimentally and theoretically identifies this bias as a key cause of catastrophic forgetting.
  • Tricks: For OCI tricks, LB and NCM yield around 64% relative improvement with small memory, while NCM and RV become more effective with larger buffers, reaching around 80% relative improvement.NCM and RV also increase running time as memory grows.
  • ODI results: In ODI, replay-based methods are comparable across memory sizes and outperform other methods, whereas GDumb is no longer effective and other OCI methods do not generalize.MIR remains strongly competitive in this setting.
  • Conclusion: Combining the strongest methods and tricks brings online continual learning closer to matching offline training for very small mini-batches.The conclusion highlights potential deployment on edge and RAM-limited devices.

Appendix B.2. Implementation Details

The appendix details implementation choices, hyperparameter tuning, datasets, and additional OCI results. It shows that trick effectiveness depends on memory size and that some methods require substantially more sensitive tuning.

  • Tuning: Hyperparameters are tuned using the first DCV tasks while ensuring each task’s data is not seen more than once.DCV is set to 2 in this work.
  • Method settings: ER uses reservoir sampling for memory updates and random retrieval with mini-batches of size 10.The retrieval batch size is independent of memory-buffer size.
  • Method settings: MIR reduces computational cost by searching among C randomly selected memory samples and tunes LR, WD, and C.The candidate subset is used for criterion search.
  • Method settings: iCaRL replaces herding-based memory updates with reservoir sampling for the online setting and uses random retrieval.Learning rate and weight decay are tuned.
  • Method settings: CN-DPM is more sensitive to hyperparameters than other methods and uses scenario- and dataset-specific grids.The tuned parameters include α and classifier chill cc.
  • Additional OCI results: The OCI figures report end-of-task average accuracy on Split CIFAR-100, Mini-ImageNet, and CORe50-NC for 1k, 5k, and 10k buffers.These figures provide results across three memory capacities.
  • Additional OCI results: On Split Mini-ImageNet, all tricks are beneficial; LB and KDC* help most with small buffers, while NCM and RV help more with large buffers.NCM is less effective than on CIFAR-100 with a 10k buffer.

Appendix C.3. More Results for ODI Setting

The appendix provides additional ODI accuracy results across three datasets and three memory-buffer sizes. The listed figures and tables organize comparisons by dataset, setting, and memory capacity.

  • ODI results: ODI results cover Mini-ImageNet-Noise, Mini-ImageNet-Occlusion, and CORe50-NI with 1k, 5k, and 10k memory buffers.Average accuracy is measured at the end of each task.
  • Supporting materials: The appendix also lists the hyperparameter grid for compared methods and the OCI trick-performance table for Split Mini-ImageNet.The table reports end-of-training average accuracy for 1k, 5k, and 10k buffers.
  • ODI figures: Figures C.5, C.6, and C.7 report end-of-task average accuracy for the three ODI datasets.Each figure compares three memory sizes.
Loading 2101.10423v4…