Source-linked AI summary
New Insights on Reducing Abrupt Representation Change in Online Continual Learning
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, Eugene Belilovsky
TL;DR
Online continual learning must incorporate new classes without disrupting representations of previously learned classes under memory and compute constraints. The paper analyzes why standard Experience Replay causes representation drift and proposes asymmetric losses that shield older representations; the resulting methods achieve strong gains over replay baselines, including a 36% average relative accuracy gain over traditional ER.
Problem
Standard Experience Replay can heavily perturb older-class representations when new classes arrive, causing disruptive updates and accuracy degradation, particularly at task boundaries and with small buffers.
Method
The paper separates incoming-data and replay losses so new classes are learned in isolation before their discrimination against older classes is learned through replay.
Results
The proposed asymmetric methods achieve strong benchmark performance, including a 36% average relative accuracy gain over traditional ER, with especially high improvements for small buffers.
Takeaways & Limitations
Asymmetric treatment of incoming and replay data mitigates old-representation drift without hindering discrimination between new and old classes.
Abstract
from arXiv · showhide
In the online continual learning paradigm, agents must learn from a changing distribution while respecting memory and compute constraints. Experience Replay (ER), where a small subset of past data is stored and replayed alongside new data, has emerged as a simple and effective learning strategy. In this work, we focus on the change in representations of observed data that arises when previously unobserved classes appear in the incoming data stream, and new classes must be distinguished from previous ones. We shed new light on this question by showing that applying ER causes the newly added classes' representations to overlap significantly with the previous classes, leading to highly disruptive parameter updates. Based on this empirical analysis, we propose a new method which mitigates this issue by shielding the learned representations from drastic adaptation to accommodate new classes. We show that using an asymmetric update rule pushes new classes to adapt to the older ones (rather than the reverse), which is more effective especially at task boundaries, where much of the forgetting typically occurs. Empirical results show significant gains over strong baselines on standard continual learning benchmarks.
1 INTRODUCTION
The paper studies representation drift in online continual learning, where replay can disrupt previously learned classes when new classes arrive. It proposes asymmetric losses that protect older representations while achieving strong benchmark gains.
- 1 INTRODUCTION: Representation drift causes large disruptions in accuracy, especially at the initial task boundary or distribution shift.The work focuses on how previously learned class features change over time.
- 1 INTRODUCTION: New-class representations can lie near old-class clusters, while standard ER heavily perturbs older representations after only a few updates.The authors attribute this to the interaction between overlapping representations and cross-entropy loss on seen and unseen classes.
- 1 INTRODUCTION: Low buffer sizes exacerbate disruption, whereas ER recovers from initial displacement only with much larger buffers.Figure 1 illustrates the resulting task-boundary performance behavior.
- 1 INTRODUCTION: The proposed approach uses separate losses for incoming and replayed data, first learning new classes in isolation before replay-based discrimination against older classes.ER-AML excludes previously learned classes from incoming-data negatives, while ER-ACE applies asymmetric cross-entropy to incoming and replay data.
- 1 INTRODUCTION: The methods achieve an average relative gain of 36% in accuracy over traditional ER, with especially high improvements in the small-buffer regime.The authors also report strong gains over replay baselines under a real-world-constraint evaluation framework.
2 RELATED WORK
Related work situates this paper in online, non-i.i.d. continual learning with shared outputs and no task identifiers. It connects the study to catastrophic forgetting and class-imbalance research in replay-based and class-incremental settings.
- 2 RELATED WORK: Continual-learning research spans sequential settings including task-incremental and online streams with changing input domains or output spaces.The paper distinguishes these settings through prior categorizations and surveys.
- 2 RELATED WORK: This work targets a single-head setting in which the learner is not told which task each sample belongs to.The stream may change at any time, making task identity unavailable during training and evaluation.
- 2 RELATED WORK: The paper relates its analysis to class-incremental methods that address implicit class imbalance when new classes are learned alongside replayed data.Its focus is the underlying cause of performance degradation in replay-based methods.
3 LEARNING SETTING AND NOTATION
The learning setting is an ongoing stream of labeled batches whose distribution can suddenly change without task identifiers. The objective is to learn incoming data while avoiding increased loss on previously learned classes, using a fixed replay memory.
- 3 LEARNING SETTING AND NOTATION: At each time step, the learner receives a labeled batch from a distribution that may suddenly switch to a new distribution.The learner is neither explicitly told when switches occur nor allowed to use task identifiers.
- 3 LEARNING SETTING AND NOTATION: The training objective minimizes loss on newly arriving data without negatively interfering with previously learned classes.The model uses parameters θ and classification loss L in this formulation.
- 3 LEARNING SETTING AND NOTATION: Experience Replay combines incoming data with samples replayed from a fixed-size memory to preserve prior-class performance.This is presented as a simple and efficient approach to reduce interference.
4 METHODS
The methods reduce representation drift by treating incoming and replayed data asymmetrically. ER-AML restricts incoming-batch negatives, while ER-ACE applies an analogous restriction to cross-entropy classes.
- ER-AML: ER-AML combines a metric-learning loss on incoming data with a cross-entropy loss on replayed buffer data.The incoming loss uses SupCon, while replay data uses a cross-entropy-type objective.
- ER-AML: Incoming samples use positives and negatives drawn from the current batch or memory, with negative selection controlling representation drift.The method considers negatives from mixed current and previous classes or only current incoming classes.
- ER-AML: Using only incoming-batch negatives avoids excessive drift by preventing sharp gradient changes on previous-class features at task boundaries.The paper measures one-iteration drift as ∥fθt(x) − fθt+1(x)∥ for normalized network outputs.
- ER-ACE: ER-ACE restricts the incoming cross-entropy denominator to classes present in the current batch, while replay uses current and previously learned classes.This restriction has an effect analogous to limiting negatives in contrastive learning.
- ER-ACE: The asymmetric loss combines incoming and replay objectives without additional computational overhead.Ccurr contains classes in the incoming batch, while Cold contains previously seen classes absent from that batch.
5 EXPERIMENTS
The experiments evaluate asymmetric replay methods on standard online continual-learning benchmarks under single-head, computation, memory, anytime, and augmentation-aware settings. The proposed methods outperform replay baselines, while evaluation also examines blurry task boundaries and compute efficiency.
- Evaluation criteria: Anytime evaluation measures performance throughout the stream rather than relying only on final accuracy.The evaluation also reports Averaged Anytime Accuracy and computation costs in FLOPs.
- Experimental setup: Experiments use Split CIFAR-10, Split CIFAR-100, and Split MiniImagenet under a single-head protocol with fixed batch and rehearsal sizes.Each method is run 10 times, with mean and standard error reported.
- Standard benchmark results: ER-AML and ER-ACE consistently outperform other methods on CIFAR-10 across buffer sizes.The comparison is conducted with shared buffer management and standard replay baselines.
- Standard benchmark results: ER-ACE outperforms the single-pass iid baseline on Mini-Imagenet and nearly reaches the equal-compute iid baseline.This result is reported without additional computation cost.
- Evaluation considerations: Data augmentation can substantially improve replay results, making equal access to augmentation important for fair comparisons.For MiniImagenet, augmentation did not improve performance in the reported experiments.
- Blurry task boundaries: The experiments also study continuously evolving streams by linearly interpolating between tasks and evaluating task-free methods.The blurry-boundary setting uses Split-CIFAR10 with incoming batches averaging two unique labels.
6 CONCLUSION
The conclusion attributes performance degradation to excessive pressure from the standard loss on old-class representations. It presents asymmetric treatment of incoming and replay data as a way to reduce drift while maintaining strong performance with minimal or no extra cost.
- 6 CONCLUSION: The standard loss applies excessive pressure to old-class representations in online continual learning.The paper identifies abrupt representation change as a source of performance degradation.
- 6 CONCLUSION: The proposed methods treat incoming and replay data asymmetrically to address representation drift.The approach does not require knowledge of the current task.
- 6 CONCLUSION: The methods achieve strong performance on long task sequences with minimal or no additional cost.The conclusion also emphasizes evaluation across many baselines and metrics.
7 REPRODUCIBILITY STATEMENT
The reproducibility statement describes a detailed implementation and hyperparameter-search setup, with fixed architectures and batch sizes for fair comparisons. It also documents baseline-specific settings and implementation differences.
- Reproducibility resources: The authors provide a codebase, grid-search results, optimal hyperparameters, and documentation intended to support reproduction.The codebase contains the results of the grid search and configuration details for methods and settings.
- Controlled comparisons: Additional experiments fix the batch and rehearsal batch sizes at 10 and keep the model architecture constant.These choices control computational cost across approaches.
- Implementation provenance: The reported DER++ results are reimplemented, whereas GDUMB results come from the authors’ public codebase.The authors also note differences from the original DER setup that affect performance comparisons.
- Method-specific details: Task-based methods use the task identifier and delay rehearsal until the second task, while ER uses a task-free implementation.The task-free ER implementation begins rehearsal when the buffer is nonempty and does not exclude current-task classes.
A.2 BLURRY TASK BOUNDARIES EXPERIMENT
The blurry-boundary experiment replaces sharply separated tasks with a smooth class-probability stream. Its sampling parameters are chosen to preserve the original average number of labels per minibatch while removing a clear task identity.
- Stream construction: The smooth stream spans 5,000 minibatches and assigns each class a time-dependent unnormalized probability.At each timestep, class probabilities are normalized before categorical sampling.
- Stream construction: The class-probability distribution uses parameters whose means depend on class sample counts.The passage defines the mean as µc = (2c−1)Nc / 2.
- Experimental control: The parameters are selected so incoming minibatches contain two unique labels on average, matching the original task-based experiment.The setting has no current task or equivalent set of current labels.
- Method eligibility: SS-IL cannot be used in this task-free setting because it requires a task identifier during training.The smooth stream therefore excludes methods that depend on explicit task identity.
B AN IN-DEPTH ANALYSIS OF SS-IL IN THE ONLINE SETTING
In the online single-head setting, SS-IL fails to learn and classify newly observed tasks, while ER learns them abruptly and ER-ACE offers a slower tradeoff. Additional analysis examines whether SS-IL’s behavior extends beyond class imbalance.
- SS-IL is analyzed as a method whose online behavior may involve more than correcting class imbalance.The section tests SS-IL when class imbalance is absent.
- ER learns the current task too abruptly, while ER-ACE incorporates new-task knowledge more slowly and SS-IL barely learns newly observed tasks.
- SS-IL cannot classify current-task samples in the single-head setting because masked incoming and rehearsal losses provide no cross-task discrimination objective.The method performs worse than random chance on current-task samples, whereas ER-ACE uses an unmasked rehearsal loss.
C OVERFITTING ON BUFFERED SAMPLES
The paper connects representation drift and buffer overfitting to replay behavior, then evaluates asymmetric methods, negative selection, and combinations with DER++. ER-ACE and ER-AML preserve stronger alignment and generally improve over ER under these analyses.
- ER-ACE retains stronger alignment between buffered and held-out representations for earlier tasks, indicating reduced buffer overfitting relative to ER.Alignment is measured by same-class cosine similarity between buffer and validation representations.
- Combining ER-ACE with DER++ yields small accuracy gains and significant gains in forgetting.
- ER produces suddenly high gradients on previous-class features at the task switch, potentially causing large representation drift.Using only incoming negatives decreases the gradient norm compared with using all negatives.
- ER-AML with SupCon and triplet losses both outperform ER, while SupCon performs better at larger buffer sizes.
- Negative selection is critical: ER-AML with all negatives performs similarly to but slightly better than ER, whereas selected incoming-class negatives greatly improve performance.
H ADDITIONAL DRIFT RESULTS
The paper reports that negative selection substantially affects representation change and observes similar behavior when comparing ER with ER-ACE.
- Negative selection significantly affects representation change, with similar behavior observed for ER versus ER-ACE.
I ANALYSIS OF THE REPRESENTATIONS DURING THE SECOND TASK
During the second task, ER displaces old-class prototypes and leaves them misaligned, whereas ER-ACE and ER-AML progressively cluster classes and align prototypes. Across varying task overlap and evaluation settings, the proposed methods show strong improvements over baselines.
- Representation evolution: After 100 iterations, ER significantly displaces old-class prototypes, while ER-ACE and ER-AML begin separating classes with prototypes near their respective classes.
- Representation evolution: After 400 iterations, ER still struggles to align prototypes, ER-ACE has clustered classes, and ER-AML clusters them more slowly.
- Representation evolution: At the end of the second task, ER-ACE and ER-AML align prototypes with class clusters, unlike ER, which retains prototype misalignment and overlap between Classes 2 and 3.
- Blurry task boundaries: Task overlap is measured by the average number of unique classes per incoming minibatch, with larger values indicating stronger overlap.
- Blurry task boundaries: Across a wide range of task blurriness levels, the proposed methods show strong improvement over other task-free baselines.
- Limited training data: Using additional training-data percentages and matched rehearsal compute, the proposed methods again outperform the baselines.The experiment uses CIFAR-10 with M = 20 and averages results over 5 runs.
- Additional benchmark results: Full results across Split CIFAR-10, Split CIFAR-100, and Split MiniImagenet, with and without augmentation, largely align with the main results and show anytime performance.