Source-linked AI summary
Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams
Matthias De Lange, Tinne Tuytelaars
TL;DR
Non-stationary streams make online prototypes outdated and cause catastrophic forgetting, while existing continual-learning setups often require task information. The paper introduces CoPE, which continually evolves prototypes in a shared latent space with replay, balanced memory, and a pseudo-prototypical proxy loss, within a two-agent learner-evaluator framework. CoPE achieves state-of-the-art performance across eight benchmarks, including three highly imbalanced streams.
Problem
Non-stationary streaming data makes online prototypes rapidly outdated, while continual learning can suffer catastrophic forgetting as neural parameters adapt over time.
Method
CoPE continually evolves class prototypes in a shared latent space using replay, balanced memory, and a pseudo-prototypical proxy loss within a learner-evaluator framework.
Results
CoPE achieves state-of-the-art performance by a significant margin on eight benchmarks, including three highly imbalanced data streams.
Takeaways & Limitations
The framework supports task-free online learning and evaluation while remaining robust to highly imbalanced data streams.
Takeaways & Limitations
Prototype updates approximate the parent-distribution mean under an assumption that is optimal for Bregman divergences.
Abstract
from arXiv · showhide
Attaining prototypical features to represent class distributions is well established in representation learning. However, learning prototypes online from streaming data proves a challenging endeavor as they rapidly become outdated, caused by an ever-changing parameter space during the learning process. Additionally, continual learning does not assume the data stream to be stationary, typically resulting in catastrophic forgetting of previous knowledge. As a first, we introduce a system addressing both problems, where prototypes evolve continually in a shared latent space, enabling learning and prediction at any point in time. In contrast to the major body of work in continual learning, data streams are processed in an online fashion, without additional task-information, and an efficient memory scheme provides robustness to imbalanced data streams. Besides nearest neighbor based prediction, learning is facilitated by a novel objective function, encouraging cluster density about the class prototype and increased inter-class variance. Furthermore, the latent space quality is elevated by pseudo-prototypes in each batch, constituted by replay of exemplars from memory. As an additional contribution, we generalize the existing paradigms in continual learning to incorporate data incremental learning from data streams by formalizing a two-agent learner-evaluator framework. We obtain state-of-the-art performance by a significant margin on eight benchmarks, including three highly imbalanced data streams.
1. Introduction
Continual learning must handle non-stationary streams while balancing retention of prior knowledge against rapid adaptation. The paper proposes task-free online prototype evolution and a learner-evaluator framework for continual learning from streams.
- Non-stationary streaming data violates iid assumptions and creates catastrophic forgetting alongside a stability–plasticity trade-off.The paper identifies this balance as a central hurdle for continual learning systems.
- Static training and testing paradigms limit continual learning, motivating independent continual optimization and evaluation from data streams.
- CoPE continually evolves class prototypes in a shared lower-dimensional latent space, keeping them representative as the stream changes.This shifts prototype-related forgetting away from the full network parameter space.
- Replay and a balancing memory population scheme make the system robust to highly imbalanced data streams.
- The pseudo-prototypical proxy loss uses batch information to support learning in the non-stationary online regime.
2. The learner-evaluator framework
The learner-evaluator framework separates continual optimization from independent evaluation and formalizes data incremental learning without task identifiers. It covers streams ranging from online to offline horizons and supports static or evolving evaluation distributions.
- The framework separates continual learning into learner and evaluator agents, replacing static training and testing phases with continual optimization and evaluation.
- The learner updates fθ from stream samples using an observable horizon D, processing batches B and bounded operational memory M.The horizon ranges from D = B for online continual learning to D = S for offline learning.
- The evaluator independently measures fθ on an evaluation stream, including potentially unobserved concepts for zero-shot performance.
- Evaluation can use static concept distributions to measure retained knowledge or evolving distributions to measure performance on the current horizon.
- Data incremental learning removes assumptions about horizon or observation order and provides no explicit task identifier t.The learner processes B = D online or infers an implicit identifier from stream statistics.
- Unlike online learning, data incremental learning evades the iid assumption and can use horizons larger than a processing batch.
3. Prior work
Prior continual-learning methods commonly depend on task information, static prototypes, or replay strategies with resource and imbalance limitations. The paper positions CoPE as an online prototype method using evolving representations, balanced replay, and batch pseudo-prototypes.
- Replay methods preserve representative data through stored samples or generative models, but task-based methods often rely on task identifiers unavailable in real-world streams.
- Existing prototypical approaches keep prototypes static between task transitions, requiring exhaustive recalculation from memory exemplars before evaluation.CoPE instead evolves prototypes online with the stream.
- Reservoir sampling approximately follows stream frequencies, which can severely reduce performance on underrepresented tasks in imbalanced streams.
- MIR retrieves replay samples using losses but requires additional forward passes and a model copy.
- Figure 2 presents continual updates to fθ and class prototypes, with PPP-loss directions increasing inter-class and reducing intra-class variance.
- CoPE’s pseudo-prototypical proxy loss combines decoupled prototypes with batch pseudo-prototypes to reduce class interference in latent space.
4. Continual Prototype Evolution
CoPE maintains evolving class prototypes in a normalized latent space while replay and pseudo-prototypes stabilize representations and separate classes. Its memory allocation is class-balanced to remain robust under imbalanced streams.
- Continual Prototype Evolution: CoPE maps batches from the stream and replay memory into a normalized low-dimensional latent space, then predicts with nearest-neighbor prototypes.Operational memory is divided into replay memory M_r and prototypical memory M_p.
- Evolving representations: Replay exemplars correct prototype estimates as the embedding network changes, while high-momentum batch updates stabilize stochastic prototype evolution.The replay batch joins an equally sized memory batch, and α≈1 is validated empirically in an ablation study.
- Memory management: CoPE divides replay memory equally among observed classes, making capacity independent of stream characteristics and avoiding computationally expensive herding.Random class-specific subsets approximate class centers of mass without recalculating feature means for changing memory sizes or network parameters.
- Pseudo-Prototypical Proxy loss: Class prototypes approximate latent-space centers of mass, providing reference points that reduce intra-class variance and increase inter-class variance.The objective uses attractor sets containing same-class examples and repellor sets containing other-class examples.
- Pseudo-Prototypical Proxy loss: The pseudo-prototypical objective formulates a binary classification problem over attractor and repellor relationships, using expected posterior probabilities and negative log-likelihood.The batch objective assumes independent instance pairs when approximating the joint probability.
- Optimal prototypes: The complementary normalized cosine similarity is efficient for full-batch similarity computation and has the cluster-mean property used by the prototype formulation.For unit vectors, squared Euclidean distance is proportional to cosine distance.
5. Experiments
The experiments evaluate CoPE on balanced and highly imbalanced continual data streams derived from three Split datasets. They vary stream imbalance, memory capacity, and task-information access while averaging results across initializations.
- Experimental setup: The study examines five balanced streams and 15 highly imbalanced variants based on Split-MNIST, Split-CIFAR10, and Split-CIFAR100.Results are averaged over 5 different network initializations.
- Imbalanced data streams: Imbalanced benchmarks remove equality assumptions on task durations by making one task substantially larger than the others across Split-MNIST, Split-CIFAR10, and Split-CIFAR100 variants.Split-MNIST and Split-CIFAR10 use a factor-10 difference, while Split-CIFAR100 uses 2.5k samples for the large task and 1k for the others.
- Buffer-size analysis: Figure 3 reports accuracies across replay-buffer sizes for balanced Split-MNIST, Split-CIFAR10, and Split-CIFAR100, with averages over observed buffer sizes.An asterisk marks learners using task information.
6. Results and discussion
CoPE consistently performs strongly on balanced streams and significantly outperforms baselines on highly imbalanced streams. Ablations indicate that pseudo-prototypes improve latent-space quality, while balanced replay and continually evolving prototypes support robustness.
- CoPE consistently achieves state-of-the-art results across the three balanced data streams.
- 0.7%, 3.7%, and at least 1.5% are CoPE’s reported improvements over competitors or baselines on Split-MNIST, Split-CIFAR10, and Split-CIFAR100, respectively.
- 2.2%, 9.2%, and 3.3% are CoPE’s gains over CoPE-CE on the three balanced benchmarks.
- CoPE significantly outperforms all baselines across 15 highly imbalanced variants, with low standard deviation indicating robustness across sequence variations.
- Including pseudo-prototypes significantly improves performance, while smaller batch sizes of 10 and 20 remain similar and larger sizes deteriorate.
7. Conclusion
The paper introduces a two-agent framework for task-free continual learning and proposes CoPE for online learning from non-stationary data streams. CoPE keeps prototypes representative as representations evolve and uses them to address catastrophic forgetting.
- The two-agent learner-evaluator framework separates continual optimization from continual evaluation and formalizes task-free data incremental learning.
- CoPE learns online from non-stationary data streams without requiring task information.
- CoPE prevents prototypes from becoming obsolete in an evolving representation space while using them to combat catastrophic forgetting.