Source-linked AI summary
Continual Unsupervised Representation Learning
Dushyant Rao, Francesco Visin, Andrei A. Rusu, Yee Whye Teh, Razvan Pascanu, Raia Hadsell
TL;DR
CURL addresses continual learning without task labels, task boundaries, or external supervision by inferring tasks in a mixture-of-Gaussians latent model. It dynamically expands to capture new concepts and uses mixture generative replay to reduce forgetting, achieving meaningful representations and competitive performance on MNIST and Omniglot.
Problem
Continual learning has largely assumed supervised or reinforcement-learning tasks with known task labels or boundaries, leaving unsupervised learning under non-stationary data as an open problem.
Method
CURL performs task inference through a mixture-of-Gaussians latent space, dynamically expands mixture components, and uses mixture generative replay to address forgetting.
Results
CURL learned meaningful class-discriminative representations without forgetting in sequential MNIST and Omniglot settings and remained competitive with prior work on i.i.d. benchmarks.
Takeaways & Limitations
CURL supports unsupervised continual representation learning across sequential, poorly bounded, and shuffled-data settings without relying on task labels.
Abstract
from arXiv · showhide
Continual learning aims to improve the ability of modern learning systems to deal with non-stationary distributions, typically by attempting to learn a series of tasks sequentially. Prior art in the field has largely considered supervised or reinforcement learning tasks, and often assumes full knowledge of task labels and boundaries. In this work, we propose an approach (CURL) to tackle a more general problem that we will refer to as unsupervised continual learning. The focus is on learning representations without any knowledge about task identity, and we explore scenarios when there are abrupt changes between tasks, smooth transitions from one task to another, or even when the data is shuffled. The proposed approach performs task inference directly within the model, is able to dynamically expand to capture new concepts over its lifetime, and incorporates additional rehearsal-based techniques to deal with catastrophic forgetting. We demonstrate the efficacy of CURL in an unsupervised learning setting with MNIST and Omniglot, where the lack of labels ensures no information is leaked about the task. Further, we demonstrate strong performance compared to prior art in an i.i.d setting, or when adapting the technique to supervised tasks such as incremental class learning.
1 Introduction
CURL addresses unsupervised continual learning, where task identity, task boundaries, and external supervision are unavailable. It performs task inference within the model, expands capacity for new concepts, and uses rehearsal-based methods to reduce forgetting across varied data regimes.
- Unsupervised continual learning removes task labels, task boundaries, and external supervision such as class labels, regression targets, or rewards.
- CURL learns task-specific representations over shared parameters while inferring tasks internally.
- Dynamic expansion lets CURL add capacity for new tasks, while additional rehearsal-based methods target catastrophic forgetting.
- The approach is evaluated with sequential, shuffled, and continuously transitioning tasks, and is also adapted to supervised learning without task knowledge or class labels.
2 Model
CURL combines a mixture-of-Gaussians latent model for task inference with dynamic component expansion and mixture generative replay. Its architecture shares lower-level representations while using component-specific heads and replay to learn new concepts and preserve earlier information.
- 2.1 Inference over tasks: CURL factorizes the generative model as p(x, y, z) = p(y)p(z | y)p(x | z), with categorical y selecting Gaussian parameters for latent z before decoding x.
- 2.1 Inference over tasks: The model uses a shared encoder, a softmax task-inference head for q(y | x), component-specific latent heads, and a shared decoder.
- 2.1 Inference over tasks: The ELBO combines expected reconstruction with KL regularisation, while marginalising the categorical variable exactly and sampling the continuous latent variable.
- 2.3 Dynamic expansion: Dynamic expansion collects poorly modelled samples and initializes a new component from the existing component most probable over that set.
- 2.3 Dynamic expansion: New components are fine-tuned on collected samples, adding parameters mainly in the encoder’s top layer while sharing low-level representations and the decoder.
- 2.4 Combatting forgetting via mixture generative replay: Mixture generative replay alternates real and generated data, tracks historical component usage, and uses generated component assignments as self-supervised signals.
- 2.4 Combatting forgetting via mixture generative replay: Replay snapshots can be updated periodically or immediately before dynamic expansion, consolidating prior information before adding a new concept.
3 Related Work
Related work spans generative models that learn structured latent spaces, continual-learning methods for mitigating forgetting, and task-agnostic approaches. Earlier task-agnostic continual-learning studies primarily addressed supervised settings, whereas CURL targets unsupervised task learning.
- Generative models: Generative-model work has used latent Gaussian mixtures and Bayesian or Dirichlet priors to capture class structure in an unsupervised latent space.
- Continual learning: Continual-learning methods address forgetting through parameter regularisation, constrained optimisation, replay, and related mechanisms.
- Task-agnostic continual learning: Task-agnostic continual-learning research considered incremental task, domain, and class learning, but the cited approaches focused on supervised learning tasks.
4 Experiments
Experiments evaluate CURL for unsupervised continual representation learning across sequential, continuous-drift, shuffled-data, and supervised settings. Ablations examine dynamic expansion and mixture generative replay, while benchmarks assess representation quality and adaptation beyond the core setting.
- Evaluation settings: Cluster accuracy and latent-space k-NN accuracy measure class-discriminative information in CURL representations.Cluster accuracy evaluates information in the categorical variable, whereas k-NN evaluates discriminability of the full latent space.
- Evaluation settings: CURL is evaluated on MNIST and Omniglot with i.i.d., sequential, and continuous-drift data presentations.Sequential data presents classes one after another; continuous drift gradually changes class ratios; i.i.d. data are shuffled.
- Continual class-discriminative representation learning: Dynamic expansion and mixture generative replay are both important for learning coherent MNIST representations without forgetting.With MGR and expansion, earlier classes remain well modelled, while confusion is concentrated more among similar classes.
- Ablation studies: 84% maximum cluster accuracy is obtained as sequential-learning component counts range from 12 to 71 under different expansion thresholds.Without expansion, accuracy is 51%; discovering 25 components with expansion reaches 77%, illustrating a capacity–memory trade-off.
- Learning with poorly-defined task boundaries: Continuous class drift generally performs better than sequential learning despite unclear task boundaries, with mixture probabilities adapting over time.Discrete changes produce sharp probability shifts, whereas continuous drift produces similar but smoother dynamics.
- External benchmarks: CURL remains competitive with prior methods on shuffled MNIST and Omniglot and can be adapted effectively to supervised splitMNIST tasks.In i.i.d. benchmarks, it is surpassed only by VaDE on MNIST and DirVAE on Omniglot; on incremental class learning, only iCARL outperforms it.
5 Conclusions
CURL addresses unsupervised continual learning when task labels and boundaries are unknown and tasks lack external supervision. On MNIST and Omniglot, it learns class-discriminative representations without forgetting and remains competitive in adapted benchmark settings.
- CURL performs task inference through a mixture-of-Gaussians latent space while dynamically expanding and using mixture generative replay.These mechanisms instantiate new concepts and minimise catastrophic forgetting.
- CURL learns meaningful class-discriminative representations without forgetting in sequential class settings, including poorly defined task boundaries.
- CURL remains competitive with previous work when adapted to unsupervised learning from i.i.d. data and supervised incremental class learning.
A.1 Generated samples
Generated samples are used to examine whether CURL preserves previously learned classes, while sample quality remains secondary to class-discriminative representation learning. Later samples retain early classes, though some reconstructions degrade while remaining identifiable.
- CURL’s primary objective is meaningful class-discriminative representations, not high-quality generated samples.The samples are examined for data variety and the quality sufficient to retain learned information.
- Later generated samples preserve classes observed early in sequential learning.
- Some previously learned symbols become degraded but remain identifiable after later classes are introduced.The passage gives zeros as an example of degraded earlier symbols.
- The authors hypothesise that capturing a class’s essence may be more useful for representation learning than pixel-perfect reconstruction.They leave a thorough analysis of this idea to future work.
A.2 Data buffers for dynamic expansion
CURL’s dynamic expansion uses poorly modelled samples to add components as distributions shift or difficult examples emerge. Buffer contents often reveal expansions associated with newly introduced classes and challenging variants.
- Dynamic expansion adds components when poorly modelled samples reach a critical size.
- Some expansion buffers contain samples from multiple classes when outliers from one class are insufficient to initialise a component alone.
- For many classes, the first expansion corresponds to a distribution change from introducing the class, while the second models outlying or challenging examples.Examples include “twos” and “curly twos,” and “threes” and “challenging threes.”
- The approach incorporates new components for both distributional shift and hard-example modelling.
A.3 Latent structure
CURL’s latent space develops relatively disjoint, temporally stable class regions as classes are introduced. Mixture components also remain reasonably aligned with individual classes, supporting the interpretation that prior representations are preserved.
- As classes are introduced incrementally, they occupy relatively disjoint latent-space regions that remain consistent over time.
- New classes do not appear to catastrophically interfere with representations learned for previous classes.
- The latent-space analysis colours points by class label or most probable mixture component across training steps.Only classes seen so far are plotted; class labels were unavailable to the model during learning.
- Mixture-component clusters cover individual classes with reasonable accuracy and maintain consistent positions over time.
- k-NN accuracy measures latent-space class discriminability without imposing a parametric class-boundary structure.Its value depends strongly on latent dimensionality, complicating comparisons across latent-space sizes.
- Raw-pixel k-NN baselines can perform surprisingly well on MNIST and Omniglot because of dataset image statistics and higher dimensionality.The passage reports approximately 3% k-NN error on MNIST for raw pixels.
B Datasets
MNIST contains handwritten digits, while Omniglot contains handwritten characters organized into 50 alphabets used as evaluation labels.
- MNIST contains handwritten samples of ten digits, with 50,000 training, 10,000 validation, and 10,000 test samples.
- Omniglot contains 20 samples from each of 1,623 characters, grouped into 50 alphabets.
- For Omniglot, each character contributes 15 training samples and 5 test samples, with alphabets serving as evaluation class labels.
C Experimental setup
Experiments use a common training protocol and dataset-specific architectures, with dynamic expansion controlled by likelihood thresholds and consolidation; external comparisons use separately specified configurations.
- Models are trained for 10^5 steps, with means and standard deviations reported across five random seeds.
- Main setup: The MNIST main setup uses an MLP shared encoder, up to K = 25 components, and 32-dimensional latent Gaussians.
- Main setup: The reported architecture was selected through a small hyperparameter sweep, with only small performance differences across alternatives.
- Dynamic expansion: Dynamic expansion flags samples below cnew = −200 as poorly explained and places them in a buffer, followed by 100-step consolidation.
- Omniglot: The Omniglot setup uses a two-layer 500-dimensional MLP encoder, K = 25 components, a 32-dimensional Gaussian, and a Bernoulli decoder trained with ADAM at learning rate 5 × 10^-4.
- External comparisons: External comparisons use separately specified architectures and capacities, including K = 10 components for one setup and K = 100 for another.