PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Robust Evaluations of Continual Learning

Sebastian Farquhar, Yarin Gal

arXiv:1805.09733v3stat.MLcs.LG

TL;DR

Continual-learning evaluations often fail to test the fundamental challenges motivating sequential learning, making some approaches appear stronger than they are. The paper formulates evaluation desiderata, analyzes standard designs, and proposes more robust experiments. It shows that existing evaluations are biased toward prior-focused approaches and that leading methods can fail when core desiderata are imposed.

  • Problem

    Current evaluations do not faithfully assess continual learning’s fundamental challenges, despite the field’s use of increasingly difficult datasets.

  • Method

    The paper defines evaluation desiderata, examines standard experiments, and proposes new designs using prior-focused, likelihood-focused, and hybrid approaches.

  • Results

    Existing evaluations are biased toward prior-focused approaches, while leading prior-focused methods fail on more robust evaluations imposing all five core desiderata.

  • Takeaways & Limitations

    Progress should be assessed with experimental designs that reflect the core challenges and real-world motivations of continual learning.

  • Takeaways & Limitations

    The five desiderata are not universally applicable, and some applications may deliberately relax particular requirements.

Abstract

from arXiv · show

Experiments used in current continual learning research do not faithfully assess fundamental challenges of learning continually. Instead of assessing performance on challenging and representative experiment designs, recent research has focused on increased dataset difficulty, while still using flawed experiment set-ups. We examine standard evaluations and show why these evaluations make some continual learning approaches look better than they are. We introduce desiderata for continual learning evaluations and explain why their absence creates misleading comparisons. Based on our desiderata we then propose new experiment designs which we demonstrate with various continual learning approaches and datasets. Our analysis calls for a reprioritization of research effort by the community.

1. Introduction

Continual learning is needed when data cannot be retained or distributions change rapidly, but current evaluations often fail to reflect these motivations. The paper proposes desiderata, diagnoses biased evaluations, and introduces more challenging experimental designs.

  • Motivation: Continual learning trains across sequential tasks while discarding prior data, yet neural networks tend to forget earlier tasks.Motivating applications include privacy-sensitive medical data and rapidly changing environments such as Mars-rover navigation.
  • Evaluation problem: Existing evaluations have major blind spots that obscure weaknesses in current continual learning approaches across commonly used datasets.The paper argues that dataset choice alone does not make an evaluation representative of continual-learning challenges.
  • Implication: The paper argues that robust evaluation design is necessary for assessing whether continual-learning methods address the field’s fundamental challenges.Its proposed designs aim to make those challenges more visible.
  • Contributions: The paper proposes dataset-independent desiderata, analyzes shortcomings in widely used evaluations, and introduces experimental designs intended to mitigate those issues.These contributions target more representative comparisons rather than a single benchmark.
  • Main result: Existing evaluations are empirically biased toward prior-focused approaches.This is presented as a central result of the paper’s analysis.

2. Continual Learning

Continual learning differs from i.i.d. supervised learning because data arrive in task-specific subsets and the model must retain performance across tasks without deliberately revisiting previous data.

  • Supervised learning: In supervised learning, parameters are learned from an i.i.d. labeled dataset to predict outputs for unseen examples.The paper uses this setting as the baseline for formalizing continual learning.
  • Continual-learning setting: Continual-learning data may be divided into disjoint subsets drawn from distinct task distributions.Tasks are encountered sequentially rather than as one pooled i.i.d. dataset.
  • Continual-learning setting: The goal is one model that predicts well on every task after sequential training without deliberately revisiting previous tasks.The model state after task t reflects training on datasets D1 through Dt.

3. Proposed Desiderata for Continual Learning Evaluations

The paper derives evaluation desiderata from diverse real-world applications rather than proposing one universal benchmark. These principles target task similarity, shared outputs, unknown task identity, restricted retraining, and long task sequences.

  • Rationale: Continual-learning applications vary widely, so specialized datasets are useful for subfields but cannot serve as an overall benchmark.The paper therefore develops sufficient desiderata that can apply across datasets and settings.
  • Core desiderata: A requires later-task inputs to resemble earlier tasks enough to sometimes trigger confident predictions of old classes.Permuted MNIST violates this condition because its random pixel rewiring is unlike many real-world shifts.
  • Core desiderata: B requires a shared output head when tasks use the same outputs, making the evaluation reflect decisions across all classes.The output-head choice substantially affects challenge difficulty.
  • Core desiderata: D disallows unconstrained retraining on old tasks, reflecting settings where returning to old data is impractical or violates privacy.The Mars-rover example requires moving back into old terrain before retraining there.
  • Core desiderata: E requires more than two tasks because success on two-task transfer does not guarantee performance over longer sequences.The paper also identifies unclear boundaries, continuous or overlapping tasks, resource constraints, and privacy guarantees as further challenges.
  • Further designs: The proposed further experiments address task-change detection through uncertainty and the trade-off between training time and accuracy.These designs seek richer evaluations without simply increasing dataset difficulty.

4. Existing Work

Existing work includes prior-focused, likelihood-focused, hybrid, and dynamic-architecture approaches, while evaluations commonly rely on MNIST variants and two-task transfers. The paper reviews these methods and designs to expose where comparisons can mislead.

  • Methods: Prior-focused methods use regularization to preserve important or “elastic” parameters across tasks.Examples include EWC, SI, VCL, Kronecker-factored Laplace approximation, and RW.
  • Methods: VCL transfers the posterior learned after one task as the prior for the next task within a variational-inference framework.Its Bayesian interpretation makes it a representative prior-focused method.
  • Methods: The approximate Bayesian loss combines a current-task log-likelihood term with a term measuring distance from the model prior.The prior covariance is estimated differently by different approaches.
  • Methods: Likelihood-focused approaches estimate likelihood on past tasks using mechanisms such as generative replay or coresets.Hybrid methods add limited likelihood estimation to prior-focused approaches.
  • Methods: Dynamic architectures can perform well but often require new models for each task, increasing computation and memory costs.The paper questions whether those growing costs solve the continual-learning problem itself.
  • Evaluations: Common evaluations emphasize Permuted MNIST, Split MNIST, multi-headed classification, and two-task transfer.These experiments are reviewed because their design choices recur throughout continual-learning research.
  • Evaluation context: Related work calls for stronger benchmarks, richer metrics, resource constraints, careful cross-validation, and privacy tracking, but does not consistently enforce all desiderata.The paper positions its critique within these earlier evaluation concerns.

5. Critical Analysis of Existing Evaluations

Common continual-learning evaluations simplify the problem in ways that favor prior-focused approaches and weaken their relevance to realistic settings. Pixel permutation, multi-headed classification, and two-task transfer each omit important aspects of continual learning.

  • Pixel permutation: Permuted MNIST creates an unrealistic best case because fully randomized inputs are unrecognizable relative to previous tasks.Real-world task shifts usually produce new situations that remain confusingly similar to old ones.
  • Pixel permutation: Permuted MNIST makes new-task examples uncertain, unlike non-randomized settings where models tend to make confident but false predictions.This difference directly affects how the model learns new tasks and may simplify continual learning.
  • Split classification: Multi-headed Split MNIST requires knowing the task and its classes, allowing models to choose among fewer labels than in single-headed evaluation.This setting is less relevant to applications because such task knowledge can make continual learning unnecessary.
  • Split classification: Prior-focused systems perform much better in multi-headed than single-headed evaluations, which can make failed approaches appear to solve continual learning.Single-headed Split MNIST tests whether predictions remain correct across all previously encountered classes and is recommended as a toy baseline.
  • Two-task transfer: Two-task transfer is not representative because realistic continual-learning applications may involve long task sequences exposing failures that remain hidden after one prior task.For EWC, Fisher estimates based on a locally accurate Taylor expansion can become inaccurate after the model moves substantially through parameter space.

6. Empirical Analysis of Existing Evaluations

The analysis shows that evaluation design, rather than dataset difficulty alone, determines whether continual learning methods are meaningfully distinguished. Experiments satisfying all five desiderata expose major differences, while common partial evaluations can favor prior-focused approaches or conceal their weaknesses.

  • Experimental approach: The study compares prior-focused, likelihood-focused, and hybrid approaches to expose which architectural aspects different evaluation designs challenge.The methods include VCL variants, EWC, and VGR.
  • Experiments with all five desiderata: Single-headed Split MNIST satisfies all five desiderata and reveals major performance differences: VCL without coresets forgets old tasks, EWC behaves similarly, and VGR performs nearly perfectly.VCL with coresets appears to derive its accuracy from retaining and retraining on coreset data.
  • Experiments with all five desiderata: Using FashionMNIST makes the single-headed experiment harder and worsens VCL relative to VGR without changing the performance ranking.The paper attributes this pattern to greater difficulty for the prior approximation than for the GAN.
  • Missing desiderata: Multi-headed Split MNIST and FashionMNIST make methods appear similarly successful, preventing clean differentiation and illustrating how missing desiderata can mislead evaluations.VCL performs slightly worse without coresets on multi-headed FashionMNIST.
  • Missing desiderata: Test-time knowledge of the correct output classes also fails to cleanly differentiate prior- and likelihood-focused methods on MNIST and FashionMNIST.Masking incorrect outputs changes evaluation without changing training, thereby weakening the test-time challenge.
  • Missing desiderata: Two-task evaluations are substantially simpler than longer sequences and can miss differences that emerge over many tasks, such as between VCL with coresets and VGR.The first two tasks in Figure 2 would not distinguish these methods, whereas the longer sequence does.

7. Discussion

The paper argues that flawed continual-learning evaluations create misleading comparisons and illusory progress, even when more realistic datasets are used.

  • Experimental design determines whether architectures are compared reproducibly and tested against continual learning’s fundamental objectives.
  • When experiments better reflect continual learning, recent leading approaches can fail even on simple datasets such as MNIST.
  • The paper calls for improving experimental design before increasing dataset realism, to address fundamental continual-learning challenges rather than show illusory progress.

Appendices

The appendices describe EWC, SI, VCL, and VGR as related continual-learning approaches, emphasizing their regularization, priors, coresets, and likelihood modeling.

  • EWC adds an L2 regularization term reflecting a Gaussian prior for each parameter.
  • EWC estimates parameter contributions using an approximate Fisher information matrix at the end of each task.
  • On Permuted MNIST, prior-focused methods were reported to perform strongly: EWC reduced forgetting, SI matched EWC, and VCL outperformed SI and EWC.
  • SI computes each parameter’s contribution to the loss gradient over the entire course of training, while its derivation explicitly extends only to two tasks.
  • VCL uses the posterior after one task as the prior for the next and can additionally train on small coresets withheld from old datasets.
  • VGR adapts the likelihood component using generative models trained to estimate likelihoods on past tasks, rather than changing priors between tasks.

A.5. Dynamic architecture approaches

The paper contrasts architecture and learning-dynamics approaches with analyses showing why Permuted MNIST can make prior-focused continual learning appear stronger than it is.

  • A.5. Dynamic architecture approaches: Task-specific architecture approaches can help in two-task transfer but may become impractical when continual learning involves many tasks and task-specific weights.
  • High entropy in Permuted Task B predictions indicates less confident, more uniform predictions than in the comparison setting.
  • Confident but false predictions on new tasks create large output-layer likelihood gradients, whereas Permuted MNIST produces unusually small gradients because its digits resemble none from the previous task.
  • In typical settings, large likelihood-term gradients dominate the prior early in training; in Permuted MNIST, the prior remains influential throughout training.
  • The comparison trains on Task A before Task B in Split and Permuted settings to assess the evaluation framework rather than continual-learning performance.
  • In early Task B training, the Split setting has larger likelihood-term gradients and a higher likelihood-to-prior ratio, so the prior has less influence and forgetting is greater.
  • On Permuted MNIST, VGR is marginally better than VCL with or without a coreset, while the coreset-only approach performs worse.

B.3. Experimental Settings for Single-headed Split MNIST

The experimental settings specify architectures, optimization, coreset selection, training schedules, and averaging procedures for Split MNIST comparisons.

  • Bayesian neural networks use the Permuted MNIST settings, except that each hidden layer has 256 weights for Split MNIST.
  • Results are averaged over 10 runs with fresh coreset selections, and standard errors are approximately one-tenth of a percent.
  • All Bayesian neural networks train for 120 epochs with batch sizes equal to the training-set size.
  • Coresets are selected per task with the k-center algorithm, withheld during training, and jointly trained for 120 epochs before evaluation.
  • VGR uses 6000 generated digits per class and caps batch sizes at 30,000, while its convolutional GAN trains for 50 epochs per MNIST class.
  • EWC uses 256-weight hidden layers, estimates Fisher information from 200 samples, and trains with SGD for 20 epochs per task.
  • Multi-headed Split MNIST uses five separately trained VCL heads, with coresets applied to each head in turn; EWC remains configured as in the single-headed setting.

B.5. Experimental Settings for Split Fashion MNIST

Split Fashion MNIST uses larger networks, and multi-headed training improves performance relative to single-headed experiments. Timing comparisons show different computational trade-offs between VCL with coresets and VGR.

  • B.5. Experimental Settings for Split Fashion MNIST: Split Fashion MNIST uses four hidden layers with 200 units for the BNN and 256 units for EWC, yet performance is lower because its classes are more diverse.Hyperparameters were deliberately not optimized.
  • B.5. Experimental Settings for Split Fashion MNIST: Retraining does not eliminate the distinction between prior-focused and likelihood-focused approaches in single-headed Split MNIST.The experiment cycled through all data five times before restarting while retaining coresets when applicable.
  • B.5. Experimental Settings for Split Fashion MNIST: Multi-headed training improves performance for all models, with VCL becoming very good and faster than VGR.This configuration allows VCL to use multi-headed training alongside coresets.
  • B.5. Experimental Settings for Split Fashion MNIST: VCL with coreset was faster than VGR, taking roughly 7 minutes on single-headed Split MNIST.Likelihood-focused approaches train generative models and use extra data, whereas prior-focused approaches may add more complicated losses or training phases.
  • B.5. Experimental Settings for Split Fashion MNIST: The computational costs scale differently: prior-focused methods scale badly with model size, while VGR scales badly with data-space complexity and very long task sequences.VGR may also be less memory intensive because generated data can be sampled on demand.

B.8. A Simpler Model Uncertainty Experiment

The paper evaluates continual-learning uncertainty by testing models on seen and unseen tasks after each training stage. VGR separates these task types cleanly, whereas VCL becomes uncertain about some older tasks, reflecting forgetting.

  • B.8. A Simpler Model Uncertainty Experiment: After each task, uncertainty is measured on data from every seen and unseen task, then thresholds generate ROC curves and AUC scores.The procedure evaluates uncertainty without tracking it throughout training.
  • B.8. A Simpler Model Uncertainty Experiment: For VGR, uncertainty is high on unseen tasks and low on previously seen tasks, indicating good calibration.Mutual information between predictions and the model posterior is normalized to 1 for each model’s most uncertain task.
  • B.8. A Simpler Model Uncertainty Experiment: For VCL, uncertainty rises on seen tasks encountered earlier, so an older task can appear nearly as uncertain as an unseen task.The fourth model is nearly as uncertain about its second task as its fifth task, which it has not seen.
  • B.8. A Simpler Model Uncertainty Experiment: VGR achieves AUC 1 while VCL achieves AUC 0.76 for distinguishing seen from unseen tasks.The paper reports that VGR can correctly detect all unseen tasks in principle, whereas VCL fails on a considerable number.
  • B.8. A Simpler Model Uncertainty Experiment: VGR’s uncertainty cut-offs produce no false positives when recognizing previously seen tasks.The resulting recognition of previously seen tasks is perfect.
  • B.8. A Simpler Model Uncertainty Experiment: VCL produces false positives because it is highly uncertain about many previously seen tasks, representing forgetting of old tasks.This uncertainty pattern limits reliable separation between old and unseen tasks.
Loading 1805.09733v3…