PaperlayerView exact revision on arXiv

Source-linked AI summary

Functional Regularisation for Continual Learning with Gaussian Processes

Michalis K. Titsias, Jonathan Schwarz, Alexander G. de G. Matthews, Razvan Pascanu, Yee Whye Teh

arXiv:1901.11356v4stat.MLcs.LG

TL;DR

Continual learning must preserve earlier task performance while learning sequentially without extensive retraining or large stored datasets. The paper uses function-space Bayesian inference with neural-network Gaussian processes and inducing-point summaries to regularise future tasks. The approach unifies parameter regularisation and rehearsal, reports strong benchmark results, and avoids brittleness associated with representation drift.

  • Problem

    Continual learning needs to avoid catastrophic forgetting and scale across tasks, while existing methods may omit uncertainty or rely on heuristic, memory-intensive observation storage.

  • Method

    The method performs approximate Bayesian inference over task-specific functions, storing posterior summaries at optimised inducing inputs and using them to regularise later tasks.

  • Results

    The method reports strong results across benchmarks, including state-of-the-art results on Permuted-MNIST and Omniglot and results close to existing methods on Split-MNIST.

  • Takeaways & Limitations

    Functional regularisation unifies parameter regularisation with rehearsal by constraining neural-network outputs and principledly compressing previous-task data.

  • Takeaways & Limitations

    Predictions based on retained weight-space posteriors can become obsolete when shared features change, and the setup assumes sequential supervised tasks with shared representations.

Abstract

from arXiv · show

We introduce a framework for Continual Learning (CL) based on Bayesian inference over the function space rather than the parameters of a deep neural network. This method, referred to as functional regularisation for Continual Learning, avoids forgetting a previous task by constructing and memorising an approximate posterior belief over the underlying task-specific function. To achieve this we rely on a Gaussian process obtained by treating the weights of the last layer of a neural network as random and Gaussian distributed. Then, the training algorithm sequentially encounters tasks and constructs posterior beliefs over the task-specific functions by using inducing point sparse Gaussian process methods. At each step a new task is first learnt and then a summary is constructed consisting of (i) inducing inputs -- a fixed-size subset of the task inputs selected such that it optimally represents the task -- and (ii) a posterior distribution over the function values at these inputs. This summary then regularises learning of future tasks, through Kullback-Leibler regularisation terms. Our method thus unites approaches focused on (pseudo-)rehearsal with those derived from a sequential Bayesian inference perspective in a principled way, leading to strong results on accepted benchmarks.

1 INTRODUCTION

Continual learning systems must learn sequential tasks while preserving earlier performance and remaining scalable. The paper addresses these challenges by regularising functions with Gaussian-process summaries and principled inducing-point selection.

  • Continual learning requires adapting to new tasks while avoiding catastrophic forgetting and scaling across an increasing number of tasks.
  • Existing parameter-regularisation methods constrain neural-network parameters, whereas replay methods store observations using often-heuristic selection procedures.
  • The proposed approach performs approximate Bayesian inference over functions, constraining predictions rather than neural-network parameters.
  • It memorises approximate posterior beliefs over task-specific functions using sparse Gaussian processes and small sets of inducing points.
  • A neural-network feature map supplies a linear-kernel Gaussian process, with earlier-layer weights optimised and final-layer weights treated Bayesianly.

2 FUNCTIONAL REGULARISATION FOR CONTINUAL LEARNING

Functional regularisation preserves earlier tasks by storing sparse Gaussian-process beliefs over task functions rather than relying only on neural-network parameters. Sequential task learning uses inducing-point summaries and KL regularisation, with weight-space inference making current-task training efficient and function-space distillation selecting representative inducing points.

  • Problem and model: Continual learning processes supervised tasks sequentially while retaining accurate predictions on earlier tasks through a shared neural representation and task-specific output functions.The model uses a deep network feature map φ(x; θ) shared across tasks, with separate task-specific weights.
  • Function-space model: A Gaussian prior over task-specific output weights induces an independent Gaussian process for each task, providing a distribution over functions.The GP kernel is defined by the dot product of neural-network feature vectors.
  • Sparse task summaries: Storing full task posteriors is expensive because each multivariate Gaussian requires O(N_i^2) parameters and the complete input set X_i.Sparse GP methods reduce these time and memory requirements by approximating each posterior with inducing variables.
  • Learning the first task: For each task, sparse variational inference learns inducing inputs and a Gaussian variational distribution by minimising a KL divergence and maximising an ELBO over feature parameters.The variational distribution q(u_1) is Gaussian, and its parameters can be optimised jointly with θ using stochastic variational inference.
  • Efficient inference and selection: Weight-space inference avoids function-space optimisation costing O(N_k^3) per step, after which the learned solution supports inducing-point selection and posterior construction.The inducing-point criterion promotes points that are mutually repulsive and evenly spread under feature-space similarity.
  • Function-space versus weight-space summaries: Function-space summaries adapt predictions as shared features change, keeping predictions at stored inducing inputs equal to their fixed mean beliefs, unlike fixed weight-space summaries.Changing θ updates kernel quantities and the mean prediction while the stored inducing-value mean remains constant.

3 DETECTING TASK BOUNDARIES USING BAYESIAN UNCERTAINTIES

The framework detects task boundaries by measuring how closely minibatch predictions resemble the prior, leveraging predictive uncertainty that depends on inputs. A statistical test on these surprise scores flags distribution changes before updating the model.

  • Predictive uncertainty increases for inputs far from observed data and eventually approaches the prior, providing a basis for detecting task changes.
  • The method compares each input’s posterior density with its prior GP density using a symmetrised KL divergence.For univariate Gaussian distributions, the divergence is computed analytically for each minibatch input.
  • Each nonnegative score ℓ_i measures surprise about an input, with smaller scores indicating greater surprise under the stated interpretation.
  • Welch’s t-test evaluates the minibatch scores before parameter updates, accounting for unequal variances.
  • Input-dependent predictive posteriors distinguish familiar-task inputs, which have reduced uncertainty, from different-task inputs, which revert toward prior uncertainty.This dependence on inputs enables change detection, unlike a posterior over weights alone.

4 EXPERIMENTS

Experiments evaluate FRCL against continual-learning methods, test inducing-point selection criteria and counts, and assess task-boundary detection. FRCL performs strongly across benchmarks, while optimisation matters more with fewer inducing points and task-boundary detection is less strong on Omniglot.

  • Experimental setup: Experiments compare FRCL with continual-learning algorithms on Split-MNIST, Permuted-MNIST, and sequential Omniglot.A simple replay-buffer BASELINE is also included alongside published results.
  • Competitive performance: FRCL achieves state-of-the-art results on Permuted-MNIST and Omniglot, while coming close to existing results on Split-MNIST.The authors report strong results across all evaluated benchmarks.
  • Inducing-point criteria: Optimisation becomes increasingly important as the number of inducing points decreases.The experiments provide statistical motivation for the trace-term criterion; a poorly chosen criterion can perform worse than random.
  • Inducing-point criteria: With 10 inducing points, trace-term optimisation consistently allocates examples across all classes and spreads them across the feature space.The objective is unsupervised and encourages repulsive inducing points under the learned feature-space similarity.
  • Task-boundary detection: On illustrative Omniglot tasks, positive Welch’s t-test results coincide with clear spikes in the monitored terms.The visualisation uses a small number of alphabets and 1000 training iterations per task.
  • Task-boundary detection: Task-boundary detection is strong on Split-MNIST and Permuted-MNIST but less strong on Omniglot, where smaller batches may produce noisier tests.The authors suggest using a larger set of previous minibatches to make the test more robust.

5 DISCUSSION

The paper presents functional regularisation for supervised continual learning as a combination of inducing-point Gaussian-process inference and deep neural networks. It reports strong empirical gains while identifying fixed or sub-linearly growing memory as a future challenge.

  • 5 DISCUSSION: The method constructs task-specific posterior beliefs on inducing inputs and uses them to regularise future learning against catastrophic forgetting.It combines inducing-point GP inference with deep neural networks.
  • 5 DISCUSSION: Functional regularisation constrains neural-network outputs rather than parameters, avoiding brittleness associated with representation drift.This provides the functional-space counterpart to parameter regularisation.
  • 5 DISCUSSION: The approach compresses previous-task data through optimised inducing-point selection, unifying parameter regularisation with replay or rehearsal.The summaries are used as compact task-specific representations.
  • 5 DISCUSSION: The method also proposes using posterior beliefs to detect task boundaries.This extends the framework beyond regularisation and rehearsal.
  • 5 DISCUSSION: A fixed or sub-linearly growing memory buffer would require compressing summaries from all previously seen tasks into one summary.This is identified as a direction for future research.
  • 5 DISCUSSION: The experiments apply the method to supervised classification, while reinforcement learning and other domains remain future applications.The paper explicitly identifies reinforcement learning as an interesting extension.

A TASK DESCRIPTIONS

The experiments use Split-MNIST, Permuted-MNIST, and sequential Omniglot as continual-learning task sequences. Multi-class versions require extending the model to multiple Gaussian-process functions per task.

  • A TASK DESCRIPTIONS: Permuted-MNIST creates one task per fixed random permutation of the input pixels in the original 10-class classification problem.Each task remains a 10-class MNIST classification task with a different pixel permutation.
  • A TASK DESCRIPTIONS: Split-MNIST consists of five binary classification tasks ordered as 0/1, 2/3, 4/5, 6/7, and 8/9.The tasks are constructed from disjoint pairs of MNIST classes.
  • A TASK DESCRIPTIONS: Sequential Omniglot treats the 50 alphabets, each containing a varying number of characters, as a sequence of distinct classification problems.The experiments use data augmentation and the train/validation/test split suggested for this continual-learning task.
  • A TASK DESCRIPTIONS: Permuted-MNIST and Omniglot require multiple Gaussian-process functions because each task may classify among multiple classes.The next section describes this multi-function extension.

B EXTENSION TO MULTI-CLASS (OR MULTIPLE-OUTPUTS) TASKS

For multi-class tasks, the method uses independent Gaussian-process functions for each class and combines them through a multi-class likelihood. Variational distributions and regularisation terms factorise or sum across classes.

  • B EXTENSION TO MULTI-CLASS (OR MULTIPLE-OUTPUTS) TASKS: A task with C_k classes is modelled using C_k independent GP draws, one for each class, combined through a likelihood such as softmax.This extends the single-function presentation to multi-class classification.
  • B EXTENSION TO MULTI-CLASS (OR MULTIPLE-OUTPUTS) TASKS: The variational distributions over inducing variables and task weights factorise across functions or classes.This factorisation makes the multi-function generalisation straightforward.
  • B EXTENSION TO MULTI-CLASS (OR MULTIPLE-OUTPUTS) TASKS: All class-specific inducing variables are evaluated on the same inducing inputs Z_k.The shared input set supports the multiple outputs for one task.
  • B EXTENSION TO MULTI-CLASS (OR MULTIPLE-OUTPUTS) TASKS: The KL regularisation term for each task is expressed as a sum over its different functions.This preserves the task-level regularisation structure in the multi-class setting.
  • B EXTENSION TO MULTI-CLASS (OR MULTIPLE-OUTPUTS) TASKS: The baseline stores an explicit replay buffer containing subsets of inputs and labels for each past task.Training uses an unbiased estimate of the full loss based on the replay buffer.
  • B EXTENSION TO MULTI-CLASS (OR MULTIPLE-OUTPUTS) TASKS: In the multi-head baseline, current and old task output weights are updated together with the feature parameters, while fresh output weights are constructed for the current task.The loss correction accounts for bias introduced by random replay-buffer subsampling.

D SELECTION OF THE INDUCING POINTS AND OPTIMISATION CRITERIA

The inducing inputs can be selected by random sampling or supervised and unsupervised optimisation criteria. Optimisation becomes more important with fewer inducing points, and a poorly chosen criterion can perform worse than random selection.

  • D SELECTION OF THE INDUCING POINTS AND OPTIMISATION CRITERIA: A random subset of training inputs provides a simple inducing-input choice, but the paper investigates structured criteria that may improve selection.The criteria are designed to choose inducing inputs more systematically than random sampling.
  • D SELECTION OF THE INDUCING POINTS AND OPTIMISATION CRITERIA: The negative average log predictive density criterion evaluates predictive density on remaining training inputs after excluding the selected inducing inputs.For classification, the predictive density is computed numerically by one-dimensional Gaussian quadrature.
  • D SELECTION OF THE INDUCING POINTS AND OPTIMISATION CRITERIA: The classification-error criterion replaces negative log predictive density with an indicator for whether the predicted label differs from the true label.The predicted label is selected by argmax over predictive probabilities.
  • D SELECTION OF THE INDUCING POINTS AND OPTIMISATION CRITERIA: Both supervised criteria assess how well the inducing inputs predict the remaining points.They therefore favour inducing sets that support prediction beyond the selected inputs.
  • D SELECTION OF THE INDUCING POINTS AND OPTIMISATION CRITERIA: The variational lower-bound criterion optimises the inducing inputs to approximate the training-data marginal likelihood.This is the criterion used by variational sparse GP training methods, with other parameters held fixed in the stated formulation.
  • D SELECTION OF THE INDUCING POINTS AND OPTIMISATION CRITERIA: The trace criterion reconstructs the full kernel matrix and promotes inducing points that are mutually repulsive and evenly spread under the feature-induced similarity.It is connected to the Nyström approximation and principal component analysis.
  • D SELECTION OF THE INDUCING POINTS AND OPTIMISATION CRITERIA: Optimisation becomes increasingly important as the number of inducing points decreases, while a poorly chosen criterion can perform worse than random on log predictive density.These comparisons are reported for selection criteria on Split-MNIST and Permuted-MNIST; Omniglot uses three inducing points per character in Figure 6.

E TASK BOUNDARY DETECTION

The boundary detector aggregates per-function Welch’s t-tests and improves robustness by testing in log-space, but it assumes clearly switching tasks with changing input distributions.

  • Detection method: Welch’s t-tests are performed separately for each function, then aggregated using the mean, median, or maximum to detect boundaries.For multiple classes, the method evaluates each function independently before applying an aggregation rule.
  • Detection method: Testing in log-space significantly improves robustness, with the maximum over functions supporting higher detection thresholds.The authors identify max aggregation in log-space as the most robust configuration among those considered.
  • Detection method: After detecting a task boundary, the method suppresses new boundary tests for the next 10 iterations.This is an implementation rule applied immediately after a boundary decision.
  • Limitations: The detector is not applicable to continua of tasks without clear switches or to constant input distributions with changing labels.Its scope is limited to settings where task changes produce detectable input-distribution shifts.
  • Ablation study: Figure 7 compares aggregation methods and log-space testing on Permuted MNIST across 10 random task permutations.The ablation isolates both the aggregation choice and the test-space choice.

F EXPERIMENTAL DETAILS

The experiments use validation-selected hyperparameters, with dataset-specific procedures for forming final test results on MNIST and Omniglot.

  • Evaluation protocol: MNIST test results use hyperparameters optimized on validation data and training on the union of the training and validation sets.This procedure is stated for the MNIST experiments.
  • Evaluation protocol: Omniglot test results are reported after training only on the training set to remain consistent with Schwarz et al. (2018).The Omniglot protocol differs from the MNIST final-training procedure.
  • Hyperparameters: Permuted MNIST hyperparameters were selected on the validation set before final test-set training.Table 5 documents the hyperparameter-selection and final-training protocol for Permuted MNIST.
  • Hyperparameters: Omniglot hyperparameters were selected on the validation set before final test-set evaluation.Table 6 documents the corresponding Omniglot hyperparameter protocol.

G COMPARISON TO VCL ON OMNIGLOT

The paper compares FRCL with VCL on sequential Omniglot using MLPs because reliable variational inference methods for Omniglot CNNs were not yet available.

  • Model choice: The Omniglot comparison with VCL uses Multi-Layer Perceptrons rather than the CNNs commonly used for Omniglot.The authors motivate this choice by the lack of reliable variational inference methods for CNNs.
  • Baseline: VCL results are obtained using code provided by the authors.This specifies the implementation source for the comparison baseline.
  • Training setup: Both algorithms use an MLP with 4 hidden layers of 256 units, ReLU activations, batch size 100, and Adam optimization.FRCL uses step size 0.0001, while VCL uses 0.001.

1. VCL: 100 training epochs per task, 50 adaptation epochs to coreset, Multi-Head

The reported VCL configuration uses 100 training epochs per task, 50 coreset-adaptation epochs, and a Multi-Head setup; the Omniglot results are presented in Table 7.

  • Results: Table 7 reports results for sequential Omniglot using an MLP.The table caption identifies the dataset sequence and model family, but does not state a numerical outcome.
Loading 1901.11356v4…