PaperlayerView exact revision on arXiv

Source-linked AI summary

Variational Continual Learning

Cuong V. Nguyen, Yingzhen Li, Thang D. Bui, Richard E. Turner

arXiv:1710.10628v3stat.MLcs.LG

TL;DR

Continual learning must adapt to evolving and newly emerging tasks without repeatedly revisiting all prior data, while avoiding catastrophic forgetting. This paper develops VCL by combining online variational inference with Monte Carlo methods for neural networks and optional coreset memory. VCL applies to deep discriminative and generative models and achieves strong performance across continual-learning tasks.

  • Problem

    Continual learning must handle sequential, possibly non-i.i.d. data and changing or newly emerging tasks while retaining knowledge from prior tasks.

  • Method

    VCL combines online variational inference with Monte Carlo variational inference for neural networks and can augment the posterior update with a small episodic coreset.

  • Results

    VCL applies to deep discriminative and generative models and achieves 90% average accuracy after 10 tasks, compared with 84% for EWC, 86% for SI, and 82% for LP.

  • Takeaways & Limitations

    The framework provides automatic continual learning without free parameters in its objective function and delivers state-of-the-art performance against previous approaches.

  • Takeaways & Limitations

    The model structure is assumed known a priori, and sharing parts of task-specific encoder networks is not investigated.

Abstract

from arXiv · show

This paper develops variational continual learning (VCL), a simple but general framework for continual learning that fuses online variational inference (VI) and recent advances in Monte Carlo VI for neural networks. The framework can successfully train both deep discriminative models and deep generative models in complex continual learning settings where existing tasks evolve over time and entirely new tasks emerge. Experimental results show that VCL outperforms state-of-the-art continual learning methods on a variety of tasks, avoiding catastrophic forgetting in a fully automatic way.

1 INTRODUCTION

Continual learning addresses sequential, non-i.i.d. data and evolving or newly emerging tasks while avoiding repeated access to all previous data. VCL applies Bayesian inference through online variational inference to balance adaptation with retention, and extends it with episodic memory.

  • Motivation: Continual learning handles continuously arriving, possibly non-i.i.d. data, changing tasks, and entirely new tasks without repeatedly revisiting all previous data.This matters because real-world tasks evolve, datasets can be too large for frequent batch updates, and related tasks may benefit from joint handling.
  • Motivation: Deep continual learning must balance adapting to recent data against retaining knowledge from old data, because excessive plasticity causes catastrophic forgetting while excessive stability impedes adaptation.
  • Approach: VCL combines online variational inference with Monte Carlo variational inference for neural networks, using Bayesian posteriors to constrain influential parameters while allowing others to change.The framework treats the previous posterior as information from earlier data and combines it with the current likelihood.
  • Scope: VCL is extended with a small episodic memory based on coreset data summarization and is demonstrated on both deep discriminative and deep generative models.The paper reports that the framework is general and yields excellent performance.

2 CONTINUAL LEARNING BY APPROXIMATE BAYESIAN INFERENCE

Continual learning can be formulated as sequential Bayesian updating, where each new likelihood modifies the previous posterior. Because exact posteriors are usually intractable, VCL uses variational projection and can preserve additional information through a coreset.

  • Bayesian updating: Bayes’ rule yields an online recursion: the posterior after each dataset combines the previous posterior with the new likelihood and then renormalizes.
  • Approximate inference: Exact posterior inference is generally intractable, so approximate inference projects the unnormalized posterior into a tractable normalized distribution at every step.The paper lists Laplace approximation, variational KL minimization, moment matching, and importance sampling as projection choices.
  • Variational inference: VCL defines its projection by minimizing KL divergence over the permitted family of approximate posteriors.
  • Coreset enhancement: Coreset VCL retains a small representative set of prior-task data to reduce information loss from repeated approximate updates and mitigate residual forgetting.The coreset acts as an episodic memory and is incorporated directly into the approximate posterior before prediction.
  • Coreset enhancement: At each task, coreset VCL updates the coreset, propagates the variational distribution using non-coreset data, and performs a final projection incorporating the current coreset for prediction.The method can select points randomly or with greedy K-center selection.

3 VARIATIONAL CONTINUAL LEARNING IN DEEP DISCRIMINATIVE MODELS

VCL is applied to deep discriminative networks, including architectures with shared parameters and task-specific heads. The framework incrementally updates task heads and shared parameters using a Gaussian mean-field posterior and Monte Carlo gradient estimates.

  • Network architectures: For discriminative continual learning, VCL supports standard single-head networks and multi-head architectures with shared lower-level parameters and task-specific output heads.The paper treats architectural advances as complementary to its learning scheme.
  • Network architectures: Multi-head models share lower-level parameters across tasks while assigning each task its own head network.
  • Assumption: The model structure is assumed known a priori rather than being automatically expanded with bespoke structure as new tasks appear.
  • Variational training: VCL represents network weights and biases with a Gaussian mean-field approximate posterior that can grow incrementally as new task heads emerge.Only heads present in the current dataset need updating, whereas shared parameters are continually updated.
  • Variational training: The expected log-likelihood is approximated with Monte Carlo and gradients are computed using the local reparameterization trick.The KL divergence between successive approximate posteriors is computed in closed form.

4 VARIATIONAL CONTINUAL LEARNING IN DEEP GENERATIVE MODELS

The paper extends VCL to deep generative models by applying posterior-based continual learning to variational autoencoders. It updates parameter uncertainty across sequential datasets while allowing shared and task-specific generative components.

  • Generative-model extension: VCL extends continual learning from discriminative models to variational autoencoders, addressing sequential data where standard batch generative-model training is unsuitable.
  • Generative-model extension: Standard approximate maximum likelihood is replaced by approximate posterior inference over neural-network parameters, preserving uncertainty needed to weight information from old data.
  • Scope: The encoder variational parameters are task-specific, while sharing parts of the encoder networks is identified as potentially beneficial but is not investigated.
  • Model architecture: The generative model can split into shared and task-specific components, with alternative choices for sharing the observation-generating network or the latent-to-intermediate head networks.One option is motivated when data share structural primitives selected by high-level latent variables.

5 RELATED WORK

Prior continual-learning methods regularize parameter updates or approximate Bayesian posteriors, while VCL retains full parameter uncertainty and avoids validation-tuned hyperparameters. The broader framework extends approximate Bayesian neural-network training and online variational inference to continual learning, including generative settings where naïve updating forgets earlier tasks.

  • Regularized continual learning: Regularized maximum-likelihood methods bias new parameters toward previous estimates using a user-selected strength and parameter-specific matrix.The regularization strength is controlled by λ_t, while Σ_t−1 encodes the relative penalty on each parameter.
  • Regularized continual learning: MAP, Laplace Propagation, Elastic Weight Consolidation, and Synaptic Intelligence differ in how they represent or estimate parameter importance across tasks.MAP does not directly provide the posterior covariance; LP propagates covariance estimates, EWC uses Fisher-information identities, and SI measures parameter importance from optimization changes.
  • Variational continual learning: VCL retains a full approximate parameter distribution, avoids validation-tuned free parameters, and is motivated by improved uncertainty estimates over MAP and Laplace methods.The paper argues that uncertainty estimates are critical for weighting information learned from old data.
  • Related Bayesian methods: Approximate Bayesian neural-network methods have mainly focused on batch learning, whereas the paper’s framework enables their application to continual learning.The related approaches include Kalman filtering, Laplace approximation, variational inference, sequential Monte Carlo, expectation propagation, and power EP.
  • Related Bayesian methods: Online variational inference had been explored previously, but not for neural networks or sets of related complex tasks.This positions VCL as an application of online variational inference to a broader neural continual-learning setting.
  • Deep generative models: Naïvely applying a VAE to each new dataset causes catastrophic forgetting, leaving the generator producing instances resembling only the most recently observed task.The passage motivates alternatives such as EWC regularization for deep generative continual learning.

6 EXPERIMENTS

The experiments evaluate VCL on discriminative and generative continual-learning tasks, comparing it with established baselines and coreset variants. Across these settings, VCL generally preserves prior-task performance and achieves competitive or superior results, with coresets often improving accuracy.

  • Experimental design: VCL is evaluated on three deep discriminative and two deep generative continual-learning tasks against EWC, diagonal LP, and SI.The comparisons include tuned baselines, whereas VCL uses a hyper-parameter-free objective.
  • Deep discriminative models: After 10 Permuted MNIST tasks, VCL reaches 90% average accuracy versus 84% for EWC, 86% for SI, and 82% for LP.Random and K-center coresets both raise VCL to 93% accuracy.
  • Deep discriminative models: At a coreset size of 5,000 examples per task, VCL reaches 95.5% accuracy after 10 tasks versus 90% for vanilla VCL.Performance improves with coreset size and asymptotes for large coresets, while the VCL-coreset combination remains advantageous.
  • Deep discriminative models: After five Split MNIST tasks, VCL achieves 97.0% average accuracy, compared with 63.1% for EWC, 98.9% for SI, and 61.2% for LP.Adding a coreset raises VCL to around 98.4% accuracy.
  • Deep discriminative models: After five Split notMNIST tasks, VCL achieves 92.0% average accuracy versus 71% for EWC, 94% for SI, and 63% for LP.Adding the random coreset improves VCL to 96% accuracy on this more challenging dataset with deeper networks.
  • Deep generative models: For deep generative models, LP, EWC, SI, and VCL remember previous tasks, with SI and VCL achieving the best visual quality on both datasets.VCL is on par with or slightly better than SI and has superior long-term memory according to the reported metrics.

7 CONCLUSION

VCL extends online variational inference into a general continual-learning framework for complex neural models. It achieves strong performance across discriminative and generative settings while remaining hyper-parameter free in its objective.

  • VCL extends online variational inference to handle general continual-learning tasks and complex neural-network models.
  • The framework applies to both deep discriminative and deep generative models.
  • Adding a small episodic memory through coreset algorithms can modestly improve VCL performance.
  • VCL achieved state-of-the-art performance against previous continual-learning approaches despite having no free parameters in its objective function.
  • The paper identifies alternative approximate-inference methods and more sophisticated episodic memories as directions for future work.

A FURTHER DETAILS FOR PERMUTED MNIST EXPERIMENT

The Permuted MNIST experiments use fully connected single-head networks and compare VCL with tuned SI, EWC, and LP baselines. Baseline hyper-parameters are selected through explicit sweeps, with performance visualized across alternative values.

  • The experiment uses single-head fully connected networks with two 100-unit ReLU hidden layers.
  • Test accuracy is measured across all observed tasks, using Adam with learning rate 10^-3, batch size 256, and 100 training epochs for VCL.
  • SI is evaluated at λ values from 0.01 to 2, with λ = 0.5 selected as the baseline.
  • EWC is evaluated at λ values from 1 to 10^4, with λ = 10^2 selected as the baseline.
  • Figures 8 and 9 show SI and EWC performance across hyper-parameter values, while Figure 10 reports the corresponding LP sweep.

B FURTHER DETAILS FOR SPLIT MNIST EXPERIMENT

The Split MNIST experiments evaluate multi-head models by task-specific and average test accuracy across repeated runs. They specify network, optimization, prior, initialization, coreset, and baseline settings for VCL and competing methods.

  • Multi-head networks use two hidden layers with 256 ReLU units each, and results average 10 runs with different random seeds.
  • Accuracy is measured separately on all observed tasks and averaged across tasks in the final figure column.
  • VCL uses Adam with learning rate 10^-3, full-batch training for 120 epochs, and a N(0, I) prior.
  • Coresets contain 40 examples from each task, while first-task optimization starts near the maximum-likelihood mean with variance 10^-6.
  • SI uses λ = 1 as its baseline, while EWC compares single-head and multi-head models across λ values from 1 to 10^4.

C FURTHER DETAILS FOR SPLIT NOTMNIST EXPERIMENT

The Split notMNIST experiment keeps the earlier Split MNIST settings while using deeper networks. It selects distinct hyper-parameters for multi-head EWC, multi-head LP, and SI.

  • The experiment uses four hidden layers with 150 hidden units each.
  • The remaining settings match those of the preceding Split MNIST experiment.
  • The selected values are λ = 10^4 for multi-head EWC, λ = 1 for multi-head LP, and λ = 0.1 for SI.

D ADDITIONAL EXPERIMENT ON A TOY 2D DATASET

The toy experiment compares VCL and EWC on two sequential binary classification tasks. After learning the second task, VCL retains good classifiers for both tasks, whereas EWC fails on both.

  • Experimental setup: The experiment uses two sequential binary classification tasks generated from Gaussian class distributions.Each task contains 200 data points, with 100 points in each class.
  • Additional comparisons: Figures 11–14 compare SI and EWC across different hyper-parameter values on Split MNIST and Split notMNIST.The EWC comparisons distinguish multi-head and single-head models.
  • Results: Both VCL and EWC perform reasonably well on the first task before the second task is observed.The comparison uses multi-head models with fully connected networks containing one hidden layer of 20 ReLU units.
  • Results: After observing the second task, EWC fails to learn classifiers for both tasks, while VCL still learns good classifiers for both.The comparison is visualized through prediction-probability contours in Figure 15.

E FURTHER DETAILS ON DEEP GENERATIVE MODEL EXPERIMENTS

The section details implementation and theoretical considerations for VCL experiments with deep generative models and Bayesian linear regression. It also describes conditions under which online variational inference matches online Laplace’s approximation.

  • Deep generative model settings: Deep generative model experiments use a learning rate of 10^-4, with 200 epochs for MNIST and 400 epochs for notMNIST.SI uses 400 epochs on MNIST; VCL initializes each variational approximation with the previous mean and log standard deviation 10^-6.
  • Deep generative model architecture: The generative model uses shared and task-specific one-hidden-layer networks with 500 hidden units, latent dimension 50, and intermediate dimension 500.Task-specific encoders use architectures symmetric to the generator.
  • Bayesian linear regression: For conjugate models with diagonal Gaussian approximations, online variational inference and online Laplace’s approximation produce the same Gaussian mean and diagonal precisions.This equivalence is illustrated for Bayesian linear regression with Gaussian prior and observation models.
  • Bayesian linear regression: The sequential variational updates for Bayesian linear regression provide closed-form updates for the mean and precisions.The model associates random D-dimensional binary inputs with random binary outputs using a weight vector W.
  • Relationship to Laplace and EWC: When the prior is ignored, the variational mean update exactly matches the corresponding online Laplace update, but the equivalence does not generally hold.The methods differ when the prior is retained or regularization constraints are accumulated.
Loading 1710.10628v3…