PaperlayerView exact revision on arXiv

Source-linked AI summary

Task Agnostic Continual Learning Using Online Variational Bayes

Chen Zeno, Itay Golan, Elad Hoffer, Daniel Soudry

arXiv:1803.10123v3stat.MLcs.LG

TL;DR

Continual-learning research often assumes task boundaries are known during training, while realistic applications may lack a task schedule. This paper presents BGD for task-agnostic continual learning and reports that it mitigates catastrophic forgetting without task-switch awareness; its variational approximation uses a diagonal Gaussian.

  • Problem

    Many continual-learning methods assume known task boundaries during training, while realistic applications may lack knowledge of the task schedule.

    Source: §1 · p. 1

  • Method

    BGD is a continual-learning algorithm for scenarios where task identity or boundaries are unknown during training and testing.

    Source: §1 · p. 1

  • Results

    BGD mitigates catastrophic forgetting without awareness of task switches.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The reported task-switch-independent mitigation is presented as relevant to learning in realistic continual-learning settings.

    Source: §6 · p. 8

    The variational approximation uses a diagonal Gaussian, an assumption the authors suggest could be relaxed for greater flexibility.

    Source: §6 · p. 8

Abstract

from arXiv · show

Catastrophic forgetting is the notorious vulnerability of neural networks to the change of the data distribution while learning. This phenomenon has long been considered a major obstacle for allowing the use of learning agents in realistic continual learning settings. A large body of continual learning research assumes that task boundaries are known during training. However, research for scenarios in which task boundaries are unknown during training has been lacking. In this paper we present, for the first time, a method for preventing catastrophic forgetting (BGD) for scenarios with task boundaries that are unknown during training --- task-agnostic continual learning. Code of our algorithm is available at https://github.com/igolan/bgd.

1. Introduction

The paper targets continual learning when task schedules are unknown, a setting relevant to realistic applications where task boundaries may not be available. It introduces BGD for task-agnostic learning and proposes finer scenario categorization and a labels trick for class learning.

Main contributions

  • The authors expand continual-learning scenario definitions to support well-defined comparisons and report first results for scenarios without task-schedule knowledge.They state that this creates two previously unaddressed scenarios.Source: §1 · p. 1
  • On Split MNIST, the labels trick improves average accuracy from ∼20% to ∼50% for all algorithms.The paper attributes the change to understanding the information available in the class-learning scenario.Source: §1 · p. 1
  • BGD is presented as the first continual-learning algorithm applicable when task identity or boundaries are unknown during both training and testing.The paper calls this setting task-agnostic continual learning.Source: §1 · p. 1
  • The labels trick is proposed to improve performance in the class-learning scenario.It uses information available in that scenario to improve performance.Source: §1 · p. 1

2. Continual learning scenarios

The section distinguishes continual-learning scenarios by task boundaries, task identity available at training and inference, and whether tasks share output heads. It also describes task-agnostic settings and a labels trick for class learning.

Previous scenario definitions

  • Task, domain, and class learning differ in test-time task identity and output heads; class learning uses separate task heads but must evaluate all heads at test time.The section assumes equal classes per task for simplicity, while noting that this assumption may be omitted.Source: §2 · p. 2

Expanding scenario definitions

  • Scenario characterization uses four questions: whether task boundaries are defined, task identity is known during training or inference, and output heads are shared.Additional questions include task duration and whether the number of tasks is known.Source: §2 · p. 2

Task-agnostic scenarios

  • Task-agnostic scenarios lack knowledge of the task distribution, and undefined boundaries may arise from gradual mixing of task samples or continuous change in the data itself.Figure 2 characterizes the gradual-transition case as a changing distribution over tasks across iterations, unavailable to the algorithm.Source: §2 · p. 2, Figure 2 · p. 2

Task agnostic continual learning

  • BGD can be applied without modification when task boundaries are unknown or undefined, unlike methods requiring notification of task switches for core actions.The paper notes that such task-switch knowledge is unavailable in task-agnostic scenarios.Source: §2 · p. 2

Labels trick

  • In class learning, nonshared heads make task identity inferable during training from the labels in the current batch.The trick trains only heads corresponding to labels present in that batch rather than all heads.Source: §2 · p. 3
  • The labels trick improves class-learning results from ∼20% to ∼50%, with the remaining error attributed to task prediction during testing.The setup is unchanged and uses no new information; the trick trains only heads corresponding to labels present in each batch.Source: §2 · p. 3

3. Related work

The section situates BGD among continual-learning approaches and highlights two differences: it needs no task-identity information and updates the posterior in closed form. It also reviews Bayesian continual-learning methods and their approximations.

Approaches for continual learning

  • Continual-learning approaches include architectural, rehearsal, regularization, and Bayesian methods; this paper focuses on regularization and Bayesian approaches without changing the architecture or replaying stored examples.Bayesian methods use a previous-task posterior as the prior for new data; VCL uses online variational inference with BBB, while Ritter et al. use a Kronecker-factored Laplace approximation.Source: §3 · p. 3

Major differences

  • BGD’s task-agnostic design contrasts with methods that depend on task switches, while its closed-form posterior update contrasts with BBB-style SGD optimization.These are the two differences the paper identifies as major.Source: §3 · p. 4
  • BGD needs no task-identity information, unlike EWC, SI, and VCL, which rely on a core action at task switches.It can be applied without modification when task boundaries are unknown or undefined.Source: §3 · p. 4
  • BGD updates the weight posterior in closed form, whereas BBB and VCL use SGD to optimize variational free energy.VCL relies on BBB.Source: §3 · p. 4

4. Theory

The theory develops BGD from online Bayesian updating and a Gaussian variational approximation, then examines how its uncertainty updates relate to loss curvature. It also states conditions for uncertainty to change monotonically and connects uncertainty-linked learning rates to task-agnostic continual learning.

4. Theory

  • Online Bayesian inference updates the posterior as data arrives, using the previous posterior as the prior for the next update.The posterior at time n follows Bayes’ rule.Source: §4 · p. 4

4.1. Online variational Bayes

  • Online variational Bayes approximates an intractable posterior with a parametric distribution and uses the previous approximate posterior as the next prior.The variational parameters are optimized by minimizing KL divergence to the true posterior.Source: §4 · p. 4, §4.1 · p. 4

4.2. Diagonal Gaussian approximation

  • BGD assumes factorized Gaussian distributions and uses unbiased Monte Carlo gradients to obtain an explicit closed-form update for the means and standard deviations.The update approximates the solution to implicit equations by evaluating the likelihood derivative using prior parameters.Source: §4.2 · p. 5

Repeat

  • Because one explicit iteration may not converge the cumulative log-likelihood, BGD repeats training-set passes until convergence and estimates expectations with Monte Carlo samples.A learning-rate hyperparameter adjusts convergence, and the algorithm uses K Monte Carlo samples.Source: §4.2 · p. 5, Repeat · p. 5

4.3. Theoretical properties of Bayesian Gradient Descent

  • BGD’s uncertainty update approximates the loss-curvature signal: uncertainty, and thus learning rate, is expected to decrease with positive curvature and increase with high negative curvature.The paper describes this as a potentially sensible way to converge to minima and escape saddles.Source: §4.3 · pp. 5–6
  • For strongly convex losses, the uncertainty sequence is monotonic decreasing; for strongly concave losses, it is monotonic increasing.The paper further states that σ_i = 0 is the only stable point in the strongly convex case.Source: §4.3 · p. 6
  • For neural networks, σ_i does not generally converge to zero, and the stable point σ_i = 0 is generally not unique.The paper reports that MNIST training for 5000 epochs did not collapse to a point estimate.Source: §4.3 · p. 6
  • For continual learning, the authors hypothesize that task-important weights have locally convex loss and low uncertainty, while less-important weights can retain higher uncertainty.They propose that BGD’s uncertainty-linked learning rates then restrict later training along less-important weights without requiring explicit task information.Source: §4.3 · p. 6

5. Experiments

The experiments evaluate BGD in continuous and discrete task-agnostic settings, test the labels trick for class learning, and report comparisons on additional benchmarks. BGD maintains high average accuracy in the continuous setting and performs on par with SI and VCL in the discrete setting despite lacking task-schedule information.

5. Experiments

  • The experiments include hyperparameter searches, but not all algorithms appear in every comparison because open-source or architecture-specific implementations were unavailable.The paper provides dataset and implementation details in appendices and makes BGD code available.Source: §5 · p. 6

5.1. Continuous task agnostic scenario

  • BGD maintains high average test accuracy across tasks in continuous task-agnostic Permuted MNIST as tasks transition slowly and boundaries remain undefined.The reported accuracy averages across tasks and three runs, with standard-deviation error bars.Source: §5.1 · p. 6, Figure 3 · p. 7

5.2. Discrete task agnostic scenario

  • The STD histograms show that the share of weights with low STD grows across tasks, while the model does not collapse to point estimation.After the first task, many weights remain near the initial STD of 0.06; later, more have much lower STD, but at least ten times the minimum σ of approximately 10^-4.Source: §5.2 · p. 7, Figure 5 · p. 7
  • In discrete task-agnostic Permuted MNIST, BGD performs on par with task-aware SI and VCL despite having no task-schedule information.With 20 epochs per task, SI achieves better performance, although all remain on par.Source: §5.2 · p. 7, Figure 4 · p. 7

5.3. “Labels trick” for class learning

  • On Split MNIST class learning, the labels trick improves results for all tested algorithms.Task identity is known during training but not testing, and tasks use separate output heads.Source: §5.3 · p. 8, Table 1 · p. 8

5.4. Additional experiments

  • Across additional experiments on several MNIST, vision, and CIFAR benchmarks, BGD results are on par with methods that use task-switch information in most cases.The evaluations include task, domain, and class learning as well as CIFAR10 and subsets of CIFAR100.Source: §5.4 · p. 8

6. Discussion and future work

The discussion presents BGD as a method for mitigating forgetting without task-switch awareness, with competitive results across basic scenarios. It also describes its broader architectural applicability and identifies the diagonal-Gaussian approximation as a possible extension point.

6. Discussion and future work

  • BGD is presented as mitigating catastrophic forgetting without awareness of task switches, a property the authors say may support adaptation when tasks are not explicitly announced.The authors frame this as enabling learning in realistic continual-learning settings.Source: §6 · p. 8
  • BGD achieves competitive results on basic scenarios while also working in task-agnostic settings and across varied datasets and architectures.The authors state that it applies to convolution layers and Batch-Normalization.Source: §6 · p. 8
  • BGD outperforms previous Bayesian methods on MNIST and is on par with SGD on CIFAR10 without Batch-Normalization.Unlike the cited analytic-approximation methods, BGD is described as straightforward to implement for different neural architectures; inference stores μ and σ per weight rather than all network copies.Source: §6 · p. 8
  • The current variational approximation uses a diagonal Gaussian; moving to non-diagonal or non-Gaussian distributions could improve flexibility but is not a trivial extension.The authors also identify accurate task prediction at test time as a possible route to closing the class-learning/task-learning performance gap.Source: §6 · p. 8

A. Derivation of (11) and (12)

The appendix derives equations (11) and (12) from the stated objective using first-order optimality conditions for the variational parameters μ_i and σ_i.

A. Derivation of (11) and (12)

  • The derivation starts from an objective combining the log variational density, the previous density, and L_n(θ).The objective is given as f(θ, φ) = log(q_n(θ|φ)) − log(q_n−1(θ)) + L_n(θ).Source: A. Derivation of (11) and (12) · p. 11
  • Equation (11) is derived using the first-order necessary condition for the optimal μ_i, followed by substitution of the derivatives.The appendix states that this substitution yields equation (11).Source: A. Derivation of (11) and (12) · p. 11
  • Equation (12) is derived using the first-order necessary condition for optimal σ_i and the condition σ_i > 0.After substituting derivatives, the appendix gives the solution as equation (12).Source: A. Derivation of (11) and (12) · p. 11

B. Proof of Theorem 1

The proof parameterizes each weight with a Gaussian perturbation and uses a strong-convexity assumption to obtain an inequality for two parameter vectors differing in coordinate i.

B. Proof of Theorem 1

  • The proof writes each parameter as θ_j = μ_j + ε_jσ_j, with ε_j drawn from a standard normal distribution.It then introduces a conditional expectation and identifies f_εi as the standard-normal density.Source: B. Proof of Theorem 1 · p. 11
  • The argument assumes L_n(θ) is continuously differentiable and strongly convex with parameter m_n > 0.Source: B. Proof of Theorem 1 · p. 11
  • The proof constructs two parameter vectors that differ by plus versus minus ε_iσ_i in coordinate i, then substitutes the resulting inequality into equation (23).Source: B. Proof of Theorem 1 · p. 11

C. Datasets

The experiments use standard image-classification datasets and task sequences derived from MNIST and CIFAR. The dataset descriptions span grayscale digits and products, color images, and several continual-learning task constructions.

MNIST datasets

  • Permuted MNIST assigns each task a different random pixel permutation, while Split MNIST groups digit classes into five pairwise-classification tasks.For example, Split MNIST's first tasks distinguish digits 0–1 and 2–3.Source: C. Datasets · p. 12

CIFAR datasets

  • CIFAR10 contains 60,000 32×32 color images in 10 classes, and CIFAR100 has 100 classes grouped into 20 superclasses.CIFAR100 images have both fine class labels and coarse superclass labels.Source: C. Datasets · p. 12

Other image datasets

  • FashionMNIST comprises 70,000 grayscale 28×28 images of fashion products across 10 categories.Its training and test sets contain 60,000 and 10,000 images, respectively.Source: C. Datasets · p. 12
  • notMNIST contains letters A–J in 10 classes, while SVHN contains RGB 32×32 house-number images in 10 digit classes.notMNIST images share MNIST's pixel size and grayscale format.Source: C. Datasets · p. 12

D. Implementation details

The section documents implementation choices across continual-learning experiments and the computational trade-offs of Monte Carlo sampling. It also specifies the settings and task sequences used for MNIST and vision-dataset evaluations.

D. Implementation details

  • BGD training estimates expected gradients with 10 Monte Carlo samples by default, and testing averages accuracy over 10 sampled networks unless stated otherwise.For continuous permuted MNIST, the authors use a two-hidden-layer width-200 network, sample with replacement, and average results over three seeds.Source: D. Implementation details · p. 12

E. Complexity

  • BGD stores both a mean and standard deviation per weight, requiring twice as many parameters as SGD; Monte Carlo samples can be generated sequentially or in parallel.Sequential generation saves memory but increases runtime, whereas parallel generation increases memory use but decreases runtime.Source: E. Complexity · p. 13
  • Runtime on CIFAR-10 grows linearly with the number of Monte Carlo iterations, while fewer iterations do not affect accuracy in the reported experiment.The timing used sequential samples and a single GeForce GTX 1080 Ti GPU.Source: E. Complexity · p. 13, Table 2 · p. 13

F.1. Spilt MNIST and permuted MNIST

  • The Split MNIST and permuted MNIST comparisons follow Hsu et al. (2018), use short training durations, and report results in Tables 5 and 6.The experiments use task-specific initial standard deviations; the Split MNIST class-learning setup averages results over ten seeds.Source: D. Implementation details · p. 12, §F.1 · p. 14

F.2. Continual learning on vision datasets

  • The vision-dataset sequence trains for 20 epochs each on MNIST, notMNIST, FashionMNIST, SVHN, and CIFAR10 using the setup of Ritter et al. (2018).The comparison uses a LeNet-like architecture with a separate final layer for each task, and results are reported in Table 7.Source: §F.2 · p. 14

F.3. Continual learning on CIFAR10/CIFAR100

On six sequential CIFAR tasks, BGD balances retention of earlier-task accuracy with performance on newer tasks, while its standard deviations tend to shrink across tasks. The paper also reports an additional simulation using the configuration of Zenke et al. (2017).

F.3. Continual learning on CIFAR10/CIFAR100

  • BGD maintains reasonable accuracy on previous CIFAR tasks while achieving high accuracy on newer ones.Figure 8 reports accuracy per task after sequential training on CIFAR10 and CIFAR100 subsets; the sequence starts with CIFAR10 and continues with five ten-class CIFAR100 subsets, each trained for 150 epochs.Source: §F.3 · p. 14, Figure 8 · p. 14
  • As more CIFAR tasks are seen, the percentage of weights with standard deviations below the initial 0.019 increases, with a reported minimum σ of approximately 10^-4.Figure 9 shows the standard-deviation histogram at the end of each task's training.Source: §F.3 · p. 14, Figure 9 · p. 14

F.4. Discrete permuted MNIST

F.5. Classification

The classification experiments compare BGD with other methods on MNIST and CIFAR10 and examine training over 5,000 epochs. BGD outperforms BBB and SGD on the reported MNIST comparison, while the extended-training results describe convergence and accuracy behavior.

F.5. Classification

  • BGD performs better than BBB and SGD on MNIST test accuracy across the network widths summarized in Table 3.The cited SGD and BBB results are reported from Blundell et al. (2015).Source: §F.5 · p. 14, Table 3 · p. 15
  • Figure 10 plots average test accuracy against the number of tasks on permuted MNIST for BGD, VCL, SI, and SGD.The caption specifies mini-batches of 256 and 20 epochs for all algorithms.Source: Figure 10 · p. 15
  • The CIFAR10 classification comparison evaluates BGD against SGD, ADAM, K-FAC, Noisy K-FAC, and Noisy ADAM using a modified VGG16 setup.Results are reported with and without Batch Normalization in Table 4.Source: §F.5 · p. 15, Table 4 · p. 15

F.6. 5000 epochs training

  • After 5,000 MNIST training epochs, the training log-likelihood cost decreases and converges to a low value, and the standard-deviation histogram converges without σ_i collapsing to zero.The authors report that the test accuracy does not drop during the extended training; Figures 11–13 show the cost, standard deviations, and accuracy curves.Source: §F.6 · p. 15, Figure 11 · p. 15, Figure 12 · p. 16
  • The authors also trained VGG11 on CIFAR10 for 5,000 epochs to examine test accuracy over extended training.The test-accuracy curve is presented in Figure 14.Source: §F.6 · p. 16, Figure 14 · p. 16
Loading 1803.10123v3…