PaperlayerView exact revision on arXiv

Source-linked AI summary

A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning

Soochan Lee, Junsoo Ha, Dongsu Zhang, Gunhee Kim

arXiv:2001.00689v2cs.LGcs.NEstat.ML

TL;DR

Continual-learning research often assumes known task boundaries, unlike realistic non-iid streams with shifting distributions. CN-DPM addresses this gap with an expansion-based Dirichlet process mixture of neural experts, and experiments report successful task-free learning for discriminative and generative tasks.

  • Problem

    Task-free continual learning must handle non-iid streams with shifting distributions when task definitions and boundaries are unavailable.

  • Method

    CN-DPM uses neural experts with discriminative classifiers and generative density estimators, expanding their number under a Bayesian nonparametric Dirichlet process mixture framework.

  • Results

    CN-DPM successfully performs task-free continual learning across discriminative and generative tasks, outperforming existing methods in multiple task-free settings.

  • Takeaways & Limitations

    The reported results identify expansion-based CN-DPM as a promising direction, especially when the number of tasks is very large.

  • Takeaways & Limitations

    Expert selection remains the main bottleneck of CN-DPM, motivating future work to improve its accuracy.

Abstract

from arXiv · show

Despite the growing interest in continual learning, most of its contemporary works have been studied in a rather restricted setting where tasks are clearly distinguishable, and task boundaries are known during training. However, if our goal is to develop an algorithm that learns as humans do, this setting is far from realistic, and it is essential to develop a methodology that works in a task-free manner. Meanwhile, among several branches of continual learning, expansion-based methods have the advantage of eliminating catastrophic forgetting by allocating new resources to learn new data. In this work, we propose an expansion-based approach for task-free continual learning. Our model, named Continual Neural Dirichlet Process Mixture (CN-DPM), consists of a set of neural network experts that are in charge of a subset of the data. CN-DPM expands the number of experts in a principled way under the Bayesian nonparametric framework. With extensive experiments, we show that our model successfully performs task-free continual learning for both discriminative and generative tasks such as image classification and image generation.

1 INTRODUCTION

The paper targets task-free continual learning, where streams are non-iid, distributions shift, and task boundaries are unavailable. It proposes CN-DPM, an expansion-based Bayesian nonparametric model evaluated on discriminative and generative tasks.

  • Task-free continual learning addresses non-iid streams whose adjacent data may be correlated and whose distributions can shift during training.
  • Most continual-learning approaches assume task divisions and boundaries are known, making task-free learning more realistic but comparatively understudied.
  • Expansion methods accommodate new data by enlarging the architecture rather than fixing its capacity beforehand, helping bypass catastrophic forgetting.
  • CN-DPM uses neural experts responsible for data subsets and expands their number through a Bayesian nonparametric framework.The model formulates task-free continual learning as online variational inference for Dirichlet process mixtures.
  • CN-DPM handles discriminative and generative continual-learning tasks, including image classification and image generation, on MNIST, SVHN, and CIFAR 10/100.

2 BACKGROUND AND RELATED WORK

The background reviews continual-learning strategies, task-free learning, and Dirichlet process mixtures. It also motivates using DPMs to add components without a prefixed number of clusters and summarizes online variational inference and discriminative extensions.

  • Continual-learning methods are grouped into regularization, replay, and expansion approaches, with hybrids combining replay and regularization.
  • Task-free continual learning removes explicit task definitions while allowing gradual domain shifts without clear boundaries, but remains largely understudied.
  • A Dirichlet process mixture clusters data without requiring the number of clusters in advance, with expected cluster count proportional to concentration parameter α.
  • Sequential Variational Approximation performs online approximate inference by sequentially estimating responsibilities and component parameters as data arrive.
  • DPMs extend to discriminative tasks by modeling p(x, y) and inducing p(y|x), with each component decomposed into conditional and marginal terms.
  • Prior work applies DPM-based component expansion to model-based reinforcement learning and online meta-learning when new tasks appear.

3 APPROACH

CN-DPM models task-free continual learning with neural experts and automatically expands the expert pool when incoming data do not fit existing components. It combines generative density estimation, discriminative prediction, responsibilities, short-term memory, and parameter sharing.

  • 3 APPROACH: The approach assumes task descriptions and boundaries are unavailable and can also address streams that cannot be split into separate tasks.
  • 3.1 CONTINUAL LEARNING AS MODELING OF THE MIXTURE DISTRIBUTION: CN-DPM models the overall distribution as a mixture whose component weights are approximated by assigned-data counts Nk/N.
  • 3.1 CONTINUAL LEARNING AS MODELING OF THE MIXTURE DISTRIBUTION: Because the expert assignment z is unknown and the expert count is unfixed, the method uses a Dirichlet process mixture with online Sequential Variational Approximation.
  • 3.2 THE CONTINUAL NEURAL DIRICHLET PROCESS MIXTURE (CN-DPM) MODEL: For each incoming sample, responsibilities determine assignment to existing experts or storage in short-term memory when a new expert is indicated.
  • 3.2 THE CONTINUAL NEURAL DIRICHLET PROCESS MIXTURE (CN-DPM) MODEL: CN-DPM combines each expert’s classifier for p(y|x, z=k) with a density estimator for p(x|z=k), enabling gating from input likelihoods.The classifier may be a neural classifier or regressor, while the density estimator may use models such as VAEs or PixelRNN.
  • 3.2 THE CONTINUAL NEURAL DIRICHLET PROCESS MIXTURE (CN-DPM) MODEL: The short-term memory fills before new-expert creation, allowing multi-epoch training during a sleep phase and later updates during a wake phase.This procedure is intended to reduce severe overfitting from training a neural expert on one example.
  • 3.2 THE CONTINUAL NEURAL DIRICHLET PROCESS MIXTURE (CN-DPM) MODEL: Parameter sharing through lateral connections addresses redundant feature learning and the lack of positive transfer between experts.

4 EXPERIMENTS

Experiments evaluate CN-DPM in task-free continual-learning scenarios without task information at training or test time, covering classification and generation benchmarks. Across settings, CN-DPM outperforms baselines, remains robust as task count or scenario length increases, and avoids catastrophic forgetting, though expert selection remains a limitation.

  • Experimental setup: CN-DPM is evaluated across four task-free continual-learning benchmarks, with task information unavailable during training and testing.Unless otherwise stated, each task is presented for one epoch with batch size 10.
  • Experimental setup: The experiments include Split-MNIST classification and generation, MNIST-SVHN domain shift, and Split-CIFAR10/100 scenarios with five and 20 tasks.Split-CIFAR100 contains 20 five-class tasks, while MNIST-SVHN uses MNIST and SVHN with shared class labels but different input domains.
  • Main results: CN-DPM outperforms the baselines by significant margins in every reported setting while using a reasonable number of parameters.Reported experimental numbers are averages over 10 runs, except the cited GSS comparison details in the Split-CIFAR10 analysis.
  • Main results: Reservoir surpasses GSS in Split-CIFAR10, but its performance degrades when each task lasts 10 epochs because repeated replay overfits the memory samples.The degradation is more severe with smaller memory sizes.
  • Main results: In Split-CIFAR100, Reservoir suffers from memory shortage across 20 tasks, whereas CN-DPM’s accuracy is more than double Reservoir’s and comparable to iid-online.The Reservoir memory contains only 50 slots per task despite a total size of 1000.
  • Component analysis: CN-DPM’s task-wise classifier accuracy changes little from initialization to the final evaluation, confirming that forgetting barely occurs in the classifiers.The analysis also reports relatively low gating accuracy, indicating room to improve density-based expert selection.
  • Discussion: CN-DPM avoids catastrophic forgetting, but choosing the correct expert becomes an additional problem, especially as the method scales to many tasks.The authors identify expert selection as the main bottleneck and describe the approach as especially promising for very large task counts.

5 CONCLUSION

The paper formulates expansion-based task-free continual learning as a Dirichlet process mixture with neural experts. CN-DPM expands experts according to the data and demonstrates performance across task-free settings, while expert selection remains a bottleneck.

  • CN-DPM formulates expansion-based task-free continual learning as learning a Dirichlet process mixture model with neural experts.
  • The main stated bottleneck is expert-selection accuracy; future directions include natural language processing and reinforcement learning.
  • CN-DPM’s effective component count is typically bounded by O(α log N), and experiments show it does not blindly increase experts.
  • Expansion avoids fixing model capacity beforehand, while redundant experts can be pruned through an introduced algorithm.

C.3 EXPERIMENTAL DETAILS

The experiments use Adam-trained CN-DPM models with task-specific learning rates, likelihood objectives, and wake-phase learning-rate decay. Common settings include weight decay, gradient clipping, and sleep-phase training of new experts.

  • All CN-DPM models use Adam, weight decay of 0.00001, and value-based gradient clipping at threshold 0.5.
  • Classifier temperatures are 0.01 for Split-MNIST and Split-CIFAR10/100, while MNIST-SVHN uses no temperature parameter.
  • During the sleep phase, new experts are trained for multiple epochs with batch size 50, and classification VAEs improve density estimation by sampling 16 latent codes.
  • Learning rates and likelihood objectives vary across classification and generation settings, with Bernoulli log-likelihood for binarized-MNIST generation and Gaussian log-likelihood for CIFAR10 VAEs.The supplied settings include classifier and VAE learning rates of 0.0001 and 0.0004 for classification, and 0.003 for generation VAEs.
  • Before the wake phase, expert learning rates are decayed by 0.003 in generation and by 0.1, 0.2, or 0.5 and 0.1 for the listed CIFAR10 configurations.

D PRUNING REDUNDANT EXPERTS

CN-DPM prunes redundant experts by comparing their data-wise log-likelihood behavior and removing smaller members of sufficiently similar pairs.

  • Expert similarity is computed as cosine similarity between vectors of per-example log-likelihoods.
  • When similarity exceeds a threshold, the expert with smaller Nk is removed and its data are reassigned to the remaining experts.
  • In a Split-MNIST example, seven experts were created, and thresholding at 0.9 selected expert pairs (2/3) and (5/6) for pruning.
  • Test accuracy marginally dropped from 87.07% to 86.01% after pruning.

E COMPARISON WITH TASK-BASED METHODS ON SPLIT-MNIST

CN-DPM is compared with task-based methods on Split-MNIST and evaluated when task boundaries are mixed rather than discrete. It performs better than task-aware regularization and replay methods and learns without clear boundaries.

  • CN-DPM was evaluated both with task information for expert selection and with expert inference based on the DPM principle.
  • Providing task information at training time can improve CN-DPM’s performance further, although task information is unavailable at test time in both settings.
  • CN-DPM performed significantly better than regularization and replay methods that exploit task descriptions, despite the task-free condition being more difficult.
  • The Split-MNIST comparison reports averages over 10 runs with standard errors, using task-based results from Hsu et al. (2018).
  • In Fuzzy-Split-MNIST, transition stages mix existing and new-task data with a linearly increasing proportion of the new task, and CN-DPM performs continual learning without task boundaries.

G GENERATION OF SAMPLES

CN-DPM generates samples by selecting an expert and then sampling from that expert’s generator. On Split-MNIST, it generated examples from all tasks without catastrophic forgetting.

  • CN-DPM samples an expert from p(z) and then samples x from that expert’s generator.The expert is selected first, conditioning the subsequent generation step.
  • After one training epoch on Split-MNIST, CN-DPM generated examples representing all tasks without catastrophic forgetting.

H EXPERIMENTS WITH LONGER CONTINUAL LEARNING SCENARIOS

Longer continual-learning scenarios compare repeated tasks with repeated full scenarios. CN-DPM improves as learning continues and determines expert growth from the task distribution rather than training length.

  • Experiments evaluate longer scenarios on Split-MNIST, Split-CIFAR10, and Split-CIFAR100 using 10 Epochs and 1 Epoch ×10 settings.Results are averaged over 10 runs with standard errors of the mean.
  • CN-DPM performance improves as the learning process is extended, whereas Reservoir accuracy drops as each task becomes longer.The Reservoir decline is attributed to overfitting replay-memory samples representing each task.
  • CN-DPM performs similarly in the 1 Epoch ×10 and 10 Epoch settings because each data point is seen 10 times in both.
  • CN-DPM determines whether to add an expert on a data-by-data basis, so expert count depends on task distribution rather than training length.

I EXPERIMENTS WITH DIFFERENT MEMORY SIZES

CN-DPM is comparatively robust to smaller replay-memory sizes and can reduce parameters through sharing without sacrificing accuracy. Its concentration parameter still requires tuning to balance performance and capacity.

  • Different memory sizes: With memory sizes of 500 and 1000, CN-DPM’s accuracy drop is relatively marginal, unlike Reservoir’s significant decline with smaller memory.The Split-CIFAR10/100 experiments use memory sizes of 500 and 1000 for comparison.
  • Concentration parameter: A higher concentration parameter α makes experts easier to create, increasing both the number of experts and accuracy as α rises.
  • Concentration parameter: At log α = −600, too few experts are created and accuracy is rather low.
  • Concentration parameter: CN-DPM requires concentration-parameter tuning to balance performance and model capacity despite automatically assigning data to experts.
  • Parameter sharing: Sharing parameters between experts reduces the number of parameters by approximately 38% without sacrificing accuracy.

L TRAINING GRAPHS

The paper presents training graphs tracking performance, expert counts, and parameter totals, alongside task-wise accuracy and average learned-task accuracy. Its comparison with CURL emphasizes differences in generative structure and replay requirements.

  • Training graphs: Training graphs report performance metrics, CN-DPM’s number of experts, and total parameters relative to baselines.Bold lines show averages across 10 runs, while faint lines show individual runs.
  • Training graphs: Task-wise accuracy plots show how each learned task changes during training and include average accuracy for learned tasks.
  • Comparison with CURL: CN-DPM’s generative process differs from CURL by generating y and z independently and maintaining a separate decoder for each cluster.
  • Comparison with CURL: CURL uses generative replay because shared components, including a single decoder and gating network, remain vulnerable to catastrophic forgetting.The cited comparison states that expansion alone is insufficient for CURL and that performance drops without generative replay.
Loading 2001.00689v2…