PaperlayerView exact revision on arXiv

Source-linked AI summary

Efficient Continual Learning with Modular Networks and Task-Driven Priors

Tom Veniat, Ludovic Denoyer, Marc'Aurelio Ranzato

arXiv:2012.12631v2cs.LG

TL;DR

Continual learning has largely focused on avoiding forgetting, while transfer and sublinear memory and compute scaling are also desirable and under-tested by common benchmarks. The paper introduces CTrL and a modular network whose task-driven prior narrows architecture search. It reports competitive standard-benchmark performance and stronger results on its new benchmark, including gains on a 100-task stream.

  • Problem

    Continual learning benchmarks have largely focused on forgetting, leaving transfer and scalability less directly evaluated, despite their potential to help learning with little labeled data.

    Source: §1 · p. 1

  • Method

    The paper evaluates continual-learning models on CTrL and uses a modular network with a data-driven prior that narrows architecture search to perturbations of a similar past task's path.

    Source: §1 · p. 1, §4.2 · p. 6

  • Results

    The model performs at least as well as state-of-the-art methods on standard benchmarks and much better on CTrL; on 100 tasks, MNTDP-D improves absolute accuracy by 18% over the independent-model baseline using less than half its memory.

    Source: §1 · p. 2, §5.3 · p. 8

  • Takeaways & Limitations

    The results suggest modularization with a restricted search space offers a trade-off between accuracy and compute and memory use.

    Source: §6 · p. 10

    The prior assumes related tasks mainly change the output distribution rather than the input distribution, and other transfer dimensions remain future work.

    Source: §3 · p. 4, §4.2 · p. 6

Abstract

from arXiv · show

Existing literature in Continual Learning (CL) has focused on overcoming catastrophic forgetting, the inability of the learner to recall how to perform tasks observed in the past. There are however other desirable properties of a CL system, such as the ability to transfer knowledge from previous tasks and to scale memory and compute sub-linearly with the number of tasks. Since most current benchmarks focus only on forgetting using short streams of tasks, we first propose a new suite of benchmarks to probe CL algorithms across these new axes. Finally, we introduce a new modular architecture, whose modules represent atomic skills that can be composed to perform a certain task. Learning a task reduces to figuring out which past modules to re-use, and which new modules to instantiate to solve the current task. Our learning algorithm leverages a task-driven prior over the exponential search space of all possible ways to combine modules, enabling efficient learning on long streams of tasks. Our experiments show that this modular architecture and learning algorithm perform competitively on widely used CL benchmarks while yielding superior performance on the more challenging benchmarks we introduce in this work.

1 INTRODUCTION

Continual learning should support transfer and sublinear memory and compute growth for related tasks, not merely resist forgetting. The paper introduces CTrL and a modular model with a task-driven prior, reporting competitive standard-benchmark performance and stronger results on its more challenging benchmark.

1 INTRODUCTION

  • A continual learner should transfer knowledge from related past tasks and scale memory and compute sublinearly as related tasks accumulate.The motivation is that accrued knowledge may help an agent learn a new task from little labeled data.Source: §1 · p. 1
  • CTrL evaluates transfer and scalability alongside accuracy and forgetting, dimensions that commonly used benchmarks do not distinguish well.The experiments are restricted to supervised learning tasks and basic transfer properties.Source: §1 · p. 1
  • The proposed model composes a handful of modules per task, borrowing modules from past tasks or training new ones.A data-driven prior restricts the exponential architecture search to local perturbations around the most relevant past-task architecture.Source: §1 · pp. 1–2
  • The model performs at least as well as state-of-the-art methods on standard benchmarks and much better on CTrL, with better transfer and scaling to streams of 100 tasks.The evaluation protocol observes each stream only once, though data from each task can be replayed multiple times.Source: §1 · p. 2

2 RELATED WORK

Prior continual-learning approaches address forgetting, capacity, or architectural growth in different ways, but may be limited on long streams or inefficiently search modular architectures. The paper positions its restricted search as a distinction from related modular methods.

Regularization-based methods

  • Regularization methods use a shared predictor and weight regularization to resist forgetting, but their fixed capacity limits long, diverse streams.The cited approaches include Fisher-information approximations and weight distance from initialization.Source: §2 · p. 2

Rehearsal-based methods

  • Rehearsal methods retain or generate past-task information, yet share the fixed-capacity limitation of regularization methods on long streams.Examples include replay buffers, generative replay, and per-class prototypes.Source: §2 · p. 2

Evolving architectures

  • Evolving architectures address limited capacity, but can grow super-linearly, return to fixed capacity, or keep expanding at inference time.The section contrasts these approaches with the paper's restricted architecture search, whose search space and RAM remain constant over time.Source: §2 · p. 2

Modular approaches

  • Prior modular approaches identify tasks with paths through modules, but use random search rather than a prior to guide architecture selection.The paper identifies Rajasegaran et al. (2019) and Fernando et al. (2017) as examples.Source: §2 · p. 3

Continual-learning benchmarks

  • Existing benchmarks may offer many tasks or realistic data without enabling precise assessment of transfer and scaling.The passage contrasts a 2,500-task MNIST-permutation stream with CORe50 and CUB-200.Source: §2 · p. 3

Related learning paradigms

  • Meta-learning emphasizes rapid adaptation, while continual learning also concerns forgetting and learning over time; static multi-task settings likewise differ in their scaling focus.The related-work discussion also notes that some modular robotics work measures only last-task performance.Source: §2 · p. 3

3 EVALUATING CONTINUAL LEARNING MODELS

The section defines accuracy, forgetting, transfer, and scalability as key evaluation dimensions for continual-learning models. It proposes task streams designed to probe transfer under different relationships between tasks and scalability over long streams.

Desirable Properties of CL models And Metrics

  • The framework formalizes tasks as ordered labeled datasets and evaluates supervised-learning predictors using accuracy.A task descriptor is provided during training and testing, and the predictor may depend on the task stream seen so far.Source: §3 · p. 3
  • The evaluation measures accuracy after learning the stream, forgetting on earlier tasks, and transfer to the final task relative to learning it in isolation.Negative transfer values indicate interference or limited plasticity; positive values indicate benefit from learning the full sequence when earlier tasks are related.Source: §3 · pp. 3–4
  • Scalability is assessed by total memory and compute used by the end of training, including resources retained to continue learning.Examples include EWC regularization parameters and experience-replay buffers.Source: §3 · p. 4

Streams

  • The proposed streams revisit an earlier task with less or more data, or with changed input or output distributions, to probe distinct transfer settings.The less-data revisit tests direct transfer; the more-data revisit tests updating knowledge of the earlier task after unrelated tasks.Source: §3 · p. 4
  • A stream of unrelated tasks probes plasticity and possible interference, while a 100-task stream of varying relatedness and data amounts probes scalability.The authors note that transfer dimensions such as compositional task descriptors and noisy conditions remain future work.Source: §3 · p. 4

4 MODULAR NETWORKS WITH TASK DRIVEN PRIOR (MNTDP)

MNTDP represents task predictors as compositions of potentially shared modules. For a new task, it adds modules and learns how to combine them while optimizing only the newly introduced parameters.

4 MODULAR NETWORKS WITH TASK DRIVEN PRIOR (MNTDP)

  • MNTDP builds predictors from modules that may be shared across related tasks, with modules in a layer differing by parameter values and modules across layers allowed to implement different functions.A module may be a parametric function such as a ResNet block.Source: §4 · p. 4

4.1 TRAINING

  • For each new task, MNTDP adds randomly initialized modules at every layer and minimizes cross-entropy over module combinations and parameters, updating only the new modules’ parameters.The search space is defined over possible ways to combine modules.Source: §4.1 · p. 4

MNTDP-S MNTDP-D

MNTDP-S and MNTDP-D differ in how they search paths and share newly added modules. A data-driven prior narrows the path search to perturbations of a similar past task’s architecture.

MNTDP-S MNTDP-D

  • In the illustrated training cycle, the method retains the architecture of the most similar past task, explores restricted perturbations, and adds parameters of selected new modules to the library.Only newly added modules are trained; the two variants differ in stochastic path selection versus parallel path training and selection.Source: Figure 2 · p. 5, MNTDP-S MNTDP-D · p. 5
  • MNTDP-S alternates REINFORCE path updates with parameter updates and can share new modules across paths; MNTDP-D exhaustively trains independent paths and selects the lowest-validation-loss path.MNTDP-S uses entropy regularization to encourage path exploration, whereas MNTDP-D uses more memory but can parallelize path training.Source: MNTDP-S MNTDP-D · pp. 5–6

4.2 DATA-DRIVEN PRIOR

  • The restricted search allows new modules to connect only to new modules in the next layer, assuming related tasks mainly change output rather than input distributions.The authors identify changed input distributions as a setting in which this assumption may not hold; module reuse is presented as the basis for transfer and sublinear scaling.Source: §4.2 · p. 6
  • The unrestricted path space grows exponentially, so the prior selects the past-task path with the best nearest-neighbor classification accuracy on current-task features and searches its perturbations.This reduces the search space from T^L to L; the designer can vary k to trade accuracy against compute and memory.Source: §4.2 · p. 6

5 EXPERIMENTS

The experiments evaluate transfer and scalability alongside accuracy and forgetting using the CTrL benchmark, which complements standard continual-learning streams. Under a task-level cross-validation protocol, MNTDP performs competitively on standard streams and shows stronger accuracy and scaling results on the more challenging evaluations.

5.1 THE CTRL BENCHMARK

  • CTrL evaluates transfer and scalability across streams built from seven vision datasets, including a 100-task Slong stream designed to test scaling and transfer with task age.Its supervised tasks include validation and test sets; Slong samples datasets, classes, and training-data amounts, favoring smaller tasks later in the stream.Source: §5.1 · pp. 6–7

5.2 METHODOLOGY AND MODELING DETAILS

  • The protocol observes each stream once, permits repeated replay of each task’s data, and cross-validates hyperparameters on that task’s validation set.This avoids using future tasks during current-task model selection; MNTDP uses a seven-module ResNet-18 variant by default and searches prior-task modules with k = 1 unless otherwise specified.Source: §5.2 · p. 7

5.3 RESULTS

  • On standard 10-task Permuted MNIST and Split CIFAR100 streams, MNTDP-D has the best average accuracy but requires more computation than independent models.MNTDP reduces to independent models without module sharing on these relatively distinct tasks, so these streams do not assess transfer or scaling well.Source: §5.3 · p. 8, Figure 3 · p. 8
  • On CTrL transfer streams, MNTDP has the best average accuracy overall, while MNTDP-D achieves the best transfer on S-, S+, Sout, and Spl.MNTDP underperforms on Sin because its prior does not cover swapping the first module; MNTDP-S uses less compute but has lower overall average accuracy.Source: §5.3 · p. 8, Table 1 · p. 9
  • On Slong’s 100 tasks, MNTDP-D improves absolute accuracy by 18% over independent models while using less than half their memory, and grows sublinearly in memory.Its runtime is close to that of independent models; Figure 4 shows MNTDP-D attaining the best performance as memory grows sublinearly.Source: §5.3 · pp. 8–9, Table 2 · p. 9, Figure 4 · p. 9
  • Randomly choosing a prior path prevents module sharing, while reducing the number of modules can lower transfer, indicating that the prior and module granularity affect performance.The reported Sout transfer metric falls from 0 to -0.1 with three modules, and task-1 performance decreases by 9% in that setting.Source: §5.3 · p. 9

6 CONCLUSIONS

The paper introduces CTrL to evaluate continual learning beyond average accuracy and forgetting, including transfer and scaling. Its modular network uses an adaptive prior over module connections, and experiments support a favorable accuracy–compute/memory trade-off.

6 CONCLUSIONS

  • CTrL broadens continual-learning evaluation beyond average accuracy and forgetting to include knowledge transfer and scaling.The authors conclude that modularization in a restricted search space is a promising avenue for continual learning and knowledge transfer.Source: §6 · pp. 9–10
  • The proposed modular network scales through an adaptive prior over the search space of possible module connections.Experiments show a desirable trade-off between accuracy and compute/memory usage.Source: §6 · p. 10

A DATASETS AND STREAMS

CTrL uses streams built from standard datasets to evaluate transfer, scalability, and sample efficiency. The accompanying learning algorithms restrict module-path search around a related past task and select a path for the new task.

A DATASETS AND STREAMS

  • CTrL uses standard datasets to build task streams, with the streams' task compositions listed in the accompanying tables.The benchmark includes transfer streams and a 100-task stream; Figure 5 shows examples from its datasets.Source: A DATASETS AND STREAMS · p. 14, Figure 5 · p. 14, Table 4 · p. 15
  • Slong mixes large tasks with 5,000 training examples and small tasks with 25, increasing the small-task share from 50% to 75% to 100% across three task ranges.The setting assesses scalability, transfer ability, and sample efficiency.Source: A DATASETS AND STREAMS · p. 14

B LEARNING ALGORITHM

  • For MNTDP-S, training data are split between updating module parameters and the path distribution, then the highest-probability architecture is fine-tuned on all task data.The split is intended to discourage the path distribution from favoring paths that overfit.Source: B LEARNING ALGORITHM · p. 18
  • MNTDP-D trains each candidate path and selects the one with the highest validation accuracy.Both algorithms search paths built from the closest past task's predictor plus newly initialized modules.Source: B LEARNING ALGORITHM · p. 18

C ARCHITECTURE OF THE MODELS AND HYPER-PARAMETERS

The experiments use shared CNN backbones, including ResNet or AlexNet, with layers grouped into modules for MNTDP and PNN. The appendix also reports training and test-time memory complexity, noting that most methods use a single path at test time.

C ARCHITECTURE OF THE MODELS AND HYPER-PARAMETERS

  • For tasks other than Permuted-MNIST, baselines and MNTDP use the same ResNet or AlexNet backbone, with layers grouped into modules for MNTDP and PNN.Permuted-MNIST uses a separately specified model.Source: C ARCHITECTURE OF THE MODELS AND… · p. 18, Table 7 · p. 18, Table 8 · p. 18
  • The one-pass stream protocol prevents stream-level hyperparameter tuning, while EWC, Online-EWC, and HAT receive that tuning as an exception.The authors note this gives those methods an unfair advantage; task-level tuning of EWC's constraint strength leads to little or no constraint and severe forgetting.Source: C ARCHITECTURE OF THE MODELS AND… · p. 19

D MEMORY COMPLEXITY OF THE MODELS

  • At test time, all methods except DEN and RCL are as fast to evaluate as an independent network because they use a single path.The appendix reports memory complexity at both training and test time, while the main paper reports overall memory consumption at the end of training.Source: D MEMORY COMPLEXITY OF THE MODELS · p. 19

E.1 METRIC

The appendix adds area under the learning curve after a fixed number of optimization steps as a learning-speed metric. It also reports results for the standard Permuted-MNIST and Split CIFAR 100 streams.

E.1 METRIC

  • LCA measures the area under the learning curve after β optimization steps to assess learning speed.A_b,t is the test accuracy on task t after b batches from that task.Source: §E.1 · p. 19

E.2 CLASSICAL STREAMS

  • The Permuted-MNIST results cover a standard stream of 10 tasks, each defined by a random permutation of MNIST input pixels.The table notes that DEN and RCL lack an LCA measure in these results and marks models using stream-level cross-validation.Source: Table 11 · p. 20
  • The Split CIFAR 100 results cover tasks composed of 10 new classes each.The table marks AlexNet-backbone models and models using stream-level cross-validation.Source: Table 12 · p. 21

E.3 THE CTRL BENCHMARK

CTrL results are presented in radar plots and companion tables, while discovered-path figures show module reuse and additions across tasks. The stream evaluations vary data volume, perturbations, distractor exposure, probe timing, and stream length.

E.3 THE CTRL BENCHMARK

  • MNTDP-D is reported as most efficient on multiple CTrL dimensions, but requires more computation than MNTDP-S.The radar plots have companion result tables.Source: Figure 6 · p. 22, §E.3 · p. 22

E.4 DISCOVERED PATHS

  • On Sout, MNTDP-D adds substantial parameters for tasks 2–5 but reuses task 1 modules for the similar final task, adding only a classifier; MNTDP-S reuses task 1 modules and adds two modules.Source: Figure 7 · p. 23, Figure 8 · p. 24

E.5 STREAM S-

  • Table 13 evaluates a stream whose last task repeats the first with an order of magnitude less data; Figure 9 compares baselines on S-.Source: Table 13 · p. 25, Figure 9 · p. 25

E.6 STREAM S+

  • Table 14 evaluates a stream whose fifth task repeats the first with an order of magnitude more data, with tasks 2–4 as distractors; Figure 10 compares baselines on S+.Source: Table 14 · p. 26, Figure 10 · p. 26

E.7 STREAM S IN

  • Table 15 tests transfer when the final task repeats the first with an input-space modification and an order of magnitude less data, after four distractors; Figure 11 compares baselines.Source: Table 15 · p. 27, Figure 11 · p. 27

E.8 STREAM S OUT

  • Table 16 tests transfer when the final task uses the first task’s classes in a different order and with an order of magnitude less data, after four distractors; Figure 12 compares baselines.Source: Table 16 · p. 28, Figure 12 · p. 28

E.9 S PL

  • Table 17 compares probe-task performance when the probe is first versus after four distractors; Figure 13 compares baselines on the plasticity stream.Source: Table 17 · p. 29, Figure 13 · p. 29

E.10 S LONG

  • Table 18 reports mean and standard error over three instances of the long evaluation stream; Figure 14 compares baselines on Slong.Source: Table 18 · p. 30, Figure 14 · p. 30

Related papers

Loading 2012.12631v2…