Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

Xilai Li; Yingbo Zhou; Tianfu Wu; Richard Socher; Caiming Xiong

Source-linked AI summary

Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, Caiming Xiong

arXiv:1904.00310v3Mar 31, 2019cs.LGcs.CV

TL;DR

Continual learning with DNNs commonly uses shared structures across tasks, which can cause catastrophic forgetting. Learn-to-grow explicitly searches task-specific structures while separating structure optimization from parameter estimation, reducing forgetting and improving performance across continual-learning experiments.

Problem
Shared DNN structures across sequential tasks can cause catastrophic forgetting, especially when tasks require different features or structures.
Method
The framework uses architecture search to choose whether each layer is reused, adapted, or newly parameterized, then estimates the model parameters.
Results
The method reduced catastrophic forgetting compared with baseline methods and improved parameter use and task-specific structure learning; on permuted MNIST, average accuracy was 96.8% with EWC versus 63.0% after sequential SGD.
Takeaways & Limitations
Explicit continual structure learning can produce sensible task-specific structures while retaining shared primitives and either avoiding or substantially alleviating catastrophic forgetting.
Takeaways & Limitations
The structure-search space may grow exponentially with the number of tasks, motivating limits on possible choices or priority-queue search.

Abstract

from arXiv · show

Addressing catastrophic forgetting is one of the key challenges in continual learning where machine learning systems are trained with sequential or streaming tasks. Despite recent remarkable progress in state-of-the-art deep learning, deep neural networks (DNNs) are still plagued with the catastrophic forgetting problem. This paper presents a conceptually simple yet general and effective framework for handling catastrophic forgetting in continual learning with DNNs. The proposed method consists of two components: a neural structure optimization component and a parameter learning and/or fine-tuning component. By separating the explicit neural structure learning and the parameter estimation, not only is the proposed method capable of evolving neural structures in an intuitively meaningful way, but also shows strong capabilities of alleviating catastrophic forgetting in experiments. Furthermore, the proposed method outperforms all other baselines on the permuted MNIST dataset, the split CIFAR100 dataset and the Visual Domain Decathlon dataset in continual learning setting.

1. Introduction

Continual learning with DNNs is challenged by catastrophic forgetting, while existing parameter- and memory-based approaches remain limited. Learn-to-grow addresses this by explicitly learning task-specific structures while retaining opportunities to share features and parameters.

DNNs often largely forget previously learned tasks after training on a new task in a sequential learning setting.
Existing approaches constrain parameter changes or store task information, but forgetting remains non-negligible as the number of tasks increases.
Current methods commonly couple structure learning with parameter estimation and apply the same model structure across tasks.
Different tasks can explore distinct structures while sharing common feature layers, potentially reducing forgetting and supporting new-task performance.
The framework explicitly separates structure learning from parameter estimation and uses architecture search to identify task-specific structures.

2. The Proposed Learn-to-Grow Framework

The framework formulates continual learning over sequentially observed tasks and grows task-specific structures on top of prior models. It searches flexible layer choices, then estimates parameters while regularizing structure and model size to avoid degenerate solutions.

2.1. Problem Definition of Continual Learning: Continual learning observes tasks sequentially while maximizing current-task performance and minimizing forgetting on earlier tasks.
2.1. Problem Definition of Continual Learning: Because earlier datasets are unavailable during later training, the ideal joint objective cannot be directly computed or minimized.
2.1. Problem Definition of Continual Learning: Keeping all previous parameters fixed avoids forgetting but relies on hand-crafted growth and forces reuse without accounting for task dissimilarity.
2.1. Problem Definition of Continual Learning: Using one parameter set with constraints such as EWC can restrict new-task accuracy and requires well-designed initial models.
2.2. Our Proposed Learn-to-Grow Framework: The method grows θt on Θt−1 and flexibly reuses prior parameters, optionally allowing reused parameters to change under elastic penalties.
2.2. Our Proposed Learn-to-Grow Framework: Architecture search selects task-specific structures from reuse, adaptation, and new layer options before parameter estimation.
2.2. Our Proposed Learn-to-Grow Framework: Structure and parameter regularizers bound model growth and prevent the degenerate solution of training completely different models for different tasks.

3. Our Implementation

The implementation separates neural structure optimization from parameter learning, using differentiable architecture search to select task-specific structures and then retraining or fine-tuning parameters. A maintained super network supports reuse, adaptation, and new layer choices, while the search can account for parameter efficiency and faces potentially exponential growth.

3. Our Implementation: The framework has separate neural structure optimization and parameter learning or fine-tuning components.Structure optimization selects the best neural structure for the current task; parameter learning estimates new parameters and may fine-tune existing ones.
3. Our Implementation: Neural architecture search optimizes wiring patterns and operations while assuming a global topology, such as ResNet, is shared across tasks.The implementation can be extended to more complex settings by applying architecture search to the first task.
3.1. Structure Optimization: A super network stores task-specific and shareable layers, with each layer offering reuse, adaptation, or new choices.Reuse shares existing parameters, adaptation adds a small additive function, and new creates parameters matching the current layer size.
3.1. Structure Optimization: The layer-level search space contains 2|S_l| + 1 choices and may grow exponentially with the number of tasks.The authors suggest limiting choices and maintaining a priority queue as one possible response, but do not find this necessary in all experiments.
3.1. Structure Optimization: Categorical layer choices are relaxed into a Softmax-weighted continuous search, after which the largest architecture weight selects the final option.Architecture weights and parameters are updated alternately using validation loss for architecture search and training loss for parameter estimation.
3.1. Structure Optimization: After search, the model is retrained on the current task, with reused parameters either fixed or fine-tuned using regularization.Fixing reused parameters can avoid forgetting completely, whereas fine-tuning preserves the possibility of positive backward transfer.

4. Experiments

Experiments evaluate whether explicit continual structure learning finds sensible task-specific architectures and reduces catastrophic forgetting across permuted MNIST and VDD. The method generally shares layers for related tasks, creates new parameters for dissimilar information, and achieves strong continual-learning performance.

Experimental setup: The experiments test whether explicit structure learning produces sensible architectures and improves continual learning on permuted MNIST and the Visual Domain Decathlon.The evaluation compares structure choices, forgetting behavior, and state-of-the-art continual-learning methods.
Sensible structures: On permuted MNIST, structure optimization selects “new” for the first layer and “reuse” for the remaining layers across all 10 tasks.This matches the expectation that pixel permutations require lower-level differences while task semantics remain shared.
Sensible structures: The learned permuted-MNIST wiring pattern performs best among the tested settings that make one of three layers “new” and the others “reuse.”The ablation compares each layer being new while the remaining layers are reused.
Sensible structures: For VDD, similar ImageNet and CIFAR-100 tasks share most layers, whereas dissimilar ImageNet and Omniglot tasks select “new” for most layers.The results indicate task structures share primitives when corresponding semantic representations are similar and spawn parameters when required information differs.
Forgetting and performance: After 10 permuted-MNIST tasks, average accuracy falls from 97.9% to 63.0% with SGD, while EWC reaches 96.8%.Fixing reused layers or fine-tuning them with l2 regularization keeps overall accuracy as high as training each task individually; removing regularization shows almost the same behavior.
Forgetting and performance: In VDD, learn-to-grow significantly outperforms a baseline sharing and updating all backbone layers, obtains the best overall results, and is best on five of nine tasks.The total model size is similar to the adapter approach, with especially large gains on VGG-Flowers and Aircraft.
Forgetting and performance: In VDD, learn-to-grow parameters move less from the first-task parameters than baseline parameters, while distances remain more consistent across layers.The authors connect smaller movements with less forgetting and attribute them to compatible learned structures and parameters requiring less tuning.

5. Related Work

Related continual-learning methods mitigate catastrophic forgetting through replay, regularization, or model expansion, while architecture search provides another route for structure optimization. The cited tables compare methods using averaged accuracy and parameter counts on permuted MNIST and split CIFAR100.

The cited permuted MNIST and split CIFAR100 tables compare averaged accuracy after 10 tasks with total parameter counts.
Replay methods store previous-task information or examples to help retain earlier performance.
Regularization methods constrain parameter updates, including EWC’s quadratic penalty on weights important to previous tasks.
Expandable approaches can add or duplicate layers, but dynamically expandable and progressive networks force new tasks to use the old structure.
Neural architecture search is relevant because the proposed structure optimization uses search methods, including DARTS’s continuous relaxation for efficiency.

6. Conclusion

The paper concludes that explicit continual structure optimization with differentiable neural architecture search supports sensible task-specific structures and mitigates catastrophic forgetting. Across three benchmark settings, the method is highly comparable or better than state-of-the-art methods.

The learn-to-grow framework explicitly incorporates continual structure optimization through differentiable neural architecture search.
Each layer can be selected as “reuse”, “adapt” or “new” when learning a new task.
Explicit structure learning produces sensible structures for new tasks in continual learning with DNNs.
Catastrophic forgetting can be completely avoided without fine-tuning reuse layers or significantly alleviated with fine-tuning.
The method is tested on permuted MNIST, visual domain decathlon and split CIFAR-100, with highly comparable or better performance than state-of-the-art methods.

Supplementary Material – Learn to Grow: A Continual Structure Learning

The supplementary material is identified by the paper’s title and lists its authors. The supplied passages provide bibliographic identity rather than supplementary technical content.

The supplementary material is titled “Framework for Overcoming Catastrophic Forgetting”.
The listed authors are Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher and Caiming Xiong.

1. Additional Experimental Details for permuted MNIST

The permuted MNIST experiments use specified architectures, implementations, data controls and comparison settings. The supplied details also report difficulty obtaining reasonable deep-generative-replay results beyond five tasks.

The MNIST setup uses three fully connected hidden layers with 300 units each and one shared output layer for the proposed method.
For fair comparison, VCL uses the official implementation with a coreset size of zero.
Deep generative replay performed reasonably through five tasks at around 96% average performance, but degraded significantly beyond five tasks.
Progressive Network and PathNet use the authors’ own implementations, while DEN, HAT, EWC and IMM use specified implementations.
All methods use the same permutations and the same subset of training data.

2. Additional Experimental Details for Split CIFAR-100

Split CIFAR-100 experiments use an AlexNet-like network with three convolutional and two fully connected layers.

The architecture has three convolutional and max-pooling stages followed by two fully connected layers.Convolutional filters increase from 64 to 128 to 256, while each fully connected layer has 2048 hidden units.

3. Additional Experiments on Visual Decathlon Dataset

Visual Domain Decathlon experiments train ten tasks sequentially while maintaining a growing super model. The framework achieves strong accuracy with parameter growth controlled through a parameter-loss scaling factor.

Ten tasks are trained in a random sequence with ImageNet fixed as the first task for comparison with prior methods.The framework maintains a super model as tasks are trained sequentially.
The experiments compare top-1 validation accuracy and total model size across parameter-loss factors β of 0.01, 0.1, and 1.0.These comparisons evaluate both predictive performance and parameter growth.
The framework achieves the best results on five of nine tasks while keeping total model size similar to the Adapter baseline.It particularly outperforms other baselines on VGG-Flowers and Aircraft, which have small data sizes.
The parameter-loss function penalizes architectural choices according to their additional parameter cost and layer-specific weight α_l.The penalty is added to the validation loss during structure search.
β = 0.1 gives the best average accuracy and limits total model size to approximately 2.3× the original model size.The scaling factor β controls the additional parameter size for each task.
Accuracy and parameter growth remain robust across four random task orderings.The first task remains ImageNet because of its large size and long training time.