Source-linked AI summary
Continual Learning with Deep Generative Replay
Hanul Shin, Jung Kwon Lee, Jaehong Kim, Jiwon Kim
TL;DR
Catastrophic forgetting makes sequential multi-task learning degrade earlier capabilities, while storing and replaying past data requires substantial memory and may be infeasible. Deep Generative Replay uses a generator and solver to create past-task input-target pairs, interleave them with new data, and retain prior-task performance across sequential image-classification settings. Its effectiveness depends heavily on generator quality.
Problem
Catastrophic forgetting degrades performance on previously learned tasks during sequential training, while storing and replaying past data requires large memory and may be infeasible.
Method
Deep Generative Replay trains a generator–solver scholar model that produces pseudo-inputs and corresponding past-task responses for interleaving with new-task data.
Results
Generative replay maintained former-task performance while learning new tasks across sequential image-classification settings, with performance no worse than replaying past real inputs in the reported domain experiment.
Takeaways & Limitations
The framework retains knowledge without revisiting actual past data and can support practical settings involving privacy issues when the generator reliably reproduces the input space.
Takeaways & Limitations
The framework's efficacy depends heavily on generator quality, and the authors observed performance loss on SVHN in one evaluated setting.
Abstract
from arXiv · showhide
Attempts to train a comprehensive artificial intelligence capable of solving multiple tasks have been impeded by a chronic problem called catastrophic forgetting. Although simply replaying all previous data alleviates the problem, it requires large memory and even worse, often infeasible in real world applications where the access to past data is limited. Inspired by the generative nature of hippocampus as a short-term memory system in primate brain, we propose the Deep Generative Replay, a novel framework with a cooperative dual model architecture consisting of a deep generative model ("generator") and a task solving model ("solver"). With only these two models, training data for previous tasks can easily be sampled and interleaved with those for a new task. We test our methods in several sequential learning settings involving image classification tasks.
1 Introduction
Continual learning in deep neural networks is undermined by catastrophic forgetting, while replaying stored past data demands memory and may be infeasible. Deep Generative Replay addresses this constraint by using a generator–solver scholar model to produce pseudo-data and targets for rehearsal.
- Catastrophic forgetting causes performance on previously learned tasks to abruptly degrade when a model trains on a new task.
- Replay-based methods can preserve prior-task performance but require large working memory and may be impractical when past data cannot be stored or accessed.
- The hippocampus is presented as a generative model rather than a simple replay buffer because reactivation can produce flexible and even false memory outcomes.
- Deep Generative Replay trains a GAN-based generator to mimic past data and pairs generated samples with responses from the previous task solver.
- The generator–solver scholar model interleaves generated input-target pairs with new-task data, allowing sequential training without revisiting actual past data.
- The framework is tested on image classification and is proposed for broader tasks when the generator reliably reproduces the input space.
2 Related Works
Related work addresses catastrophic forgetting through replay, parameter regularization, task-specific architecture, pseudorehearsal, and generative modeling. The proposed framework distinguishes itself by learning past input distributions and jointly training on generated prior data with real current data.
- Catastrophic forgetting describes the degradation of previously learned knowledge caused by training neural networks on new samples and objectives.
- Regularization methods reduce interference by limiting changes to consolidated network weights when previous-task data are unavailable.
- Task-specific output layers reduce interference at the outputs, but changes in shared earlier layers can still harm older-task performance.
- Pseudorehearsal feeds a task network pseudoinputs and pseudo-targets from a memory network to preserve old input-output patterns without real past data.
- Generative replay differs by generating fake inputs from the learned past input distribution and jointly optimizing them with real current data.
- Generative models, including variational autoencoders and GANs, are designed to generate samples resembling complex real distributions such as images.
3 Generative Replay
Generative replay trains a scholar as a generator–solver pair that sequentially learns new tasks while retaining prior knowledge through generated data and responses. The procedure mixes current real examples with replayed examples from the previous scholar.
- Model definition: A scholar is a generator–solver pair in which the generator produces real-like samples and the solver performs the task sequence.The solver is parameterized by θ, while the generator supplies samples for replay.
- Sequential training: Sequential training uses each new scholar to learn the current task and the knowledge of the previous scholar.Training a single scholar while referring to its recent copy is equivalent to training a sequence of scholars.
- Generator training: The new generator mixes current inputs with replayed inputs to reconstruct the cumulative input space.The real-to-replayed mixing ratio depends on the desired importance of new tasks relative to older tasks.
- Solver training: The new solver jointly learns from current input-target pairs and replayed inputs labeled by the previous solver.Its replay loss compares the current solver’s output with the previous solver’s response on generated inputs.
- Implementation and initialization: The framework uses a task-suitable solver and a GAN-trained generator, while allowing any deep generative model to serve as the generator.In the first task, replay losses are omitted because no previous solver or replayed data exists.
4 Experiments
Across independent tasks, related domains, and disjoint classes, generative replay preserved earlier knowledge while learning new tasks. Its effectiveness depended on replaying meaningful input distributions and extended to LwF augmentation.
- 4.1 Learning independent tasks: Higher average accuracy followed when replayed inputs more closely resembled real data, while random noise or meaningless fake inputs failed to preserve performance.The comparison included exact replay, generative replay, and poorly matching replay inputs.
- 4.1 Learning independent tasks: Generative replay maintained former-task performance during sequential MNIST pixel-permutation training, unlike a solver trained without replay.Average accuracy remained near full performance on trained tasks, whereas naive sequential training incurred catastrophic forgetting.
- 4.2 Learning new domains: The generator produced samples resembling both MNIST and SVHN as training progressed on the second domain.Samples were examined after 1000, 2000, 5000, 10000, and 20000 training iterations.
- 4.2 Learning new domains: On MNIST and SVHN learned in either order, generative replay retained the first domain while achieving the second domain’s task.Performance was no worse than replaying past real inputs paired with predicted responses, although no-replay training performed slightly better on the new task.
- 4.2 Learning new domains: Generative replay augmented LwF by greatly tempering the loss of SVHN accuracy during shared-parameter fine-tuning, while both methods performed highly on MNIST.The comparison used LwF-GR against original LwF after training on SVHN and then MNIST.
- 4.3 Learning new classes: For five sequential MNIST tasks containing two disjoint classes each, only generative replay reached test performance close to the upper bound.Reconstructing output distributions without meaningful inputs did not retain prior classes; reconstructing both input and output distributions did.
5 Discussion
Deep generative replay retains prior knowledge by rehearsing generated input-target pairs, enabling flexible balancing of old and new tasks without revisiting past data. Its effectiveness depends heavily on generator quality, while combining it with EWC or LwF is not straightforward.
- Deep generative replay: Deep generative replay generates and rehearses fake data that mimics former training examples for sequential learning.A scholar model combines a generator and solver as a task knowledge base.
- Related approaches: EWC and LwF alleviate forgetting through parameter protection or shared-parameter training, but impose constraints that can trade off new-task and old-task performance.The discussion also notes that these approaches may require larger networks or additional training constraints.
- Deep generative replay: The framework maintains former knowledge using input-target pairs produced from saved networks, allowing flexible knowledge transfer and easier balancing of old and new task performance.Joint optimization can achieve full performance when the generator recovers former input spaces.
- Limitations and future work: Generative replay efficacy depends heavily on generator quality, with performance loss observed on SVHN under the reported setting.The authors identify improved generative modeling as a route to better performance on more complex domains.
- Related approaches: The authors state that EWC, LwF, and generative replay address memory retention at different levels but cannot be straightforwardly combined.They propose that a future mixture of the three frameworks might improve continual learning.