Source-linked AI summary
Deep Generative Dual Memory Network for Continual Learning
Nitin Kamra, Umang Gupta, Yan Liu
TL;DR
Sequential learning causes neural networks to forget earlier tasks, challenging continual learning under limited memory. The paper addresses this with a hippocampus-inspired short-term memory, a neocortex-inspired long-term memory, and generative replay. Experiments report better retention and reduced forgetting across task sequences, with reconstruction also improving robustness to noisy and occluded inputs.
Problem
Sequentially trained neural networks forget previously learned tasks, making catastrophic forgetting a central challenge for continual learning.
Method
The paper uses dual generative memories: short-term memories rapidly learn incoming tasks, while a long-term memory consolidates prior knowledge through generative replay during sleep or downtime.
Results
DGDMN and DGR outperform other baselines on average accuracy while achieving the least negative backward transfer, mitigating catastrophic forgetting; the LTM is also more robust to noise and occlusion.
Takeaways & Limitations
Generative replay and dual memories support long-term performance retention, while periodic consolidation connects the architecture to complementary learning systems and sleep-related memory processes.
Takeaways & Limitations
The paper leaves incorporating synaptic consolidation into the dual-memory architecture for future work and assumes the generator can produce reliable samples and reconstruct incoming samples accurately.
Abstract
from arXiv · showhide
Despite advances in deep learning, neural networks can only learn multiple tasks when trained on them jointly. When tasks arrive sequentially, they lose performance on previously learnt tasks. This phenomenon called catastrophic forgetting is a fundamental challenge to overcome before neural networks can learn continually from incoming data. In this work, we derive inspiration from human memory to develop an architecture capable of learning continuously from sequentially incoming tasks, while averting catastrophic forgetting. Specifically, our contributions are: (i) a dual memory architecture emulating the complementary learning systems (hippocampus and the neocortex) in the human brain, (ii) memory consolidation via generative replay of past experiences, (iii) demonstrating advantages of generative replay and dual memories via experiments, and (iv) improved performance retention on challenging tasks even for low capacity models. Our architecture displays many characteristics of the mammalian memory and provides insights on the connection between sleep and learning.
1. Introduction
Sequential task learning causes catastrophic forgetting because models violate the iid-learning assumption, motivating replay-based approaches. Inspired by complementary learning systems, the paper proposes dual memories that consolidate past experiences through generative replay.
- Sequentially trained models forget previously learned tasks, making catastrophic forgetting a central obstacle to continual learning.
- Because sequential data violate the iid assumption of gradient-based learning, replaying earlier experiences can help restore the required sampling conditions.
- Generative replay maintains a model over past samples, reducing memory demands and avoiding explicit decisions about which examples to retain or discard.
- Complementary learning systems separate rapid hippocampal learning from slower neocortical consolidation of knowledge across tasks.
- The proposed architecture uses short-term and long-term generative memories, transferring replayed samples during downtime to consolidate prior tasks while learning new ones.
2. Problem Description
The paper formalizes continual supervised learning as sequentially arriving tasks with unknown total duration and limited storage. It evaluates retention using average accuracy and backward transfer, where strongly negative transfer indicates forgetting.
- Each task provides supervised samples drawn iid from its own distribution, while task distributions may differ completely and arrive sequentially.
- The total number of tasks is unknown in advance, and the architecture must predict labels for test samples from any task observed so far.
- Limited storage is smaller than the total training set, making it infeasible to store and replay every sample directly.
- Average accuracy measures performance across tasks, while backward transfer measures how learning later tasks affects earlier-task performance.
- Large negative backward transfer indicates catastrophic forgetting, so successful continual learning seeks high accuracy with minimally negative or positive transfer.
3. Deep Generative Dual Memory Network
Deep Generative Replay combines new samples with generated samples from prior tasks, while the dual-memory network separates rapid task acquisition from long-term consolidation. Short-term memories handle recent tasks and the long-term memory consolidates them through replay during sleep.
- 3.1. Deep Generative Replay: Deep Generative Memory combines a generator, a feedforward learner, and a task-descriptor dictionary to represent and update learned tasks.
- 3.1. Deep Generative Replay: Deep Generative Replay jointly trains on incoming samples and generated samples from previous tasks, balancing them according to memory age and capacity.
- 3.1. Deep Generative Replay: The replay procedure reconstructs incoming samples with a variational autoencoder before learner training, improving robustness to noise and occlusion.
- 3.2. Dual memory networks: A dual-memory network uses a large long-term memory for consolidated prior knowledge and multiple small short-term memories that rapidly learn individual new tasks.
- 3.2. Dual memory networks: When a task was previously consolidated, the long-term memory reconstructs samples and labels for its short-term memory; after short-term memories are exhausted, sleep consolidates tasks into long-term memory.
- 3.2. Dual memory networks: At test time, the relevant short-term memory predicts recent tasks when available, otherwise prediction is deferred to the long-term memory.
- 3.2. Dual memory networks: Task descriptors identify previously observed tasks or memory locations, but reconstruction error can provide recognition when descriptors are unavailable.
4. Experiments
Experiments show that generative replay and dual memory reduce catastrophic forgetting across sequential image-classification tasks. DGDMN retains more accuracy than DGR on long task sequences while training faster, supporting periodic consolidation through separate short- and long-term memories.
- Experimental setup: Experiments evaluate sequential image classification on Permnist, Digits, TDigits, Shapes, and Hindi, using shared-capacity networks to expose forgetting.The study compares DGDMN with baselines representing representational overlap, learning slowdown, and experience replay.
- Accuracy and forgetting curves: NN and DropNN forget catastrophically, while EWC improves retention but rapidly slows learning and stagnates after Task 3.The authors report that DGR and DGDMN can learn all tasks sequentially with the same learner networks.
- Accuracy and forgetting curves: DGR and DGDMN retain performance on Digits by replaying generated samples, whereas most baselines learn only the current task and immediately forget previous tasks.Digits is especially difficult because each task has a single label, encouraging overfitting to the incoming task.
- Accuracy and forgetting curves: DGR and DGDMN consistently outperform baselines in retained average accuracy and achieve the least negative backward transfer across datasets.Backward transfer directly measures forgetting, so the results indicate reduced inter-task interference.
- Long task sequences: On 40-task TDigits, DGDMN forgets gradually, retains more accuracy than DGR, and trains faster because its STM consolidates less often using less erroneous samples.DGDMN stays around 90% average accuracy on the most recently observed 10 tasks, while DGR eventually fails on this metric.
- Connections to complementary learning systems and sleep: The experiments connect DGDMN’s dual memories and periodic sleep-like consolidation with scalable continual learning and improved retention over DGR.The paper presents this connection as an interpretation of why separate memories and downtime consolidation help sequential learning.
5. Analysis and discussion
The analysis examines DGDMN’s robustness, scalability, and relevance to streaming data, highlighting reconstructive robustness and task-descriptor-free recognition while noting open questions about synaptic consolidation.
- Hyperparameters: DGDMN’s two hyperparameters, κ and nSTM, have intuitive interpretations and can be tuned with simple heuristics.κ controls the minimum LTM fraction reserved for incoming tasks, while nSTM controls consolidation-cycle frequency.
- Resilience to noise and occlusion: DGDMN’s LTM is more robust to noise and occlusion because its denoising reconstructive properties support reconstruction of corrupted inputs.The comparison evaluates jointly trained LTM and NN models on noisy and occluded Digits images.
- Choice of generative model: The architecture is agnostic to the underlying generative model if it generates reliable samples and accurately reconstructs incoming samples.The paper mentions VAEs and several GAN variants as possible choices depending on the modeled domain.
- Learning from streaming data: DGDMN can recognize task samples through a reconstructive generative model without task descriptors, supporting continual learning from directly streaming data.The paper presents this as applicable to streaming domains and as a possible basis for future reinforcement-learning applications.
- Open directions: The authors leave incorporating synaptic consolidation into the dual-memory architecture without causing stagnation to future work.This limitation connects the architecture’s complementary-learning-systems inspiration with an unimplemented biological mechanism.
6. Conclusion
The paper concludes that DGDMN uses dual memories and generative replay to consolidate past experiences and retain performance during continual learning. Experiments indicate that generative replay and the dual-memory design scale well and improve retention even for low-capacity models.
- Conclusion: DGDMN emulates complementary learning systems through dual memories and maintains consolidated long-term memory using generative replay.The conclusion presents these as the architecture’s central mechanisms for averting catastrophic forgetting.
- Conclusion: Experiments show that generative replay performs best for long-term performance retention and scales with a dual-memory architecture.The reported conclusion is based on the paper’s experiments across continual-learning settings.
- Conclusion: The architecture improves performance retention on challenging tasks even when model capacity is low.This is stated as a principal scope of the reported empirical advantage.
7. Appendix A
The appendix examines revision, task-structure representations, generated samples, and recognition without task descriptors. DGDMN supports revision and descriptor-free recognition, while sequential latent representations and older generated samples retain limitations.
- Repeated tasks and revision: DGDMN improves previously learned tasks during revision, whereas EWC stagnates and does not benefit from revisiting some tasks.DGDMN learns all tasks through Task 6 and then improves after revising Tasks 1 and 6.
- Additional datasets: DGR and DGDMN learn Shapes and Hindi task structure sequentially, while all baselines exhibit catastrophic forgetting.The same pattern appears in the corresponding average forgetting curves.
- Jointly versus sequentially learned structure: The LTM latent space segregates the ten Digits classes under both joint and sequential training, with roughly similar neighboring clusters.The absolute cluster locations differ between the two t-SNE visualizations.
- Jointly versus sequentially learned structure: The authors caution that these t-SNE results do not establish identical latent representations for shared task structure and defer fuller investigation.This is an explicit limitation of the visualization-based comparison.
- Generated samples: Sequentially trained LTM samples are sharp for recently learned digits but blurred for earlier digits, indicating partial forgetting in generated samples.Jointly trained LTM samples are described as quite sharp.
- Recognition without task descriptors: DGDMN recog uses reconstruction loss and a threshold to recognize tasks without descriptors, achieving ACC = 0.766 and BWT = −0.197 across all tasks.With γdgm = 1.55, the variant shows very little degradation relative to descriptor-based DGDMN.
8. Appendix B
The appendix documents datasets, figures, training settings, model configurations, and DGDMN hyperparameters. It emphasizes long-sequence evaluation, consolidation trade-offs, and implementation choices governing memory and replay.
- Datasets: TDigits contains 40 transformed-MNIST tasks and is used to evaluate continual learning over longer task sequences.The transformations include original, mirrored, upside-down, and main-diagonal-reflected digit images.
- Figures: The reported figures plot task-wise accuracy, forgetting, latent embeddings, generated digits, robustness, and training time across the evaluated datasets.The figure captions specify axes or panels for Digits, Shapes, Hindi, TDigits, and related analyses.
- Training settings: Models use RMSProp with learning rate 0.001, batch size 128, and distinct epoch budgets for joint, sequential, and generative training.Generative models use gradient clipping and are trained for 25 epochs regardless of task or dataset.
- Model configurations: The experiments compare DGDMN with neural, dropout, rehearsal, elastic-weight-consolidation, and generative-replay baselines using matched classifier architectures where specified.The appendix also describes dataset-specific hidden-layer sizes and generative-model architectures.
- Hyperparameters: DGDMN uses κ to reserve capacity for incoming tasks and nSTM to control consolidation frequency.Increasing nSTM improves learning speed but can increase forgetting because fewer previous-task samples participate in consolidation.
- Memory configuration: The DGR and DGDMN memory capacity Nmax equals total dataset size for most datasets but is halved for TDigits to study long-term learning.nSTM is set to 2 for Digits, Permnist, and Shapes, 5 for TDigits, and 2 for Hindi.