Source-linked AI summary
Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
Nicolas Y. Masse, Gregory D. Grant, David J. Freedman
TL;DR
Sequential learning often causes ANNs to forget earlier tasks when new training alters important connection weights. The paper adds task-specific context-dependent gating and combines it with synaptic stabilization, enabling high performance across large sequential task sets with little forgetting.
Problem
Sequentially trained ANNs often suffer catastrophic forgetting because new-task learning alters connection weights important for previous tasks.
Method
The paper uses task-specific context-dependent gating to activate sparse, mostly non-overlapping unit patterns, combined with synaptic stabilization.
Results
The combined method successfully learned 100 sequential MNIST and ImageNet tasks with little forgetting and enabled recurrent networks to learn 20 tasks with high accuracy.
Takeaways & Limitations
Combining complementary neuroscience-inspired mechanisms provides a simple, computationally inexpensive approach for continual learning in feedforward and recurrent ANNs.
Abstract
from arXiv · showhide
Humans and most animals can learn new tasks without forgetting old ones. However, training artificial neural networks (ANNs) on new tasks typically cause it to forget previously learned tasks. This phenomenon is the result of "catastrophic forgetting", in which training an ANN disrupts connection weights that were important for solving previous tasks, degrading task performance. Several recent studies have proposed methods to stabilize connection weights of ANNs that are deemed most important for solving a task, which helps alleviate catastrophic forgetting. Here, drawing inspiration from algorithms that are believed to be implemented in vivo, we propose a complementary method: adding a context-dependent gating signal, such that only sparse, mostly non-overlapping patterns of units are active for any one task. This method is easy to implement, requires little computational overhead, and allows ANNs to maintain high performance across large numbers of sequentially presented tasks when combined with weight stabilization. This work provides another example of how neuroscience-inspired algorithms can benefit ANN design and capability.
1 Introduction
The paper addresses catastrophic forgetting in sequentially trained ANNs by combining neuroscience-inspired context-dependent gating with synaptic stabilization. This combination enables learning many tasks with little forgetting.
- Sequential training can cause catastrophic forgetting because new learning alters connection weights optimized for previous tasks.
- Synaptic stabilization alone may be insufficient for continual learning across large numbers of tasks, motivating complementary algorithms.
- Context-dependent gating activates sparse, mostly non-overlapping unit patterns for each task through a task-specific signal projected onto hidden neurons.
- Across 100 sequential MNIST permutations and 100 sequential ImageNet tasks, gating or stabilization alone was partially effective, whereas their combination supported all tasks with little forgetting.
- Combined gating and stabilization also enabled recurrent networks trained with supervised or reinforcement learning to learn 20 neuroscience tasks with high accuracy.
2 Results
The results test whether neuroscience-inspired stabilization methods scale beyond short task sequences. On sequential permuted-MNIST tasks, unprotected networks rapidly lose performance as more permutations are learned.
- The study evaluates feedforward networks on 100 sequential permuted-MNIST tasks, an input-reformatting problem with unchanged input and output semantics but changing pixel locations.
- 98.5% single-permutation accuracy fell to 52.5% after 10 permutations and 19.1% after 100 without synaptic stabilization.
- Synaptic Stabilization: Synaptic intelligence and elastic weight consolidation reduce forgetting by penalizing changes to parameters according to their importance for previous tasks.
A B Network with synaptic stabilization
Synaptic stabilization protects weights important for earlier tasks, while context-dependent gating limits which units can change for each task. Combining these mechanisms preserves accuracy across many sequential tasks by maintaining stability while retaining flexibility for new learning.
- Network with synaptic stabilization: Synaptic stabilization methods penalize changes to connection weights and biases according to their importance for previously learned tasks.EWC estimates parameter importance from output sensitivity, whereas SI uses the relationship between loss gradients and parameter changes.
- Network with synaptic stabilization: Context signals identify the current task, but contextual information alone remains insufficient to prevent forgetting across many tasks.With stabilization plus a context signal, mean permuted-MNIST accuracy reached 89.6% with SI and 87.3% with EWC, yet remained below single-task accuracy after 100 tasks.
- Context-Dependent Gating: 80% gating with XdG combined with stabilization reduced forgetting across 100 tasks, lowering accuracy from 98.2% on the first task to 95.4% on average.Across 500 tasks, XdG with SI retained a mean accuracy of 90.7%, compared with 54.9% for SI alone.
- Analyzing the interaction between XdG and synaptic stabilization: XdG helps stabilization balance protecting important synapses with retaining flexibility to adjust less important synapses for new tasks.Compared with SI alone, XdG produced larger overall synaptic changes while confining larger adjustments more strongly to synapses with low importance.
- XdG on the ImageNet Dataset: On ImageNet with differing output-class assignments across tasks, XdG combined with stabilization enabled learning all 100 tasks with higher mean accuracy than either method alone.Mean accuracy was 50.7% with SI plus XdG and 52.4% with EWC plus XdG, versus 28.1% with XdG alone.
3 Discussion
The study argues that combining context-dependent gating with synaptic stabilization alleviates catastrophic forgetting across many sequential tasks. It also identifies transfer learning and modular representations as important directions and boundaries for extending the method.
- 3 Discussion: XdG combined with synaptic stabilization alleviates catastrophic forgetting in feedforward and recurrent networks across many sequential tasks.The approach applies to networks trained with supervised or reinforcement learning.
- 3 Discussion: The method is simple to implement and has little computational overhead.
- 3.1 Transfer learning: XdG likely does not support transfer learning in its current form.Selective gating might be modified to activate reusable task-relevant building blocks, but this possibility remains speculative.
- 3.1 Transfer learning: Transfer learning requires algorithms that identify applicable network modules and compare the current task or context with previously learned contexts.The authors state that the proposed method lacks this capability.
- 3.1 Transfer learning: Diffuse representations of learned information may make relevant circuits difficult to activate, motivating strategies that encourage modular representations.The authors connect modular representations with the broader goal of continual and transfer learning.
- 3.2 Related methods: The discussion places XdG alongside earlier stabilization, modular-expansion, and gating methods, while noting that the study focuses on networks whose size cannot be augmented for each task.Related approaches include Progressive Neural Networks, Learning Without Forgetting, PathNet, and methods that gate weights or units.
4 Methods
The study combines context-dependent gating with synaptic stabilization methods and evaluates these approaches in feedforward and recurrent networks using supervised and reinforcement learning.
- Network training and testing: The networks were trained with TensorFlow, Adam, task-specific epoch or batch schedules, and separate test batches for MNIST and ImageNet.The optimizer used η = 0.001, β1 = 0.9, and β2 = 0.999, with optimizer state reset between tasks.
- Network architectures: The feedforward experiments used fully connected and convolutional architectures, while recurrent experiments used LSTM networks for cognitive tasks.The ImageNet model fixed convolutional parameters before training its fully connected layers on 100 tasks.
- Reinforcement learning: Reinforcement-learning RNNs used actor-critic training with value, policy, and entropy terms in the loss function.The entropy term encourages exploration, while α and β control the influence of entropy and value losses.
- Synaptic stabilization: XdG was paired with SI or EWC, which stabilize important parameters by penalizing changes from their values before a new task.EWC estimates importance with diagonal Fisher information, whereas SI estimates importance from parameter changes and loss gradients.
- Importance estimation: For both stabilization methods, parameter importance at the start of a task accumulates importance across completed tasks.The reinforcement-learning adaptation of SI instead uses changes in parameter values and mean reward because estimated future rewards can be inaccurate early in training.
- Hyperparameter selection: The study selected stabilization and gating hyperparameters by comparing candidate values and using those yielding the greatest mean classification accuracy.The tested parameters included c and ζ for SI, c for EWC, and multiple MNIST input-dropout settings.
Supplemental Information
The supplemental experiments describe twenty cognitive tasks and compare gating configurations and methods across extended sequential-task evaluations.
- Cognitive task set: The twenty cognitive tasks were grouped into Go, Anti-Go, Decision-Making, Delayed Decision-Making, and Matching categories.These tasks used coherent motion stimuli in eight possible directions and required either responses or withholding responses.
- Go and Anti-Go tasks: Go tasks required responses toward a presented motion direction, whereas Anti-Go tasks required responses 180 degrees opposite to it.The task variants differed in stimulus timing and whether fixation had to be maintained.
- Decision-making tasks: Decision-making tasks required selecting stronger motion, attending to task-relevant locations, or integrating multiple stimuli.Delayed variants changed stimulus timing and relative strengths compared with the corresponding decision-making tasks.
- Matching tasks: Matching tasks required responding when two motion stimuli matched either exactly or by category, with standard and opposite-direction response rules.The stimuli were separated by variable delays, and matching occurred on 50% of trials.
- Gating configurations: Across 100 permuted-MNIST tasks, SI combined with context-dependent gating achieved peak mean accuracy when 80%–86.7% of hidden units were gated per task.The evaluated gating levels were 50%, 67%, 75%, 80%, and 86.7%.
- Extended MNIST evaluation: Across 500 MNIST permutations, the comparison used SI alone versus SI with context-dependent gating applied to 92.2% (11/12) of units per task.
- Method comparison: Across 100 MNIST permutations, XdG with stabilization reached 95.8% mean accuracy versus 93.0% for HAT under the supplemental comparison settings.Both methods were trained for 100 epochs per task, and HAT used λ = 0.75.