Source-linked AI summary
Continual Learning of Context-dependent Processing in Neural Networks
Guanxiong Zeng, Yang Chen, Bo Cui, Shan Yu
TL;DR
Deep neural networks typically learn fixed input-output mappings, limiting context-dependent processing. This paper combines OWM and CDP to let one network learn numerous mappings continually, including with as few as ~10 samples per mapping.
Problem
Current deep neural networks primarily learn fixed input-output mappings rather than flexibly mapping identical stimuli to different actions across contexts.
Method
The approach combines orthogonal weights modification to protect prior knowledge with context-dependent processing to modulate sensory representations and reuse mappings across contexts.
Results
A single network learned up to thousands of context-dependent mapping rules sequentially without interference, with fewer than 10 samples per class sufficient for Chinese characters.
Takeaways & Limitations
The study demonstrates that a highly compact network can acquire many context-dependent mappings sequentially in an online and continual manner.
Takeaways & Limitations
The approach is demonstrated with supervised learning, and approaching network capacity may require larger regularization settings or additional layers.
Abstract
from arXiv · showhide
Deep neural networks (DNNs) are powerful tools in learning sophisticated but fixed mapping rules between inputs and outputs, thereby limiting their application in more complex and dynamic situations in which the mapping rules are not kept the same but changing according to different contexts. To lift such limits, we developed a novel approach involving a learning algorithm, called orthogonal weights modification (OWM), with the addition of a context-dependent processing (CDP) module. We demonstrated that with OWM to overcome the problem of catastrophic forgetting, and the CDP module to learn how to reuse a feature representation and a classifier for different contexts, a single network can acquire numerous context-dependent mapping rules in an online and continual manner, with as few as $\sim$10 samples to learn each. This should enable highly compact systems to gradually learn myriad regularities of the real world and eventually behave appropriately within it.
INTRODUCTION
The study addresses context-dependent cognitive control by proposing OWM and CDP to let one neural network continually learn diverse mapping rules. OWM protects prior knowledge, while CDP enables context-specific mappings even for identical stimuli.
- Motivation: Humans and non-human primates can map the same sensory stimulus to different actions depending on goals, environments, and internal states.This flexibility is described as cognitive control, enabling appropriate behavior across many situations with a limited behavioral repertoire.
- Approach: The proposed approach combines orthogonal weight modification (OWM) with a context-dependent processing (CDP) module for progressive context-dependent learning.OWM protects previously acquired knowledge, whereas CDP allows contextual information to modulate sensory-feature representations.
- Results: Up to thousands of different mapping rules could be learned sequentially without interference.This result is attributed to OWM’s protection of previously acquired knowledge.
- Results: ∼10 samples were sufficient to learn each mapping rule.The study reports this sample efficiency alongside continual acquisition of numerous mapping rules.
- Results: A single network can learn different context-specific mappings for even identical stimuli.The CDP module enables contextual information to determine how sensory features and mappings are processed.
1 ORTHOGONAL WEIGHTS MODIFICATION (OWM)
OWM enables continual learning by projecting weight updates into a subspace that preserves previously learned mappings, while avoiding storage of all prior inputs. Across benchmark, character, and image tasks, it supports scalable sequential learning, few-shot category acquisition, and continual end-to-end training, with capacity limits emerging as task demands increase.
- Benchmark evaluations: OWM was evaluated on sequential shuffled and disjoint MNIST tasks using ReLU feedforward networks trained across entire multilayer architectures.These experiments tested continual recognition of handwritten digits across multiple sequential mappings.
- Orthogonal weights modification: OWM projects standard backpropagation updates into a subspace supporting previously learned tasks, preserving the existing weight configuration during new-task training.The implemented update is ΔW = κPΔWBP, where P is computed iteratively from prior input vectors and need not require storing them all.
- Scalability: 3,755 Chinese characters and natural images from ImageNet were used to test whether OWM could scale to many mappings and more complex inputs.The character task used CASIA-HWDB1.1, while the reported results indicated that system performance approached the front-end feature extractor’s limit.
- Few-shot continual learning: < 10 samples per class were sufficient for satisfactory Chinese-character performance when learning new mappings with the OWM-trained classifier.Comparisons with other methods in the same task also supported better performance with fewer training samples.
- Capacity: Performance remained stable until network size or task count reached a threshold, after which it declined, indicating an approach to network capacity.Capacity was examined by reducing network size for a task or increasing the number of tasks for a fixed network.
- End-to-end continual learning: OWM also supported continual end-to-end training of feature extractor and classifier across five CIFAR-10 tasks, each introducing two classes, although performance remained below that with pre-trained features.This result advances online learning by reducing the distinction between DNN training and application phases.
2 CONTEXT DEPENDENT PROCESSING MODULE
The CDP module enables context-dependent learning by allowing contextual information to interact with sensory information. Inspired by primate PFC architecture, it processes both inputs before an OWM-trained classifier to select task-relevant sensory features.
- Context-dependent learning requires contextual information to interact properly with sensory information.
- Inspired by primate PFC architecture, the CDP module receives sensory inputs and contextual information to choose features relevant to the present task.The PFC receives both sensory and contextual information and uses them to guide action.
- The CDP module is placed before the OWM-trained classifier and is fed sensory feature vectors together with contextual information.
DISCUSSION
The study presents a compact architecture that combines context-dependent processing with continual-learning mechanisms to acquire many context-dependent mappings sequentially. Its results emphasize preserving prior knowledge while retaining plasticity, while noting limitations of alternative approaches and potential extensions beyond supervised learning.
- Contextual information can select task-relevant sensory features, enabling artificial neural networks to exhibit flexible context-dependent behavior.The discussion relates this function to primate prefrontal cortex and reports that a similar architecture achieves it in neural networks.
- Efficient, scalable continual learning is crucial both for flexible context-dependent processing and for learning new tasks as they are encountered.The key requirement is preserving previously acquired knowledge while maintaining plasticity for subsequent learning.
- Context-dependent gating can separate task-specific subnetworks, but requiring the same contextual information during training and testing limits practical applicability.This limitation distinguishes the discussed approach from a recently proposed continual-learning solution.
- The study demonstrates that a highly compact network can be taught many context-dependent mappings sequentially.The authors also state that OWM may apply beyond supervised learning and relate it to EWC, which has been implemented in reinforcement learning.
METHODS
The methods combine orthogonal weights modification (OWM) with a context-dependent processing (CDP) module to support sequential learning while reusing representations across contexts. OWM protects previously learned tasks through recursively updated projectors, while CDP changes feature representations using contextual signals and fixed input weights.
- OWM algorithm: OWM uses an orthogonal projector in each layer’s input space to reduce catastrophic interference during sequential learning.The projectors are recursively updated from current inputs and the previous task’s projector, avoiding the matrix inversion required by the original definition.
- OWM algorithm: The OWM procedure initializes weights and projectors, applies standard backpropagation, updates weights, and iteratively updates projectors before proceeding to the next task.An alternative updates projectors after every batch with a decaying α, treating each batch as a separate task and accelerating processing without extra storage or data reloading.
- OWM capacity: OWM capacity is quantified by normalized projector rank, with whole-network capacity approximated by summing the normalized ranks across layers.When the network approaches its capacity limit, the method proposes increasing α or adding layers such as the CDP module to preserve previously learned knowledge.
- CDP module: The CDP layer modulates feature vectors with non-negative context signals, while its randomly initialized input weights remain fixed and its remaining weights are trained with OWM.For identical inputs, task-specific controlling signals alter the output representation; the layer therefore rotates feature vectors while preserving information content and supports sequential learning of different tasks.
- CDP module: 90.93% average accuracy with hyperbolic tangent and 90.05% with logistic activation were close to 90.38% with ReLU on the same tasks.The study used ReLU, although the formulation also permits hyperbolic tangent or logistic activation functions.
COMPETING INTERESTS
The Institute of Automation, Chinese Academy of Sciences has submitted pending patent applications covering the OWM algorithm and CDP module.
- COMPETING INTERESTS: Two pending patent applications cover the OWM algorithm and CDP module, filed by the Institute of Automation, Chinese Academy of Sciences.The OWM application is PCT/CN2019/083355, and the CDP application is PCT/CN2019/083356.