Source-linked AI summary
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Liyuan Wang, Xingxing Zhang, Hang Su, Jun Zhu
TL;DR
Continual learning seeks to let AI systems adapt to dynamic, lifelong data while limiting catastrophic forgetting and operating efficiently. This survey unifies theoretical objectives, method taxonomies, and application-specific adaptations, concluding that the field offers broad developments and opportunities for more human-like adaptability.
Problem
Continual learning must adapt to sequentially changing data while preserving prior knowledge, despite limited access to old data and resource constraints.
Method
The paper provides a comprehensive survey bridging basic settings, theoretical foundations, representative methods, and practical applications.
Results
The survey synthesizes stability-plasticity and generalizability objectives, organizes methods into five categories, and analyzes adaptations to realistic application challenges.
Takeaways & Limitations
The holistic perspective supports subsequent exploration of continual learning, including cross-directional opportunities and interdisciplinary connections.
Takeaways & Limitations
Replay can introduce privacy issues and linearly increasing resource overhead, while generative replay adds substantial overhead and faces forgetting and expressiveness limitations.
Abstract
from arXiv · showhide
To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as continual learning, provides a foundation for AI systems to develop themselves adaptively. In a general sense, continual learning is explicitly limited by catastrophic forgetting, where learning a new task usually results in a dramatic performance degradation of the old tasks. Beyond this, increasingly numerous advances have emerged in recent years that largely extend the understanding and application of continual learning. The growing and widespread interest in this direction demonstrates its realistic significance as well as complexity. In this work, we present a comprehensive survey of continual learning, seeking to bridge the basic settings, theoretical foundations, representative methods, and practical applications. Based on existing theoretical and empirical results, we summarize the general objectives of continual learning as ensuring a proper stability-plasticity trade-off and an adequate intra/inter-task generalizability in the context of resource efficiency. Then we provide a state-of-the-art and elaborated taxonomy, extensively analyzing how representative methods address continual learning, and how they are adapted to particular challenges in realistic applications. Through an in-depth discussion of promising directions, we believe that such a holistic perspective can greatly facilitate subsequent exploration in this field and beyond.
1 INTRODUCTION
Continual learning addresses adaptation to dynamic, incrementally arriving data while limiting catastrophic forgetting and balancing stability, plasticity, generalizability, and resource efficiency. This survey organizes theoretical foundations, representative methods, realistic applications, and emerging directions into a unified perspective.
- Motivation: Continual learning acquires, updates, accumulates, and exploits knowledge from incremental contents such as new skills, examples, environments, and contexts.The literature also uses incremental learning and lifelong learning for this setting without a strict distinction.
- Challenges: Dynamic data distributions create catastrophic forgetting, because adapting to new distributions generally reduces the ability to capture old ones.This reflects a stability-plasticity trade-off in which excessive plasticity interferes with memory stability, and vice versa.
- Methods: Continual learning methods are grouped into regularization-, replay-, optimization-, representation-, and architecture-based approaches.The survey connects these categories through their effects on optimization and knowledge distillation.
- Applications: Realistic applications introduce scenario complexity and task specificity, including missing task identities, tiny or one-pass batches, limited labels, and non-classification domains.The survey analyzes adaptations for vision, generation, reinforcement learning, natural language processing, and ethics-related settings.
- Scope: The survey aims to provide a comprehensive, up-to-date synthesis spanning basic setups, theory, methods, applications, trends, and interdisciplinary connections.It positions this breadth as a basis for subsequent work in continual learning and neuroscience-related directions.
2 SETUP
The setup section formalizes continual learning as sequential learning from dynamic task distributions under limited access to old data, and distinguishes scenarios by batch arrival and task-identity availability. It evaluates performance through overall accuracy, memory stability, learning plasticity, transfer, and resource-related measures.
- 2.1 Basic Formulation: Continual learning trains on task data arriving incrementally or simultaneously, often with no or limited access to old samples and without guaranteed labels or task identities.A task is defined through its training samples and distribution, with matching training and testing distributions assumed.
- 2.2 Typical Scenario: Typical scenarios include Instance-, Domain-, Task-, Class-, and Task-Free Continual Learning, distinguished by batch structure, label-space changes, and task-identity availability.Domain-incremental learning does not require task identities, while task-incremental learning provides them during training and testing.
- 2.2 Typical Scenario: Continual learning extends beyond supervised settings to zero-shot, few-shot, semi-supervised, open-world, and unsupervised or self-supervised scenarios.These variants depend on the availability of input data and labels for each task.
- 2.3 Evaluation Metric: Overall performance is commonly measured by average accuracy and average incremental accuracy, with evaluation using either task-specific or cumulative class output spaces.AA reflects current overall performance, whereas AIA additionally reflects historical performance.
- 2.3 Evaluation Metric: Memory stability is assessed with forgetting measure and backward transfer, while learning plasticity is assessed with intransience measure and forward transfer.Negative backward transfer usually reflects forgetting; other metrics cover task-specific performance, representation forgetting, flatness, anytime inference, storage, and computation.
3 THEORETICAL FOUNDATION
Theoretical analyses frame continual learning as balancing stability and plasticity while preserving generalization across tasks. They motivate Bayesian, regularization, replay, gradient-based, and other methods that manage task distributions and parameter updates under sequential learning.
- Stability-Plasticity Trade-off: Continual learning must balance stability and plasticity because old training sets become inaccessible while new tasks arrive sequentially.Excessive plasticity or stability can compromise the other, making balanced capture of old and new task distributions difficult.
- Stability-Plasticity Trade-off: Replay approximates old task distributions with stored samples or generative models, but increases resource overhead and can introduce privacy and forgetting concerns.More replayed samples can improve old-task performance, while generative models add resource demands and may themselves forget or lack expressiveness.
- Stability-Plasticity Trade-off: Bayesian continual learning propagates the previous posterior as the next prior, allowing the current posterior to be computed using only the current training set.Because exact posteriors are generally intractable, online Laplace approximation and variational inference approximate them with tractable distributions.
- Stability-Plasticity Trade-off: Online Laplace approximation recursively updates posterior modes and precision matrices, while practical implementations approximate curvature with the Fisher information matrix and simplified forms.Hessian computation is often inefficient in high-dimensional parameter spaces, and its approximation is not guaranteed to be positive semi-definite under the Gaussian assumption.
- Generalizability Analysis: Optimization-based methods directly manipulate gradient directions, using old samples or curvature-informed constraints to control updates without necessarily storing old data.Gradient projection can constrain task-specific optimization within a region measured using the prior curvature encoded by a precision matrix.
- Generalizability Analysis: Generalization depends on loss-landscape flatness and task-distribution similarity: synergistic tasks can improve each task, whereas competing tasks can deteriorate shared-solution performance.These analyses motivate solutions that combine an appropriate stability-plasticity trade-off with adequate intra- and inter-task generalizability.
4 METHOD
The survey organizes continual learning methods into five approaches and analyzes their motivations, implementations, and empirical properties. These approaches address forgetting through regularization, replay, optimization, representations, or task-adaptive architectures.
- The taxonomy comprises regularization-based, replay-based, optimization-based, representation-based, and architecture-based approaches.The survey presents this as an elaborated taxonomy of representative methods, including their motivations, implementations, and empirical properties.
- Regularization-based Approach: Regularization-based methods add explicit penalties that preserve old-model parameters or behaviors while learning new tasks.Weight regularization commonly stores a frozen copy of the old model and penalizes important parameter changes; function regularization uses distillation.
- Replay-based Approach: Replay-based methods approximate old data distributions through experience replay, generative replay, or feature replay.Experience replay stores selected samples, generative replay produces samples with generative models, and feature replay recovers old feature distributions.
- Representation-based Approach: Representation-based methods create robust features through self-supervised learning and pretraining, which can remain relatively stable for downstream continual learning.Strong pretraining provides representations that are generalizable to downstream tasks, while pre-trained representations may be fixed during task-specific learning.
- Architecture-based Approach: Architecture-based methods construct task-specific or adaptive parameters through parameter allocation, modular networks, or model decomposition.These approaches can reduce inter-task interference, but may affect scalability and inter-task generalizability or require task identities.
5 SCENARIO COMPLEXITY IN APPLICATION: THE CASE OF VISUAL CLASSIFICATION
This section organizes scenario complexity in visual-classification continual learning around task identity, data scarcity, and online, task-free streams. It also surveys representative strategies for class-incremental and resource-constrained learning.
- Task-Agnostic Inference: Task-agnostic class-incremental inference is more natural but harder than task-incremental learning because task identity is unavailable at test time.Class-incremental learning separates within-task prediction from task-identity prediction, the latter being closely related to OOD detection.
- Task-Agnostic Inference: Experience replay addresses class-incremental forgetting end to end, while feature and label distillation are limited by the new samples’ biased, old-class-OOD distribution.Many state-of-the-art methods therefore combine replay with knowledge distillation across data, feature, and label spaces.
- Task-Agnostic Inference: Data-free class-incremental learning replaces retained samples with synthetic replay, model inversion, or class-wise feature statistics to reduce storage and privacy costs.Model-inversion methods commonly add knowledge distillation to compensate for lost information.
- Scarcity of Labeled Data: Few-shot continual learning studies sequential novel classes with only small labeled datasets, where limited data exacerbates overfitting of representations to later tasks.Representative responses preserve representation topology, distill exemplar relations, or selectively update or stabilize parameters.
- Scarcity of Labeled Data: Semi-supervised and unsupervised continual learning exploit partially labeled, external unlabeled, or incrementally arriving unlabeled data for replay, consistency, distillation, and representation learning.General continual learning further covers task-free and one-pass online streams without explicit task boundaries.
6 TASK SPECIFICITY IN APPLICATION
This section surveys continual learning beyond visual classification, emphasizing how task-specific outputs and data structures create distinct forgetting challenges. It covers detection, segmentation, generation, reinforcement learning, language, and efficiency-oriented updates.
- Incremental Object Detection: Incremental object detection must locate and identify old and new object instances that can co-occur, unlike classification with one object instance per sample.Knowledge distillation is a recurring strategy across multiple detector architectures.
- Continual Semantic Segmentation: Continual semantic segmentation faces background shift when only new-class annotations are available, causing old classes to be treated as background.Adaptive distillation from the old model helps distinguish unannotated old classes from background pixels.
- Conditional Generation: Continual learning for conditional generation commonly uses generative replay to recover previously learned data distributions, often retaining a frozen old generator during new training.GAN-based methods combine replay with regularization, distillation, parameter allocation, or model decomposition.
- Reinforcement Learning: Continual reinforcement learning addresses changing distributions between and within tasks because states, actions, and environments interact over time.Its benchmarks span continuous control, maze navigation, and video games, varying widely in task type and computation overhead.
- Natural Language Processing: Continual NLP adapts common continual-learning scenarios and methods to pretrained transformers, motivating parameter-efficient adaptor- and prompt-tuning.The surveyed applications include DIL, TIL, CIL, OCL, and CPT.
- Beyond Task Performance: Resource-efficient updates also matter within ordinary single-task training because gradient optimization can make networks forget observed samples and require repetitive training.Experience replay of hard examples and orthogonal gradient projection improve one-pass visual-classification performance, while within-task forgetting affects reinforcement learning and GAN stability.
7 DISCUSSION
The discussion reframes continual learning as balancing stability with plasticity and improving generalizability across changing distributions. It also highlights expanding applications and connections to robustness, foundation models, multimodality, embodied AI, and neuroscience.
- Observation of Current Trend: A desirable continual-learning solution balances memory stability for old tasks with learning plasticity for new tasks and adequate inter-task generalizability.Recent work increasingly addresses plasticity and generalizability rather than focusing only on preserving old knowledge.
- Observation of Current Trend: Applications are becoming more diverse, but existing work often emphasizes basic benchmarks and baseline methods, leaving method specialization and generality evaluation for future work.The survey identifies robotics, graph learning, and bioimaging among additional scenarios to explore.
- Cross-Directional Prospect: Diffusion models offer a new target for continual generative learning, and their conditional-generation performance may improve generative replay.The paper also identifies multimodal information as a potential additional source of stabilization and task-identity prediction.
- Cross-Directional Prospect: Foundation-model pretraining creates demands for efficient updates, while fine-tuning can cause forgetting of pretrained knowledge and therefore requires specialized continual-learning strategies.Transformer architectures likewise require designs that overcome forgetting while preserving task specificity.
- Interdisciplinary Connections: Neuroscience informs continual-learning design through synaptic stabilization, connection expansion and pruning, and complementary hippocampal-neocortical learning functions.These biological mechanisms motivate regularization, architectural flexibility, and complementary systems for rapid experience acquisition and structured generalization.
8 CONCLUSION
The survey bridges theory, methods, and applications while organizing continual learning’s objectives and practical challenges. It concludes that the field’s broadening activity supports further work toward adaptable lifelong AI systems.
- Conclusion: The paper presents an up-to-date comprehensive survey bridging continual-learning theory, methods, and applications.It summarizes general objectives and particular challenges while analyzing how representative methods address them.
- Conclusion: The survey observes growing interest across AI, accompanied by new understandings, diversified applications, and cross-directional opportunities.It frames this holistic perspective as supporting subsequent exploration of continual learning.