Source-linked AI summary
Continual Learning with Pre-Trained Models: A Survey
Da-Wei Zhou, Hai-Long Sun, Jingyi Ning, Han-Jia Ye, De-Chuan Zhan
TL;DR
Continual learning must absorb streaming knowledge without catastrophic forgetting, while existing surveys mainly address methods built without pre-trained models. This paper surveys PTM-based continual learning, organizes methods into three categories, and evaluates representative approaches across seven datasets while examining comparison fairness. It concludes by proposing batch-agnostic inference for fairer comparisons and identifying future directions for the field.
Problem
Existing continual-learning surveys primarily focus on algorithms that do not incorporate pre-trained models, despite PTM-based continual learning becoming a central research area.
Method
The paper systematically surveys PTM-based continual learning, taxonomizes methods into prompt-based, representation-based, and model mixture-based categories, and evaluates representatives across seven benchmark datasets.
Results
The survey compares methods across seven benchmark datasets and raises batch-agnostic inference as a fair comparison protocol.
Takeaways & Limitations
The survey provides an up-to-date synthesis of PTM-based continual learning and highlights future directions including LLMs, multimodal learning, and restricted computational resources.
Takeaways & Limitations
Prompt-based methods can suffer from prompt-selection and prompt-level forgetting, while fixed or expanding prompt pools create representation or train-test mismatch constraints.
Abstract
from arXiv · showhide
Nowadays, real-world applications often face streaming data, which requires the learning system to absorb new knowledge as data evolves. Continual Learning (CL) aims to achieve this goal and meanwhile overcome the catastrophic forgetting of former knowledge when learning new ones. Typical CL methods build the model from scratch to grow with incoming data. However, the advent of the pre-trained model (PTM) era has sparked immense research interest, particularly in leveraging PTMs' robust representational capabilities. This paper presents a comprehensive survey of the latest advancements in PTM-based CL. We categorize existing methodologies into three distinct groups, providing a comparative analysis of their similarities, differences, and respective advantages and disadvantages. Additionally, we offer an empirical study contrasting various state-of-the-art methods to highlight concerns regarding fairness in comparisons. The source code to reproduce these evaluations is available at: https://github.com/sun-hailong/LAMDA-PILOT
1 Introduction
Continual learning addresses streaming data by acquiring new tasks while retaining prior knowledge, but catastrophic forgetting remains a central challenge. This survey focuses on pre-trained-model continual learning because existing surveys largely overlook it and current results suggest strong potential.
- Motivation: Continual learning adapts continuously to streaming data when privacy and storage constraints prevent permanent retention.The model must evolve as data arrives rather than retain all historical streaming data.
- Motivation: Catastrophic forgetting occurs when learning new knowledge significantly reduces performance on previously learned tasks.Addressing this phenomenon has motivated extensive continual-learning research.
- PTM-based CL: Traditional continual learning starts from randomly initialized weights, whereas PTM-based methods initialize from models pre-trained on extensive, informative datasets.Both approaches learn from sequential task streams, but their initial model setup differs.
- Research Gap: Existing continual-learning surveys primarily cover algorithms that do not incorporate pre-trained models.The survey identifies PTM-based continual learning as an emerging area requiring updated coverage.
- Contributions: The survey provides a comprehensive review, categorizes PTM-based methods into three subcategories, evaluates representatives on seven datasets, and examines comparison fairness.It also identifies challenges and future research directions.
2 Preliminaries
Continual learning trains on a sequence of task-specific datasets while aiming to perform well on all seen tasks without forgetting earlier knowledge. The paper uses ImageNet-pre-trained ViT models as representative PTMs, whose generalizable features support frozen-backbone or lightweight adaptation strategies.
- Continual Learning: Continual learning presents tasks sequentially, with training at stage b restricted to the current dataset D_b.The model is evaluated across all encountered tasks after learning each new task.
- Continual Learning: The continual-learning objective is to acquire knowledge from all seen tasks while minimizing expected risk and avoiding forgetting.Performance must remain strong on both new and previously learned tasks.
- Variations of CL: Class-incremental and task-incremental learning introduce new classes across tasks, but only task-incremental learning supplies the task identity during testing.Domain-incremental learning instead retains the label space while introducing domain shifts such as cartoon versus oil-painting images.
- Pre-Trained Models: PTM-based continual-learning methods commonly use ImageNet21K-pre-trained Vision Transformers as embedding functions because of their strong representation ability.The paper therefore treats ViT as its representative pre-trained model.
- Pre-Trained Models: A ViT divides images into non-overlapping patches and uses the final [CLS] token as the recognition feature.The classification model is decomposed as f(x) = W^Tϕ(x), with ϕ as the embedding function and W as the classification head.
- Pre-Trained Models: Pre-trained models provide generalizable feature extractors, enabling continual-learning algorithms to operate on frozen backbones or use lightweight tuning.Prompt tuning and adapter learning adapt downstream behavior while preserving pre-trained weights.
3 Continual Learning with PTMs
PTM-based continual learning studies are organized into prompt-based, representation-based, and model mixture-based methods, each exploiting different aspects of pre-trained models. The section describes their mechanisms and highlights trade-offs involving forgetting, efficiency, representation quality, memory, computation, and comparison fairness.
- PTM-based continual learning is taxonomized into prompt-based, representation-based, and model mixture-based methods.The categories reflect different ideas for addressing continual learning with pre-trained models.
- Prompt-based Methods: Prompt-based methods tune lightweight prompts while keeping pre-trained weights unchanged to preserve generalizability and alleviate forgetting.Prompt pools support instance-specific retrieval, with methods differing in retrieval, task sharing, attention-based combination, and prompt generation.
- Prompt-based Methods: Prompt selection matches query features with keys or combines prompt components using attention, while DAP generates instance-specific prompts through an MLP.CODA-Prompt uses weighted prompt construction, whereas DAP produces generation parameters from task predictions.
- Prompt-based Methods: Prompt-based methods face selection concentration, matching- and prompt-level forgetting, fixed-pool capacity limits, growing-pool train-test mismatch, and fairness concerns.The section also notes that some prompt-based methods can underperform a simple prototype-based baseline.
- Representation-based Methods: Representation-based methods use frozen pre-trained features to construct class prototypes, while feature concatenation adds task-specific information through parameter-efficient adaptation.SimpleCIL replaces classifier weights with class prototypes; ADAM and EASE concatenate generalized and adapted representations, with EASE combining multiple task-specific backbones.
- Representation-based Methods: Representation-based approaches can duplicate features across models and may insufficiently bridge domain gaps when downstream tasks involve multiple domains.Continually adjusting the backbone may be more suitable for extracting task-specific features in that setting.
- Model Mixture-based Methods: Model mixture-based methods derive predictions by merging or ensembling multiple models, but ensembles require historical-model storage and merging large backbones adds computation.The parameters to merge remain an open problem, leaving merging solutions heuristic and hand-crafted.
4 Experiments
The experiments evaluate representative PTM-based continual-learning methods across seven benchmarks and reveal important fairness concerns in comparison protocols.
- SimpleCIL outperforms typical prompt-based methods such as L2P and DualPrompt, supporting the strong representation ability of PTMs.The finding also suggests that more complex learning systems do not necessarily guarantee better performance.
- DAP’s improvements depend on batch voting information that effectively uses task identity during inference.Its instance-specific prompt parameters are generated by voting among instances from the same batch, simplifying the task.
- With testing batch size set to 1, DAP without batch information degrades drastically and performs worse than L2P.This motivates batch-agnostic inference for fair comparisons because models should be tested independently.
5 Future Directions
The survey identifies future directions for continual learning with PTMs in large language models, multimodal systems, and resource-constrained deployments.
- Continual learning could progressively update LLMs with new concepts instead of repeatedly retraining them on updated datasets.The paper relates this setting to lifelong model editing and notes its relevance to changing information such as global events.
- Multimodal continual learning extends beyond visual recognition by adapting vision-language models such as CLIP to multimodal tasks.PROOF introduces a cross-modal fusion module for continual learning with CLIP and other vision-language models.
- Resource-efficient continual learning is needed for deploying and training PTMs on edge devices with limited computational resources.The survey highlights local training and inference, including LLM-based personal-assistant smartphone applications, as a relevant setting.
6 Conclusion
The paper surveys continual learning with pre-trained models, evaluates representative methods across seven datasets, and emphasizes fairer batch-agnostic comparison.
- The survey categorizes continual-learning methods with pre-trained models into three taxonomic categories.It presents an up-to-date summary of recent work and identifies future research directions.
- The authors conduct extensive experiments across seven benchmark datasets for holistic evaluation among methods from the categories.
- The paper proposes a fair comparison protocol using batch-agnostic inference.