Source-linked AI summary
Learning to Prompt for Continual Learning
Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas Pfister
TL;DR
Continual learning must adapt to sequentially changing data without catastrophically forgetting prior knowledge, while common solutions rely on rehearsal buffers or task identity. L2P keeps a pretrained backbone frozen and learns an instance-wise queried prompt pool that manages shared and task-specific knowledge. Across several continual-learning settings, it consistently outperforms prior state-of-the-art methods and remains competitive with rehearsal-based methods without a buffer.
Problem
Continual learning must train a single model on non-stationary sequential data, but prior approaches often rely on rehearsal buffers or task identity to mitigate catastrophic forgetting.
Method
L2P freezes a pretrained backbone and learns a key-value prompt pool whose instance-wise query mechanism selects prompts to encode shared and task-specific knowledge.
Results
L2P consistently outperforms previous state-of-the-art methods across multiple continual-learning benchmarks and remains competitive with rehearsal-based methods without a rehearsal buffer.
Takeaways & Limitations
L2P provides a prompt-based continual-learning framework applicable to task-agnostic settings without known task identity or boundaries.
Takeaways & Limitations
The method is demonstrated on vision models, assumes pretrained sequence-based models, and is evaluated in task-agnostic settings using a synthetic Gaussian-scheduled CIFAR-100 benchmark.
Abstract
from arXiv · showhide
The mainstream paradigm behind continual learning has been to adapt the model parameters to non-stationary data distributions, where catastrophic forgetting is the central challenge. Typical methods rely on a rehearsal buffer or known task identity at test time to retrieve learned knowledge and address forgetting, while this work presents a new paradigm for continual learning that aims to train a more succinct memory system without accessing task identity at test time. Our method learns to dynamically prompt (L2P) a pre-trained model to learn tasks sequentially under different task transitions. In our proposed framework, prompts are small learnable parameters, which are maintained in a memory space. The objective is to optimize prompts to instruct the model prediction and explicitly manage task-invariant and task-specific knowledge while maintaining model plasticity. We conduct comprehensive experiments under popular image classification benchmarks with different challenging continual learning settings, where L2P consistently outperforms prior state-of-the-art methods. Surprisingly, L2P achieves competitive results against rehearsal-based methods even without a rehearsal buffer and is directly applicable to challenging task-agnostic continual learning. Source code is available at https://github.com/google-research/l2p.
1. Introduction
Continual learning must handle sequentially shifting data while limiting catastrophic forgetting, but existing approaches often depend on rehearsal buffers or task identity. L2P addresses these constraints with a dynamically selected prompt-pool memory for a frozen backbone and performs strongly across challenging settings.
- Continual learning trains one model on non-stationary data, causing deterioration on previously learned data through catastrophic forgetting.
- Prior methods often rely on rehearsal buffers, whose effectiveness deteriorates with smaller buffers and whose use can be incompatible with privacy-sensitive scenarios.
- L2P learns a shared prompt pool that stores task-specific knowledge and dynamically selects prompts from instance-wise input features without requiring task identity or boundaries.
- The prompt pool explicitly separates shared knowledge from task-specific knowledge, reducing interference while preserving model plasticity without requiring a rehearsal buffer.
- The work introduces prompting as a new perspective for continual learning and reports applicability to the challenging task-agnostic setting.
- L2P consistently outperforms prior state-of-the-art methods across class-, domain-, and task-agnostic continual-learning benchmarks, including without rehearsal buffers.
2. Related Work
Related continual-learning methods trade off plasticity, memory requirements, architectural complexity, and access to task identity. L2P is positioned against these approaches as a prompt-based alternative that avoids directly storing past data and targets settings where task identity is unknown.
- Continual learning: Regularization-based methods limit updates to important parameters and avoid storing past examples, but may perform poorly in challenging settings or complex datasets.
- Continual learning: Rehearsal-based methods store older samples in a buffer and achieve strong benchmark performance, but performance generally deteriorates as buffer size decreases.
- Continual learning: Rehearsal becomes inapplicable when data privacy prevents retaining past examples.
- Continual learning: Architecture-based methods use separate or selected task-specific components, but most require task identity at test time and therefore do not fit task-agnostic settings.
- Continual learning: CTN still requires test-time task identity, whereas DualNet requires a rehearsal buffer to work.
- Prompting for transfer learning: Prompt tuning and prefix tuning use learnable prompts with fewer additional parameters than methods such as Adapter and LoRA, but prompting was mainly designed for individual transfer-learning tasks.
3. Prerequisites
The paper frames continual learning as sequential prediction under non-stationary data and introduces prompt-based learning as conditioning for frozen sequence models. L2P extends this idea by selecting prompts dynamically rather than assigning one prompt per known task.
- Continual learning protocols: Continual learning trains models on non-stationary data from sequential tasks, with previous-task data potentially unavailable during future training.
- Continual learning protocols: Task-incremental learning assumes task identity is known at test time, unlike class- and task-agnostic settings where the identity may be unknown.
- Prompt-based learning: Prompt-based learning conditions pretrained models with task-specific prompt functions instead of traditional supervised fine-tuning.
- Prompt-based learning: Prompt tuning prepends learnable prompt parameters to input tokens while keeping the large pretrained model frozen.
- Prompt-based learning: For vision transformers, images become patch-token sequences, and independent prompts are prepended before the shared model processes downstream tasks.
- Prompt-based learning: Because independently trained prompts require task identity, L2P instead uses a prompt pool and dynamically queries suitable prompts for each input.
4. Learning to Prompt (L2P)
L2P uses a prompt pool and instance-wise key-value querying to select task-relevant prompts without requiring task identity or boundaries. The selected prompts are concatenated with input embeddings and optimized jointly with a frozen pre-trained model and classifier.
- 4.1. From prompt to prompt pool: The prompt pool stores composable prompts so similar inputs can share knowledge while dissimilar tasks retain more independent representations.This addresses the tension between knowledge sharing and catastrophic forgetting associated with a single shared prompt.
- 4.1. From prompt to prompt pool: The pool is P = {P1, P2, · · · ; PM}, where each prompt has token length Lp and embedding size D matching the input features.A selected subset of N prompts is concatenated with the embedded input, with 1 ≤ N ≤ M.
- 4.2. Instance-wise prompt query: Each input is matched against learnable prompt keys, and the query retrieves the top-N keys to determine which prompts are selected.The key-value design associates each prompt value with a learnable key and uses a matching score γ, for which cosine distance works well.
- 4.2. Instance-wise prompt query: Instance-wise prompt querying makes L2P task-agnostic by avoiding the need for task boundaries during training and task identity at test time.The query is produced from frozen pre-trained features and selects prompts specifically for each input.
- 4.2. Instance-wise prompt query: An optional training-time task-boundary prior penalizes frequently selected prompts to encourage more diverse prompt selection.This extension is intended especially for tasks with high diversity; the penalty is not used at test time.
- 4.3. Optimization objective for L2P: After prompt selection, the adapted embedding is processed by the remaining pre-trained model and classifier under an end-to-end training objective.The pre-trained model components are used in the optimization pipeline after the selected prompts are inserted.
- 4.3. Optimization objective for L2P: The objective combines softmax cross-entropy with a surrogate loss that pulls selected keys toward their corresponding query features.The scalar λ controls the weight of the key-query surrogate term.
- 4.3. Optimization objective for L2P: Prompt-location hidden vectors are average-pooled before the classification head to form the representation used for prediction.The pooled vectors correspond to the N ·Lp selected prompt locations in the transformed input.
5. Experiments
L2P is evaluated across class-, domain-, and task-agnostic continual-learning settings against baselines spanning fine-tuning, regularization, rehearsal, and architecture-based methods. It consistently performs strongly, including without rehearsal buffers, while ablations and prompt-selection analyses examine the mechanisms behind its results.
- Experimental setup: L2P is compared across class-incremental, domain-incremental, and task-agnostic settings using established continual-learning benchmarks and evaluation protocols.The experiments use Split CIFAR-100, 5-datasets, CORe50, and Gaussian scheduled CIFAR-100, with accuracy, forgetting, or final test accuracy reported according to setting.
- Baselines: The comparison includes naive fine-tuning, regularization-based, rehearsal-based, and architecture-based continual-learning methods, alongside L2P and L2P-R.L2P is evaluated without a rehearsal buffer, while L2P-R adds one for comparison with rehearsal-based methods.
- Main results: L2P consistently outperforms competing methods on class-incremental benchmarks in both average accuracy and forgetting.With smaller buffers, its margin increases; with no buffer, it remains stronger than regularization methods and outperforms almost all rehearsal methods when buffers are small.
- Main results: L2P outperforms architecture-based methods on Split CIFAR-100 when performance is measured by difference from the backbone-specific upper bound.The comparison uses Diff = Upper-Bound Acc - Method Acc, where lower values indicate closer performance to the upper bound.
- Main results: L2P achieves the best reported performance in both domain-incremental and task-agnostic experiments, including the task-agnostic setting with zero buffer.In task-agnostic learning, L2P ranks best among methods with and without rehearsal buffers; in domain-incremental learning, it remains ahead of the compared methods.
- Analysis: Ablations show that the prompt pool and learnable keys are important, while prompt selection adapts sharing to task similarity.Similar Split CIFAR-100 tasks largely share prompts, whereas diverse 5-datasets tasks use more task-specific prompts; prompt-pool and learnable-key removal both degrade performance.
6. Conclusion
L2P introduces prompt-based continual learning through a shared prompt pool, allowing one pre-trained model to adapt sequentially while mitigating catastrophic forgetting. The method achieves strong performance without rehearsal or task identity across several continual learning settings.
- L2P uses a shared prompt pool to adapt a single pre-trained model to sequential tasks.The framework introduces prompt-based learning for continual learning.
- L2P mitigates catastrophic forgetting without requiring rehearsal.
- L2P achieves strong performance on class-incremental and domain-incremental continual learning problems.
- L2P handles task-agnostic settings where previous methods are incapable.
A. Potential negative societal impact
The paper identifies risks associated with deploying L2P, including inherited bias from its pre-trained backbone and security concerns in safety-critical applications.
- L2P may carry bias and fairness issues from the pre-trained model used as its backbone.The paper recommends checking the pre-trained model for such issues.
- Deploying L2P in safety-critical systems such as autonomous driving may expose it to adversarial attacks.The paper recommends future robustness testing and corresponding defense techniques.
- The paper recommends testing L2P robustness and designing defenses against potential security concerns.
B. Limitations
The evaluation focuses on vision models and assumes pre-trained sequence-based models, while realistic task-agnostic continual learning remains insufficiently benchmarked.
- L2P is demonstrated on vision models, while exploration of other modalities remains future work.
- L2P assumes access to pre-trained sequence-based models.
- Generalizing L2P to other vision architectures such as ConvNets remains an open research direction.
- Gaussian scheduled CIFAR-100 is synthetic and remains far from realistic task-agnostic continual learning scenarios.The paper calls for more complex benchmarks to evaluate real-world capability.
C. Dataset details and licensing information
The experiments use class-incremental, domain-incremental, and task-agnostic image-classification benchmarks with differing sequential distribution shifts. The listed datasets have mixed licensing availability.
- Split CIFAR-100 divides CIFAR-100 into 10 class-incremental tasks with 10 disjoint classes per task.The tasks share similarities because they originate from one dataset.
- The 5-datasets benchmark sequences CIFAR-10, MNIST, Fashion-MNIST, SVHN, and notMNIST as diverse class-incremental tasks.Sequential training is challenging because the tasks are diverse.
- CORe50 is a domain-incremental object-recognition dataset containing 50 objects across 11 domains.Eight domains provide 120,000 training samples, while the remaining data form a 45,000-sample test set.
- Gaussian scheduled CIFAR-100 represents task-agnostic learning through gradual distribution shifts without explicit task boundaries.Methods must adapt without task-specific information during training or inference.
- CIFAR-10, CIFAR-100, and Fashion-MNIST are licensed under the MIT license.
- MNIST is licensed under Creative Commons Attribution-Share Alike 3.0, and CORe50 is licensed under Creative Commons Attribution 4.0.
- Licensing information is unavailable for SVHN and notMNIST.
D. Algorithm details
Algorithm 1 trains L2P sequentially across tasks by selecting prompts through query–key matching, prepending them to input embeddings, and updating the prompt system and classifier. During prediction, loss calculation is replaced by label prediction, and task-boundary information can optionally alter key lookup.
- Prediction: At prediction time, the procedure replaces loss calculation with label prediction and can optionally use equation 4 instead of top-N key lookup when task-boundary prior is known.The default prediction procedure does not require this optional task-boundary modification.
- Initialization: L2P initializes a pre-trained embedding layer, self-attention layers, classification layer, prompt pool, and prompt keys before sequential task training.The procedure specifies task count, per-task epochs, learning rate, and balancing parameter as inputs.
- Prompt selection: For each batch sample, the algorithm computes a query feature and retrieves the top-N keys by minimizing their matching scores.The selected keys determine which prompts are used for that sample.
- Prompted prediction: The selected prompts are prepended to the input embedding sequence before the model computes the per-sample loss.Prompt selection and embedding construction occur independently for each input sample.
- Batch loss: The algorithm accumulates per-sample losses into a batch loss while recording the selected keys and prompts for the batch.These batch-level sets are then used during the update stage.
- Parameter updates: It updates keys, prompts, and classifier parameters using gradients of the batch loss after processing the selected key–prompt pairs.The updates use learning rate η and apply separately to K, P, and φ.