PaperlayerView exact revision on arXiv

Source-linked AI summary

DualPrompt: Complementary Prompting for Rehearsal-free Continual Learning

Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, Tomas Pfister

arXiv:2204.04799v2cs.LGcs.CV

TL;DR

Continual learning needs to acquire sequential tasks without forgetting, yet strong rehearsal-based methods require buffers that raise privacy and memory concerns. DualPrompt learns complementary general and expert prompts attached to a pre-trained model, and achieves state-of-the-art rehearsal-free class-incremental performance while introducing Split ImageNet-R.

  • Problem

    Rehearsal buffers support continual learning but are problematic when privacy concerns or memory constraints prevent storing past data.

  • Method

    DualPrompt learns disjoint G-Prompt and E-Prompt spaces encoding task-invariant and task-specific instructions for a pre-trained backbone.

  • Results

    DualPrompt sets state-of-the-art performance across multiple benchmarks and surpasses rehearsal-based methods with relatively large buffers.

  • Takeaways & Limitations

    Split ImageNet-R provides a more challenging benchmark for evaluating rehearsal-free continual learning under high intra-class diversity.

  • Takeaways & Limitations

    Prompt-attaching configurations are selected empirically, while more advanced automatic configuration search remains future work.

Abstract

from arXiv · show

Continual learning aims to enable a single model to learn a sequence of tasks without catastrophic forgetting. Top-performing methods usually require a rehearsal buffer to store past pristine examples for experience replay, which, however, limits their practical value due to privacy and memory constraints. In this work, we present a simple yet effective framework, DualPrompt, which learns a tiny set of parameters, called prompts, to properly instruct a pre-trained model to learn tasks arriving sequentially without buffering past examples. DualPrompt presents a novel approach to attach complementary prompts to the pre-trained backbone, and then formulates the objective as learning task-invariant and task-specific "instructions". With extensive experimental validation, DualPrompt consistently sets state-of-the-art performance under the challenging class-incremental setting. In particular, DualPrompt outperforms recent advanced continual learning methods with relatively large buffer sizes. We also introduce a more challenging benchmark, Split ImageNet-R, to help generalize rehearsal-free continual learning research. Source code is available at https://github.com/google-research/l2p.

1 Introduction

Continual learning seeks to learn sequential tasks without catastrophic forgetting, but rehearsal buffers create privacy and memory constraints. DualPrompt addresses this with complementary prompts, achieving strong rehearsal-free class-incremental performance and introducing Split ImageNet-R.

  • Class-incremental learning is harder than task-incremental learning because task identity is unknown at test time.
  • Rehearsal-based methods are effective but become impractical under privacy concerns or tight memory budgets.
  • DualPrompt uses disjoint G-Prompt and E-Prompt spaces to encode task-invariant and task-specific instructions.
  • Split ImageNet-R has high intra-class diversity, making small buffers insufficient to represent past experiences.
  • 20% of the total training data in rehearsal buffers was needed by rehearsal-based methods to achieve competitive average accuracy, whereas DualPrompt used no rehearsal buffer.
  • Properly attaching prompts to the pre-trained backbone is reported as crucial to continual-learning effectiveness.
  • DualPrompt set state-of-the-art performance on multiple benchmarks and surpassed rehearsal-based methods with relatively large buffers.

2 Related work

Prior continual-learning methods use regularization, rehearsal, or architecture changes, but important limitations remain in challenging class-incremental settings. DualPrompt instead uses complementary prompts with a pre-trained backbone and consistently outperforms L2P.

  • Regularization-based methods mitigate forgetting in simpler task-incremental settings but perform inadequately in challenging class-incremental settings or datasets.
  • Architecture-based methods isolate task parameters, but many assume task identity, target convolutional architectures, or require model expansion or division.
  • Rehearsal-based methods achieve strong class-incremental performance, but deteriorate as buffer size decreases and are unsuitable for privacy-sensitive data.
  • DualPrompt avoids rehearsal buffers by using a pre-trained backbone and complementary prompts for task-invariant and task-specific instructions.
  • DualPrompt consistently outperforms L2P, whose single prompt pool does not distinguish common features from task-unique features.

3 Prerequisites

The paper studies rehearsal-free class-incremental learning with a pre-trained sequence model and unknown task identity at test time. Prompt tuning supplies lightweight instructions to a frozen transformer, while DualPrompt adds task-aware prompt selection.

  • 3.1 Continual learning problem setting: Continual learning trains one model on a sequence of tasks while previous-task data is unavailable during future-task training.
  • 3.1 Continual learning problem setting: The setting assumes clear task boundaries, sudden task switches, unknown task identity at test time, and an available pre-trained sequence model.
  • 3.1 Continual learning problem setting: Unlike rehearsal-based methods, the setup does not require a rehearsal buffer.
  • 3.2 Prompt-based learning: Prompt-based learning adds instructions to pre-trained models so they conditionally perform downstream tasks.
  • 3.2 Prompt-based learning: In vision transformers, prompt parameters are prepended to image embeddings while the pre-trained backbone remains frozen as a general feature extractor.
  • 3.2 Prompt-based learning: DualPrompt selects an expert prompt using a task key and attaches it with a shared general prompt to multiple transformer attention layers.

4 DualPrompt

DualPrompt attaches complementary general and expert prompts to selected layers of a pretrained backbone, using configurable prompting functions and joint optimization to learn task-invariant and task-specific instructions. Its design separates prompt positions and task selection while avoiding rehearsal buffers.

  • Prompt components: G-Prompt is shared across tasks, whereas each task has an E-Prompt and an associated learnable key.The key is updated with a matching loss and used at test time to select the best-matched task prompt.
  • Design choices: Prompt attachment positions and combination functions are treated as configurable design choices that are searched or evaluated empirically.The paper assumes contiguous selected MSA-layer indices, while more advanced automatic configuration search is left for future work.
  • Complementary prompts: DualPrompt uses disjoint G-Prompt and E-Prompt spaces to encode task-invariant and task-specific instructions, respectively.The two prompt types can be attached to different, potentially non-overlapping MSA-layer ranges.
  • Prompting functions: The method attaches prompts by modifying MSA-layer inputs, supporting prompt-tuning and prefix-tuning variants with different effects on query, key, and value representations.Prompt Tuning concatenates the prompt to all three inputs and increases output length, while Prefix Tuning adds separate prompt portions to keys and values and preserves output length.
  • Training objective: DualPrompt jointly trains the prompts, task keys, and classification head using classification loss plus a weighted matching loss.The scalar λ balances the matching loss against cross-entropy classification loss.

5 Experiments

Experiments on Split ImageNet-R and Split CIFAR-100 evaluate DualPrompt against prompt-based, rehearsal-based, regularization-based, and architecture-based methods. DualPrompt consistently performs strongly without buffered data, while complementary prompts and their placement support reduced forgetting.

  • Evaluation benchmarks: Experiments use Split ImageNet-R, Split CIFAR-100, and additional five-dataset evaluations, with Average accuracy and Forgetting as primary metrics.Split ImageNet-R contains 10 tasks of 20 classes; Split CIFAR-100 contains 10 tasks of 10 classes.
  • Comparison with state-of-the-arts: DualPrompt outperforms all compared methods consistently, including non-rehearsal methods and rehearsal-based methods with large buffers.The comparison includes regularization-, rehearsal-, prompt-, and architecture-based approaches.
  • Comparison with state-of-the-arts: 3%-7% margin on Average accuracy separates DualPrompt from L2P among non-rehearsal methods.The passage attributes this result to the complementary two-prompt design and reduced catastrophic forgetting.
  • Architecture-based comparison: 4.34% difference to upper-bound is achieved by DualPrompt on Split CIFAR-100, the best result among compared architecture-based methods.The metric measures the gap between method accuracy and the upper-bound accuracy of the used architecture.
  • Prompt placement: 2nd and 5th MSA layers are the empirically best attachment positions for G- and E-Prompts on Split ImageNet-R.The broader search found shallower placement for G-Prompt and deeper placement for E-Prompt.
  • Backbone analysis: Pre-trained backbones alone do not guarantee continual-learning performance, as general methods still show substantial forgetting and DynaER performs worse with a larger pretrained ResNet.The authors present effective use of large backbones as an open issue for traditional architecture-based methods.
  • Ablation study: G-Prompt captures task-invariant knowledge at shallower layers, whereas E-Prompt captures task-specific knowledge at deeper layers.Ablations report that combining both prompts improves performance by selectively decoupling these knowledge types, while multi-layer prompting adds representation power.
  • Prompt visualization: E-Prompts are well-separated in t-SNE visualization, while G-Prompts are relatively centered, matching their task-specific and shared roles.Each visualized prompt vector has dimension 768; E-Prompts come from the final model and G-Prompts from task-trained snapshots.

6 Conclusion

DualPrompt provides rehearsal-free continual learning in the challenging class-incremental setting by attaching complementary prompts to a pretrained model. The paper also introduces Split ImageNet-R and reports strong performance with lower additional memory than prior approaches.

  • Conclusion: DualPrompt achieves rehearsal-free continual learning when task identity is unknown at test time.The method attaches complementary prompts to a pretrained model to learn decoupled knowledge.
  • Conclusion: Split ImageNet-R is introduced as a new continual-learning benchmark alongside evaluations on widely used benchmarks.The benchmark is intended to support validation of rehearsal-free continual learning.
  • Conclusion: DualPrompt sets state-of-the-art performance across all reported metrics while requiring much lower additional memory than prior architecture-based and rehearsal-based methods.The conclusion presents simplicity, flexibility, and strong performance as reasons to use DualPrompt as a framework for future prompt-based research.

B Experimental details

The experiments use fixed optimization and preprocessing settings across benchmarks, search prompt locations on Split ImageNet-R, and align comparison protocols around pretrained ViT models. Training duration varies by benchmark to support convergence.

  • Optimization: 0.005 learning rate, Adam with β1 = 0.9 and β2 = 0.999, and batch size 128 are used for DualPrompt across benchmarks.The balancing factor is set to λ = 1 in equation 6.
  • Training schedule: 5 epochs per task are used for Split CIFAR-100 and five-datasets, while Split ImageNet-R uses 50 epochs per task.The authors state that these durations ensure convergence and disentangle forgetting from possible underfitting.
  • Prompt configuration: startg = 1, endg = 2, starte = 3, and ende = 5 are selected for prompt attachment across datasets.The settings are found to perform consistently well during validation-based search; prompt lengths are Lg = 5 and Le = 20.
  • Comparison protocol: All methods start from the same ImageNet-pretrained ViT-B/16, while DualPrompt and L2P keep the backbone frozen.Other methods use fully trainable pretrained models because the authors observed limited learning capacity with frozen backbones.

C Evaluation metrics

The paper evaluates continual learning with Average Accuracy and Forgetting after training on each task. Average Accuracy captures both learning capacity and catastrophic forgetting, whereas Forgetting specifically measures catastrophic forgetting.

  • Average Accuracy A_t averages evaluation scores across the t tasks after training on task t.S_t,τ denotes the evaluation score on task τ after training on task t.
  • Forgetting F_t averages each prior task’s maximum score decrease after training on task t.The calculation compares the current score with the maximum score previously attained for each earlier task.
  • Average Accuracy is the overall continual-learning metric because it reflects both learning capacity and reduced catastrophic forgetting.
  • Forgetting serves specifically as a measure of catastrophic forgetting rather than the overall evaluation metric.

D Details of comparing methods

The comparison covers regularization-, rehearsal-, prompt-, and architecture-based methods, using established baselines and implementation choices intended to support fair evaluation. Split ImageNet-R illustrates why rehearsal-based methods need substantial buffers in this setting.

  • EWC and LwF represent the regularization-based methods included in the comparison.
  • ER, GDumb, BiC, DER++, and Co2L represent the rehearsal-based comparison methods, with medium and large buffer sizes selected from prior recommendations.
  • L2P is treated as the current state-of-the-art prompt-based method, and DualPrompt uses a similar number of additional parameters for fair comparison.
  • Architecture-based comparison methods are based on ResNet-18, while reported results or original implementations and hyperparameters are used where applicable.
  • Images sharing a class label in Split ImageNet-R can differ substantially, supporting the finding that rehearsal-based methods require large buffers to perform well.

F Searching for multi-layered prompts

The appendix searches contiguous prompt placements and prompt lengths on Split ImageNet-R, selecting configurations that balance accuracy with parameter efficiency. The resulting placement combination is reported to work consistently across benchmark datasets, although the search is not exhaustive.

  • F Searching for multi-layered prompts: The multi-layered E-Prompt search assumes prompt-attached MSA layers are contiguous and includes cases with or without the best single-layer position.
  • F Searching for multi-layered prompts: The best multi-layered E-Prompt placement is starte = 3, ende = 5, while attaching it to every MSA layer gives comparable accuracy.
  • F Searching for multi-layered prompts: The 3-to-5 E-Prompt placement is chosen because it uses fewer additional parameters than attaching prompts to every MSA layer.
  • F Searching for multi-layered prompts: The multi-layered G-Prompt search fixes the E-Prompt placement and prefers including the second MSA layer.
  • F Searching for multi-layered prompts: The best G-Prompt placement is startg = 1, endg = 2, whereas sharing prompts across all MSA layers has a negative effect on accuracy.
  • F Searching for multi-layered prompts: The selected combination startg = 1, endg = 2, starte = 3, ende = 5 is reported to work well across benchmark datasets despite a non-exhaustive search.
  • G Searching for prompt length: Grid search over prompt lengths selects Lg = 5 and Le = 20 on Split ImageNet-R, and this combination is used for other datasets.

H Additional results on 5-datasets

The additional 5-datasets evaluation tests continual learning across five diverse image-classification datasets. DualPrompt is reported to consistently outperform competing methods, while the authors frame this setting as a proof of concept for large task diversity.

  • The 5-datasets benchmark contains CIFAR-10, MNIST, Fashion-MNIST, SVHN, and notMNIST.
  • The benchmark evaluates continual learning when task diversity is large, providing a more comprehensive evaluation setting.
  • A buffer size of 500 is already considered large for 5-datasets, although its tasks are easier than those in Split CIFAR-100 and ImageNet-R.
  • DualPrompt consistently outperforms competing methods on 5-datasets.
  • The authors present the 5-datasets results as a proof of concept because such extreme task diversity is considered uncommon in real-world continual learning.

I Relationship between query accuracy and performance

DualPrompt remains robust when query-based task selection mismatches the true task identity, achieving accuracy close to perfect matching despite imperfect query accuracy.

  • DualPrompt is compared using a query strategy against selection of E-Prompt with known test-time task identity.The comparison evaluates how query accuracy relates to performance.
  • Despite imperfect matching accuracy, DualPrompt remains robust to mismatching.
  • DualPrompt achieves accuracy very close to perfect matching even when query accuracy is not high.The passage attributes this robustness to the method’s design, including task-invariant instructions.
Loading 2204.04799v2…