PaperlayerView exact revision on arXiv

Source-linked AI summary

A Unified Continual Learning Framework with General Parameter-Efficient Tuning

Qiankun Gao, Chen Zhao, Yifan Sun, Teng Xi, Gang Zhang, Bernard Ghanem, Jian Zhang

arXiv:2303.10070v2cs.CV

TL;DR

The paper addresses the limited flexibility of prompt-only PET methods for continual learning after pre-training. It proposes LAE, which calibrates online PET learning, accumulates knowledge in an offline PET module, and ensembles both experts. LAE consistently outperforms prior state-of-the-art approaches across CIFAR100 and ImageNet-R.

  • Problem

    Existing PET-based continual-learning methods mainly use prompt tuning, limiting flexibility to Transformer architectures and relying on increasingly noisy task-specific selection.

  • Method

    LAE uses calibrated online PET learning, momentum-based accumulation into an offline PET module, and online/offline expert ensembling.

  • Results

    LAE consistently achieves superior incremental performance to previous state-of-the-art approaches on CIFAR100 and ImageNet-R.

  • Takeaways & Limitations

    LAE converts compatible PET methods into an efficient memory-free continual-learning approach without accessing old data.

Abstract

from arXiv · show

The "pre-training $\rightarrow$ downstream adaptation" presents both new opportunities and challenges for Continual Learning (CL). Although the recent state-of-the-art in CL is achieved through Parameter-Efficient-Tuning (PET) adaptation paradigm, only prompt has been explored, limiting its application to Transformers only. In this paper, we position prompting as one instantiation of PET, and propose a unified CL framework with general PET, dubbed as Learning-Accumulation-Ensemble (LAE). PET, e.g., using Adapter, LoRA, or Prefix, can adapt a pre-trained model to downstream tasks with fewer parameters and resources. Given a PET method, our LAE framework incorporates it for CL with three novel designs. 1) Learning: the pre-trained model adapts to the new task by tuning an online PET module, along with our adaptation speed calibration to align different PET modules, 2) Accumulation: the task-specific knowledge learned by the online PET module is accumulated into an offline PET module through momentum update, 3) Ensemble: During inference, we respectively construct two experts with online/offline PET modules (which are favored by the novel/historical tasks) for prediction ensemble. We show that LAE is compatible with a battery of PET methods and gains strong CL capability. For example, LAE with Adaptor PET surpasses the prior state-of-the-art by 1.3% and 3.6% in last-incremental accuracy on CIFAR100 and ImageNet-R datasets, respectively. Code is available at \url{https://github.com/gqk/LAE}.

1. Introduction

The paper targets continual learning after pre-training, where prompt-based PET methods limit flexibility and task-specific selection becomes noisy. LAE generalizes PET-based continual learning through calibrated learning, knowledge accumulation, and expert ensembling.

  • Continual Learning is needed for models that must incorporate new knowledge without forgetting previously learned information.
  • Existing PET-based continual-learning methods mainly use prompt tuning, restricting flexibility because prompts cooperate only with Transformer architectures.
  • LAE reshapes general PET methods into a unified continual-learning framework with Learning, Accumulation, and Ensemble designs.
  • Learning: Adaptation calibration aligns different PET modules by addressing their differing adaptation and forgetting speeds.
  • Accumulation and Ensemble: LAE accumulates task knowledge in one offline PET module and ensembles online and offline experts for newer and historical tasks.
  • Experiments: LAE consistently achieves superior incremental performance on CIFAR100 and ImageNet-R benchmarks.

2. Related Works

The paper situates LAE among parameter-efficient adaptation methods and memory-free continual learning, emphasizing compatibility with multiple PET modules.

  • Adapter-Tuning, Prompt-Tuning, Prefix-Tuning, and related methods adapt pre-trained models with additional parameters while preserving efficient inference.
  • LAE is framed as a unified framework for Memory-Free Continual Learning that incorporates various PET module types.

3. Preliminaries

The preliminaries define class-incremental learning and review PET modules that freeze the pre-trained model while tuning compact additions, including Adapter, LoRA, Prefix, and Prompt variants.

  • Continual Learning Formulation: Class-Incremental Learning sequentially adds tasks with non-overlapping categories while retaining previously learned categories.
  • Parameter-Efficient Tuning: PET freezes the pre-trained model and tunes a small number of additional learnable parameters called a PET module.
  • Adapter: Adapters add residual down- and up-projection modules that can be inserted into layers of pre-trained CNN or Transformer models.
  • LoRA: LoRA represents weight updates as the product of two small matrices and adds that low-rank update during the adapted layer's forward pass.
  • Prefix and Prompt: Prefix and Prompt methods use learnable tokens prepended to Transformer inputs or attention keys and values.
  • Scaled PET variants: The paper introduces learnable scale parameters for parallel Adapter and LoRA variants, recovering the original forms when the scale equals 1.

4. Methodology

LAE extends a frozen-model PET baseline into continual learning by calibrating adaptation speed, accumulating knowledge with an offline expert, and combining online and offline predictions.

  • Naive Baseline: The baseline sequentially trains one PET module for arriving tasks while keeping the pre-trained model frozen.
  • Naive Baseline: Local cross-entropy masks logits for old categories and empirically performs better than global cross-entropy with large pre-trained models.
  • Learning with calibrated speed: LAE calibrates PET adaptation speeds relative to classifiers to improve stability-plasticity balance across PET modules.
  • Learning with calibrated speed: Prefix adapts more slowly than Adapter because its attention contribution can produce smaller gradients, so LAE compensates and rescales Prefix updates.
  • Learning with calibrated speed: Calibrated Prefix reaches adaptation speed and performance nearly equivalent to Adapter, while the strategy can be extended to other PET modules.
  • Accumulation: An offline PET module accumulates online knowledge through exponential moving average, retaining historical information while the online module learns new tasks.
  • Ensemble: Because task identity is unknown at inference, LAE ensembles experts built from online and offline PET modules to combine newer- and older-task strengths.
  • Scope: LAE can be applied to any pre-trained network architecture whose PET modules are compatible with that model.

5. Experiment

Experiments evaluate LAE across CIFAR100 and ImageNet-R using standardized continual-learning protocols, benchmark comparisons, ablations, and module-placement studies. LAE consistently improves incremental performance across PET modules, datasets, inference strategies, and model architectures.

  • Datasets and evaluation: Experiments use ImageNet21k-pretrained models on CIFAR100 and ImageNet-R, split into 10 tasks with mean and standard deviation reported over three class orders.Evaluation uses last incremental accuracy and average incremental accuracy.
  • Benchmark results: LAE with six PET modules surpasses DualPrompt and ESN by about 1.5% in CIFAR100 last incremental accuracy A10.All approaches use the same ImageNet21k-pretrained ViT-B/16.
  • Benchmark results: More than 3.5% improvement over DualPrompt in ImageNet-R last incremental accuracy A10 demonstrates the benefit of adaptation-speed calibration on the harder benchmark.The calibration addresses faster adaptation by Adapter and LoRA than Prefix.
  • Benchmark results: LAE with all three PET types outperforms L2P and DualPrompt at almost all learning phases, with a wider margin in 20-task experiments.The task-by-task results assess performance throughout continual learning rather than only at the final task.
  • Ablation study: Removing calibrated-speed learning causes the largest ablation drop, while removing accumulation and ensemble lowers last incremental accuracy by 2.15%.The ablation uses ImageNet-R with Adapter10.
  • Ablation study: Expert-ensemble inference is more robust across phases, although offline-only inference can outperform it as historical-task performance dominates later.The comparison evaluates online, offline, and ensemble inference strategies.
  • Ablation study: Gradient compensation and learnable scaling each improve Prefix calibration, and their combined gain is approximately additive.The ablation is conducted on ImageNet-R with Prefix20.
  • Module placement: Inserting PET modules into shallow blocks performs better than deeper placement, while six initial Transformer blocks are best and five are nearly equivalent.LAE’s default attaches modules to the first five Transformer blocks.

6. Conclusion

The paper proposes LAE to adapt pre-trained models continually with general PET methods through three designs, and reports improved performance without accessing old data. It also identifies efficiency, ensemble quality, and larger-task validation as future limitations.

  • 6. Conclusion: LAE converts PET methods into Memory-Free Continual Learning through learning, accumulation, and ensemble designs.The framework continuously adapts a pre-trained model without accessing old data.
  • 6. Conclusion: LAE significantly outperforms previous state-of-the-art approaches in the reported experiments.
  • 6. Conclusion: Future work includes more efficient knowledge accumulation and improved expert-model ensembling.
  • 6. Conclusion: LAE has not been verified with larger numbers of tasks because suitable large-scale datasets without pre-training overlap are lacking.

Supplementary Materials

The supplementary materials provide additional experimental details, results, and an investigation of prompt learning and prompt-pool selection.

  • Supplementary Materials: Section A provides additional experimental details.
  • Supplementary Materials: Section B provides additional experimental results.
  • Supplementary Materials: Section C investigates prompt learning and selection from pools in prompt-pool-based approaches.

A. Additional Experimental Details

The additional experimental details describe data augmentation, evaluation protocol, and the 20-task benchmark configurations and comparisons.

  • A. Additional Experimental Details: Training uses random resizing, normalization, and horizontal flipping, while inference uses resizing followed by a central crop.The procedures are described as consistent with the compared approaches for fair comparisons.
  • A. Additional Experimental Details: The incremental accuracy equation averages task accuracies over the total number of tasks.
  • A. Additional Experimental Details: The evaluation includes three different class orders, while prior L2P, DualPrompt, and ESN reports used one class order.
  • A. Additional Experimental Details: Tables VII and VIII report 20-task results for CIFAR100 and ImageNet-R using PET modules inserted into the first five ViT-B/16 transformer blocks.

B. Additional Experimental Results

The additional results examine longer-term continual learning, comparisons with CODA-Prompt, EMA sensitivity, computational cost, and parameter counts. They report stronger long-term performance and practical efficiency, while identifying evaluation-setting differences in comparisons.

  • B. Additional Experimental Results: LAE is evaluated on 20-task CIFAR100 and ImageNet-R benchmarks using mean and standard deviation across three runs and class orders.
  • B. Additional Experimental Results: LAE is described as mitigating forgetting and improving the stability-plasticity balance in longer-term continual learning.
  • B. Additional Experimental Results: The CODA-Prompt comparison addresses differences in data splits, pre-training, fine-tuning, and training strategies by adopting CODA-Prompt’s settings.
  • B. Additional Experimental Results: EMA weight decay controls the trade-off between absorbing unstable new knowledge and absorbing new knowledge effectively.The paper uses α = 0.9999 in its experiments.
  • B. Additional Experimental Results: LAE requires two forward passes per inference sample, has computational costs on par with L2P, DualPrompt, and CODA-Prompt, and maintains a constant parameter count across tasks.
  • B. Additional Experimental Results: LAE’s task-by-task results are compared with L2P, DualPrompt, ENS, and different PET modules in two 20-task benchmarks.

C. Prompt Learning and Selection from Pool

Prompt-pool continual learning depends on both learning effective task prompts and selecting the right prompts at inference. DualPrompt’s selection accuracy deteriorates as tasks accumulate, while LAE’s calibrated PET modules address limitations observed with Prompt/Prefix tuning.

  • Prompt-pool approaches: L2P shares prompts through a pool, whereas DualPrompt learns task-specific E-Prompts and selects matched prompts during inference.These methods represent prompt-tuning approaches for continual learning.
  • Prompt-pool challenges: Prompt-pool performance depends on learning task knowledge while retaining prior knowledge and accurately selecting the best-matched prompts.The selection ability is described as especially critical because learned prompts are useful only when the appropriate prompts are selected for each inference sample.
  • Prompt selection: Below 50%: DualPrompt’s E-Prompt selection accuracy by the 10th learned task, after being completely accurate on the first task.The reported trend shows gradually worsening selection as the number of learned tasks increases.
  • PET comparison: LAE outperforms DualPrompt on the first task with Adapter, LoRA, or Prefix, despite using fewer, equivalent, or slightly more parameters, respectively.The comparison is reported on the two evaluated datasets and suggests Prompt/Prefix is less effective than Adapter and LoRA for learning new knowledge there.
  • PET calibration: Calibrating Prefix modules is necessary because DualPrompt may fail to learn optimal first-task E-Prompts without this calibration.The authors use this observation to motivate exploring and calibrating different Parameter-Efficient Tuning methods.
  • Task-specific prompt sets: DualPrompt’s 2nd–10th E-Prompt sets produce very small differences in last and average incremental accuracy when evaluated across all tasks.The evaluation compares task-specific prompt sets and reports similar performance despite the baseline using one Prefix set.
Loading 2303.10070v2…