PaperlayerView exact revision on arXiv

Source-linked AI summary

Hierarchical Decomposition of Prompt-Based Continual Learning: Rethinking Obscured Sub-optimality

Liyuan Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, Jun Zhu

arXiv:2310.07234v1cs.LG

TL;DR

Prompt-based continual learning performs strongly with supervised pre-training, but its effectiveness under realistic self-supervised pre-training is limited by difficulties in encoding and inferring task-specific knowledge. The paper decomposes this objective into hierarchical components and proposes HiDe-Prompt, which achieves leading performance across continual learning benchmarks and pre-training paradigms, including leads of up to 15.01% on Split CIFAR-100 and 9.61% on Split ImageNet-R.

  • Problem

    Supervised-pretraining gains from prompt-based continual learning do not reliably transfer to self-supervised pre-training, where task-specific knowledge is harder to encode and infer.

  • Method

    HiDe-Prompt decomposes the continual learning objective into within-task prediction, task-identity inference, and task-adaptive prediction, then optimizes them using task-specific prompts, representation statistics, and contrastive regularization.

  • Results

    HiDe-Prompt achieves superior performance across pre-training paradigms, with up to 15.01% and 9.61% leads on Split CIFAR-100 and Split ImageNet-R, respectively.

  • Takeaways & Limitations

    The approach provides a framework for explicitly optimizing hierarchical continual learning components across varied pre-training paradigms.

  • Takeaways & Limitations

    The method assumes adequate pre-training representations, mainly targets transformer backbones, and freezes the backbone so downstream adaptation cannot enrich or update pre-trained knowledge.

Abstract

from arXiv · show

Prompt-based continual learning is an emerging direction in leveraging pre-trained knowledge for downstream continual learning, and has almost reached the performance pinnacle under supervised pre-training. However, our empirical research reveals that the current strategies fall short of their full potential under the more realistic self-supervised pre-training, which is essential for handling vast quantities of unlabeled data in practice. This is largely due to the difficulty of task-specific knowledge being incorporated into instructed representations via prompt parameters and predicted by uninstructed representations at test time. To overcome the exposed sub-optimality, we conduct a theoretical analysis of the continual learning objective in the context of pre-training, and decompose it into hierarchical components: within-task prediction, task-identity inference, and task-adaptive prediction. Following these empirical and theoretical insights, we propose Hierarchical Decomposition (HiDe-)Prompt, an innovative approach that explicitly optimizes the hierarchical components with an ensemble of task-specific prompts and statistics of both uninstructed and instructed representations, further with the coordination of a contrastive regularization strategy. Our extensive experiments demonstrate the superior performance of HiDe-Prompt and its robustness to pre-training paradigms in continual learning (e.g., up to 15.01% and 9.61% lead on Split CIFAR-100 and Split ImageNet-R, respectively). Our code is available at \url{https://github.com/thu-ml/HiDe-Prompt}.

1 Introduction

Prompt-based continual learning leverages frozen pre-trained transformers and task-adaptive prompts, but its strong supervised-pretraining results do not transfer reliably to realistic self-supervised pre-training. HiDe-Prompt addresses this gap by decomposing the objective into hierarchical components and explicitly optimizing them.

  • Motivation: Prompt-based continual learning freezes a pre-trained transformer backbone and uses task-adaptive prompts to steer representation learning.These methods seek to accommodate both task-sharing and task-specific knowledge while reducing catastrophic forgetting.
  • Problem: Self-supervised pre-training exposes unresolved weaknesses in current prompt-based continual learning.Task-specific knowledge is difficult to encode in instructed representations and infer from uninstructed representations at test time.
  • Approach: The continual learning objective decomposes into within-task prediction, task-identity inference, and task-adaptive prediction.This decomposition provides the theoretical basis for diagnosing and addressing sub-optimality.
  • Approach: HiDe-Prompt explicitly optimizes the hierarchical components using task-specific prompt ensembles, representation statistics, and contrastive regularization.The method is designed particularly for the challenges observed under self-supervised pre-training.
  • Results: Across continual learning benchmarks and pre-training paradigms, HiDe-Prompt achieves state-of-the-art performance without rehearsal.The paper presents this as a broad empirical contribution rather than a result limited to one benchmark.

2 Related Work

Continual learning combines strategies such as stabilization, replay, optimization control, distributed representations, and task-specific parameters to address catastrophic forgetting. Prompt-based methods adapt pre-trained knowledge efficiently, but their advantage under supervised pre-training remains uncertain under self-supervised pre-training.

  • Continual Learning: Continual learning strategies address catastrophic forgetting through parameter stabilization, replay, optimization control, distributed representations, and task-specific parameters.Their performance varies across continual learning settings.
  • Self-Supervised Learning and Pre-Training: Self-supervised learning is widely used in pre-training because explicit labels are scarce and expensive in many real-world applications.Instance discrimination with contrastive learning is described as a dominant strategy.
  • Prompt-Based Approach: Prompt-based continual learning constructs and infers adaptive prompts to leverage pre-trained knowledge for downstream tasks.Earlier methods show strong performance under supervised pre-training.
  • Prompt-Based Approach: Whether prompt-based advantages persist under realistic self-supervised pre-training remains an open question.Related evidence also indicates that self-supervised pre-training challenges approaches requiring backbone fine-tuning.

3 Preliminary Analysis

Prompt-based continual learning uses frozen vision-transformer backbones with prompts that incorporate task knowledge and infer suitable prompts at test time. Empirical analysis shows that approaches performing strongly with supervised pre-training degrade under self-supervised pre-training, where representation specificity and task-identity inference become weaker.

  • 3.1 Formulation of Prompt-Based Continual Learning: Continual learning trains sequentially on task datasets and evaluates on corresponding test samples, with task identity unavailable in domain- and class-incremental settings.The formulation uses a pre-trained backbone and output layer to predict labels for samples from previous tasks.
  • 3.1 Formulation of Prompt-Based Continual Learning: Prompt-based vision methods prepend a small prompt to representations in a frozen pre-trained vision transformer.The prompt adapts generalizable backbone representations without updating backbone parameters.
  • 3.1 Formulation of Prompt-Based Continual Learning: Prompt Tuning prepends prompts to query, key, and value representations, whereas Prefix Tuning inserts separate key and value prefixes.Prompt Tuning increases output sequence length, while Prefix Tuning preserves the input representation dimension.
  • 3.1 Formulation of Prompt-Based Continual Learning: Existing approaches construct task-adaptive prompts and infer suitable prompts or prompt combinations at test time.L2P uses a prompt pool and selects prompts through key-query similarity, while DualPrompt and S-Prompt use task-specific structures.
  • 3.2 Empirical Study of Pre-Training Paradigms: Under Sup-21K, task-specific prompts outperform task-sharing prompts, but all four baselines degrade substantially under self-supervised pre-training.The performance differences between prompt architectures also become smaller in the self-supervised setting.
  • 3.2 Empirical Study of Pre-Training Paradigms: Self-supervised pre-training produces higher similarity between uninstructed and instructed representations and lower task-identity prediction accuracy.Less than 40% of task identities are correctly predicted in the reported analysis, while single-head prediction can compensate for some errors.

4 Theoretical Foundation and Our Approach

The paper decomposes continual-learning performance into within-task prediction, task-identity inference, and task-adaptive prediction, then designs HiDe-Prompt to optimize these components explicitly using prompts and representation statistics.

  • Hierarchical Decomposition: The continual-learning objective is decomposed into task-identity inference and within-task prediction, with task-adaptive prediction added as a separate objective over all observed classes.
  • Theoretical Foundation: Theorem 1 bounds loss error by max{δ + ϵ, η} when WTP, TII, and TAP cross-entropies are bounded by δ, ϵ, and η.
  • Theoretical Foundation: Theorem 2 states that low continual-learning loss implies the existence of WTP, TII, and TAP models whose individual loss errors are no greater than that loss.
  • HiDe-Prompt: HiDe-Prompt models uninstructed and instructed representations with class-conditional distributions, then uses auxiliary layers for TII and TAP with pseudo-representations sampled from those distributions.
  • HiDe-Prompt: HiDe-Prompt improves WTP with an expandable task-specific prompt pool and prompt ensemble, while contrastive regularization reduces overlap among instructed representations.

5 Experiment

Experiments compare prompt-based continual-learning methods across benchmarks and pre-training paradigms, showing that HiDe-Prompt generally achieves the strongest accuracy and forgetting results, especially under self-supervised pre-training.

  • Overall Performance: HiDe-Prompt generally achieves the highest FAA and CAA and the lowest FFM across evaluated pre-training paradigms and benchmarks.
  • Overall Performance: 15.01% and 9.61% are the reported maximum leads on Split CIFAR-100 and Split ImageNet-R under self-supervised pre-training.
  • Extended Benchmarks: 12.63% and 30.44% are the reported maximum leads on 5-Datasets and Split CUB-200 when inter-task differences and fine-grained classification are evaluated.
  • Efficiency: HiDe-Prompt uses 16141MB on Split CIFAR-100 compared with 25325MB for CODA-Prompt, while maintaining comparable computation cost to most baselines.
  • Ablation Study: Ablations show that WTP, TII, TAP, and contrastive regularization each contribute to performance, with component improvements generally larger under self-supervised pre-training.

6 Discussion and Conclusion

The discussion frames continual learning as a hierarchical objective and presents a general decomposition-based approach, while acknowledging assumptions that limit its scope.

  • The framework uses uninstructed and instructed representation statistics to explicitly optimize the hierarchical components.
  • The approach is presented as a general framework for parameter-efficient fine-tuning methods including prompts, adapters, LoRA, and FiLM.
  • The method assumes adequate pre-training, is mainly applicable to transformer backbones, and freezes the backbone during prompt-based adaptation.
  • The continual learning objective is decomposed into within-task prediction, task-identity inference, and task-adaptive prediction.
  • For domain-incremental learning, the loss error is bounded by max{δ + ϵ + log t, η} when the three hierarchical component losses satisfy their stated bounds.

Proof of Theorem 3

Theorem 3 characterizes domain-incremental learning through separate hierarchical error terms and bounds the overall loss using their combined contributions.

  • Improving within-task prediction or task-identity inference alone is insufficient because the objective also directly requires task-adaptive prediction.
  • The final domain-incremental objective jointly maximizes class-domain prediction and task-adaptive prediction.
  • Task-identity inference is distributed over observed domains, whereas task-adaptive prediction is distributed over observed classes.
  • Theorem 3 bounds the loss error by max{δ + ϵ + log t, η} when the expected WTP, TII, and TAP losses meet their respective limits.

A.3 Task-Incremental Learning (TIL)

In task-incremental learning, the task identity is provided at test time, so task-identity inference disappears and task-adaptive prediction reduces to within-task prediction.

  • Task-incremental learning predicts within-task classes conditioned on the provided task identity.
  • Theorem 5 states that expected task-identity inference loss is zero and task-adaptive prediction degrades into within-task prediction.
  • If expected within-task prediction loss is at most δ, the task-incremental loss error lies in [0, δ].

Proof of Theorem 5

Theorem 5’s proof applies the supplied task identity to reduce the task-incremental objective to within-task prediction alone.

  • Task-incremental prediction conditions on the known task identity and predicts the corresponding within-task class.
  • The loss decomposes into within-task prediction and task-identity inference, with the latter equal to zero when the task identity is provided.
  • The resulting task-incremental loss error is bounded by δ when the expected within-task prediction loss is at most δ.
  • Conversely, if the task-incremental loss error is at most ξ, there exists a within-task predictor whose loss is at most ξ.

B Impact of Pre-Training on Continual Learning

The paper argues that pre-training changes the continual-learning objective: task-adaptive prediction becomes an additional requirement beyond within-task prediction and task-identity inference. Its analysis distinguishes uninstructed and instructed representation spaces, explaining why the two prediction routes differ under pre-training.

  • Theoretical formulation: The paper formulates the hierarchical components as θ-conditional probabilities, with θ representing pre-trained knowledge in the initialization.The components are defined for within-task prediction, task-identity inference, and task-adaptive prediction.
  • Theoretical distinction: Pre-trained parameters contain useful downstream knowledge, so they should typically be stabilized or frozen and included in the continual-learning objective.This differs from training from scratch, where randomly initialized parameters must be substantially changed.
  • Theoretical distinction: Pre-trained continual learning requires task-adaptive prediction in addition to within-task prediction and task-identity inference.Without accounting for pre-training, task-adaptive prediction reduces to the product of task-identity inference and within-task prediction.
  • Representation spaces: Prompt-based continual learning separates task-adaptive prediction from task-identity inference times within-task prediction because prompts alter the semantic space of representations.Task-adaptive prediction uses instructed representations, whereas the other product uses uninstructed representations.

C Implementation Details

The experiments use established prompt architectures and a common pre-trained ViT-B/16 training setup, while evaluating continual learning with final accuracy, historical accuracy, and forgetting metrics.

  • Prompt architectures: Baselines follow their original prompt architectures, including specified prompt counts, lengths, and Top-N key settings.L2P uses M=30 prompts, prompt length 5, and N=5 Top-N keys; DualPrompt uses task-sharing prompts of length 5.
  • Training regime: Training uses a pre-trained ViT-B/16 backbone, Adam optimization, batch size 128, and cosine-decayed learning rates.CODA-Prompt uses learning rate 0.001, while other approaches use 0.005 before epoch selection across pre-training paradigms.
  • Evaluation: The study evaluates final average accuracy, cumulative average accuracy, and final forgetting measure.FAA measures final performance, CAA reflects historical performance, and FFM measures catastrophic forgetting.
  • Compute: Experiments run on multiple GPU configurations for Split CIFAR-100, Split ImageNet-R, 5-Datasets, and Split CUB-200.The hardware includes Tesla P100-SXM2, NVIDIA A100, and NVIDIA GeForce RTX 3090 GPUs.

D Extended Results

Extended experiments examine baseline reproduction, learning-rate sensitivity, statistical modeling, and representation visualizations across datasets and pre-training paradigms. They report that statistical modeling remains effective with simplified representations, while benchmark reproduction and implementation differences require care.

  • Reproduction caveats: Baseline comparisons are affected by checkpoint, data-split, implementation, and reproduction differences.CODA-Prompt’s Split ImageNet-R results could not be reproduced exactly, while DualPrompt’s 5-Datasets implementation remains unresolved.
  • Baseline sensitivity: CODA-Prompt performance depends heavily on reduced learning rates, especially on Split ImageNet-R.The extended evaluation compares learning-rate choices using FAA.
  • Statistical modeling: 91.74% FAA on Split CIFAR-100 and 73.97% FAA on Split ImageNet-R show that reducing covariance to variance remains essentially consistent under Sup-21K.The result comes from the statistical-modeling ablation under an adequately strong supervised pre-trained model.
  • Statistical modeling: Fewer than 10 centroids per class, averaging around 5, achieve comparably strong performance while improving efficiency and generality.The centroids are selected with KNN, with a maximum of 10 per class.
  • Representation visualization: t-SNE visualizations compare uninstructed and instructed representations for Sup-21K and iBOT-21K, with colors denoting classes.The visualizations support examining representation compactness and differentiation across pre-training paradigms.
  • Additional benchmark: Table 8 reports overall continual-learning performance on 5-Datasets, selected for its large inter-task differences.The extended evaluation keeps prompt-architecture implementations consistent with those used on Split CIFAR-100 and Split ImageNet-R.
Loading 2310.07234v1…