PaperlayerView exact revision on arXiv

Source-linked AI summary

On Transferability of Prompt Tuning for Natural Language Processing

Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, Jie Zhou

arXiv:2111.06719v2cs.CL

TL;DR

Prompt tuning uses few trainable parameters but takes longer to train than fine-tuning. This paper empirically studies prompt transfer across tasks and models, finding that transfer can accelerate PT and improve performance, while prompt stimulation of PLMs is associated with transferability.

  • Problem

    Prompt tuning can use few trainable parameters, but requires much more training time than fine-tuning, motivating study of ways to improve its efficiency.

    Source: §1 · pp. 1–2

  • Method

    The paper empirically analyzes prompt transfer across tasks and models and uses transferred or projected prompts to initialize prompt tuning.

    Source: §1 · p. 2

  • Results

    Prompts transfer effectively to similar tasks, and transferred or projected prompts used as initialization can accelerate PT and improve performance; activation overlap reflects transferability better than embedding similarities.

    Source: §1 · p. 2, §6.2 · p. 8, §7 · p. 9

  • Takeaways & Limitations

    The findings suggest prompt transfer is promising for improving PT and that transfer research should attend to how prompts stimulate PLMs.

    Source: §1 · p. 2, §7 · p. 9

Abstract

from arXiv · show

Prompt tuning (PT) is a promising parameter-efficient method to utilize extremely large pre-trained language models (PLMs), which can achieve comparable performance to full-parameter fine-tuning by only tuning a few soft prompts. However, PT requires much more training time than fine-tuning. Intuitively, knowledge transfer can help to improve the efficiency. To explore whether we can improve PT via prompt transfer, we empirically investigate the transferability of soft prompts across different downstream tasks and PLMs in this work. We find that (1) in zero-shot setting, trained soft prompts can effectively transfer to similar tasks on the same PLM and also to other PLMs with a cross-model projector trained on similar tasks; (2) when used as initialization, trained soft prompts of similar tasks and projected prompts of other PLMs can significantly accelerate training and also improve the performance of PT. Moreover, to explore what decides prompt transferability, we investigate various transferability indicators and find that the overlapping rate of activated neurons strongly reflects the transferability, which suggests how the prompts stimulate PLMs is essential. Our findings show that prompt transfer is promising for improving PT, and further research shall focus more on prompts' stimulation to PLMs. The source code can be obtained from https://github.com/thunlp/Prompt-Transferability.

1 Introduction

Prompt tuning can match fine-tuning performance with few trainable parameters, but takes longer to converge. This paper studies prompt transfer across tasks and models as a way to improve PT’s efficiency and effectiveness, and examines what signals transferability.

1 Introduction

  • PT takes much more training time than fine-tuning, motivating efforts to improve its efficiency.The paper’s prompt-transfer approach builds on the idea that prompts concentrate task knowledge because they are PT’s only tuned parameters.Source: §1 · pp. 1–2
  • Trained prompts transfer well to similar tasks, while initialization from transferred prompts can accelerate PT and improve its performance.Cross-model transfer uses projectors to map prompts into another PLM’s semantic space.Source: §1 · p. 2
  • The study examines prompt transfer across 17 NLP tasks and two PLM series, using direct reuse across tasks and projectors across models.Cross-model direct reuse is intractable because different PLMs have inconsistent semantic spaces.Source: §1 · p. 2
  • Activation-based prompt similarity correlates with transferability better than embedding-distance metrics, suggesting that how prompts stimulate PLMs matters.The authors suggest future work focus on PLM responses to prompt stimulation rather than prompt embedding properties.Source: §1 · p. 2

2 Related Work

Related work establishes prompt tuning as a parameter-efficient approach, while highlighting its convergence-time cost and the limited exploration of transferability for PT. This paper extends prior cross-task work by emphasizing efficiency, additional indicators, and cross-model transfer.

2 Related Work

  • Soft prompts are trainable embeddings, and PT can match full-parameter fine-tuning on PLMs with billions of parameters, but takes longer to converge.The cited result is from Lester et al. (2021).Source: §2 · p. 3
  • Earlier knowledge-transfer studies largely target fine-tuning, leaving PT’s transferability and transfer methods worth exploring.The passage cites intermediate-task transfer work including Phang et al. (2018), Pruksachatkun et al. (2020), Gururangan et al. (2020), Wang et al. (2019a), Vu et al. (2020), and Poth et al. (2021).Source: §2 · p. 3
  • Compared with Vu et al. (2021), this work emphasizes PT efficiency, investigates more transferability indicators, and includes cross-model transfer.Vu et al. (2021) explore cross-task prompt initialization and cosine-similarity metrics, focusing on PT effectiveness.Source: §2 · p. 3

3 Preliminary

The paper studies prompt tuning with PLM parameters frozen, across diverse downstream tasks and representative PLM families. The preliminary section specifies the tuning setup, task coverage, and model scope.

3 Preliminary

  • The preliminary section introduces the prompt-tuning setup and identifies the downstream tasks and models used in the experiments.The section’s stated scope is PT, downstream tasks, and investigated models.Source: §3 · p. 3

3.1 Prompt Tuning

  • PT prepends virtual soft prompts to inputs and trains them while freezing the PLM parameters to generate desired downstream-task outputs.The objective maximizes the likelihood of the desired output; only the prompts are learnable.Source: §3.1 · p. 3

3.2 Investigated NLP Tasks

  • The evaluation covers 17 NLP tasks grouped into six types, including sentiment analysis, inference, ethical judgment, paraphrase identification, question answering, and summarization.The task groups include multiple datasets; implementation and metric details are provided in the appendix.Source: §3.2 · pp. 3–4

3.3 Investigated Models

  • Experiments use RoBERTa and T5, representing masked-language-modeling and sequence-to-sequence pretraining, and mainly report their largest versions.RoBERTa is not used for multi-token QA and summarization outputs; those tasks are studied with T5.Source: §3.3 · p. 4

4 Cross-Task Transfer

Cross-task prompt transfer is strongest among tasks of the same type, while transfer with initialization generally reduces training time and preserves or improves performance. Some cross-type transfers also work to an extent.

4.1 Zero-shot Transfer Performance

  • Directly reused prompts generally transfer well between same-type tasks, but poorly between different types, where performance often resembles random initialization.Some cross-type transfers, such as QA and summarization prompts to sentiment analysis, work to some extent.Source: §4.1 · p. 4
  • The section studies zero-shot performance by training prompts on a source task, reusing them directly on target tasks, and evaluating the targets.The paper frames this as an empirical study of cross-task prompt transferability.Source: §4.1 · p. 4, §4 · p. 4

4.2 Transfer with Initialization

  • TPTTASK initializes prompts with well-trained prompts from the most similar task before starting PT.The cross-task method is intended to improve PT’s effectiveness and efficiency.Source: §4.2 · p. 4
  • TPTTASK generally takes less training time and mostly matches or exceeds vanilla PT’s performance.It initializes the target task with prompts from the source task that achieved the best zero-shot transfer performance.Source: §4.2 · p. 4

5 Cross-Model Transfer

The paper investigates cross-model prompt transfer from RoBERTaLARGE to T5XXL, where direct prompt reuse is infeasible because the PLMs have different embedding spaces. It evaluates learned projectors for zero-shot transfer and as prompt-tuning initializations.

5 Cross-Model Transfer

  • Directly reusing prompts across PLMs is infeasible because their embedding spaces differ, motivating projection into the target PLM’s semantic space.The experiments focus on the difficult RoBERTaLARGE-to-T5XXL setting; other settings are reported in the appendix.Source: §5 · p. 4, §5.1 · p. 5

5.1 Cross-Model Prompt Projection

  • Projectors are trained either to minimize distance from a parallel prompt or by tuning projected prompts with task-specific supervision on the target PLM.The projector is trained on selected tasks and is intended to generalize to other tasks.Source: §5.1 · pp. 4–6

5.2 Zero-shot Transfer Performance

  • Task Tuning projectors generalize to unseen tasks of the same type, but not different-type tasks; Distance Minimizing falls back to random performance on unseen tasks.The paper links Distance Minimizing’s limitation to embedding distances’ weak correlation with transferability and calls for universal cross-model projections.Source: §5.2 · p. 6

5.3 Transfer with Initialization

  • For tasks matching the projector-training task type, TPTMODEL mostly matches or exceeds vanilla PT performance with much less training time.Table 2 reports cross-model results from RoBERTaLARGE to T5XXL.Source: §5.3 · p. 6, Table 2 · p. 6

6 Exploring Transferability Indicator

The section compares prompt-embedding and model-stimulation similarity metrics as indicators of transferability. It finds that activated-neuron overlap tracks cross-task zero-shot transfer better than embedding similarities, while its correlation weakens as T5 grows.

6.1 Prompt Similarity Metric

  • Embedding metrics compare concatenated prompt-token embeddings or average pairwise token similarities, including a position-invariant variant.The metrics include Euclidean and cosine similarities.Source: §6.1 · p. 7
  • Model-stimulation similarity measures overlap in activated feed-forward neurons; prompts are run without task input sentences, using output-relevant activation states from selected layers.The experiments use the top three layers; T5 uses the decoder’s first position and RoBERTa uses [MASK].Source: §6.1 · p. 7

6.2 Experimental Results

  • The section evaluates whether prompt similarities distinguish same- and different-task prompts and whether they align with cross-task zero-shot transfer.Transfer alignment is assessed by ranking source prompts using similarity and zero-shot performance, then computing Spearman rank correlation.Source: §6.2 · p. 8
  • All five similarity metrics distinguish same-task prompts from different-task prompts, suggesting task-specific prompt clusters and distinct PLM activations.Same-task pairs use prompts trained with different random seeds.Source: Table 3 · p. 7, §6.2 · p. 8
  • Activated-neuron overlap (ON) aligns better with cross-task zero-shot transfer than embedding similarities, but performs worse on T5XXL than RoBERTaLARGE.The authors report ON correlation scores of 36.9% on T5XXL, rising to 46.3% when activation states from three random-seed prompts are intersected.Source: §6.2 · p. 8
  • For cross-model transfer, ON indicates that projected prompts resemble original prompts when projector-training tasks share a type, matching the zero-shot transfer pattern; Caverage does not.The comparison uses prompts projected with Task Tuning and original prompts trained on T5XXL.Source: §6.2 · pp. 8–9, Table 5 · p. 8

7 Conclusion

The conclusion reports that soft prompts transfer across tasks and models, and that prompt initialization can improve prompt tuning. It also identifies how prompts stimulate PLMs as important to transferability.

7 Conclusion

  • The paper’s empirical analysis examines prompt transferability across tasks and models.The conclusion frames the analyses and model-stimulation idea as support for further research on transferable and efficient prompt tuning.Source: §7 · p. 9
  • Soft prompts transfer to similar tasks without training and can be projected into other PLMs; using prompts from other tasks or PLMs as initialization accelerates training and improves effectiveness.The conclusion describes these findings across cross-task and cross-model settings.Source: §7 · p. 9
  • The authors find that how prompts stimulate PLMs matters for transferability.They present this as a basis for further research on transferable and efficient prompt tuning.Source: §7 · p. 9

A.1 Dataset and Task

The experiments cover six NLP task types and use task-appropriate metrics. They apply a shared prompt-tuning setup across classification and generation tasks.

A.1 Dataset and Task

  • The study covers sentiment analysis, natural language inference, ethical judgment, paraphrase identification, question answering, and summarization.The selected datasets span these six task types.Source: §A.1 · p. 13

A.2 Evaluation Metrics

  • Classification tasks use accuracy, while question answering uses F1 and summarization uses ROUGE-L.Source: §A.2 · p. 13

A.3 Prompt Tuning Setting

  • All tasks use AdamW with learning rate 0.001 and soft prompts of length 100, which are randomly initialized and optimized during training.Source: §A.3 · p. 13

A.4 Label Tokens

  • Classification outputs are selected by ranking label-token likelihoods; generation tasks use annotated answers as desired outputs.The classification label tokens are listed in Table 6.Source: §A.3 · p. 13, §A.4 · p. 13, Table 6 · p. 13

B Cross-Task Transfer

Cross-task prompt transfer is generally poor between different task types, including across model sizes. Unifying classification label tokens does not generally improve this transferability, and the appendix defines two speedup measures for training comparisons.

B.1 More Zero-shot transfer performance

  • Prompt transfer between different task types is generally poor across RoBERTa and T5 model sizes.Figure 5 reports relative zero-shot performance, with source tasks in rows and target tasks in columns.Source: §B.1 · p. 13, Figure 5 · p. 14

B.2 Unifying Label Tokens

  • Unifying label tokens across task types generally does not improve transferability on RoBERTaBASE.The authors infer that different task types require distinct abilities.Source: §B.2 · p. 15, Figure 6 · p. 15

B.3 Speedup Calculation

  • Convergence speedup divides PT convergence time by TPT convergence time; comparable-result speedup divides PT convergence time by the time TPT reaches comparable performance.Convergence is identified when loss stops dropping and evaluation score stops increasing for 300 steps.Source: §B.3 · p. 15

C.1 Implementation Details of Projector

The cross-model projector maps source prompts into a target model’s prompt space. Additional experiments examine transfer across model scales and frameworks, while LayerNorm is introduced to address optimization issues with projected prompts.

C.1 Implementation Details of Projector

  • The projector is a two-layer perceptron that maps a concatenated source prompt to the target PLM’s hidden-size prompt space.The experiments include several model sizes and heterogeneous frameworks, including BERTBASE to RoBERTaBASE, RoBERTaBASE to RoBERTaLARGE, and T5BASE to T5XXL.Source: §C.1 · pp. 15–16

C.2 More Zero-shot Transfer Performance

  • Cross-model zero-shot transfer results across additional model pairs are consistent with the results reported in Section 5.2.Table 7 covers transfers between different scales and heterogeneous frameworks.Source: §C.2 · p. 16, Table 7 · p. 16

C.3 Technical Details of TPTMODEL

  • Adding LayerNorm lets projected prompts work well for TPTMODEL and achieve better performance and speedup.Without LayerNorm, projected prompts can be hard to optimize, particularly in small-scale PLMs; zero-shot performance is similar with or without LayerNorm.Source: §C.3 · p. 16, Table 8 · p. 17, Figure 7 · p. 17

D.1 Effectiveness of Similarity Metrics

The appendix examines how prompt-similarity measures relate to task types and zero-shot transferability. It reports that correlations decline as PLM size increases, can be raised using ONI, and that activated neurons are more shared in bottom than top layers.

D.1 Effectiveness of Similarity Metrics

  • All five similarity metrics distinguish prompt pairs from the same task, same-type tasks, and different-type tasks.Table 9 reports average metric values for these prompt-pair groups.Source: §D.1 · p. 17, Table 9 · p. 18

D.2 Correlation Between Prompt Transferability and Prompt Similarity

  • The appendix reports Spearman correlations between prompt similarity and zero-shot transferability by task type across additional PLM sizes.Tables 10 and 11 show results for T5 and RoBERTa, respectively.Source: §D.2 · p. 18, Table 10 · p. 18, Table 11 · p. 18

D.3 PLMs’ Redundancy Influence Indicators

  • Correlation between transferability and prompt similarity drops as PLM size increases; ONI raises the ON correlation score by intersecting activation states from three random-seed prompts.Source: §D.3 · p. 18, Table 10 · p. 18

D.4 Overlapping Rate of Activated Neurons in Different Layers

Related papers

Loading 2111.06719v2…