PaperlayerView exact revision on arXiv

Source-linked AI summary

Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts

Xiangyang Liu, Tianxiang Sun, Xuanjing Huang, Xipeng Qiu

arXiv:2210.11292v2cs.CL

TL;DR

Prompt tuning can lag other methods, and its training cost remains high because back-propagation traverses the model. Late Prompt Tuning (LPT) uses late, instance-aware prompts; experiments report competitive or better performance than state-of-the-art PETuning methods and full model tuning, with faster training and lower memory cost.

  • Problem

    Prompt tuning lags other tuning methods, while back-propagation through the entire model means its training cost is not significantly reduced.

    Source: §1 · pp. 1–2

  • Method

    LPT uses a prompt generator to create an independent prompt for each input and inserts it into an intermediate PTM layer.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    12.4 points over prompt tuning and 5.0 points over model tuning in average performance across ten text-classification tasks with 100 training samples; overall, LPT is comparable or better across evaluated scenarios.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    LPT combines late, instance-aware prompts with competitive performance, faster training, and lower memory cost in the evaluated full-data and few-shot scenarios.

    Source: §7 · pp. 8–9

    The evaluation did not consider PTMs with billions or more parameters or natural language generation tasks.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Prompt tuning is a parameter-efficient tuning (PETuning) method for utilizing pre-trained models (PTMs) that simply prepends a soft prompt to the input and only optimizes the prompt to adapt PTMs to downstream tasks. Although it is parameter- and deployment-efficient, its performance still lags behind other state-of-the-art PETuning methods. Besides, the training cost of prompt tuning is not significantly reduced due to the back-propagation through the entire model. Through empirical analyses, we shed some light on the lagging performance of prompt tuning and recognize a trade-off between the propagation distance from label signals to the inserted prompt and the influence of the prompt on model outputs. Further, we present Late Prompt Tuning (LPT) that inserts a late prompt into an intermediate layer of the PTM instead of the input layer or all layers. The late prompt is obtained by a neural prompt generator conditioned on the hidden states before the prompt insertion layer and therefore is instance-dependent. Through extensive experimental results across various tasks and PTMs, we show that LPT can achieve competitive performance to full model tuning and other PETuning methods under both full-data and few-shot scenarios while possessing faster training speed and lower memory cost.

1 Introduction

Prompt tuning is efficient in trainable parameters but lags other PETuning methods, while its training speed and memory savings over full model tuning are limited. LPT inserts an instance-dependent prompt in an intermediate layer, and the paper reports competitive performance with lower training costs.

1 Introduction

  • Prompt tuning trails advanced PETuning methods in performance and convergence, and its roughly 17,000× reduction in trainable parameters yields only a 1.5× training-speed increase and 29.8% lower memory cost.P-tuning v2 improves performance by inserting prompts in every hidden layer but is difficult to optimize and needs more training steps.Source: §1 · pp. 1–2
  • The paper identifies a trade-off: shortening the label-signal path can improve prompt learning, but the prompt must still influence model outputs adequately.This motivates inserting the prompt in an intermediate layer and generating it from pre-insertion hidden states.Source: §1 · p. 2
  • LPT inserts an instance-dependent prompt into an intermediate PTM layer, avoiding gradient computation for model parameters below that layer.The prompt generator conditions the prompt on hidden states before insertion.Source: §1 · p. 2
  • 12.4 points over prompt tuning and 5.0 points over model tuning in average performance across ten tasks; LPT is also 2.0× faster and uses 56.6% less memory than model tuning.These few-shot results use 100 training samples per task on RoBERTaLARGE.Source: Figure 1 · p. 1, §1 · p. 2
  • The authors report evaluating LPT across full-data and few-shot settings, ten text-classification tasks, and three PTMs.They characterize its performance as comparable to adapter-based tuning and full model tuning while reducing training costs.Source: §1 · p. 2

2 Related Work

Related work includes adapter-based and prompt-based PETuning, alongside methods that alter biases or weight updates. These approaches differ in where trainable components are placed and how prompts are obtained.

Adapter-based tuning

  • Adapter-based methods insert trainable modules between model layers; Adapter uses bottleneck modules, AdapterDrop removes lower-layer adapters, and Compacter compresses adapters with low-rank and hypercomplex operations.The passage reports comparable-to-model-tuning results when data is sufficient, but weaker few-shot performance.Source: §2 · p. 2

Prompt-based tuning

  • Prompt-based methods insert soft prompts into embeddings or hidden layers, while BBT uses derivative-free optimization and some methods cast downstream tasks as pre-training tasks.Prompt tuning and P-tuning prompt word embeddings; Prefix-tuning and P-tuning v2 prompt every hidden layer.Source: §2 · p. 3
  • Instance-aware prompt approaches include IDPG, Context-tuning, and IPL, while BitFit tunes bias terms and LoRA optimizes low-rank decomposition matrices.The passage notes that the described instance-aware methods encode inputs with an extra encoder, increasing training time and inference latency.Source: §2 · p. 3

3 Problem Formulation

The paper formulates model tuning, prompt tuning, and LPT through how inputs are represented and where prompts are inserted. For prompt tuning, labels are mapped to vocabulary words and predicted from the final masked-token state.

3 Problem Formulation

  • Model-tuning inputs are embedded with special tokens, and the final [CLS] hidden state is used to predict the label.Single-sentence and sentence-pair inputs use different [SEP]-delimited forms.Source: §3 · p. 3
  • Prompt tuning prepends a randomly initialized soft prompt to embedded inputs and uses task-specific manual templates with a [MASK] token.Original labels are mapped to vocabulary words, and the final [MASK] hidden state predicts the label.Source: §3 · p. 3
  • LPT uses a prompt generator to produce an independent prompt for each input and inserts it at an intermediate PTM layer called the prompt layer.Source: §3 · p. 3

4 Why Prompt Tuning Performs Poorly?

Experiments examine prompt insertion at different RoBERTaLARGE layers and find that performance and convergence vary with prompt-layer placement. The findings support a trade-off between shortening the path from label signals and retaining prompt influence on model outputs.

Pilot experiments

  • Prompting at layers 12–14 achieves the highest performance, while layer 13 has the fastest convergence among the tested layers 1, 13, and 24.Figure 2 reports mean and standard deviation over three random seeds.Source: §4 · p. 3, Figure 2 · p. 4
  • When the propagation path is shortened too much, prompt–input interaction weakens, reducing the prompt’s influence on outputs and accompanying performance decline.Source: §4 · p. 4

Task-related information in hidden states

  • For prompt layer 13, estimated mutual information between hidden states and labels increases through the forward pass; the last-layer estimate is highest among the three prompt-layer settings.The other two settings collapse, especially on RTE.Source: §4 · p. 4

Conclusion of analysis

  • The authors conclude that the long propagation path in traditional prompt tuning is associated with poor performance and low convergence, while properly shortening it can improve performance.Source: §4 · p. 4

5 LPT: Late Prompt Tuning

LPT combines a prompt inserted into an intermediate layer with an instance-aware prompt generator. It uses hidden states before insertion to generate prompts, and the paper explores generator designs and prompt-layer selection.

5 LPT: Late Prompt Tuning

  • LPT combines late prompts with instance-aware generation, using pre-insertion hidden states to create an independent prompt for each instance.This design also avoids gradient calculation for model parameters below the prompt layer.Source: §5 · p. 4
  • Figure 3 contrasts naive and pooling prompt generators and illustrates LPT’s forward and backward pass.Source: §5 · p. 4, Figure 3 · p. 5

5.1 Prompt Generators

  • NPG uses a feed-forward bottleneck generator to create an independent prompt for each instance.Source: §5.1 · p. 5
  • NPG’s up-projection parameter count grows with prompt length, motivating the pooling prompt generator.The generator uses a bottleneck, with its up-projection weights scaling with prompt length.Source: §5.1 · p. 5
  • PPG pools input sequences from length n to prompt length l between down- and up-projection, with average and max pooling variants.Source: §5.1 · p. 5

5.2 How to Determine Prompt Layer?

  • Figure 4 tracks performance across prompt layers for three generators on RoBERTaLARGE, reporting means and standard deviations over three seeds.Source: Figure 4 · p. 5
  • NPG performance declines significantly at layers 14–24, while APPG and MPPG retain high performance near the output layer, especially on TREC.The authors attribute this difference to higher-layer hidden states helping generate better prompts, while NPG uses only [CLS].Source: §5.2 · p. 5
  • The paper selects layer 13 for RoBERTaLARGE to balance performance across generators with relatively low training costs.Source: §5.2 · p. 5

6 Experiments

The experiments evaluate LPT across ten classification tasks, three pretrained models, full-data and few-shot settings, and efficiency and prompt-layer analyses. LPT’s prompt-generator variants compare favorably with prompt-based baselines, while the results also examine performance trade-offs across data regimes and prompt layers.

6.1 Evaluation Datasets

  • Evaluation covers five single-sentence and five sentence-pair classification tasks, including six GLUE tasks and four other popular tasks.The tasks include MPQA, MR, Subj, and TREC.Source: §6.1 · p. 5

6.2 Experiment Settings

  • RoBERTaLARGE and DeBERTaLARGE use the 13th prompt layer, while GPT2LARGE uses the 19th, except where noted.Full-data and few-shot scenarios are evaluated on all three models.Source: §6.2 · p. 6

6.3 Baselines

  • Baselines include model tuning, adapter-based and prompt-based methods, BitFit, and LoRA; IDPG is compared only in its single-layer S-IDPG-PHM version without supplementary training.Adapter and AdapterDrop are the adapter-based baselines; Prompt Tuning, P-tuning v2, and IDPG are prompt-based baselines.Source: §6.3 · p. 6

6.4 Main Results

  • With 100 training samples, LPT w/ NPG beats model tuning by 5 points and Adapter by 7.1 points; prompt-generator variants also outperform prompt-based baselines in full-data evaluations.LPT w/ NPG outperforms all baselines in both few-shot settings; in full-data results, generator variants outperform prompt-based methods and are comparable with AdapterDrop, with one-third its tunable parameters for MPPG and APPG.Source: §6.4 · pp. 6–7, Table 1 · p. 6, Table 2 · p. 7

6.5 Efficiency Evaluation

  • On RoBERTaLARGE and GPT2LARGE, LPT w/ NPG beats AdapterDrop by 20.1 and 7 points, respectively, and LPT offers a better trade-off between training budget and performance.The efficiency evaluation reports accuracy, tunable parameters, training speed, and memory cost; it uses the RTE dataset.Source: §6.5 · pp. 7–8, Table 5 · p. 8

6.6 Analyses

  • Across DeBERTaLARGE and GPT2LARGE, the most intermediate prompt layer is optimal, especially for LPT w/ NPG.The visualization shows similar instance-aware prompts for similar sentence representations and different prompts for dissimilar representations.Source: §6.6 · p. 8

7 Conclusion

The paper identifies a trade-off between how far label signals must propagate to the prompt and how strongly the prompt influences outputs. It introduces LPT with late, instance-aware prompts, reporting competitive performance alongside higher training speed and lower memory cost.

7 Conclusion

  • The authors identify a trade-off between label-signal propagation distance to the prompt and the prompt’s influence on model outputs.Source: §7 · p. 8
  • LPT combines late and instance-aware prompts to improve prompt tuning.The conclusion reports results across full-data and few-shot scenarios.Source: §7 · p. 9
  • Across the evaluated settings, LPT achieves comparable or better performance than state-of-the-art PETuning methods and full model tuning, with higher training speed and lower memory cost.Source: §7 · p. 9

Limitations

The evaluation covers diverse NLU tasks on three PTMs, but not PTMs with billions or more parameters or NLG tasks.

Limitations

  • The study evaluates diverse NLU tasks on RoBERTaLARGE, DeBERTaLARGE, and GPT2LARGE, but excludes PTMs with billions or more parameters and NLG tasks.The authors identify investigation of other backbone models and task types as future work.Source: Limitations · p. 9

Ethics Statement

The paper reports no attached privacy or ethical issues for its widely used datasets and describes a classifier-based estimate of mutual information between hidden states and labels.

Ethics Statement

  • The authors report that the widely used datasets have no attached privacy or ethical issues to their knowledge.Source: Ethics Statement · p. 9

A Details for Mutual Information Estimation

  • Because mutual information cannot be calculated directly, the authors estimate it using a classifier trained on hidden states and original labels.They use classifier performance to estimate I(h, y), training 24 classifiers for RoBERTaLARGE's 24 hidden layers and using a six-layer RoBERTaLARGE classifier to speed training.Source: A Details for Mutual Information… · p. 12

B Datasets

The dataset section describes original GLUE splits for six datasets and proportion-based development-set sampling for four others.

B Datasets

  • SST-2, MNLI, MRPC, QNLI, QQP, and RTE use their original GLUE data splits.Source: B Datasets · p. 12
  • For four other datasets, development samples are selected from training data with per-label counts proportional to the original training-set label distribution.Source: B Datasets · p. 12

C Implementation Details

Implementation details specify shared training budgets and model-specific hyperparameters, and the appendix reports a prompt-layer performance–efficiency trade-off on RTE with RoBERTaLARGE.

C Implementation Details

  • All methods use the same number of training epochs or steps, and adapter-based methods use down-projection size m = 16.Source: C Implementation Details · p. 12
  • Prompt length is 20 for prompt tuning and P-tuning v2, and 5 for S-IDPG-PHM and LPT with NPG.The paper also searches lengths {10, 15, 20} for LPT with MPPG and APPG; LPT's down-projection size is 128.Source: C Implementation Details · pp. 12, 14
  • Manual templates and label words are provided for RoBERTa and DeBERTa, while GPT2 uses its own templates and label words.For GPT2, [MASK] marks positions to predict because its vocabulary lacks a [MASK] token.Source: Table 8 · p. 13, Table 9 · p. 13, C Implementation Details · p. 14

D Efficiency Evaluation on Different Prompt Layers

C Implementation Details

Related papers

Loading 2210.11292v2…