PaperlayerView exact revision on arXiv

Source-linked AI summary

PPT: Pre-trained Prompt Tuning for Few-shot Learning

Yuxian Gu, Xu Han, Zhiyuan Liu, Minlie Huang

arXiv:2109.04332v3cs.CL

TL;DR

Few-shot prompt tuning is substantially weaker than full-model tuning despite its efficiency, motivating better soft-prompt initialization. PPT pre-trains prompts through unified classification tasks and reports performance comparable to or better than full-model tuning while retaining parameter efficiency.

  • Problem

    Prompt tuning is comparable to full-model tuning with sufficient data but performs much worse in few-shot settings, limiting its low-resource applicability.

  • Method

    PPT pre-trains soft prompts with self-supervised tasks after unifying classification tasks into a common multiple-choice format, then uses them to initialize downstream prompt tuning.

  • Results

    PPT significantly outperforms other prompt-tuning baselines and performs comparably to or better than full-model tuning across full-data and few-shot settings.

  • Takeaways & Limitations

    Pre-trained prompt initialization improves few-shot adaptation while preserving the parameter efficiency of prompt tuning.

  • Takeaways & Limitations

    The authors identify adapting unified-task pre-training beyond soft prompts and to other parameter-efficient tuning approaches as future work.

Abstract

from arXiv · show

Prompts for pre-trained language models (PLMs) have shown remarkable performance by bridging the gap between pre-training tasks and various downstream tasks. Among these methods, prompt tuning, which freezes PLMs and only tunes soft prompts, provides an efficient and effective solution for adapting large-scale PLMs to downstream tasks. However, prompt tuning is yet to be fully explored. In our pilot experiments, we find that prompt tuning performs comparably with conventional full-model fine-tuning when downstream data are sufficient, whereas it performs much worse under few-shot learning settings, which may hinder the application of prompt tuning in practice. We attribute this low performance to the manner of initializing soft prompts. Therefore, in this work, we propose to pre-train prompts by adding soft prompts into the pre-training stage to obtain a better initialization. We name this Pre-trained Prompt Tuning framework "PPT". To ensure the generalization of PPT, we formulate similar classification tasks into a unified task form and pre-train soft prompts for this unified task. Extensive experiments show that tuning pre-trained prompts for downstream tasks can reach or even outperform full-model fine-tuning under both full-data and few-shot settings. Our approach is effective and efficient for using large-scale PLMs in practice.

1 Introduction

Prompt tuning adapts large PLMs efficiently by training only soft prompts, but its random initialization performs poorly in few-shot settings. PPT addresses this gap by pre-training prompts on unified classification tasks, retaining parameter efficiency while reaching or exceeding full-model tuning.

  • Motivation: Prompt tuning freezes the PLM and learns continuous soft prompts, avoiding the cost of fine-tuning and storing the entire model for each task.Soft prompts replace discrete hard prompts and are learned end-to-end.
  • Motivation: PT is comparable to FT with sufficient downstream data but performs much worse under few-shot settings.The few-shot gap may hinder prompt tuning in low-resource scenarios.
  • Motivation: Pilot experiments attribute weak few-shot PT partly to soft-prompt initialization and show that verbalizer choice strongly affects performance.Simply using concrete word embeddings does not improve performance, while combining soft and hard prompts helps but remains insufficient.
  • PPT framework: PPT pre-trains soft prompts with self-supervised tasks on large unlabeled corpora across sentence-pair, multiple-choice, and single-text classification formats.The framework uses format-specific pre-training tasks to obtain a better downstream initialization.
  • Results: Experiments on three 11B PLMs show that PPT improves PT substantially, can reach or outperform FT, reduces few-shot variance, and retains PT’s parameter efficiency.The reported models are T5-XXL, mT5-XXL, and CPM-2.

2 Pilot Experiments

Pilot experiments examine hybrid prompts, verbalizer selection, and concrete-word initialization for few-shot PT. These factors affect performance, but none closes the gap with FT, motivating PPT’s pre-trained initialization.

  • Experimental setup: The pilot study evaluates hybrid prompt tuning, verbalizer selection, and real-word initialization for few-shot PT.The experiments use T5-XXL with 100 tunable soft prompt tokens and 32-sample training sets.
  • Hybrid Prompt Tuning: Hard prompts improve PT on SST-2, but the resulting systems still underperform FT.The study compares manually designed and auto-generated hard prompts combined with soft prompts.
  • Verbalizer Selection: Verbalizer choices significantly influence PT performance, with common words that express label meanings generally working well.This observation guides verbalizer selection for PPT.
  • Real Word Initialization: For 11B models, real-word initialization has little or negative impact on few-shot PT performance.This differs from prior observations on models smaller than 3B parameters.
  • Summary: The tested enhancement strategies do not make PT comparable with FT under few-shot settings, despite affecting PT performance.The findings motivate a better prompt initialization strategy.

3 Pre-trained Prompt Tuning (PPT)

PPT pre-trains soft prompts on self-supervised classification tasks, then transfers them as initializations for downstream prompt tuning. It unifies several task formats through multiple-choice classification to improve generalization across classification settings.

  • 3.1 Overview: PPT converts classification into text-to-text prediction using a pattern-verbalizer pair, with prompts added before the transformed input and labels mapped to vocabulary tokens.The transformed input preserves a mask token for prediction, while the verbalizer maps task labels to concrete tokens.
  • 3.1 Overview: Prompt tuning can match full-model fine-tuning with full data but struggles in few-shot settings because effective soft prompts are difficult to learn from initialization.Pilot experiments found existing initialization strategies had little or negative impact on large-scale PLM prompt tuning.
  • 3.1 Overview: PPT pre-trains soft prompts on unlabeled-corpus self-supervised tasks related to downstream task groups, keeping the PLM parameters fixed during prompt pre-training.The resulting prompts initialize downstream optimization for tasks within the corresponding group.
  • 3.2.1 Sentence-Pair Classification: Sentence-pair pre-training extends next-sentence prediction to three labels representing coherent, similar, and irrelevant sentence relations.Adjacent sentences, nonadjacent sentences from the same document, and sentences from different documents provide the three relation types.
  • 3.2.2 Multiple-Choice Classification: Multiple-choice pre-training teaches prompts to select the adjacent sentence from six candidates, allowing the resulting pattern-verbalizer pair to transfer to many multiple-choice tasks.For reading comprehension, the passage and question are concatenated into the query.
  • 3.2.3 Single-Sentence Classification: Single-sentence pre-training uses pseudo-labels from a separately fine-tuned RoBERTaBASE model, filtering corpus sentences with low classification probability.The procedure is designed around five-class sentiment labels and can select subsets for tasks with fewer labels.
  • 3.3 Unifying Task Formats: The single-sentence approach is limited across domains and label counts, motivating a unified format that represents all classification tasks as multiple-choice classification.Sentence pairs become queries with three options, while single-text tasks use the input sentence and concrete labels as options.
  • 3.3 Unifying Task Formats: Unified prompt pre-training varies option numbers from 2 to 16 and option lengths from 50 to 20 before applying the prompts to sentence-pair, multiple-choice, and single-sentence tasks.The paper sets 16 labels because they cover most benchmarks, while noting that more labels are applicable to other tasks.

4 Experiments

Across few-shot and full-data experiments, PPT generally improves prompt-tuning performance, often matching or exceeding FT while retaining parameter efficiency. Its unified variant extends this advantage to classification tasks with more labels, and pre-trained initialization improves sample efficiency and convergence.

  • Main results: PPT outperforms Vanilla PT and LM Adaption on most datasets, while Hybrid PPT further improves results when hard prompts are added.The same complementarity is observed on BoolQ, RACE-m, LCQMC, and C3.
  • Main results: PPT outperforms FT on all Chinese datasets and most English datasets.The authors attribute this pattern to prompt pre-training bridging a gap between masked language modeling and downstream tasks.
  • Main results: PPT produces lower variances on most datasets, keeping variance at a low level across all datasets.Vanilla PT is unstable in few-shot learning; SST-2 variance reaches 15.5 under some random seeds.
  • Unified PPT: Unified PPT achieves the best performance on single-text classification tasks with more than 5 labels, exceeding FT by a large margin.Unified PPT unifies classification formats as multiple-choice classification and remains competitive with PPT and Hybrid PPT.
  • Sample Efficiency: For 32 to 128 samples, PPT is consistently better than PT, while the three methods gradually converge at 256 samples.This trend is reported on the RACE-m and CB datasets.
  • Training Efficiency: Pre-trained initialization speeds up PPT convergence, addressing PT's slower convergence despite PT being faster per optimization step.The convergence comparison is made among FT, Vanilla PT, and PPT.

5 Related Works

Prior work adapts PLMs through task-oriented or prompt-oriented fine-tuning, while automated prompting and soft prompts address limitations of manually designed discrete prompts. Few-shot learning motivates these developments because low-resource settings are important for stable PLM use.

  • Task-oriented fine-tuning adds task-specific heads and tunes both the PLM and heads using task-specific objectives.
  • Prompt-oriented fine-tuning converts downstream tasks into language-modeling problems by inserting language prompts.
  • Knowledge probing used language triggers to induce PLMs to generate relational facts, stimulating later prompt-based methods.
  • Automatic prompt-generation methods reduce manual design effort but remain restricted to discrete spaces that are usually sub-optimal.
  • Soft prompts use continuous learnable embeddings, typically randomly initialized, and can be combined with hard prompts or tuned independently.
  • Few-shot learning is important for stable and effective PLM use because long-tail distributions are common in real-world applications.

6 Conclusion and Future Work

The paper presents PPT, which unifies downstream classification tasks, pre-trains prompts for the resulting formats, and then applies those prompts to downstream tuning. Future work extends this strategy beyond classification and prompt tuning.

  • PPT unifies downstream tasks into several formats, pre-trains prompts with format-specific self-supervised tasks, and initializes downstream prompt tuning with them.
  • Extensive experiments show PPT significantly outperforms prompt-tuning baselines and performs comparably to or better than full-model tuning.
  • Future work includes unified task formats and corresponding pre-training objectives for language generation and relation extraction.
  • Future work also evaluates other parameter-efficient tuning approaches and studies whether unified task pre-training benefits PLMs themselves.

A Dataset Information

The evaluation uses English datasets from GLUE and SuperGLUE, along with additional single-text classification datasets, and substitutes original validation sets when test sets are unavailable.

  • Unavailable public test sets led the authors to use original validation sets for testing, following prior work.
  • English experiments include SST-2, BoolQ, CB, RTE, SST-5, and Yahoo datasets.

B PVPs for Chinese Tasks

Chinese classification tasks are converted into prompt-based formats analogous to the English setup. The appendix specifies sentence-pair, multiple-choice, and single-sentence classification patterns, plus hard-prompt terminology in the table caption.

  • Sentence-Pair Classification: Chinese sentence-pair classification takes an input pair x = (s1, s2) with label list Y = [0, 1, 2].
  • Multiple-Choice Classification: Chinese multiple-choice classification represents an input as a query plus six candidate sentences.
  • Multiple-Choice Classification: The multiple-choice label mapping uses the Chinese sequence [一, 二, 三, 四, 五, 六].
  • Single-Sentence Classification: Chinese single-sentence classification uses a single input sentence and a sentiment-classification pattern analogous to the English scenario.
  • Table 7 distinguishes hard prompts for hybrid prompt tuning by single-sentence, multiple-choice, and sentence-pair classification formats.

C Training Details

The experiments compare full-model tuning and prompt tuning under fixed training settings, while pre-training prompts uses task-specific data and a unified option configuration. Additional procedures cover task construction, pseudo-labeling, and hybrid prompt selection.

  • Downstream Adaptation: Full-Model Tuning updates all model parameters, whereas Prompt Tuning freezes the model and updates only soft prompts.Both methods use batch size 16 and train for 50 epochs.
  • Downstream Adaptation: Prompt Tuning requires learning-rate searches in [5e-3, 1e-2, 2e-2, 5e-2], and the authors report that it is harder to train than Full-Model Tuning.The learning rate is selected using validation performance.
  • Prompt Pre-training: Prompt pre-training uses an inverse square root scheduler with learning rate 0.1, batch size 256, maximum length 512, and at most 200,000 steps.Five percent of the data is reserved for validation, with evaluation every 2,000 steps and selection by lowest validation loss.
  • Task Construction: Sentence-pair and multiple-choice pre-training data are constructed from next-sentence relations, document membership, and controlled option sampling.Unified PPT samples option counts uniformly from 2 to 16; input lengths and filtering rules are specified for each format.
  • Task Construction: Single-sentence classification uses RoBERTaBASE pseudo-labels, confidence thresholds across five labels, and validation-selected annotation checkpoints.The selected checkpoint reaches 70.53 validation accuracy at the fifth epoch.
  • Hybrid Prompting: Hybrid PPT directly reuses task-format-specific hard prompts selected through Prompt Tuning pilot experiments.The formats include sentence-pair, multiple-choice, and single-sentence classification.

E Training Consumption

The study compares the time and memory costs of Full-Model Tuning and Prompt Tuning. Prompt Tuning has shorter single-step optimization and lower GPU memory use, while Pre-trained Prompt Tuning has the same downstream consumption as Prompt Tuning.

  • Cost Comparison: Pre-trained Prompt Tuning has the same downstream consumption as Prompt Tuning.Prompt pre-training adds external cost, but it is performed once for reuse across multiple tasks.
  • Cost Comparison: Prompt Tuning has shorter single-step optimization time and lower GPU memory usage than Full-Model Tuning.The explanation is that optimization updates only prompt parameters, avoiding momentum and gradients for other parameters.
  • Resource Mechanism: Prompt Tuning’s lower resource use follows from updating only prompt parameters during optimization.Other model parameters do not require momentum or gradient storage.
Loading 2109.04332v3…