PaperlayerView exact revision on arXiv

Source-linked AI summary

SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer

Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, Daniel Cer

arXiv:2110.07904v2cs.CL

TL;DR

Large language models make separate full-model adaptation costly, while Prompt Tuning leaves substantial gaps at smaller model sizes. SPoT transfers source-task soft prompts to initialize target prompts and uses prompt embeddings to retrieve transferable sources. It improves Prompt Tuning, matches or exceeds Model Tuning across SuperGLUE model sizes, and can use 27,000× fewer task-specific parameters.

  • Problem

    Separate full-model adaptation is prohibitively expensive for very large models, while Prompt Tuning has large gaps behind Model Tuning below 11B parameters.

  • Method

    SPoT learns prompts on source tasks, transfers them to initialize target prompts, and uses task-prompt embeddings with similarity retrieval to select transferable sources.

  • Results

    Across SuperGLUE model sizes, SPoT matches or outperforms Model Tuning and improves Prompt Tuning, including 27,000× fewer task-specific parameters.

  • Takeaways & Limitations

    Prompt transfer can improve frozen-model adaptation, and task prompts can identify source tasks likely to benefit novel targets.

  • Takeaways & Limitations

    Prompt Tuning may be outperformed by other parameter-efficient adaptation methods in specific situations, so extending SPoT beyond Prompt Tuning remains open.

Abstract

from arXiv · show

There has been growing interest in parameter-efficient methods to apply pre-trained language models to downstream tasks. Building on the Prompt Tuning approach of Lester et al. (2021), which learns task-specific soft prompts to condition a frozen pre-trained model to perform different tasks, we propose a novel prompt-based transfer learning approach called SPoT: Soft Prompt Transfer. SPoT first learns a prompt on one or more source tasks and then uses it to initialize the prompt for a target task. We show that SPoT significantly boosts the performance of Prompt Tuning across many tasks. More remarkably, across all model sizes, SPoT matches or outperforms standard Model Tuning (which fine-tunes all model parameters) on the SuperGLUE benchmark, while using up to 27,000x fewer task-specific parameters. To understand where SPoT is most effective, we conduct a large-scale study on task transferability with 26 NLP tasks in 160 combinations, and demonstrate that many tasks can benefit each other via prompt transfer. Finally, we propose an efficient retrieval approach that interprets task prompts as task embeddings to identify similar tasks and predict the most transferable source tasks for a novel target task.

1 Introduction

SPoT addresses the practical cost of adapting large frozen language models by transferring learned soft prompts from source tasks to target tasks. It improves Prompt Tuning, matches or outperforms Model Tuning on SuperGLUE across model sizes, and uses task-prompt transferability to guide source-task retrieval.

  • Fine-tuning separate 100B+ model instances for downstream tasks is prohibitively expensive, motivating parameter-efficient adaptation methods.
  • Prompt Tuning learns small task-specific prompts for frozen models, but below 11B parameters it retains large gaps relative to Model Tuning.
  • SPoT trains prompts on one or more source tasks and uses them to initialize prompts for downstream target tasks.
  • +10.1 and +2.4 point average accuracy improvements are obtained on SuperGLUE with T5 Base and T5 XXL, respectively, while SPoT remains competitive with or outperforms Model Tuning across model sizes.
  • The paper studies transferability across 26 NLP tasks in 160 source-target combinations and finds that many tasks benefit from prompt transfer.
  • Task prompts are interpreted as task embeddings, enabling similarity-based retrieval of source tasks likely to yield positive transfer for a novel target.

2 Improving PROMPTTUNING with SPOT

SPoT inserts source prompt tuning before target adaptation while keeping the pretrained model frozen. Experiments show substantial gains over vanilla Prompt Tuning, strong SuperGLUE performance across model sizes, and efficient transfer guided by source-task choices.

  • 2 Improving PROMPTTUNING with SPOT: SPoT learns a prompt on source tasks and uses it to initialize each target prompt, preserving a frozen model and small task-specific storage.
  • 2.1 Experimental setup: The evaluation uses frozen T5 models spanning 60M to 11B parameters and compares SPoT with vanilla Prompt Tuning, Model Tuning, and Multi-Task Model Tuning.
  • 2.1.2 Evaluation datasets: Experiments evaluate downstream performance on diverse GLUE and SuperGLUE tasks using validation results after a fixed training schedule.
  • 2.1.3 Data for source prompt tuning: Source prompts are trained from unsupervised C4, supervised MNLI or SQuAD, or mixtures spanning multiple NLP benchmarks and task families.
  • 2.2 Effect of SPOT: +4.4 and +10.1 point average accuracy improvements over vanilla Prompt Tuning are obtained on GLUE and SuperGLUE, respectively, with longer tuning also improving performance and omitted longer tuning improving stability.
  • 2.2 Effect of SPOT: SQUAD and some task-family mixtures yield strong transfer, whereas mixing all datasets is not best, possibly because task interference or negative transfer can hurt target performance.
  • 2.2 Effect of SPOT: 91.2 is SPoT’s best XXL average score, 1.1 points above Multi-Task Model Tuning, despite 27,000× fewer task-specific parameters.
  • 2.2 Effect of SPOT: SPoT reaches 89.2 on the SuperGLUE leaderboard, compared with GPT-3 at 71.8 and fully fine-tuned T5 XXL at 89.3, while tuning 27,000× fewer parameters.

3 Predicting task transferability

The paper studies when soft prompt transfer benefits target tasks and develops task-embedding retrieval to select promising source prompts efficiently. Across 26 NLP tasks, task similarity often reflects transferability, but other factors also matter.

  • 3.3 Predicting transferability via similarity: 69% source-task search-space reduction retains 90% of the best-case quality gain through task-embedding retrieval.The method ranks source prompts by embedding similarity and selects targeted candidates for novel target tasks.
  • 3.1 Measuring transferability: 160 source-target combinations across 16 source and 10 low-resource target datasets provide the transferability study's experimental setting.The target datasets contain fewer than 10K training examples, and experiments use T5 BASE with three random seeds.
  • 3.1 Measuring transferability: 58.9% relative error reduction is achieved by MNLI → CB, the largest reported gain among the transfer pairs.The average score increases from 92.7 to 97.0; other highlighted transfers include MNLI → COPA at 29.1% and RECORD → WSC at 20.0%.
  • 3.1 Measuring transferability: Best-source selection raises the average score across 10 target tasks from 74.7 to 80.7.The results also report positive transfer between similar tasks and between some relatively dissimilar tasks.
  • 3.2 Defining task similarity through prompts: Prompt-derived task embeddings cluster many semantically related tasks and can distinguish task type from shared dataset domain.Examples include QA, sentiment, NLI, semantic similarity, paraphrasing, and commonsense clusters; QNLI is not closely linked to SQUAD despite its dataset origin.
  • 3.3 Predicting transferability via similarity: Task-embedding similarity significantly correlates with transferability on four of 10 target tasks, while other factors may also influence transfer.Significant correlations are reported for STS-B, CB, WSC, and RTE; the study identifies data size, task difficulty, and domain similarity as possible additional factors.

4 Related Work

SPoT builds on parameter-efficient adaptation and prompt-based learning by transferring soft prompts between tasks. It differs from approaches relying on handcrafted or specialized pre-training tasks by using existing tasks and supporting transfer across mismatched task types or formats.

  • Parameter-efficient transfer learning: Parameter-efficient adaptation seeks to reduce the cost of applying large pre-trained language models by updating only small model components or task-specific modules.The related work covers compression, partial-model updates, and task-specific modules.
  • Task-transfer prediction: +2.8 points: BEST OF TOP-k with k = 3 improves over Prompt Tuning from scratch in the reported task-transferability comparison.The table also reports that larger k values up to 15 retain most oracle-selection benefits.
  • Prompt-based learning: Prompt-based learning progressed from manually designed prompts toward learned soft prompts because hard prompts are sensitive to prompt choice and often sub-optimal.Soft prompts are learnable parameters injected into the model.
  • Prompt transfer: SPoT uses existing tasks as source tasks and can transfer across mismatches in task type or input/output format, unlike concurrent methods tailored to specific downstream-task types.This design is intended to be more extensible to novel downstream tasks.
  • Task transferability: Prior task-transfer studies report benefits from data-rich, reasoning-intensive, or target-similar source tasks and have also examined predicting transferability.These findings motivate selecting source tasks rather than treating all transfer pairs equally.

5 Limitations & Future work

The paper identifies open questions about extending SPoT beyond Prompt Tuning and notes that prompt embeddings do not capture every factor influencing transferability. It also highlights Prompt Tuning’s efficiency and simplicity as continuing advantages.

  • Future work: SPoT’s extension to other parameter-efficient adaptation methods remains an open question because those methods may outperform Prompt Tuning in specific situations.The paper proposes testing whether an analogous transfer approach works for those methods.
  • Prompt Tuning advantages: Prompt Tuning requires less than 0.01% task-specific parameters for most model sizes, making it highly parameter-efficient.The paper presents this as an advantage over other methods with learnable parameters.
  • Prompt Tuning advantages: Prompt Tuning does not modify the internal model architecture, enabling mixed-task inference and facilitating transfer learning between tasks.The paper contrasts this simplicity with methods that add components to Transformer layers.
  • Future work: Task-prompt embeddings do not capture all factors influencing task transferability, so other task-embedding methods remain future work.This limits the current retrieval formulation’s coverage of transferability.

6 Conclusion

The paper concludes that SPoT improves prompt-based transfer learning while retaining parameter efficiency. Its experiments show broad task transferability and support task-embedding retrieval for selecting useful source tasks.

  • Main findings: SPoT matches or exceeds Model Tuning across SUPERGLUE model sizes while remaining more parameter-efficient.The conclusion states that scale is not necessary for Prompt Tuning to match Model Tuning.
  • Task transferability: A large-scale study indicates that tasks can benefit one another through prompt transfer in various scenarios.The study examines transferability across many task relationships.
  • Task retrieval: Task prompts can be interpreted as task embeddings to formalize task similarity and identify source tasks that may benefit novel targets.The proposed retrieval approach measures task similarity for source-task selection.

A Full results for Figure 1

The appendix reports SUPERGLUE results alongside experiments spanning diverse NLP datasets and task families. Table 4 compares model- and prompt-tuning methods across model sizes, with SPoT matching or outperforming Model Tuning throughout.

  • Evaluation coverage: The experiments cover GLUE, SUPERGLUE, NLI, paraphrasing and semantic similarity, sentiment analysis, question answering, commonsense reasoning, translation, summarization, and generation.The dataset inventory includes both labeled and unlabeled sources, including C4.
  • Evaluation coverage: The question-answering datasets include SQUAD, NEWSQA, TRIVIAQA, SEARCHQA, HOTPOTQA, and NaturalQuestions within MRQA.These datasets represent the QA portion of the evaluation inventory.
  • SUPERGLUE results: Table 4 compares model-tuning and prompt-tuning methods on SUPERGLUE across model sizes using means and standard deviations over three random seeds.The comparison includes SPoT, vanilla Prompt Tuning, GPT-3, Model Tuning, and Multi-task Model Tuning.
  • SUPERGLUE results: SPoT outperforms vanilla Prompt Tuning and GPT-3 by a large margin while matching or outperforming Model Tuning across all model sizes.At XXL size, SPoT also outperforms Multi-task Model Tuning.
  • Evaluation coverage: The commonsense-reasoning datasets include αNLI, COSMOSQA, HellaSwag, PIQA, SocialIQA, and Winogrande within RAINBOW.The inventory also includes machine translation datasets across English-German, English-French, and English-Romanian.
  • Evaluation coverage: The summarization and generation coverage includes AESLC, BillSum, CNN/DailyMail, WikiLingua, Gigaword, Multi-News, Newsroom, SAMSum, XSum, and GEM tasks.These datasets broaden evaluation beyond classification and inference.

C Additional training details

The experiments use class-based prompt initialization, single and mixed source-task datasets, and standard model-tuning baselines with specified hyperparameters.

  • Prompt tokens are initialized with embeddings representing an enumeration of output classes.
  • C4, MNLI, and SQUAD serve both as individual source tasks and as components of mixed-task settings.
  • Table 5 reports results for the SPOT XXL submission alongside public SUPERGLUE leaderboard competitors.
  • The model-tuning baselines use a 0.001 learning rate, Adafactor with restored pre-training states, 0.1 dropout, and 216 tokens per batch.

D Details of our SUPERGLUE submission

SPOT achieves a strong SUPERGLUE leaderboard result while tuning far fewer task-specific parameters than fully fine-tuned systems and other parameter-efficient methods.

  • The leaderboard comparison includes competitors that generally tune more than 3B parameters directly on final tasks, while only three prior submissions tune under 1M.
  • Compared with WARP, SPOT tunes 410K rather than 25K parameters and benefits from 50× more frozen parameters.
  • SPOT nearly matches fully fine-tuned T5 XXL at 89.3 while tuning 27,000× fewer parameters.
  • SPOT outperforms T5 on three of eight SUPERGLUE tasks: CB, COPA, and RTE.

E Task transferability results

The transferability experiments show that source-task prompts can improve target-task performance, with positive gains documented across many task pairs.

  • The transferability study includes results involving the T5-3B and T5-LARGE models.
  • Prompt initialization from a source task can provide significant gains on a target task.
  • Table 7 lists source-to-target transfers achieving more than 10% relative error reduction.

G Correlation between task similarity and task transferability

Task embeddings are used to represent task relationships and examine whether embedding similarity tracks transferability.

  • Table 6 organizes transfer results with source tasks as rows and target tasks as columns, highlighting positive and best transfers.
  • Table 7 reports positive transfers exceeding 10% relative error reduction using s →t source-to-target notation.
  • Task embeddings from similar tasks form clusters, while embeddings from different prompt-tuning runs of the same task are linked.
  • Figure 8 relates source-target embedding cosine similarity to relative error reduction on the target task.
Loading 2110.07904v2…