Source-linked AI summary
Multitask Prompt Tuning Enables Parameter-Efficient Transfer Learning
Zhen Wang, Rameswar Panda, Leonid Karlinsky, Rogerio Feris, Huan Sun, Yoon Kim
TL;DR
Existing prompt-tuning methods leave open how to exploit cross-task knowledge while adapting large pretrained models with few parameters. MPT distills multiple task-specific source prompts into a shared transferable prompt and uses low-rank multiplicative updates for target tasks, outperforming strong baselines across 23 NLP datasets, including full finetuning in some cases.
Problem
Existing prompt-tuning methods typically learn task-specific soft prompts, leaving cross-task knowledge underused in multitask transfer.
Method
MPT decomposes each source prompt into a shared prompt matrix and a rank-one task-specific matrix, then learns the shared component through knowledge distillation for transfer to target tasks.
Results
Across 23 NLP datasets, MPT outperforms state-of-the-art prompt-transfer methods, improves SuperGLUE by 16.3% over PT, and sometimes exceeds full finetuning with 0.035% tunable parameters per task.
Takeaways & Limitations
MPT enables parameter-efficient transfer across diverse NLP benchmarks while tuning substantially fewer task-specific parameters than competing multitask prompt-transfer methods.
Abstract
from arXiv · showhide
Prompt tuning, in which a base pretrained model is adapted to each task via conditioning on learned prompt vectors, has emerged as a promising approach for efficiently adapting large language models to multiple downstream tasks. However, existing methods typically learn soft prompt vectors from scratch, and it has not been clear how to exploit the rich cross-task knowledge with prompt vectors in a multitask learning setting. We propose multitask prompt tuning (MPT), which first learns a single transferable prompt by distilling knowledge from multiple task-specific source prompts. We then learn multiplicative low rank updates to this shared prompt to efficiently adapt it to each downstream target task. Extensive experiments on 23 NLP datasets demonstrate that our proposed approach outperforms the state-of-the-art methods, including the full finetuning baseline in some cases, despite only tuning 0.035% as many task-specific parameters.
1 INTRODUCTION
Parameter-efficient tuning seeks task-level performance without duplicating the hundreds of millions or billions of parameters in modern pretrained language models. MPT addresses prompt-transfer limitations by distilling multiple source prompts into one transferable prompt and adapting it efficiently to target tasks.
- Full task-specific finetuning is difficult to scale because modern pretrained language models contain hundreds of millions or billions of parameters.
- Prompt tuning freezes pretrained model parameters and learns task-specific continuous prompt vectors, but it can lag behind full finetuning and be sensitive to initialization.
- Prior prompt-transfer methods pretrain soft prompts on source tasks and use similarity-based initialization for target-task finetuning.
- MPT learns one transferable prompt from multiple source tasks through prompt decomposition and knowledge distillation instead of retrieving or aggregating source prompts.
- MPT improves SuperGLUE by 16.3% over vanilla prompt tuning while tuning 77.6K rather than 232K task-specific prompt parameters.
- Across 23 NLP datasets, MPT outperforms state-of-the-art prompt-transfer methods and sometimes exceeds full finetuning while tuning only 0.035% as many task-specific parameters.
2 RELATED WORK
Related work spans parameter-efficient model updates, multitask transfer, and knowledge distillation. These lines of research provide the methodological context for MPT’s combination of prompt tuning, cross-task transfer, and distillation.
- Parameter-efficient transfer learning: Parameter-efficient methods reduce adaptation cost through trainable layers, bias-only updates, sparse updates, soft prompts, prefixes, or task-conditioned hyperprompts.
- Multitask learning: Multitask learning studies solving related tasks jointly and transferring models trained on multiple source tasks to target tasks.
- Knowledge distillation: Knowledge distillation has been applied to improve performance and efficiency in model compression, transfer learning, translation, question answering, and retrieval.
3 APPROACH
MPT learns a transferable shared prompt from multiple source tasks, decomposes task prompts into shared and low-rank components, and adapts the shared prompt efficiently to target tasks through knowledge distillation and multiplicative updates.
- Prompt tuning: Prompt tuning freezes the pretrained language model and learns a small prompt matrix prepended to input embeddings instead of updating all model parameters.The prompt matrix has dimensions l × d, matching the token-embedding dimension d.
- Prompt decomposition: MPT decomposes each source-task prompt into a shared matrix and a task-specific low-rank matrix, separating general information from task-specific knowledge.The task-specific matrix is represented as a rank-one outer product of vectors and combined with the shared matrix through elementwise multiplication.
- Prompt distillation: MPT first trains task-specific teacher prompts, then distills their output distributions and hidden states into a shared prompt with task-specific vectors.The total objective combines the aggregated source-task language-model loss with weighted logit and hidden-state distillation losses.
- Target adaptation: MPT adapts the learned shared prompt to a target task by initializing a low-rank task-specific update and optimizing it with the shared prompt.Target adaptation uses the Hadamard product of the shared prompt and the target task’s low-rank matrix, with separate learning rates for shared and task-specific parameters.
- Parameter-efficiency: For τ target tasks, MPT tunes (l × d) + (l + d)τ parameters and can compress each trained target prompt into a single l × d matrix.The shared prompt is counted once across the target-task group, while each task contributes l + d task-specific parameters.
4 EXPERIMENTS
Experiments across diverse NLP benchmarks show that MPT improves performance over prompt-tuning and other parameter-efficient baselines while using substantially fewer task-specific parameters. Ablations indicate that prompt decomposition, distillation, joint component updates, and stochastic task sampling contribute to performance, with prompt length requiring careful tuning.
- Full-dataset adaptation: MPT establishes state-of-the-art parameter-efficient results on GLUE and SuperGLUE, improving over vanilla PT by 13% and 16%, respectively.It also outperforms full finetuning on both benchmarks while tuning 0.035% as many task-specific parameters.
- Full-dataset adaptation: MPT improves PT by +2.8% on MRQA and +13.5% on Others while adding only 0.01% more task-specific parameters.On WinoGrande, Yelp, SciTail, and PAWS, MPT reaches 85.5% average accuracy versus BitFit’s 84.7% while updating 10× fewer task-specific parameters.
- Model scaling: MPT remains effective across T5 model sizes from 60M to 770M parameters while achieving the best parameter efficiency.The scaling study compares MPT with full finetuning, Adapters, PT, and ATTEMPT on SuperGLUE.
- Ablation studies: Prompt decomposition alone improves SuperGLUE by 3.5% over the baseline, while adding prompt distillation improves average performance by 1.1%.Combining decomposition and distillation yields the best average SuperGLUE performance of 74.1%.
- Ablation studies: Using 300 prompt vectors reaches 76.8% on SuperGLUE, but increasing length to 400 causes an absolute 1.8% accuracy drop.The authors suggest that the decline may result from overfitting.
- Ablation studies: Updating both shared and task-specific prompt components is important: updating only the shared component reaches 62.5%, versus 71.3% when updating only task-specific vectors.The two components use different learning rates during target adaptation.
- Ablation studies: Removing stochastic task sampling lowers SuperGLUE performance from 74.1% to 73.7%, showing a slight benefit from this multitask training strategy.The ablation evaluates the same MPT training setup without stochastic sampling within each mini-batch.
5 CONCLUSION
MPT learns a transferable prompt by distilling knowledge from multiple source tasks and decomposing each task prompt into shared and rank-one task-specific components. The shared prompt is then adapted efficiently to diverse downstream tasks, sometimes outperforming full finetuning with far fewer task-specific parameters.
- MPT distills knowledge from multiple source tasks to learn a single transferable prompt.
- MPT represents each task prompt as the Hadamard product of a shared prompt matrix and a rank-one task-specific matrix.
- The shared prompt is transferred and adapted to target tasks for parameter-efficient transfer learning across diverse NLP benchmarks.
- MPT sometimes outperforms full finetuning while tuning far fewer task-specific parameters.
A EXPERIMENTAL SETUP
The experiments train source prompts on a mixture of source tasks, distill encoder and decoder hidden states, and reuse the shared prompt for target adaptation. Target-specific initialization uses averaged source task-specific vectors, followed by training on small datasets.
- Source prompts are trained for 5 epochs using examples-proportional mixing and stochastic task sampling.
- Prompt distillation uses hidden-state loss from both the encoder and decoder of T5.
- Target adaptation reuses the shared prompt and initializes the target task-specific vector with averaged source task-specific vectors.
- Target adaptation trains for 20 epochs on small data.
- During target adaptation, the task-shared and task-specific components use learning rates of 0.3 and 0.4, respectively.
- When original test sets are unavailable, development sets are used for testing, with additional splits for small training sets.