PaperlayerView exact revision on arXiv

Source-linked AI summary

WARP: Word-level Adversarial ReProgramming

Karen Hambardzumyan, Hrant Khachatrian, Jonathan May

arXiv:2101.00121v2cs.CL

TL;DR

The paper addresses how to transfer pretrained language-model capabilities across tasks without storing or training large task-specific models. WARP learns input-level prompt and verbalizer embeddings, achieving strong GLUE and SuperGLUE results with very few trainable parameters. Its parameter savings can support serving many user-specific sentence-classification models from one pretrained model.

  • Problem

    Existing transfer methods either store task-specific model parameters or rely on very large language models for prompt-based few-shot transfer.

  • Method

    WARP learns task-specific prompt and verbalizer embeddings that are appended around the input while the pretrained language model remains frozen.

  • Results

    WARP achieves an 81.6 test score on GLUE with up to 25K trainable parameters per task and outperforms GPT-3 on two SuperGLUE tasks using 32 examples.

  • Takeaways & Limitations

    WARP can reduce task-specific storage and serve many user-specific models by storing one pretrained language model plus small task-specific parameters.

  • Takeaways & Limitations

    Different random seeds can produce different results because training-example order affects optimization, even when parameters are manually initialized.

Abstract

from arXiv · show

Transfer learning from pretrained language models recently became the dominant approach for solving many NLP tasks. A common approach to transfer learning for multiple tasks that maximize parameter sharing trains one or more task-specific layers on top of the language model. In this paper, we present an alternative approach based on adversarial reprogramming, which extends earlier work on automatic prompt generation. Adversarial reprogramming attempts to learn task-specific word embeddings that, when concatenated to the input text, instruct the language model to solve the specified task. Using up to 25K trainable parameters per task, this approach outperforms all existing methods with up to 25M trainable parameters on the public leaderboard of the GLUE benchmark. Our method, initialized with task-specific human-readable prompts, also works in a few-shot setting, outperforming GPT-3 on two SuperGLUE tasks with just 32 training samples.

1 Introduction

WARP introduces optimized input prompts as a parameter-efficient alternative to task-specific heads, fine-tuning, adapters, and massive few-shot prompting. It achieves strong benchmark and few-shot results with substantially fewer trainable parameters.

  • 1 Introduction: Adapters reduce task-specific parameters by adding new weights at every pretrained-model layer while freezing the original parameters.
  • 1 Introduction: Prior task-description and priming methods demonstrate prompt-based transfer but require language models with 1.5B and 175B parameters, respectively.
  • 1 Introduction: WARP learns task-specific prompts that reprogram a pretrained language model for downstream NLP tasks.The method is inspired by adversarial reprogramming, which changes a model’s task through input-level perturbations.
  • 1 Introduction: 25K trainable parameters per task yield an 81.6 test score on the GLUE Leaderboard, surpassing submissions using up to three orders of magnitude more trainable parameters.
  • 1 Introduction: WARP also supports few-shot learning with manually initialized prompts and outperforms GPT-3 on two SuperGLUE tasks using 32 examples.

2 Related Work

Related work spans parameter-efficient adaptation, task reformulation, automatic prompt search, and adversarial reprogramming. WARP combines these directions by optimizing input-level prompts in continuous embedding space while retaining human-informed initialization.

  • 2 Related Work: Adapters add task-specific parameters throughout the Transformer, whereas WARP learns task-specific prompts at the input level.
  • 2 Related Work: GPT-3 performs few-shot transfer through contextual input-output analogies without fine-tuning, but its examples are limited by context size and its model is massive.
  • 2 Related Work: PET reformulates tasks as Cloze problems and fine-tunes masked language models for few-shot learning without requiring huge models such as GPT-3.
  • 2 Related Work: Unlike PET, WARP finds prompts using training examples, while still benefiting from careful initialization based on task knowledge.
  • 2 Related Work: Adversarial reprogramming originally used trainable image padding to make ImageNet classifiers perform MNIST and CIFAR-10 classification.
  • 2 Related Work: Unlike AutoPrompt’s vocabulary-space search, WARP optimizes word embeddings continuously, providing more degrees of freedom for downstream tasks.

3 WARP

WARP optimizes prompt and verbalizer embeddings around input text so a frozen masked language model predicts task labels. Only these embeddings are trained and stored, while the pretrained model remains unchanged.

  • 3 WARP: WARP searches continuous prompt and verbalizer embedding spaces so a masked language model predicts the desired class token.The template inserts prompt embeddings into predefined positions, and the model output is scored against verbalizer tokens.
  • 3 WARP: The masked-language-model decoder is replaced by a linear layer formed from the verbalizer embeddings.
  • 3 WARP: Stochastic gradient descent inserts special prompt and mask tokens into task-specific templates before optimizing the downstream task objective.
  • 3 WARP: The cross-entropy objective compares masked-language-model outputs with verbalizer tokens assigned to the task classes.
  • 3 WARP: Only prompt and verbalizer embeddings are trainable; the language model body and original input-token embeddings remain untouched.
  • 3 WARP: WARP uses RoBERTa-large for GLUE and ALBERT-xxlarge-v2 for few-shot experiments, with Adam optimization and mixed-precision training.

4 Experiments on GLUE

WARP is evaluated across GLUE’s nine natural language understanding tasks using single-task training and task-specific prompt adaptations. It achieves strong leaderboard performance with very few trainable parameters, while prompt size and initialization affect results.

  • Task formulations: GLUE tasks are mainly sentence or sentence-pair classification problems, requiring few modifications for WARP adaptation.SST-2 uses a [MASK] after the sentence, with trainable prompt tokens both prepended and appended.
  • Test-set evaluation: WARP’s average GLUE performance exceeds models using up to three orders of magnitude more trainable parameters.The evaluation uses GLUE test-set results, with WNLI included in the averaged score although its results are not shown.
  • Task formulations: For sentence-pair tasks, prompt tokens may surround or separate the sentences, while [MASK] is placed between them.MNLI uses matched accuracy for validation and the same model for its mismatched version.
  • Task formulations: STS-B is handled as regression with a regression head, Mean Squares Error optimization, Pearson correlation validation, and score clipping to [1, 5].Unlike classification tasks, STS-B does not use verbalizer-token embeddings.
  • Prompt-size analysis: Reducing trainable prompt parameters lowers performance, but WARP remains functional.WARP0 uses no prompt parameters and obtains sentence representations from the masked-language-model head; this pooling is significantly better than the linear-classifier alternative.
  • Prompt initialization: Manual prompt initialization improves WARP8 on RTE, MRPC, and STS-B, indicating potential in the low-data regime.The shared initialization uses a human-readable prompt template and is especially beneficial on tasks with relatively few training examples.

5 Few-Shot Experiments

WARP is evaluated in a supervised 32-example setting on two FewGLUE tasks, using manually initialized prompts and model-selection procedures designed for limited data. It outperforms PET and GPT-3 while using far fewer trainable parameters, though iPET remains stronger.

  • 5.2 Initialization: Manual prompt initialization transfers task-specific knowledge into WARP and is intended to help when training data are limited.The few-shot prompts use PVP-derived prompt and verbalizer embeddings, with ALBERT as the pretrained model.
  • 5.1 Setup: 32 examples per task define the FewGLUE setting used to compare WARP with GPT-3, PET, and iPET.The experiments use a purely supervised setup and do not use the 20,000 additional unlabeled examples.
  • 5.3 Model Selection: Different random seeds can produce different results because training-example order varies across epochs.The ensemble of 20 WARP runs is used to reduce the effect of random seeds.
  • 5.4 Results: WARP outperforms PET and GPT-3 on both tasks but remains behind iPET.The comparison uses systems evaluated in a similar 32-example few-shot setup.

6 Discussion

The discussion examines learned prompt embeddings, comparisons with AutoPrompt, and WARP’s parameter-storage advantages for serving many classification tasks. WARP can improve with more data while retaining a compact task-specific footprint.

  • 6.1 Interpreting tokens learned by WARP: WARP’s learned prompt tokens are generally not meaningful in task context, whereas verbalizer embeddings can be more interpretable.Examples include MNLI’s contradiction embedding near “Unless” and SST-2’s class embeddings near “defective” and “important.”
  • 6.2 Comparison with AutoPrompt: With a few hundred SST-2 training samples or fewer, WARP and AutoPrompt do not differ significantly; WARP performs better with more samples.Figure 4 reports median performance over 10 runs, with minimum and maximum shown by error bars.
  • 6.2 Comparison with AutoPrompt: Manually initialized WARP performs very well with just 100 examples, while WARP succeeds on RTE and CB even without manual initialization.The manual-initialization result is compared with a manually designed prompt.
  • 6.3 Real-world applications: For N tasks with at most C classes, WARP stores M + NE(C + K) parameters, with K denoting trainable prompt tokens.This contrasts with storing at least MN parameters for separately fine-tuned models.
  • 6.3 Real-world applications: WARP can serve many user-specific models in parallel by storing one pretrained language model and adding little task-specific overhead.Inputs from different tasks can share a forward pass, supporting machine-learning-as-a-service systems.

7 Conclusion

WARP transfers knowledge by appending optimized input embeddings to pretrained language models. It achieves strong GLUE and few-shot SuperGLUE results with a small trainable footprint and can reduce storage for many classification tasks.

  • 7 Conclusion: WARP transfers knowledge by appending carefully optimized embeddings to the input text.The method is presented as an alternative to conventional task-specific layers and full-model fine-tuning.
  • 7 Conclusion: WARP outperforms existing methods with substantially more trainable parameters on GLUE and performs impressively in two 32-example SuperGLUE tasks.On sentiment analysis, its performance is comparable to fully fine-tuned language models.
  • 7 Conclusion: WARP can save substantial storage in software applications serving large numbers of sentence-classification tasks.The storage advantage follows from keeping task-specific information in compact input-level prompts.

A Hyperparameters

The experiments search learning rates, epoch counts, and prompt initialization strategies, then apply additional training and batching settings. [MASK]-based initialization performs better than random initialization.

  • A Hyperparameters: Prompt embeddings are initialized either from the [MASK] token or randomly using RoBERTa embedding statistics.The random option uses a normal distribution whose mean and variance come from RoBERTa’s word-embedding matrix.
  • A Hyperparameters: [MASK] initialization performs better than random initialization across the selected settings.The hyperparameter search took roughly four days on two Titan V GPUs.
  • A Hyperparameters: Training disables Transformer dropout, uses AdamW without weight decay, normalizes gradients to 1.0, and caps batches at 2,048 tokens.Sequences are truncated to 512 tokens and mixed precision is enabled.
  • A Hyperparameters: The best-performing model hyperparameters for each task are reported in Table 5.The table distinguishes [MASK]-initialized prompts from prompts initialized with an MNLI solution.

B Learned Tokens

The learned task prompts use multiple inserted segments and trainable verbalizer tokens, with initialization varying by task and related tasks inheriting MNLI-based prompts.

  • Prompt structure: Most tasks use three prompt segments around two input sentences, while single-sentence tasks concatenate the latter two segments.Each task also learns one verbalizer token per output class.
  • Prompt structure: Task-specific prompts include trainable verbalizer tokens, one for each output class.
  • Initialization: RTE, MRPC, and STS-B were initialized from pretrained MNLI models, producing prompts similar to MNLI’s.
  • Initialization: Other tasks were initialized with [MASK] tokens, and CoLA’s final model remained relatively close to its initialization.
  • Embedding interpretation: Table 6 identifies the closest words to prompt and verbalizer embeddings using cosine distance, with bold [MASK] positions marking prediction outputs.
Loading 2101.00121v2…