PaperlayerView exact revision on arXiv

Source-linked AI summary

Prefix-Tuning: Optimizing Continuous Prompts for Generation

Xiang Lisa Li, Percy Liang

arXiv:2101.00190v1cs.CL

TL;DR

Storing a separately fine-tuned language model for every task is costly because fine-tuning updates all parameters. Prefix-tuning instead trains a small continuous prefix while freezing the language model, maintaining comparable full-data performance and outperforming fine-tuning in low-data and extrapolation settings.

  • Problem

    Fine-tuning downstream tasks requires updating and storing all parameters of large pretrained language models, making task-specific deployment expensive.

  • Method

    Prefix-tuning prepends trainable continuous task-specific vectors that subsequent tokens can attend to as virtual tokens while the language model remains frozen.

  • Results

    Learning 1000x fewer parameters preserves comparable full-data performance and outperforms fine-tuning in low-data and extrapolation settings across table-to-text generation and summarization.

  • Takeaways & Limitations

    Prefix-tuning offers a parameter-efficient alternative that can support multiple natural language generation tasks with one unchanged language model.

  • Takeaways & Limitations

    The reason for prefix-tuning’s gains in extrapolation remains an open question.

Abstract

from arXiv · show

Fine-tuning is the de facto way to leverage large pretrained language models to perform downstream tasks. However, it modifies all the language model parameters and therefore necessitates storing a full copy for each task. In this paper, we propose prefix-tuning, a lightweight alternative to fine-tuning for natural language generation tasks, which keeps language model parameters frozen, but optimizes a small continuous task-specific vector (called the prefix). Prefix-tuning draws inspiration from prompting, allowing subsequent tokens to attend to this prefix as if it were "virtual tokens". We apply prefix-tuning to GPT-2 for table-to-text generation and to BART for summarization. We find that by learning only 0.1\% of the parameters, prefix-tuning obtains comparable performance in the full data setting, outperforms fine-tuning in low-data settings, and extrapolates better to examples with topics unseen during training.

1 Introduction

Prefix-tuning addresses the storage cost of task-specific fine-tuning by freezing the pretrained language model and learning a small continuous prefix that guides generation. Across table-to-text and summarization, it uses substantially less storage while retaining comparable full-data performance and improving average low-data performance.

  • Motivation: Fine-tuning requires updating and storing all language-model parameters, making separate task-specific model copies potentially prohibitively expensive.This storage burden arises from the large size of current pretrained language models.
  • Method: Prefix-tuning freezes Transformer parameters and optimizes continuous task-specific vectors that subsequent tokens can attend to as virtual tokens.Unlike prompting, these prefixes consist entirely of free parameters rather than real tokens.
  • Efficiency and modularity: Prefix-tuning stores only one large Transformer copy plus a learned prefix for each task, yielding a small additional-task overhead and modular support for multiple tasks.For table-to-text, the task-specific prefix contains 250K parameters; separate prefixes can also be trained for different users.
  • Results: 1000x fewer parameters are stored by prefix-tuning than by fine-tuning.The comparison is reported for the evaluated table-to-text and abstractive summarization tasks.
  • Results: On full datasets, prefix-tuning and fine-tuning are comparable for table-to-text, while prefix-tuning has a small summarization degradation.In low-data settings, prefix-tuning on average outperforms fine-tuning on both tasks.

2 Related Work

Prior work primarily adapts pretrained language models through full fine-tuning, lightweight trainable modules, or prompting. Prefix-tuning differs by optimizing continuous, task-specific prefixes for language generation rather than discrete prompts or instance-specific vectors.

  • Fine-tuning for natural language generation: Fine-tuning pretrained language models is the prevailing approach for natural language generation, including table-to-text, summarization, translation, and dialogue generation.Examples include T5 for table-to-text and BERT or BART for summarization.
  • Lightweight fine-tuning: Lightweight fine-tuning freezes most pretrained parameters and adds small trainable modules, with architecture and parameter-subset selection as central challenges.Existing approaches include training binary masks that ablate selected model weights.
  • Prompting: Prompting prepends instructions or examples to inputs, but bounded Transformer context limits how much training data in-context learning can exploit.GPT-3 uses manually designed prompts for task adaptation under a 2048-token context limit.
  • Prompting: Unlike discrete-trigger methods such as AutoPrompt, prefix-tuning optimizes continuous prefixes and targets language generation tasks.AutoPrompt searches for discrete trigger-word sequences for masked language models.
  • Continuous vectors: Unlike prior sentence-reconstruction work that learns input-specific continuous vectors, prefix-tuning learns one task-specific prefix applicable to all instances of a task.This distinction enables its application to natural language generation tasks beyond sentence reconstruction.

3 Problem Statement

The paper formulates conditional generation as predicting an output sequence y from context x, focusing on table-to-text generation and summarization. It defines autoregressive Transformer-based language-modeling notation and contrasts decoder-only and encoder-decoder formulations before introducing fine-tuning.

  • Task formulation: Conditional generation maps context x to output sequence y, with table-to-text and summarization as the two target tasks.For table-to-text, x is a linearized data table and y is a textual description; for summarization, x is an article and y is a short summary.
  • Autoregressive model: The decoder-only formulation uses an autoregressive language model pφ(y | x) based on the Transformer architecture.The concatenated sequence z = [x; y] is indexed into context and output positions, while activations depend on the current token and past left-context activations.
  • Autoregressive model: At each time step, the model concatenates activations from all Transformer layers into hi ∈ R^d and uses the last layer to predict the next-token distribution.The next-token distribution is computed with a softmax over vocabulary logits from the final-layer activation.
  • Fine-tuning: Fine-tuning initializes the pretrained parameters φ and updates them using a log-likelihood objective.Under this framework, pφ is a trainable language-model distribution.

4 Prefix-Tuning

Prefix-tuning is presented as an alternative to fine-tuning for conditional generation, using trainable prefix parameters while keeping language-model parameters fixed. The prefix steers subsequent activations through left-context influence, and reparameterization improves optimization stability.

  • Intuition: Continuous prompt embeddings provide a more expressive alternative to discrete-token prompts by propagating their effects through Transformer layers and subsequent tokens.The approach is motivated by prompting intuition: an appropriate context can steer generation without changing model parameters.
  • Method: Prefix-tuning prepends a trainable prefix to autoregressive or encoder-decoder inputs while preserving the model’s conditional-generation objective.For autoregressive models, z = [PREFIX; x; y]; for encoder-decoder models, z = [PREFIX; x; PREFIX′; y].
  • Method: The language-model parameters φ remain fixed, while the prefix parameters θ are the only trainable parameters.Prefix parameters are stored in a trainable matrix Pθ with dimensions |Pidx| × dim(hi).
  • Method: Prefix activations affect all activations to their right because they remain in the left context of subsequent positions.This lets the trainable prefix influence activations even when those activations are not themselves prefix positions.
  • Optimization: Directly optimizing Pθ is unstable and slightly reduces performance, so prefix-tuning reparameterizes it through a large feedforward network.Preliminary experiments found direct optimization sensitive to learning rate and initialization; after training, the reparameterization parameters can be discarded.

5 Experimental Setup

The experiments evaluate prefix-tuning on three table-to-text datasets and a news summarization dataset, using standard task-specific metrics. Comparisons include fine-tuning, partial fine-tuning, adapter-tuning, and reported state-of-the-art systems.

  • Datasets: The table-to-text experiments use E2E, WebNLG, and DART, ordered by increasing complexity and size.E2E covers one restaurant-review domain, WebNLG covers 14 domains, and DART is open-domain with Wikipedia tables.
  • Datasets: E2E contains approximately 50K examples with eight fields and 22.9-token average outputs, evaluated using BLEU, NIST, METEOR, ROUGE-L, and CIDEr.The official evaluation script is used and multiple test references may correspond to one source table.
  • Datasets: DART contains 82K open-domain examples with 21.6-token average outputs and reports BLEU, METEOR, TER, MoverScore, BERTScore, and BLEURT.Its examples combine data from WikiSQL, WikiTableQuestions, E2E, and WebNLG through manual or automated conversion.
  • Baselines: For table-to-text, prefix-tuning is compared with fine-tuning, top-two-layer fine-tuning, adapter-tuning, and reported state-of-the-art results; summarization uses fine-tuned BART as a baseline.The reported systems include Shen et al. (2019) on E2E and Kale (2020) on WebNLG.
  • Implementation: The experiments use GPT-2Medium and GPT-2Large for linearized table inputs and BARTLarge for summarization, with source articles truncated to 512 BPE tokens.The implementation uses Hugging Face Transformers, AdamW, and a linear learning-rate scheduler; hyperparameters include epochs, batch size, learning rate, and prefix length.

6 Main Results

Prefix-tuning adapts GPT-2 and BART effectively while learning only a small task-specific prefix instead of updating the full language model. It is especially competitive in table-to-text, low-data, and unseen-topic settings.

  • Table-to-text generation: 4.1 BLEU average improvement over ADAPTER (0.1%) demonstrates prefix-tuning’s advantage under matched parameter budgets.Prefix-tuning also achieves results comparable or better than fine-tuning (100%) and adapter-tuning (3.0%).
  • Low-data settings: 2.9 BLEU average improvement over fine-tuning occurs in low-data regimes, although the gap narrows as dataset size increases.The low-data experiments use dataset sizes {50, 100, 200, 500}, five sampled datasets per size, and two training random seeds.
  • Extrapolation: Prefix-tuning achieves better extrapolation than fine-tuning under all metrics for both table-to-text and summarization.This advantage appears on XSUM and in the ‘U’ columns for WebNLG; adapter-tuning also shows comparable extrapolation performance.

7 Intrinsic Evaluation

Intrinsic evaluations show that prefix-tuning benefits from an intermediate prefix length, greater expressivity than embedding-only or discrete prompting, prefix placement at the sequence beginning, and real-word initialization, especially in low-data settings.

  • Prefix length: Performance increases with prefix length up to 200 for summarization and 10 for table-to-text, then slightly drops.Longer prefixes lower training loss beyond these thresholds but slightly worsen test performance, suggesting overfitting.
  • Prefix length: Longer prefixes have a negligible impact on inference speed because attention over the entire prefix is parallelized on GPUs.
  • Embedding-only tuning: Embedding-only tuning significantly underperforms full prefix-tuning because optimizing only word embeddings is insufficiently expressive.The expressive-power ordering is discrete prompting < embedding-only ablation < prefix-tuning.
  • Prefixing versus infixing: Infix-tuning slightly underperforms prefix-tuning because it can influence only the activations of y, whereas prefix-tuning can affect x and y.Prefix-tuning places trainable activations before x and y, while infix-tuning places them between x and y.
  • Prefix initialization: Real-word initialization significantly improves generation over random initialization in low-data settings, with task-relevant words performing slightly better than task-irrelevant words.Random initialization produces low performance with high variance, while real-word activations preserve the pretrained language model as much as possible.

8 Discussion

Prefix-tuning is especially useful for independently trained personalized models at large scale, while preserving the shared language model enables efficient batching across users. Freezing pretrained parameters is also associated with better extrapolation to unseen domains, although the cause remains unresolved.

  • Scalability and personalization: Millions of users can be treated as independent tasks, allowing prefix-tuning to scale personalized model training while preserving user-data separation.The discussion identifies user privacy as a setting requiring independently trained personalized models.
  • Scalability and personalization: Prefix-tuning enables batching queries from different users by prepending each personalized prefix while leaving the shared language-model computation unchanged.This makes mixed-user batching computationally efficient on a cloud GPU device.
  • Generalization: Prefix-tuning and adapter-tuning both show significant performance gains when extrapolating to domains unseen during training, but the reason remains an open question.Both methods preserve pretrained language-model parameters, which may support generalization beyond training domains.
  • Parameter efficiency: Prefix-tuning attains good generation performance by updating a very small prefix, echoing evidence that low-dimensional reparameterizations can match full-space fine-tuning.The related work attributes this broader finding to intrinsic-dimension results.

9 Conclusion

The paper proposes prefix-tuning as a lightweight alternative to fine-tuning for natural language generation, using a trainable continuous prefix. Despite learning 1000x fewer parameters, it maintains comparable full-data performance and outperforms fine-tuning in low-data and extrapolation settings.

  • 9 Conclusion: Prefix-tuning is proposed as a lightweight alternative to fine-tuning that prepends a trainable continuous prefix for NLG tasks.The method replaces full model updates with optimization of a task-specific continuous prefix.
  • 9 Conclusion: 1000x fewer parameters are learned than with fine-tuning while maintaining comparable performance in the full-data setting.This result demonstrates parameter efficiency without sacrificing full-data performance.
  • 9 Conclusion: Prefix-tuning outperforms fine-tuning in both low-data and extrapolation settings.The reported gains extend beyond full-data training to limited-data and extrapolation scenarios.

A Supplementary Material … A.4 Qualitative Examples for Extrapolation

The supplementary material documents training hyperparameters, extends low-data and initialization analyses, and provides qualitative WebNLG examples showing extrapolation failures on unseen categories. These results reinforce prefix-tuning’s low-data advantages and the importance of real-word initialization.

  • A.1 Hyperparameters: Table 5 reports the hyperparameters used to train the models in the experiment section.This supplementary table documents the experimental training configuration.
  • A.2 Additional Results for Low-data Settings: Figure 6 plots training size against generation metrics for prefix-tuning and fine-tuning, supplementing the low-data performance curves.The plots cover summarization with ROUGE-1, ROUGE-2, and ROUGE-L, and table-to-text with NIST, METEOR, and CIDEr.
  • A.3 Additional Results for the Initialization Experiment: Random initialization significantly underperforms initialization with real words, while task-relevant words attain slightly better generation scores than task-irrelevant words.Task-relevant examples include “summarization” and “table-to-text”; task-irrelevant examples include “elephant” and “banana.”
  • A.4 Qualitative Examples for Extrapolation: For unseen WebNLG categories, both prefix-tuning and fine-tuning tend to undergenerate or generate untruthfully.Undergeneration omits full table contents, while untruthful generation is inconsistent with table contents.
  • A.2 Additional Results for Low-data Settings: Prefix-tuning outperforms fine-tuning in low-data regimes while requiring many fewer parameters.Figure 6 reports this pattern across the plotted summarization and table-to-text generation metrics.
  • A.3 Additional Results for the Initialization Experiment: Figure 7 shows that initializing the prefix with activations of real words significantly outperforms random initialization with 100 training data.This comparison is made in a low-data setting.
  • A.4 Qualitative Examples for Extrapolation: Table 6 presents qualitative WebNLG examples from six unseen-category cases and two seen-category cases.The examples compare prefix-tuning and finetuning outputs against their source tables.
  • A.4 Qualitative Examples for Extrapolation: Prefix-tuning tends to undergenerate more often than generate untruthfully on unseen categories.The supplied qualitative-results passage contrasts this tendency with fine-tuning’s behavior.
Loading 2101.00190v1…