PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards a Unified View of Parameter-Efficient Transfer Learning

Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, Graham Neubig

arXiv:2110.04366v3cs.CLcs.LG

TL;DR

Adapting increasingly large pretrained language models by duplicating all parameters for every task is prohibitively expensive, while the ingredients shared by parameter-efficient methods remain poorly understood. The paper unifies these methods as hidden-state modifications, studies their design choices across four NLP task types, and transfers favorable elements to new variants. These variants achieve comparable results to full fine-tuning while tuning fewer parameters.

  • Problem

    The relationships and critical ingredients among parameter-efficient tuning methods are poorly understood, despite the cost of full fine-tuning for large models and many tasks.

  • Method

    The paper frames parameter-efficient methods as modifications to hidden representations of frozen pretrained models and compares them across shared design dimensions.

  • Results

    Experiments across machine translation, summarization, language understanding, and classification show that transferred design elements produce more effective parameter-efficient variants.

  • Takeaways & Limitations

    The unified framework identifies design choices and enables variants that match full fine-tuning performance with fewer tuned parameters.

  • Takeaways & Limitations

    The paper does not examine whether parameter-efficient fine-tuning changes bias or information leakage.

Abstract

from arXiv · show

Fine-tuning large pre-trained language models on downstream tasks has become the de-facto learning paradigm in NLP. However, conventional approaches fine-tune all the parameters of the pre-trained model, which becomes prohibitive as the model size and the number of tasks grow. Recent work has proposed a variety of parameter-efficient transfer learning methods that only fine-tune a small number of (extra) parameters to attain strong performance. While effective, the critical ingredients for success and the connections among the various methods are poorly understood. In this paper, we break down the design of state-of-the-art parameter-efficient transfer learning methods and present a unified framework that establishes connections between them. Specifically, we re-frame them as modifications to specific hidden states in pre-trained models, and define a set of design dimensions along which different methods vary, such as the function to compute the modification and the position to apply the modification. Through comprehensive empirical studies across machine translation, text summarization, language understanding, and text classification benchmarks, we utilize the unified view to identify important design choices in previous methods. Furthermore, our unified framework enables the transfer of design elements across different approaches, and as a result we are able to instantiate new parameter-efficient fine-tuning methods that tune less parameters than previous methods while being more effective, achieving comparable results to fine-tuning all parameters on all four tasks.

1 INTRODUCTION

Parameter-efficient tuning reduces the cost of adapting large pretrained language models, but the methods’ shared ingredients and relationships remain unclear. This paper unifies these methods and uses the framework to identify effective designs and construct stronger variants.

  • Motivation: Full fine-tuning requires a separate parameter copy for each task, becoming prohibitively expensive as pretrained language models grow larger.PLMs range from hundreds of millions to hundreds of billions or even trillions of parameters.
  • Prior approaches: Parameter-efficient methods train only a small number of added parameters while freezing most pretrained parameters, often using less than 1% of the original model parameters.Examples include adapters, prefix tuning, prompt tuning, and low-rank updates.
  • Research questions: The paper asks how parameter-efficient methods are connected, which design elements matter, and whether effective ingredients can transfer across approaches.These questions target the poorly understood sources of the methods’ effectiveness.
  • Unified framework: The unified framework represents methods as modifications to frozen-model hidden representations and decomposes them by dimensions such as modification function, insertion position, and integration.The framework also supports transferring design choices to create variants such as multi-head adapters.
  • Evaluation: Experiments span text summarization, machine translation, text classification, and general language understanding benchmarks to evaluate existing methods and proposed variants.The study examines both design choices and cross-approach transfers.

2 PRELIMINARIES

Transformers combine multi-head self-attention and feed-forward sublayers within stacked blocks, while parameter-efficient methods add trainable modules or updates around these components with the pretrained model frozen.

  • Transformer architecture: Transformer models consist of L stacked blocks, each containing multi-head self-attention and a fully connected feed-forward network.Residual connections and layer normalization follow these sublayers.
  • Transformer architecture: Multi-head attention applies attention in parallel across separately parameterized heads and concatenates their outputs.Each head operates in a lower-dimensional space, typically with d_h = d/N_h.
  • Parameter-efficient methods: Unless otherwise specified, these methods tune only added parameters while keeping the pretrained language model frozen.Other approaches include tuning bias vectors or learning sparse parameter-update vectors.
  • Parameter-efficient methods: Adapters insert bottleneck modules that down-project, apply a nonlinear function, and up-project hidden states within transformer layers.The bottleneck dimension is r, and adapter placement can occur after attention and FFN sublayers or only after the FFN sublayer.
  • Parameter-efficient methods: Prefix tuning prepends trainable prefix vectors to attention keys and values at every layer, while prompt tuning prepends trainable vectors only to first-layer input embeddings.Prefix vectors are split across attention heads.
  • Parameter-efficient methods: LoRA represents updates to pretrained weight matrices with trainable low-rank factors and applies them to query and value projections in multi-head attention.Its update uses W + ΔW = W + W_downW_up with rank controlled by r.

3 BRIDGING THE GAP – A UNIFIED VIEW

The paper unifies parameter-efficient tuning methods by viewing them as modifications to hidden representations of frozen pretrained language models. This view exposes shared design dimensions and supports transferring design elements to create new variants.

  • 3.1 A CLOSER LOOK AT PREFIX TUNING: Prefix tuning can be rewritten as a position-wise modification of the original attention output through linear interpolation.The modification combines standard attention with a prefix-derived vector using the scalar λ(x).
  • 3.1 A CLOSER LOOK AT PREFIX TUNING: Prefix tuning and adapters have similar functional forms, but prefix tuning performs weighted addition while adapters use unweighted addition.The two methods differ in how their modification vector is composed with the original representation.
  • 3.1 A CLOSER LOOK AT PREFIX TUNING: Prefix tuning’s prefix count l plays a role analogous to the adapter bottleneck dimension r by limiting the rank of the modification vector.The modification is a linear combination of the same l or fewer basis vectors for any input x.
  • 3.2 THE UNIFIED FRAMEWORK: The unified framework represents adapters, prefix tuning, and LoRA as learning a modification vector ∆h applied to selected hidden representations.Methods vary according to the function computing ∆h, the representation modified, the insertion form, and the composition function.
  • 3.2 THE UNIFIED FRAMEWORK: The framework identifies common functional structure across methods: projection down, a nonlinear transformation, and projection up, with LoRA using an identity transformation.It also distinguishes sequential adapter insertion from the parallel insertion equivalent of prefix tuning and LoRA.
  • 3.3 TRANSFERRING DESIGN ELEMENTS: Transferring design elements across methods yields Parallel Adapter, Multi-head Parallel Adapter, and Scaled Parallel Adapter variants.These variants transfer insertion, multi-head capacity, and LoRA-style composition choices into adapters.

4 EXPERIMENTS

Experiments evaluate parameter-efficient tuning across summarization, translation, language understanding, and classification, then analyze insertion form, modified representation, composition, and design integration. The results show that effective choices depend on task capacity: FFN modifications perform best at larger budgets, while multi-head attention modifications are strongest at 0.1% parameters.

  • 4.1 GENERAL SETUP: The study evaluates XSum, en-ro translation, MNLI, and SST2 using pretrained BART, mBART, and RoBERTa, reporting task metrics and tuned parameters relative to full fine-tuning.XSum uses ROUGE, en-ro uses BLEU, and MNLI/SST2 use accuracy; MNLI and SST2 results are medians over five random runs.
  • 4.2 THE RESULTS OF EXISTING METHODS: Fewer than 1% tuned parameters can achieve competitive MNLI and SST2 performance, but substantial gaps from full fine-tuning remain on XSum and en-ro even above 10%.The authors caution that results from simpler or encoder-only benchmarks may not generalize to diverse benchmarks.
  • 4.3 WHICH INSERTION FORM?: Parallel adapters outperform sequential adapters, with PA (ffn) exceeding SA (ffn) by 1.7 R-2 points on XSum and 0.8 BLEU points on en-ro.Prefix tuning, which uses parallel insertion, also outperforms attention sequential adapters.
  • 4.4 WHICH MODIFIED REPRESENTATION – ATTENTION OR FFN?: FFN modification outperforms attention modification across cases, and LoRA (ffn) improves LoRA (attn) by 1 R-2 point on XSum.The only stated exception is ffn-PA with 2.4% parameters; the authors suggest FFNs utilize added capacity more effectively.
  • 4.4 WHICH MODIFIED REPRESENTATION – ATTENTION OR FFN?: At 0.1% parameters, multi-head prefix tuning and MH PA (attn) outperform other methods by at least 1.6 BLEU points, while reducing prefix length from 200 to 30 costs 0.4 BLEU.PA (attn) loses 1.9 points under the same reduction, and gating adds 0.3 points for prefix tuning.
  • 4.5 WHICH COMPOSITION FUNCTION?: Scaling improves additive composition: Scaled PA exceeds vanilla parallel adapters by 0.56 ROUGE-2 points on XSum, while LoRA’s advantage disappears when scaling is removed.The authors conclude that scaling is better than vanilla addition and readily transferable across methods.
  • 4.6 AN EFFECTIVE INTEGRATION BY TRANSFERRING FAVORABLE DESIGN ELEMENTS: MAM Adapter combines small-budget prefix attention with larger-budget scaled parallel-adapter FFN modification and matches full fine-tuning while using fewer parameters.The design transfers favorable elements identified in earlier experiments and achieves the best performance among the compared variants on both tasks.

5 DISCUSSION

The paper presents a unified framework for parameter-efficient tuning methods and uses transferred design elements to build a more effective model. The authors report that this model matches full fine-tuning and aim to guide future research.

  • 5 DISCUSSION: The unified framework connects several parameter-tuning methods and enables design elements to be transferred across approaches.The framework supports instantiating new variants rather than treating existing methods as unrelated techniques.
  • 5 DISCUSSION: The resulting model matches the performance of full fine-tuning while remaining within the parameter-efficient tuning framework.The authors present this as evidence that transferred techniques can improve parameter-efficient methods.

ETHICS STATEMENT

The paper acknowledges both broad ethical concerns surrounding language models and uncertainty about how parameter-efficient fine-tuning affects them. It also discusses environmental trade-offs from the proposed methods.

  • Language models raise concerns about biased or discriminative text, private-information leakage, and environmental impact.
  • The effects of parameter-efficient fine-tuning on bias and information leakage have not been examined in prior work.
  • The final MAM Adapter requires 100% - 150% of full fine-tuning’s training time, while inference time is roughly the same.The longer training time is attributed to the additional epochs typically needed for parameter-efficient tuning.

REPRODUCIBILITY STATEMENT

The paper reports reproducibility resources, dataset and model implementation details, and constraints used to compare bottleneck dimensions across methods.

  • Complete experimental settings, including batch sizes, optimizers, and learning rates, are detailed in Appendix A, and source code is public.The authors state these resources should be sufficient to reproduce the results.
  • The experiments use BARTLARGE and mBARTLARGE for summarization and translation, and RoBERTaBASE for MNLI and SST2.mBARTLARGE is pretrained on 25 languages.
  • Bottleneck dimension 1024 is tested only for FFN modification because prefix tuning at that dimension exceeds 48GB of GPU memory.Attention modification is capped at 512 for a relatively fair comparison with prefix tuning.

A.2 TRAINING AND EVALUATION

Training uses Adam with polynomial learning-rate decay, while task-specific warmup and beam-search decoding settings govern optimization and generation.

  • All tasks use Adam and a polynomial learning-rate scheduler that linearly decays the learning rate during training.
  • Warmup is omitted for machine translation and summarization, while classification warms up linearly for the first 6% of training steps.
  • Beam search uses 6 beams for machine translation and 5 for summarization, with task-specific minimum and maximum generation lengths.The generation-length ranges are (10, 60) for summarization and (1, 200) for machine translation.

A.3 OTHER EXPERIMENTAL DETAILS

Additional implementation details cover prefix-tuning reparameterization, LoRA and adapter initialization, and the training hyperparameters summarized in Table 8.

  • Prefix tuning reparameterizes prefix vectors with an MLP comprising a small embedding matrix and a large feedforward network.The shared parameters across layers are described as conducive for learning.
  • LoRA initializes W_down with Kaiming uniform values and W_up with zeros, whereas adapters use BERT initialization.The same LoRA initialization is also used for scaled PA because it was found beneficial.
  • Table 8 summarizes training hyperparameters for parameter-efficient tuning methods across the four tasks.The table defines lr as learning rate and ls as label smoothing.

B COMPUTATION OF TUNABLE PARAMETERS

This section defines how tunable parameters are counted for parameter-efficient methods across encoder-decoder and encoder-only pretrained models. It combines per-sub-layer overheads across layers, with method-specific parameterizations for adapters, prefix tuning, and LoRA.

  • Tunable-parameter counts depend on where a module is inserted and how it is parameterized.
  • For encoder-decoder models, one encoder layer and one decoder layer provide the per-layer parameter overhead used in the computation.
  • The parameter count for attention and FFN components is combined as |Θ| = |Θ|attn + |Θ|ffn when applicable.
  • LoRA approximates each weight-matrix update with Wdown and Wup, so it typically uses more parameters than other methods at the same r.
  • Prompt tuning prepends l tunable vectors at the input layer and uses l × d parameters.
  • Table 11 illustrates tunable-parameter counts for prefix tuning, adapter variants, and LoRA with BART/MBART models using L = 12.

C FULL RESULTS ON DIFFERENT BOTTLENECK DIMENSIONS

This section reports test-set performance for abstractive summarization and machine translation. The listed benchmarks are XSum and WMT EN-RO.

  • The table reports performance on the test sets of abstractive summarization and WMT EN-RO translation.
  • XSum is the benchmark identified for abstractive summarization.
  • WMT EN-RO is the benchmark identified for translation.
Loading 2110.04366v3…