PaperlayerView exact revision on arXiv

Source-linked AI summary

One Network, Many Masks: Towards More Parameter-Efficient Transfer Learning

Guangtao Zeng, Peiyuan Zhang, Wei Lu

arXiv:2305.17682v2cs.CLcs.AI

TL;DR

Fine-tuning language models across tasks and existing PETL methods can require substantial storage. PROPETL shares one PETL prototype across layers and tasks using binary masks, and the paper reports better performance than other PETL methods with significantly fewer parameters.

  • Problem

    Fine-tuning and existing PETL methods can require substantial storage when models are adapted across many tasks.

    Source: §1 · p. 1

  • Method

    PROPETL shares a single prototype PETL module across layers and tasks, using binary masks to select different subnetworks.

    Source: §1 · p. 2, §3.3 · p. 4

  • Results

    PROPETL achieves better performance than other PETL methods while using significantly fewer parameters.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors conclude that prototype sharing and layer- and task-specific masks improve parameter efficiency while maintaining performance comparable to conventional PETL methods.

    Source: §7 · p. 9

    Training PROPETL requires underlying 32-bit mask scores, so it uses slightly more training memory than existing PETL methods and can take longer to converge than full fine-tuning.

    Source: Limitations · p. 9

Abstract

from arXiv · show

Fine-tuning pre-trained language models for multiple tasks tends to be expensive in terms of storage. To mitigate this, parameter-efficient transfer learning (PETL) methods have been proposed to address this issue, but they still require a significant number of parameters and storage when being applied to broader ranges of tasks. To achieve even greater storage reduction, we propose PROPETL, a novel method that enables efficient sharing of a single PETL module which we call prototype network (e.g., adapter, LoRA, and prefix-tuning) across layers and tasks. We then learn binary masks to select different sub-networks from the shared prototype network and apply them as PETL modules into different layers. We find that the binary masks can determine crucial information from the network, which is often ignored in previous studies. Our work can also be seen as a type of pruning method, where we find that overparameterization also exists in the seemingly small PETL modules. We evaluate PROPETL on various downstream tasks and show that it can outperform other PETL methods with approximately 10% of the parameter storage required by the latter.

1 Introduction

Fine-tuning and conventional PETL methods can impose substantial storage costs across many tasks. PROPETL shares one prototype network across layers and tasks, using binary masks to select subnetworks; experiments report better performance with fewer parameters than conventional PETL methods.

1 Introduction

  • Conventional fine-tuning updates and saves the entire model, while PETL methods add parameters that remain substantial across a broader range of tasks.The paper notes that PETL modules are commonly applied at multiple layers.Source: §1 · p. 1
  • The paper argues that subnetwork structure, alongside parameters, contributes to modeling ability, motivating its use of masks in PETL.It cites work finding that an untrained model's subnetwork can perform well without parameter updates.Source: §1 · p. 2
  • Binary masks let PROPETL dispatch parameters from one shared prototype into different layer- and task-specific subnetworks.A connection pruned in one layer can be used in another with a different mask.Source: §1 · p. 2
  • PROPETL achieves better performance than other PETL methods while using significantly fewer parameters.The introduction also describes the method as dramatically reducing parameter storage.Source: §1 · p. 2
  • The authors frame PROPETL as an alternative view of pruning and sharing, with masks deciding when to discard or share parameters.They hope this perspective will inspire further exploration.Source: §1 · p. 2

2 Related Work

The related work situates PROPETL among parameter-efficient transfer learning, pruning, and multi-task learning. It highlights the storage costs of per-layer and per-task PETL parameters and the potential value of sharing and subnetwork structure.

2 Related Work

  • The paper surveys parameter-efficient transfer learning, pruning methods, and multi-task learning as three related fields.These areas provide the context for combining parameter sharing with pruning.Source: §2 · p. 2

2.1 Parameter-Efficient Transfer Learning

  • Adapter-tuning, prefix-tuning, and LoRA add trainable parameters, but their storage grows with the number of layers and tasks.The paper proposes sharing these additional parameters across layers and tasks, drawing inspiration from ALBERT (Lan et al., 2020a).Source: §2.1 · p. 2

2.2 Pruning Methods

  • Prior PETL-and-pruning methods discard redundant parameters but do not share retained ones, limiting sparse subnetworks without accuracy compromise.Figure 2 depicts PROPETL deriving subnetworks by pruning connections from a prototype network.Source: §2.2 · p. 3, Figure 2 · p. 3

2.3 Multi-Task Learning

  • Multi-task learning trains one model across tasks, but can face catastrophic forgetting and imbalance between high- and low-resource tasks.Related approaches include task-specific adapters and Hyperformer, which generates adapter weights from task embeddings.Source: §2.3 · p. 3

3 Methods

PROPETL freezes the pre-trained language model and shares one prototype PETL network across layers, using learned masks to select layer- and task-specific subnetworks. The method measures storage in bits and extends mask sharing across tasks.

3.1 Preliminaries

  • PETL freezes the pre-trained language model and maximizes label likelihood by updating only added parameters; Bit-Level Storage accounts for parameter bit lengths.The paper groups parameters by bit length to measure storage.Source: §3.1 · pp. 3–4

3.2 Shared Prototype Network

  • A single prototype network is shared across layers, while distinct binary masks select subnetworks to preserve layer-specific capacity.The shared prototype reduces the parameter count from nL to L; each layer’s subnet is formed by element-wise masking.Source: §3.3 · p. 4, §3.2 · p. 4

3.3 Masked Sub-Networks

  • Layer-specific masks are learned from floating-point scores by top-k% thresholding, with straight-through gradients, while the prototype network is jointly optimized.After training, PROPETL discards the scores and retains the shared prototype and binary masks; its stated storage reduction is from around pL to p(1 + L/32).Source: §3.3 · p. 5

3.4 Hybrid Masks for Multi-Task Learning

  • For multi-task learning, layer masks and task masks are combined with a logical OR to form each task’s layer-specific hybrid mask.For T tasks and N layers, the method uses one PROPETL module, N layer masks, and T task masks.Source: §3.4 · p. 5

4 Experimental Setup

Experiments cover language understanding, summarization, and translation, using RoBERTaBASE or T5BASE depending on the setting. Evaluation uses task-specific metrics and different run counts for GLUE versus generation.

Datasets

  • The evaluation spans GLUE language-understanding tasks, XSum summarization, and WMT16 Ro-En translation.The reported generation metrics are ROUGE-2 on XSum and BLEU on Ro-En.Source: §4 · pp. 5–6

Backbones

  • RoBERTaBASE is used for single-task GLUE, while T5BASE is used for generation and multi-task learning; layer norms remain frozen.Only PROPETL and the classification head are tuned for GLUE, while only PROPETL is tuned for generation and multi-task learning.Source: §4 · p. 5

PETL Modules and PROPETL

  • The setup uses Pfeiffer adapters with bottleneck dimension 64, prefix length 64, and LoRA bottleneck dimension and scaling factor 32; PROPETL uses α=48 and sparsity ratio 0.5.For multi-task learning, layer and task mask sparsity ratios are each set to 0.3 to target an expected hybrid-mask ratio around 0.5.Source: §4 · pp. 5–6

Evaluation

  • GLUE results are means over three random runs, while XSum and Ro-En results use one run because generation training has high overhead.For GLUE, the evaluation procedure uses validation and test splits that vary by task sample count.Source: §4 · p. 6

5 Results

Across multi-task GLUE and single-task benchmarks, PROPETL generally matches or improves on comparison methods with less bit-level storage, though results vary by task and variant.

5.1 Single-Task Learning

  • On single-task GLUE, PROPETLAdapter raises the adapter average score by 0.95 and PROPETLLoRA improves the LoRA score by 0.41, while PROPETL uses 1/9 the inference bit-level storage (0.11% v.s. 0.95%).PROPETLAdapter scores 86.60 v.s. 86.16 for the fully fine-tuned model; PROPETLPrefix does not outperform prefix-tuning.Source: §5.1 · p. 6
  • On XSum, PROPETLAdapter matches regular adapter performance with less bit-level storage and is competitive with fully fine-tuned T5 above 1.6% storage, but neither adapter reaches that model on Ro-En.The authors suggest Ro-En may require additional tunable parameters because translation knowledge may be underrepresented in T5 pre-training; they note this challenge also occurs with other PETL methods.Source: §5.1 · pp. 6–7

5.2 Multi-Task Learning

  • On single-task T5BASE results, PROPETLAdapter with bottleneck dimension 64 surpasses fully fine-tuned T5; versus Hyperformer++, it uses 26× less task-specific bit-level storage (0.011% vs. 0.29% per task).PROP­ETLAdapter can reach fine-tuned performance with as few as 0.002% task-specific bit-level storage.Source: §5.2 · p. 7
  • At bottleneck dimension 6, PROPETLAdapter maintains performance comparable to fully fine-tuned T5 (85.32 v.s. 85.47) with 0.001% bit-level storage per task, while adapter tuning falls from 85.48 to 83.94.The authors report that T5 3B experiments align with the T5-base conclusions.Source: §5.2 · p. 7, Table 3 · p. 7

6 Discussion

Across XSum, Ro-En, and GLUE, PROPETLAdapter reaches fully fine-tuned performance earlier and performs better than adapters at fixed storage. Ablations and sparsity tests show the value of combining sharing with masking and choosing a balance between subnetwork size and distinctiveness.

How does PROPETL Scale Differently to Adapter?

  • At fixed task-specific storage, PROPETLAdapter performs better than adapter and reaches fully fine-tuned performance earlier across the reported scaling curves.The comparison covers XSum, Ro-En, and GLUE; performance gradually converges near the fully fine-tuned level as storage increases.Source: Figure 3 · p. 8, §6 · p. 8

Which is More Important, Sharing or Masking?

  • Neither masking without sharing nor sharing without masking matches PROPETL, while random masks yield the poorest performance.The authors attribute the advantage to masks adding structural information and sharing expanding each subnetwork at a fixed number of fine-tunable parameters.Source: §6 · p. 8, Table 4 · p. 9

How is PROPETL Conceptually Related to PETL?

  • PROPETL is framed as PETL-on-PETL: binary masks prune different structures from a prototype PETL module shared across layers and tasks, making task-specific parameters an order of magnitude smaller than conventional PETL modules.The masks function like PETL modules applied to the shared prototype network.Source: §6 · p. 9

How do Sub-Networks’ Size and Structure Affect Each Other?

  • PROPETL performs best at sparsity ratio k% = 0.5; performance declines as k% increases beyond 0.5, and unpruned sharing at k% = 1.0 performs worst among the tested sparsity levels.The authors suggest that denser subnetworks use more parameters up to 0.5, while further overlap reduces layer-specific structural information.Source: §6 · pp. 8–9, Figure 4 · p. 9

7 Conclusion and Future Work

The paper concludes that PROPETL shares prototype PETL modules across layers and tasks while maintaining binary masks for each layer and task. Experiments show comparable performance to fully fine-tuned models and conventional PETL methods with a much smaller fraction of storage.

Conclusion and Future Work

  • Experiments show PROPETL achieves comparable performance to fully fine-tuned models and conventional PETL methods with a much smaller fraction of storage.Future work proposed by the authors includes studying mask interpretability and applying the method to language-model pre-training to reduce overall parameters.Source: §7 · p. 9

Limitations

Although PROPETL’s binary masks reduce storage, training still requires their underlying 32-bit scores and uses slightly more memory than existing PETL methods. Fine-tuning takes similar time to conventional PETL modules and normally takes longer to converge than full fine-tuning.

Limitations

  • Training requires 32-bit scores for each mask, so PROPETL uses slightly more training memory than existing PETL methods.The binary masks themselves require significantly less storage than other PETL networks.Source: Limitations · p. 9
  • PROPETL fine-tuning takes similar training time to conventional PETL modules.Source: Limitations · p. 9
  • PROPETL normally takes longer to converge than a fully fine-tuned model.Source: Limitations · p. 9

A Implementation Details of the Three Variants of PROPETL

The appendix describes how PROPETL applies binary masks to adapter, LoRA, and prefix modules, and gives storage calculations for each variant. It also reports that masking helps adapter and LoRA but not reparameterized prefix tuning.

A.1 PROPETLAdapter

  • Adapter masks prune the down-sampling and up-sampling weights, while the adapter itself uses those projections with an intervening non-linear layer.The original Houlsby adapter is inserted after attention and feed-forward layers; the Pfeiffer variant is inserted only after feed-forward layers.Source: §A.1 · pp. 11–12

A.2 PROPETLLoRA

  • LoRA uses down- and up-sampling projections without a non-linear layer, and PROPETL prunes its network with binary masks.LoRA is applied to query and value representations; PROPETLLoRA uses a larger scaling factor α than LoRA to remedy reduced output feature norm.Source: §A.2 · p. 12

A.3 PROPETLPrefix

  • Masking reparameterized prefixes underperforms: unlike adapter and LoRA masks, these masks operate on output activations, and the authors report that they fail to locate suitable structures.The authors report that adapter and LoRA improve when 50% of parameters are pruned, while prefix performance drops.Source: §A.3 · p. 12, Table 5 · p. 12

B.1 Overview

  • PROP​ETL storage calculations count a shared 32-bit prototype network plus binary masks, whose storage is approximately 1/32 of a 32-bit PETL module.The calculation assumes one shared PETL network containing p BLS and a model with L layers; the specific module changes the formula.Source: §B.1 · p. 12, Table 6 · p. 12

B.2 PROPETLAdapter

  • For PROPETLAdapter, storage is 32 · (2 · bn · d + bn + d) for the prototype plus 2 · bn · d · L for masks.Bias terms are not masked, and the shared-network calculation does not apply the pruning ratio k%.Source: §B.2 · p. 13

B.3 PROPETLLoRA

  • For PROPETLLoRA, storage is 32 · 4 · bn · d for the prototype plus 4 · bn · d · L for masks.LoRA has no bias term; under only-mask settings, storage is 32 · 4 · k% · bn · d · L.Source: §B.3 · p. 13

B.4 PROPETLPrefix

  • For PROPETLPrefix, storage is 32 · 2 · l · d for the prototype plus 2 · l · d · L for masks; under only-mask settings it is 2 · k% · l · d · L.Source: §B.4 · p. 13

C Experimental Details

C.1 Datasets

This section describes the evaluation datasets and implementation settings, then reports that OR-combining layer and task masks performs best among the tested mask-combination methods.

C.1 Datasets

  • GLUE covers sentence-level understanding tasks, XSum tests single-document abstractive summarization, and WMT16 Ro-En evaluates Romanian-to-English translation.The XSum input averages 431.07 tokens and the target is a short, single-sentence summary.Source: §C.1 · p. 13

C.2 Implementation Details

  • RoBERTa experiments use a PETL-module learning rate of 1e-4, batch size 128, and weight decay 0.1; GLUE runs for 10 epochs on large tasks and 20 on small tasks.For T5, the authors do not perform an exhaustive hyperparameter search and use the same hyperparameters for full fine-tuning, adapter tuning, and PROPETL.Source: §C.2 · p. 14, Single and Multi-Task Learning on T5BASE · p. 14

Single and Multi-Task Learning on T5BASE

D.1 Choice of Mask Combining Methods

  • OR-combining the layer and task masks achieves the best performance among the tested methods on T5 multi-task GLUE.The authors explain that this includes parameters carrying both layer and task information in the forward pass.Source: §D.1 · p. 14, Table 11 · p. 14

D.2 Choice of Sharing and Masking

The ablation compares sharing and masking settings at similar storage levels. PROPETL matches a higher-storage mask-only setup on GLUE, while on Ro-En the mask-only adapter improves BLEU at substantially higher storage cost.

D.2 Choice of Sharing and Masking

  • On GLUE, PROPETL matches the mask-only setup using 0.11% storage versus 0.49%, despite the latter requiring more than 4 times the bit-level storage.The compared settings use bottleneck dimensions and sparsity ratios selected to provide similar storage levels, with an additional mask-only setup using the same values as PROPETL.Source: §D.2 · pp. 14–15
  • On Ro-En, masking adapter modules alone while keeping k% and bn unchanged improves performance by 0.65, at around 7× more parameter storage.The comparison concerns the more challenging Ro-En dataset.Source: §D.2 · p. 15, Table 15 · p. 15

E Additional Results

Additional experimental results are provided in table form for results from Figure 3.

E Additional Results

Related papers

Loading 2305.17682v2…