Source-linked AI summary
AdapterFusion: Non-Destructive Task Composition for Transfer Learning
Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, Iryna Gurevych
TL;DR
Sequential fine-tuning and multi-task learning face catastrophic forgetting and task-balancing or retraining constraints when sharing knowledge across tasks. AdapterFusion separates adapter-based knowledge extraction from knowledge composition, and consistently outperforms fully fine-tuned models while composing adapters efficiently across tasks. Its scope includes observed gains from single-task and multi-task adapters, with further investigation needed for last-layer adapter removal.
Problem
Sequential fine-tuning and multi-task learning make cross-task transfer difficult because of catastrophic forgetting, task balancing, and the need for joint retraining.
Method
AdapterFusion first trains task-specific adapters with the pretrained model fixed, then uses a Fusion layer to combine representations from multiple adapters for a target task.
Results
AdapterFusion consistently outperforms fully fine-tuned target-task models and works with adapters trained in both single-task and multi-task setups.
Takeaways & Limitations
Separating knowledge extraction from composition avoids common multi-task pitfalls, allows adapters to be trained in parallel and reused, and selects task-relevant adapter knowledge.
Takeaways & Limitations
The observed trend from omitting last-layer adapters remains preliminary and is identified for future investigation.
Abstract
from arXiv · showhide
Sequential fine-tuning and multi-task learning are methods aiming to incorporate knowledge from multiple tasks; however, they suffer from catastrophic forgetting and difficulties in dataset balancing. To address these shortcomings, we propose AdapterFusion, a new two stage learning algorithm that leverages knowledge from multiple tasks. First, in the knowledge extraction stage we learn task specific parameters called adapters, that encapsulate the task-specific information. We then combine the adapters in a separate knowledge composition step. We show that by separating the two stages, i.e., knowledge extraction and knowledge composition, the classifier can effectively exploit the representations learned from multiple tasks in a non-destructive manner. We empirically evaluate AdapterFusion on 16 diverse NLU tasks, and find that it effectively combines various types of knowledge at different layers of the model. We show that our approach outperforms traditional strategies such as full fine-tuning as well as multi-task learning. Our code and adapters are available at AdapterHub.ml.
1 Introduction
AdapterFusion addresses the limitations of sequential fine-tuning and multi-task learning by separating task-specific knowledge extraction from its composition. It combines adapters trained for different tasks and outperforms full fine-tuning and adapter-based single-task and multi-task alternatives across diverse NLU tasks.
- 1 Introduction: Sequential fine-tuning can cause catastrophic forgetting, while multi-task learning requires joint access to tasks and difficult task balancing.Multi-task models may overfit low-resource tasks and underfit high-resource tasks.
- 1 Introduction: Adapters add small task-specific parameter sets while keeping the pretrained language model fixed, enabling separate training for multiple tasks.The approach stores task-specific knowledge without updating the shared pretrained parameters.
- 1 Introduction: AdapterFusion uses a two-stage algorithm that first trains task adapters and then combines their representations through a Fusion layer for the target task.The architecture separates adapter training from representation combination.
- 1 Introduction: AdapterFusion is evaluated on 16 diverse NLU tasks, including sentiment analysis, commonsense reasoning, paraphrase detection, and textual entailment.The evaluation covers multiple types of natural language understanding tasks.
- 1 Introduction: AdapterFusion improves over multi-task adapter training and outperforms fully fine-tuned models and adapter-based single-task and multi-task setups.The comparisons include both traditional full fine-tuning and adapter-based alternatives.
2 Background
The background defines transfer learning across multiple classification tasks and reviews sequential fine-tuning, multi-task learning, and adapter-based approaches. Adapters preserve shared pretrained parameters while encoding task-specific information, but their separate weights prevent downstream composition without AdapterFusion.
- 2 Background: The transfer-learning goal is to use a set of N classification tasks to improve performance on a target task.The tasks may have labelled datasets of varying sizes and different loss functions.
- 2.1.1 Sequential Fine-Tuning: Sequential fine-tuning updates all model weights task by task but can suffer catastrophic forgetting, especially beyond two sequential tasks.The order of fine-tuning is also part of the procedure.
- 2.1.2 Multi-Task Learning: Multi-task learning trains all tasks simultaneously to learn a shared representation, but adding tasks requires simultaneous access and combining unequal tasks is challenging.Different task sizes and loss functions require heuristic approaches for effective combination.
- 2.2 Adapters: Adapters keep pretrained parameters fixed and introduce small task-specific parameter sets that encode representations in the shared model.Adapter training can be parallelized across tasks and stores knowledge in designated model parts.
- 2.2 Adapters: Adapter parameters can be considerably smaller than the pretrained model, reaching 3.6% of its parameters in one cited architecture.This supports more compact storage of task-specific knowledge.
- 2.2.3 Adapters in Practice: Adapters can perform on-par with or slightly below full model fine-tuning, but distinct adapter weights prevent downstream tasks from using multiple extracted information sources.This limitation motivates a two-stage method for sharing information across adapters.
3 AdapterFusion
AdapterFusion separates task-specific knowledge extraction from knowledge composition, then dynamically combines multiple adapters for a target task without changing the shared pretrained model or adapter parameters.
- 3 AdapterFusion: AdapterFusion addresses the limitation that adapters preserve task knowledge but prevent downstream tasks from using multiple extracted information sources.It is designed to share information across adapters trained on different tasks.
- 3.1 Learning algorithm: The first stage trains ST-A or MT-A adapters for each of N tasks.These adapters encode task-specific representations while the pretrained parameters remain shared.
- 3.1 Learning algorithm: The second stage fixes the pretrained parameters Θ and adapters Φ, then learns Fusion parameters Ψ to combine the N task adapters for the target task.This separates knowledge extraction from knowledge composition.
- 3 AdapterFusion: Separating extraction and composition addresses catastrophic forgetting, task interference, and training instabilities.The composition stage learns to combine stored adapter information while keeping the extracted parameters fixed.
- 3.2 Components: At each transformer layer, AdapterFusion uses Query, Key, and Value matrices to compute contextual adapter activations.The transformer output supplies the query, while each adapter output supplies inputs to the key and value transformations.
- 3.2 Components: Given the input context, AdapterFusion learns a parameterized mixer that identifies and activates the most useful adapter.The resulting composition is a dynamic function of the target task data.
4 Experiments
The experiments evaluate AdapterFusion across 16 diverse NLU datasets and compare single-task, multi-task, and Fusion-based adapter settings using accuracy.
- 4 Experiments: 16 diverse NLU datasets are used to evaluate AdapterFusion, with accuracy as the scoring metric.The study covers multiple task types, including commonsense reasoning, sentiment analysis, and natural language inference.
- 4 Experiments: The experiments compare Fusion using ST-A with ST-A alone to investigate catastrophic forgetting.They also compare Fusion using ST-A with MT-A to test interference between tasks.
- 4 Experiments: MT-A models with and without Fusion are compared to assess whether AdapterFusion remains useful when base adapters already leverage all tasks simultaneously.This tests the versatility of Fusion beyond single-task adapter training.
- 4.1 Setup: BERT-base-uncased is used as the pretrained language model in all experiments.Single-task adapters are trained for all datasets, while multi-task adapters are trained on all datasets simultaneously.
- 4.1 Setup: Single-task adapters use reduction factors {2, 16, 64}, learning rate 0.0001, AdamW, linear decay, and early stopping for a maximum of 30 epochs.The reduction factor controls the bottleneck size; factor 64 reduces BERT Base's hidden size to 12.
- 4.1 Setup: AdapterFusion uses a learning rate of 5e −5 and trains for a maximum of 10 epochs with early stopping.Query and Key are randomly initialized, while Value starts near an identity transformation and is regularized with an l2-norm.
5 Results
Across 16 datasets, adapters and AdapterFusion provide competitive transfer-learning strategies, with Fusion generally improving over its component adapters and especially benefiting smaller datasets. Fusion with ST-A is the most promising configuration, while Fusion with MT-A offers smaller gains despite additional training effort.
- 5.1 Adapters: 10 points is the average accuracy drop from training only a prediction head instead of fully fine-tuning BERT.This comparison motivates more complex adaptation approaches.
- 5.1 Adapters: 0.66% is the average improvement of ST-A over full fine-tuning, while MT-A improves over full fine-tuning by 0.4% on average.MT-A nevertheless shows drops exceeding 2% on CSQA and MRPC.
- 5.2 AdapterFusion: 6.5% for RTE and 5.64% for MRPC are the substantial gains from Fusion with ST-A on datasets with fewer than 5k training instances.Smaller datasets benefit more from Fusion than datasets with more than 40k training instances.
- 5.2 AdapterFusion: 1.27% and 1.25% are the average improvements from Fusion with ST-A and Fusion with MT-A, respectively.Fusion with MT-A produces smaller improvements because the model already includes a shared parameter set.
- 5.2 AdapterFusion: For 15/16 tasks, Fusion with ST-A remains the same or improves over the task’s pretrained adapter, with gains on 10/16 tasks.Fusion with ST-A also improves over MT-A on 11/16 tasks.
- 5.2 AdapterFusion: Fusion with ST-A allows adapters to be trained in parallel and new tasks to be added without complete model retraining.The paper identifies this configuration as its most promising approach to sharing information across tasks.
6 Analysis of Fusion Activation
AdapterFusion’s activation patterns show that the model selectively uses task adapters, with transfer patterns varying across target tasks and BERT layers. Small datasets often rely on adapters trained on larger datasets, while layer 12 distributes activation more broadly.
- 6 Analysis of Fusion Activation: Tasks that do not benefit from Fusion tend to strongly activate their own adapter at every layer.This supports selective extraction of information from other adapters when it benefits the target task.
- 6 Analysis of Fusion Activation: MNLI and QQP are widely used by other tasks, including small-dataset tasks such as CB, RTE, and MRPC.The analysis identifies MNLI and QQP as useful intermediate tasks for multiple targets.
- 6 Analysis of Fusion Activation: Layer 12 has considerably more distributed activations across tasks than earlier layers.The paper suggests this may reflect the final adapters’ proximity to the prediction head and less comparable representations.
7 Contemporary Work
Contemporary work extends parameter-efficient transfer learning beyond adapters, while efficiency studies examine the computational cost and scalability of AdapterFusion. These studies report linear overhead with more adapters and investigate pruning or minibatch subsets to reduce it.
- 7 Contemporary Work: Sparse diff vectors, bias-only tuning, and prefix-tuning are contemporaneous parameter-efficient alternatives for adapting pretrained models.These approaches target different parameter subsets or task settings, including natural language generation.
- 7 Contemporary Work: Adding more tasks to AdapterFusion produces a linear increase in training and inference cost.Related work proposes approaches to mitigate this computational overhead.
8 Conclusion and Outlook
AdapterFusion separates knowledge extraction from composition to combine task information while avoiding catastrophic forgetting, task interference, and complete retraining when tasks are added. It consistently outperforms fully fine-tuned models, with small-data tasks often benefiting from large-data task adapters.
- AdapterFusion separates knowledge extraction from composition, avoiding catastrophic forgetting and interference between tasks.
- AdapterFusion consistently outperforms fully fine-tuned models on target tasks and supports both single-task and multi-task adapters.
- Small-dataset tasks more often rely on large-dataset task adapters, producing the largest performance gains in the experiments.
- Pruning less-activated adapters is reported to preserve inference performance while improving inference speed, suggesting a scalability direction.
- Preliminary results suggest that omitting adapters in the last layer may improve cross-lingual transfer, but this remains future work.
A.1 Datasets
The evaluation spans diverse NLU datasets covering commonsense reasoning, sentiment analysis, natural language inference, sentence relatedness, argument classification, and reading comprehension.
- Commonsense Reasoning: The commonsense reasoning evaluation includes sentence- and document-level classification and multiple-choice datasets requiring general, social, or passage-based reasoning.
- Sentiment Analysis: Sentiment analysis uses IMDb for long movie reviews and SST-2 for short movie reviews.
- Natural Language Inference (NLI): Natural language inference covers MultiNLI, SciTail, SICK, RTE, and Commitment Bank across entailment, contradiction, neutrality, relatedness, and three-class entailment settings.
- Sentence Relatedness: Sentence relatedness evaluation uses MRPC for paraphrase equivalence and QQP for duplicate-question detection.
- Misc: Additional tasks include three-way argument classification and binary reading-comprehension classification for yes-or-no questions.
A.2 What Is The Best Adapter Setup?
The study exhaustively searches adapter architecture choices and identifies a common high-performing configuration across SST-2, Argument, and CSQA. Results also compare adapter capacity and pretrained-model settings.
- The architecture search varies adapter placement and count, layer normalization, residual connections, reduction factors, and nonlinearities.
- The same adapter setup achieves the best results across SST-2, Argument, and CSQA.
- A single adapter after the feed-forward layer outperforms other settings and uses half as many newly introduced adapters as the Houlsby configuration.
- The selected single-task architecture is used for subsequent experiments with a learning rate of 1e-4.
- AdapterFusion activations are analyzed across all BERT-base layers, with target tasks as rows and adapters as columns.
- Reduction factors 2, 16, and 64 correspond to dense adapter dimensions 384, 48, and 12, respectively.
- With RoBERTa-base, datasets with less data particularly benefit from AdapterFusion, while reduction factor 2 performs poorly.