Source-linked AI summary
Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning
Zhaojiang Lin, Andrea Madotto, Pascale Fung
TL;DR
The paper tackles the cost of storing a separate large fine-tuned model for every language-generation task. It introduces a shared-backbone VLM with compact task-specific adapters and embeddings, finding that under 3% additional parameters can maintain or improve whole-model fine-tuning performance across five tasks.
Problem
Fine-tuning large generative language models requires a separate model for each downstream task, which is unsuitable for low-memory and low-power applications.
Method
VLM shares a pre-trained language-model backbone while using residual adapters, task embeddings, and optional sentence-level knowledge distillation for task adaptation.
Results
Across five diverse generation tasks, less than 3% task-specific parameters maintained or improved performance relative to fine-tuning the whole model.
Takeaways & Limitations
A single model can support multiple generation tasks while retaining task-specific behavior through compact parameters, with frozen backbones additionally supporting sequential extension.
Abstract
from arXiv · showhide
Fine-tuning pre-trained generative language models to down-stream language generation tasks has shown promising results. However, this comes with the cost of having a single, large model for each task, which is not ideal in low-memory/power scenarios (e.g., mobile). In this paper, we propose an effective way to fine-tune multiple down-stream generation tasks simultaneously using a single, large pre-trained model. The experiments on five diverse language generation tasks show that by just using an additional 2-3% parameters for each task, our model can maintain or even improve the performance of fine-tuning the whole model.
1 Introduction
The paper addresses the inefficiency of fine-tuning separate large models for each generation task, especially in resource-constrained settings. It proposes sharing one pre-trained backbone while learning compact task-specific parameters.
- Fine-tuning achieves strong downstream performance but requires a separate model for each task, which is problematic when computation and memory are limited.
- The study targets one pre-trained backbone shared across conversational question answering, summarization, translation, dialogue, and task-oriented generation.
- The parameter-sharing scheme keeps the pre-trained model as shared parameters and learns task-specific parameters for each dataset.
- Residual adapters and task embeddings model task-specific parameters, while distillation and multi-task or continual-learning strategies are explored.
- Less than 3% task-specific parameters can maintain or improve performance relative to fine-tuning the whole model.
2 Related work
Prior work established the effectiveness of pre-trained language models for generation and classification, but generation studies generally fine-tuned each task separately. Adapter research introduced scalable task-specific modules across vision, classification, translation, and controllable generation.
- Pre-trained generative models are effective for language generation, while bidirectional pre-trained models improve several downstream classification tasks.
- Earlier generation studies considered fine-tuning each task individually, requiring a separate model for each task.
- Residual adapters originated in multiple visual-domain learning and were later used for scalable transfer across text classification and neural machine translation.
- Related controllable-generation work introduced plug-and-play methods for steering language-model outputs.
3 Methodology
The Versatile Language Model combines a pre-trained language-model backbone with low-rank residual adapters and task embeddings for task-specific generation behavior. It also uses task-dependent distillation to improve adaptation under large distribution shifts.
- VLM consists of a pre-trained language-model backbone plus low-rank residual adapters and task embeddings specialized for each generation task.
- Residual Adapters: Residual adapters apply layer normalization and a bottleneck autoencoder with a residual connection to steer the backbone toward downstream tasks.
- Residual Adapters: The adapter bottleneck dimension m is tunable, allowing capacity to match the complexity of the target task.
- Task Embedding: Task-specific segment embeddings adapt unconditional generation to conditional tasks and encode dialogue structure by alternating System and User embeddings.
- Knowledge Distillation: Sentence-level distillation first trains a full GPT-2 teacher, replaces gold targets with greedy-decoded outputs, and then trains the student VLM on those targets.
4 Experiments
Experiments evaluate VLM across five diverse generation tasks against full GPT-2 fine-tuning, LM-Head tuning, reference systems, and SOTA models. The results show strong parameter efficiency, with adapters, distillation, task embeddings, and backbone-training choices affecting performance and continual learning.
- Experimental setup: Five datasets cover chit-chat dialogue, neural machine translation, summarization, conversational question answering, and task-oriented natural language generation.The evaluation uses task-appropriate metrics, including perplexity, F1, BLEU, ROUGE, NIST, METEOR, and CiDER.
- Experimental setup: VLM is compared with separately fine-tuned GPT-2, LM-Head tuning, reference models, and state-of-the-art systems.Table 1 reports the experimental results for these model categories.
- Adapter efficiency: 13% additional adapter parameters incur a 0.4% performance loss compared with fine-tuning a separate GPT-2 model.Adapters are also reported as more effective than fine-tuning the LM-Head in both performance and added-parameter count.
- Training strategies: Knowledge distillation is especially useful for NMT and SUM when adapter performance trails full fine-tuning, but has no impact when adapter performance is already close or better.Distillation replaces gold targets with teacher-generated targets, simplifying translation or summarization for a low-capacity adapter with 4% parameters.
- Training strategies: Removing task embeddings causes a substantial performance drop because the model struggles to identify sequence boundaries and distinguish input segments.The affected structures include NLG attributes and the document-question distinction in CoQA.
- Backbone settings: Multi-task VLM improves CoQA F1 by 3%, while a frozen GPT-2 backbone supports sequential learning without changing the original weights.The trainable-backbone setting can transfer knowledge among tasks, whereas the frozen setting can be extended to tasks learned one at a time.
5 Conclusion
The paper presents one Versatile Language Model for five diverse natural language generation tasks. It finds residual adapters effective, distillation helpful for difficult tasks, and a frozen backbone competitive while supporting extension to future tasks without full retraining.
- The Versatile Language Model learns five diverse natural language generation tasks in a single model.
- Residual adapters outperform fine-tuning other model components such as the LM-Head, while distillation reduces the performance gap on difficult summarization and translation tasks.
- A frozen backbone offers competitive performance and can be extended to future tasks without full retraining.
A.1 Model details
VLM combines a shared GPT-2 backbone with task-specific embeddings and adapters selected for each task. The task embeddings represent input structures, while the adapter provides task-specific modeling.
- VLM shares one GPT-2 backbone across tasks.
- Task embeddings model different input structures for each task.
- The model selects the corresponding adapter for each task.
A.2 Experiment details
The experiments evaluate VLM across five generation tasks using task-specific datasets, metrics, and training settings. VLM requires around 80%-90% of the GPU memory of full GPT-2 fine-tuning.
- Training setup: Training uses Adam with task-specific epochs, batch sizes, sequence-length limits, and input segment embeddings.Segment embeddings distinguish structures such as articles and summaries, source and target languages, dialogue roles, and response attributes.
- Datasets: Five tasks are evaluated: summarization, neural machine translation, persona dialogue, conversational question answering, and task-oriented natural language generation.The tasks use CNN/Daily-Mail, IWSLT, Persona-Chat, CoQA, and the E2E NLG challenge datasets.
- Evaluation: Each task uses task-appropriate evaluation metrics, including ROUGE, BLEU, perplexity, F1, Consistency, NIST, METEOR, and CiDER.NLG reports a normalized average score over multiple metrics.
- Computational cost: 80%-90% GPU memory is required for VLM relative to full GPT-2 fine-tuning because VLM updates only a small ratio of parameters.Both models have similar training cost, and training speed is reported on a single GTX 1080 Ti.
- Computational cost: Training speed ranges from 0.2h/epoch for NLG to 7.5h/epoch for summarization.Reported speeds are 1.6h/epoch for NMT, 1.5h/epoch for dialogue, and 5.0h/epoch for QA.
A.3 Detailed Results
The detailed-results section reports task results in Tables 2-6 and uses greedy decoding for all tasks.
- Detailed results: Results for each task are reported in Tables 2-6 using a greedy decoding strategy throughout.The section presents detailed results for the five evaluated tasks.
A.4 Example
The examples section includes architecture and performance figures, task-result tables, and generated examples for conversational and natural-language-generation tasks.
- Architecture: Figure 3 depicts VLM sharing a GPT-2 backbone while selecting task embeddings and the corresponding adapter for each task.The figure provides a detailed view of the model's task-specific components.
- Performance comparison: Figure 4 compares performance across different ratios of additional parameters and reports task-dependent effects of knowledge distillation.Distillation does not improve NLG, while gold targets outperform distilled targets for dialogue and QA.
- Task results: Tables 2-6 report results for summarization, Persona Chat, CoQA, NMT, and NLG.The supplied table captions identify the task-result tables, while separate tables contain task examples.
- CQA examples: The CQA example passages show alternative English renderings of a German input involving relationships with people in their twenties.The examples include literal, revised, repetitive, and more fluent outputs.
- Task examples: Tables 7-12 provide examples for NMT, summarization, CQA, dialogue, and NLG.The examples are organized by task in the corresponding example tables.