Source-linked AI summary
To Tune or Not to Tune? Adapting Pretrained Representations to Diverse Tasks
Matthew E. Peters, Sebastian Ruder, Noah A. Smith
TL;DR
The paper asks how pretrained representations should be adapted to target tasks, rather than focusing primarily on pretraining objectives. It compares feature extraction and fine-tuning with ELMo and BERT across diverse NLP tasks, finding that their relative performance depends on pretraining–target task similarity and motivating practical adaptation guidelines.
Problem
Prior NLP work has focused mainly on pretraining objectives, while comparatively few studies examine how best to adapt pretrained representations to target tasks.
Method
The paper empirically compares frozen feature extraction and direct fine-tuning for ELMo and BERT across diverse NLP tasks.
Results
The relative performance of fine-tuning and feature extraction depends on the similarity between pretraining and target tasks, with comparable performance in most cases except highly similar or dissimilar tasks.
Takeaways & Limitations
Adaptation choices should account for the relationship between pretraining and target tasks, and the paper provides practical recommendations for NLP practitioners.
Takeaways & Limitations
The study’s information-theoretic analysis includes a caveat that earlier estimators produced non-zero values that appeared less reliable in higher dimensions.
Abstract
from arXiv · showhide
While most previous work has focused on different pretraining objectives and architectures for transfer learning, we ask how to best adapt the pretrained model to a given target task. We focus on the two most common forms of adaptation, feature extraction (where the pretrained weights are frozen), and directly fine-tuning the pretrained model. Our empirical results across diverse NLP tasks with two state-of-the-art models show that the relative performance of fine-tuning vs. feature extraction depends on the similarity of the pretraining and target tasks. We explore possible explanations for this finding and provide a set of adaptation guidelines for the NLP practitioner.
1 Introduction
The paper shifts attention from pretraining objectives to adaptation, comparing frozen feature extraction with fine-tuning across diverse NLP tasks. It finds that the relative performance of these approaches depends on pretraining–target task similarity and offers practitioner guidelines.
- Sequential inductive transfer learning separates pretraining, which learns general-purpose input representations, from adaptation to a new task.
- Feature extraction freezes pretrained weights, whereas fine-tuning unfreezes and adapts the pretrained parameters to a new task.
- Feature extraction supports task-specific architectures and can be computationally cheaper because features need to be computed only once.
- Fine-tuning is convenient because it can adapt a general-purpose representation to many different tasks.
- Across seven diverse tasks, the study compares ELMo and BERT to characterize when either adaptation approach substantially outperforms the other.
- Feature extraction and fine-tuning have comparable performance in most cases, except when source and target tasks are highly similar or highly dissimilar.The paper also provides practical adaptation guidelines.
2 Pretraining and Adaptation
The paper situates adaptation within work on universal pretrained representations, contextual word vectors, sentence embeddings, and pretrained transformers. It contrasts fixed representations with adaptation strategies for downstream tasks.
- The paper focuses on pretraining tasks intended to induce universal representations suitable for any downstream task.
- Pretrained word vectors are commonly fixed and fed into task-specific models, while contextual representations have improved over noncontextual vectors.
- Sentence embedding methods learn sentence representations using objectives such as next-sentence prediction, NLI, or combinations of objectives.
- ULMFiT proposes fine-tuning techniques including triangular learning-rate schedules and discriminative fine-tuning with lower rates for lower layers.
- BERT combines word and sentence representations through masked language modeling and next-sentence prediction in a large pretrained transformer.
- BERT is adapted to word- and sentence-level tasks with task-specific layers.
3 Experimental Setup
The experiments compare ELMo and BERT adaptation approaches across sequence-labeling, classification, and sentence-pair tasks. The setup uses task-specific architectures or output layers and evaluates task-appropriate metrics.
- ELMo and BERT are evaluated as representatives of two high-performing pretraining settings.
- The target tasks include named entity recognition, sentiment analysis, natural language inference, paraphrase detection, and semantic textual similarity.
- The datasets cover newswire entity annotations, movie-review sentiment, broad- and narrow-domain inference, paraphrase detection, and human-judged sentence similarity.
- The adaptation setup uses task-specific architectures for some tasks and task-specific output layers for others, with an extensive hyper-parameter search for fair comparison.
- For feature extraction, contextual representations from all layers are combined linearly and passed to task-specific models.
- ELMo uses max pooling and softmax for text classification, cross-sentence bi-attention for sentence pairs, and a CRF layer for named entity recognition.
- The reported metrics are accuracy, matched accuracy, Pearson correlation, and span F1, depending on the task.
4 Results
Across seven diverse NLP tasks, adaptation performance varies with the relationship between pretraining and target tasks. Closely aligned objectives transfer better, while distant tasks can expose model-architecture limitations.
- ELMo and BERT significantly outperform Skip-thoughts on most tasks, except that Skip-thoughts is similar to ELMo on semantic textual similarity.Skip-thoughts uses a next-sentence prediction objective similar to BERT.
- Performance differences between feature extraction and fine-tuning are generally small, but vary by task and include notable exceptions.
- Earlier NLP and computer-vision studies likewise report better transfer between similar tasks, although prior NLP work examined only classification-task transfers.
- BERT shows strong performance on closely aligned tasks, particularly semantic textual similarity, whereas ELMo performs poorly on more distant sentence-pair tasks.The reported pattern links next-sentence prediction in BERT with STS, and language modeling in ELMo with sentence-pair tasks.
- The contrast between strong aligned-task transfer and poor distant-task transfer may partly reflect the suitability of the model architecture's inductive bias for sentence-pair tasks.
5 Analyses
The analyses examine how model architecture, sentence-pair encoding, added parameters, target-domain distance, and representation layers affect adaptation. They find that fine-tuning is especially useful when task-relevant information is not already concentrated in pretrained representations, while some settings require specialized adaptation procedures.
- Sentence-pair adaptation: Fine-tuning ELMo through cross-sentence bi-attention provides the best sentence-pair performance, supporting explicit modeling of sentence interactions.The authors contrast this with sequential processing in the ELMo LSTM and with Transformer-based adaptation.
- Sentence-pair adaptation: Separately encoding BERT sentence pairs lowers performance, indicating that BERT representations encode cross-sentence relationships.
- Additional parameters: Additional parameters are key for one NER adaptation setting but hurt performance with the other.
- Adaptation procedures: Gradual unfreezing is required for fine-tuning to match feature extraction in one evaluated setting.
- Target domain: Target-domain distance shows no significant correlation with adaptation performance for the evaluated MNLI domains.The analysis compares Jensen–Shannon divergence between BERT’s training domains and MNLI domains.
- Representation layers: Diagnostic-classifier performance generally increases at higher layers, while fine-tuning improves performance at every layer.For CoLA and SST-2, pretrained performance rises toward the last layers; for MRPC and STS-B, it is mostly flat after layer four.
- Representation layers: Sentence-pair information is mostly flat after the fourth pretrained layer, whereas single-sentence information increases toward the final layers.This distribution may explain why fine-tuning is particularly useful for sentence-pair tasks.
- Mutual information: Pretrained representations have near-zero mutual information with labels, while fine-tuned representations show substantially higher mutual information.For MRPC, mutual information rises gradually; for single-sentence classification, it rises sharply in the last layers.
6 Conclusion
The paper concludes that adaptation performance depends on the similarity between pretraining and target tasks. It also presents practical recommendations for adapting pretrained representations, while noting an estimator-related limitation in the mutual-information analysis.
- Across diverse datasets, the relative performance of fine-tuning and feature extraction depends on pretraining–target task similarity.
- The paper provides practical recommendations for adapting pretrained representations to NLP tasks.
- Earlier mutual-information estimators produce non-zero values for the same random-label settings, suggesting lower reliability in higher dimensions.
A Experimental details
The experiments use extensive development-set hyperparameter tuning, Adam with weight-decay correction, and publicly available pretrained ELMo and BERT models.
- All experiments tune learning rate, dropout ratio, weight decay, and training epochs on development sets before test evaluation.Fine-tuning also varies triangular schedules, gradual unfreezing, and discriminative learning rates.
- All models use Adam with the weight-decay fix.
- The study uses the original two-layer bidirectional ELMo and 12-layer uncased BERT-base, with cased BERT for NER.
A.1 Feature extraction
Feature extraction freezes pretrained representations and feeds learned combinations of their layers into task-specific models. The experiments isolate contextual word representations while using task-specific architectures for NER, sentiment analysis, and sentence-pair tasks.
- Feature-based models used only ELMo or BERT representations, excluding other pretrained word representations to isolate contextual representation effects.
- All pretrained layers were combined using learned scalar weights before being passed to task-specific models.The feature-based setups used a BiLSTM-based model for NER, a bi-attentive classifier for sentiment analysis, and ESIM for sentence-pair tasks.
- The NER feature-extraction setup used a two-layer bidirectional LSTM, with a CRF output for ELMo.ELMo feature extraction treated each sentence independently.
- Additional pooling operations for composing BERT word pieces and attention or pooling mechanisms did not add parameters or provide further gains.Alternative word-piece pooling operations were tested, but they did not provide additional gains.
- For BERT sentence-pair feature extraction, both sentences were processed jointly, which improved performance over extracting each sentence separately.
A.2 Fine-tuning
Fine-tuning adapted pretrained models with task-specific optimization choices. The setups included discriminative learning rates, gradual unfreezing, standard BERT scheduling, and attention-based sentence-pair modeling.
- Fine-tuning ELMo used discriminative learning rates that decreased by 0.4× at each successive layer.For SST-2 and NER, weights were gradually unfrozen from the top layer, adding one layer per epoch until all weights were trainable.
- Gradual unfreezing was beneficial for ELMo fine-tuning on SST-2 and NER.
- BERT fine-tuning used the default learning-rate schedule from Devlin et al. (2018).
- For sentiment analysis, max pooling performed slightly better than average pooling on the development set when composing ELMo states.
- The sentence-pair fine-tuning mechanism used bi-attention similar to the attention mechanism in the feature-based ESIM model.