Source-linked AI summary
Sentence Encoders on STILTs: Supplementary Training on Intermediate Labeled-data Tasks
Jason Phang, Thibault Févry, Samuel R. Bowman
TL;DR
Fine-tuning pretrained sentence encoders can be brittle, especially with limited target-task data. STILTs adds supervised training on a data-rich intermediate task before target fine-tuning, yielding gains across encoders and GLUE tasks, including a 1.4-point improvement for BERT and greater benefits in data-constrained settings.
Problem
Fine-tuning can be brittle with limited target-task data because encoders must adapt to new distributions and labels without overfitting or catastrophic forgetting.
Method
STILTs adds a supplementary training stage on a data-rich labeled intermediate task between unsupervised pretraining and target-task fine-tuning.
Results
STILTs improves performance across many tasks and three sentence encoders; for BERT, it improves the reported GLUE baseline by 1.4 points and sets the state of the art.
Takeaways & Limitations
Benefits are strongest for sentence-pair tasks and tasks with little training data, while STILTs also reduces degenerate runs across random restarts.
Takeaways & Limitations
The experiments use only three epochs for supplementary training and fine-tuning, so more careful tuning of training duration might improve performance, especially for data-constrained tasks.
Abstract
from arXiv · showhide
Pretraining sentence encoders with language modeling and related unsupervised tasks has recently been shown to be very effective for language understanding tasks. By supplementing language model-style pretraining with further training on data-rich supervised tasks, such as natural language inference, we obtain additional performance improvements on the GLUE benchmark. Applying supplementary training on BERT (Devlin et al., 2018), we attain a GLUE score of 81.8---the state of the art (as of 02/24/2019) and a 1.4 point improvement over BERT. We also observe reduced variance across random restarts in this setting. Our approach yields similar improvements when applied to ELMo (Peters et al., 2018a) and Radford et al. (2018)'s model. In addition, the benefits of supplementary training are particularly pronounced in data-constrained regimes, as we show in experiments with artificially limited training data.
1 Introduction
STILTs adds supervised intermediate-task training between unsupervised pretraining and target-task fine-tuning to reduce brittleness and improve GLUE performance, especially with limited target-task data.
- Motivation: Fine-tuning can be brittle because the encoder must adapt to new inputs and labels while avoiding overfitting or catastrophic forgetting.
- Approach: STILTs inserts a second pretraining stage using data-rich labeled tasks before final target-task fine-tuning.The pipeline is unsupervised pretraining, intermediate-task training, then target-task evaluation on GLUE.
- Approach: Four intermediate tasks are evaluated: MNLI, SNLI, QQP, and fake-sentence detection.
- Findings: STILTs yields significant gains across most GLUE tasks and across BERT, GPT, and ELMo.
- Findings: For 24-layer BERT with 5k examples, degenerate runs decrease from five to one on SST and from two to none on STS.
- Findings: STILTs improves performance more strongly when target-task training data is artificially limited to 1k or 5k examples.
2 Related Work
Related work examines unsupervised pretraining, fine-tuning, diagnostic sentence representations, multitask learning, and supervised refinement of learned embeddings.
- Sentence-encoder pretraining: Prior work finds that language-model pretraining can perform reliably well for syntactic target tasks.
- Sentence-encoder pretraining: Fine-tuning entire pretrained models has produced state-of-the-art results on several benchmarks, motivating this paper’s focus on whole-model fine-tuning.
- Sentence representations: Diagnostic tasks emphasize preserving lexical content information in sentence-to-vector encoders.
- Multitask learning: Multitask learning is more likely to help when the target task plateaus quickly while the auxiliary task continues improving.
- Representation refinement: Embedding retrofitting refines representations learned by unsupervised methods using labeled resources before target-task use.
3 Methods
The methods evaluate supplementary training across three pretrained encoders, four intermediate tasks, and nine GLUE targets, including full-data and data-constrained settings.
- Pretrained sentence encoders: The study applies STILTs to BERT, GPT, and ELMo, which differ in architecture, objectives, and pretraining corpora.
- Target tasks and evaluation: Evaluation covers nine GLUE target tasks, including entailment, paraphrase, similarity, sentiment, and acceptability classification.
- Target tasks and evaluation: Test-set STILTs models are selected by development-set performance, and reported GLUE aggregates use QNLIv1.
- Intermediate-task training: STILTs adds intermediate-task training between standard pretraining and target-task fine-tuning.
- Intermediate-task training: The four intermediate tasks are MNLI, SNLI, QQP, and custom fake-sentence detection.
- Training details: BERT experiments use 20 random restarts for limited-data tasks and report the best validation-set model.
4 Results
STILTs improves GLUE performance across many tasks and models, with especially pronounced gains under limited-data conditions, while also reducing instability across random restarts. The approach sets a GLUE state of the art for BERT and GPT, though CoLA and SST are exceptions and training-duration tuning remains a potential improvement.
- STILTs boosts performance across many GLUE tasks, with gains generally larger for ELMo and GPT than for BERT.
- 1.4 points: BERT with STILTs improves on the reported GLUE baseline and sets the benchmark state of the art.
- 76.9 GLUE score: GPT with STILTs improves 2.8 points over baseline and narrows the gap with 12-layer BERT's 78.3.
- More than 10 points: several tasks improve when training data is artificially limited, showing stronger benefits in data-constrained settings.
- CoLA and SST deteriorate with supplementary training, while the experiments use only three supplementary-training and fine-tuning epochs.
- STILTs significantly reduces performance variance across random restarts, largely by producing fewer degenerate runs.
5 Discussion
STILTs improves many downstream tasks across sentence encoders, but its benefits vary by task and model. It also reduces instability and adds little computational overhead, while task selection remains important.
- STILTs improves performance on many downstream tasks across three sentence encoders with different architectures and pretraining schemes.
- Sentence-pair tasks benefit more than single-sentence tasks, while low-data tasks show especially large improvements.
- BERT gains eight points on RTE when supplementary training uses the related MNLI task.
- Supplementary training adds little complexity because it uses the same fine-tuning infrastructure, but intermediate-task choice can make results ineffective or counterproductive.The authors recommend trying several closely related data-rich tasks and selecting among them using development data.
- STILTs reduces degenerate training runs for BERT with limited data, lowering computational cost in some settings.With 5k examples, degenerate runs fall from five to one on SST and from two to none on STS.
6 Conclusion
The initial STILTs investigation finds gains across BERT, GPT, and ELMo, improved stability in low-data settings, and especially pronounced benefits under data constraints.
- STILTs produces significant gains across BERT, GPT, and ELMo and reaches the state of the art on GLUE with BERT.
- STILTs significantly stabilizes training in unstable contexts, including BERT tasks with little data.
- Data-constrained regimes yield up to 10-point score improvements on some intermediate/target task pairs.
- The work remains an initial investigation into when STILTs applies productively and how to predict successful intermediate-target task combinations.
A ELMo on STILTs
The ELMo experiments use task-specific pooling, projection, and logistic-regression components alongside the shared encoder. Table 4 reports development-set GLUE results with and without STILTs under full target-task training data.
- ELMo task-specific parameters include layer weights, task weights, maxpooling with projection, and logistic regression.
- The shared encoder processes the two input sentences independently before constructing pairwise features for the task classifier.
- Table 4 compares GLUE development-set results with and without STILTs when each target task uses its full training data.The table caption notes that A.Ex excludes MNLI and QQP because they overlap with intermediate tasks.