Source-linked AI summary
Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning
Beliz Gunel, Jingfei Du, Alexis Conneau, Ves Stoyanov
TL;DR
Cross-entropy fine-tuning of pre-trained language models can suffer from poor generalization and instability, particularly with limited labeled data. The paper adds a supervised contrastive objective that groups same-class representations and separates different-class representations, improving few-shot GLUE performance over RoBERTa-Large cross-entropy fine-tuning and increasing robustness to noisy data and transfer to related tasks.
Problem
Cross-entropy fine-tuning can have poor generalization, limited robustness to noisy labels and adversarial examples, and unstable runs when supervised data is limited.
Method
The paper adds a supervised contrastive learning term to the fine-tuning objective, pulling same-class examples together and pushing different-class examples apart.
Results
The combined objective significantly improves several GLUE tasks in 20-, 100-, and 1000-example settings over RoBERTa-Large cross-entropy fine-tuning, with up to 10.7 points of improvement in the 20-example setting.
Takeaways & Limitations
The objective also produces models that are more robust to varying training-data noise and generalize better to related tasks with limited labeled data without extra architecture, augmentation, memory banks, or unsupervised data.
Takeaways & Limitations
The authors hypothesize that larger batch sizes improve performance, but leave this for future work because it requires additional engineering effort.
Abstract
from arXiv · showhide
State-of-the-art natural language understanding classification models follow two-stages: pre-training a large language model on an auxiliary task, and then fine-tuning the model on a task-specific labeled dataset using cross-entropy loss. However, the cross-entropy loss has several shortcomings that can lead to sub-optimal generalization and instability. Driven by the intuition that good generalization requires capturing the similarity between examples in one class and contrasting them with examples in other classes, we propose a supervised contrastive learning (SCL) objective for the fine-tuning stage. Combined with cross-entropy, our proposed SCL loss obtains significant improvements over a strong RoBERTa-Large baseline on multiple datasets of the GLUE benchmark in few-shot learning settings, without requiring specialized architecture, data augmentations, memory banks, or additional unsupervised data. Our proposed fine-tuning objective leads to models that are more robust to different levels of noise in the fine-tuning training data, and can generalize better to related tasks with limited labeled data.
1 INTRODUCTION
The paper addresses poor generalization, noise robustness, and instability associated with cross-entropy fine-tuning, especially when labeled data is limited. It introduces supervised contrastive learning for fine-tuning and reports improvements across few-shot, noisy-data, and related-task settings.
- Motivation: Cross-entropy fine-tuning can generalize poorly, resist noisy labels and adversarial examples, and remain unstable across runs when supervised data is limited.These shortcomings motivate alternatives that improve both generalization and fine-tuning stability.
- Approach: The proposed objective adds supervised contrastive learning, bringing same-class examples closer while separating examples from different classes.The approach is designed to identify important dimensions in hidden representations during few-shot fine-tuning.
- Results: The SCL-augmented objective significantly improves several GLUE classification tasks over RoBERTa-Large fine-tuned with cross-entropy alone in 20-, 100-, and 1000-example settings.The method uses no specialized architecture, memory bank, data augmentation, or additional unsupervised data.
- Results: 10.7 points is the largest reported few-shot improvement, obtained on a subset of GLUE tasks with 20 labeled examples.The cited subset comprises SST-2, QNLI, and MNLI.
- Results: The objective improves robustness across varying noise levels and generalization to related tasks with limited labeled data.The paper also reports a 2.9-point improvement on Amazon-2 after transfer from an SST-2 task model.
2 APPROACH
The approach combines cross-entropy with supervised contrastive learning to bring same-class examples together and separate different-class examples during fine-tuning.
- The supervised contrastive term captures similarities among same-class examples and contrasts them with examples from other classes.
- The method operates on normalized encoder representations and uses temperature τ to control class separation.
- The loss can support multi-class classification and can use encoders such as pre-trained language models.
- The overall objective is a weighted average of cross-entropy and the proposed supervised contrastive learning loss.
- Lower temperature increases the influence of harder-to-separate examples, effectively creating harder negatives.
- Unlike self-supervised contrastive objectives, the method contrasts labeled examples for supervised final-task learning rather than augmented views.
3 RELATED WORK
Related work describes limitations of cross-entropy, prior approaches to fine-tuning stability, and the development of contrastive objectives across supervised and self-supervised settings.
- Traditional Machine Learning and Theoretical Understanding: Prior analyses associate cross-entropy with poor generalization from poor margins and limited robustness to noisy labels or adversarial examples.
- Contrastive Learning: Contrastive learning research spans self-supervised, semi-supervised, and supervised methods, primarily in computer vision.
- Stability and Robustness of Fine-tuning Pre-trained Language Models: Recent fine-tuning research targets generalization and stability in pre-trained language models, which can overfit limited labeled task data.
4 EXPERIMENTAL SETUP
The experiments evaluate the objective across GLUE sentence and sentence-pair classification tasks, using few-shot sampling, repeated seeds, and RoBERTa-Large training procedures.
- Datasets and Training Details: The evaluation uses GLUE datasets covering both single-sentence and sentence-pair classification tasks.
- Datasets and Training Details: Few-shot experiments sample training examples from the original GLUE training sets and construct validation and test splits from the validation data.
- Datasets and Training Details: Experiments use 10 seeds and report average test accuracy, standard deviation, and p-values relative to the baseline.
- Datasets and Training Details: The setup uses RoBERTa-Large with Adam, learning rate 1e-5, batch size 16 by default, dropout 0.1, and sweeps over λ and τ.
- Constructing Augmented Noisy Training Datasets: Augmented noisy training datasets are created through back-translation, with temperature controlling the amount of noise.
5 ANALYSIS AND RESULTS
Across few-shot, noisy-data, full-data, representation, and transfer experiments, adding supervised contrastive learning to cross-entropy improves RoBERTa performance and robustness. The gains are largest with very limited labels and on noisier or related-task settings, while larger batches improve performance but add engineering and speed trade-offs.
- Few-shot learning: 10.7 points improvement on QNLI, 3.4 points on MNLI, and 2.2 points on SST-2 with 20 labeled examples.The CE+SCL objective significantly improves over RoBERTa-Large with cross-entropy across the reported few-shot datasets; gains decrease as labeled data increases.
- Representation analysis: SCL produces more compact same-label CLS-embedding clusters than CE alone, whose embeddings are close to random in the 20-example SST-2 setting.Figure 2 compares CE-only and CE+SCL tSNE representations, with blue denoting positive and red negative examples.
- Robustness to noisy data: 7 points average improvement on MNLI, 2.5 points on QNLI, and 0.4 points on SST-2 across augmented noisy training sets.Higher noise yields particularly large gains: 7.7 points on MNLI at T=0.7 and 4.2 points on QNLI at T=0.9.
- Full-dataset results: 3.1 points improvement on MRPC, 3.5 points on QNLI, and 1.2 points averaged across six datasets in full-data experiments.Only the MRPC and QNLI results are statistically significant; the other dataset results are reported as findings for completeness.
- Batch-size ablation: Increasing CE+SCL batch size from 16 to 256 improves SST-2 by 0.3 points, CoLA by 0.8, QNLI by 0.4, and MNLI by 1.3.The authors leave larger-batch investigation for future work because it requires additional engineering; SCL has negligible speed impact at batch size 16.
- Generalization to related tasks: 2.9 points improvement on Amazon-2 and 0.4 points on Yelp-2 when transferring an SST-2 task model trained with SCL.The transfer experiment uses 20 labeled examples per class and also reports a significant reduction in variance across training-set samples.
6 CONCLUSION
The paper proposes supervised contrastive learning for fine-tuning pre-trained language models and reports improvements in few-shot GLUE performance, noise robustness, and related-task generalization.
- The proposed fine-tuning objective adds a supervised contrastive learning term to pre-trained language model training.
- The objective significantly improves performance over a strong RoBERTa-Large baseline across multiple GLUE datasets in few-shot settings.
- The resulting models are more robust to different levels of noise in the training data.
- The resulting models can generalize better to related tasks with limited labeled task data.
- Future work will study principled and automated NLP data augmentation for extending the objective to semi-supervised and self-supervised settings.
A APPENDIX
The appendix presents embedding visualizations and tabular evaluations comparing cross-entropy with and without supervised contrastive learning, alongside alternative objectives and efficiency analyses.
- Figure 3 visualizes learned CLS embeddings on SST-2 test data for 20, 100, and full-dataset labeled-example settings.
- The figure compares cross-entropy with and without the supervised contrastive learning term using blue positive and red negative examples.
- Table 8 compares RoBERTa-Large fine-tuning with cross-entropy with and without supervised contrastive learning, CE+CE, and a two-stage method.
- Table 9 reports performance and fine-tuning speed for RoBERTa-Base across batch sizes, using average updates per second as the speed measure.
- Table 10 reports few-shot GLUE test accuracies for N=20, 100, and 1000 labeled examples, including standard deviations and p-values.