Source-linked AI summary
Pretrained Transformers Improve Out-of-Distribution Robustness
Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, Adam Dziedzic, Rishabh Krishnan, Dawn Song
TL;DR
The paper examines whether pretrained Transformers generalize beyond IID NLP data and builds a benchmark to test that question across realistic distribution shifts. Across seven datasets, Transformers generally decline less and detect OOD examples better than earlier models, although robustness remains incomplete and its causes are not fully established.
Problem
High IID accuracy does not establish OOD robustness, while shifted or unknown examples require models to generalize or trigger conservative fallback policies.
Method
The study constructs a benchmark of realistic shifts across seven NLP datasets and evaluates traditional models and pretrained Transformers for OOD generalization and detection.
Results
Pretrained Transformers generally have smaller IID/OOD performance declines and better OOD detection than traditional NLP models, many of which perform near or worse than chance at detection.
Takeaways & Limitations
Pretrained Transformers are moderately robust across varied NLP distribution shifts, but future work remains necessary to improve robustness.
Takeaways & Limitations
The paper does not fully establish why pretrained Transformers are more robust; diverse pretraining data is presented as only a partial explanation.
Abstract
from arXiv · showhide
Although pretrained Transformers such as BERT achieve high accuracy on in-distribution examples, do they generalize to new distributions? We systematically measure out-of-distribution (OOD) generalization for seven NLP datasets by constructing a new robustness benchmark with realistic distribution shifts. We measure the generalization of previous models including bag-of-words models, ConvNets, and LSTMs, and we show that pretrained Transformers' performance declines are substantially smaller. Pretrained transformers are also more effective at detecting anomalous or OOD examples, while many previous models are frequently worse than chance. We examine which factors affect robustness, finding that larger models are not necessarily more robust, distillation can be harmful, and more diverse pretraining data can enhance robustness. Finally, we show where future work can improve OOD robustness.
1 Introduction
The paper asks whether pretrained Transformers remain robust when NLP data shifts beyond the IID setting. It introduces a systematic study of OOD generalization and detection across model types.
- Train-test mismatches arise because datasets incompletely characterize distributions and test data can drift over time.
- High IID accuracy does not establish OOD robustness, especially when models rely on spurious cues and annotation artifacts.
- The study decomposes OOD robustness into generalizing to shifted examples and detecting examples that do not belong to known classes.
- The benchmark induces shifts in writing style, topic, and vocabulary across sentiment analysis, entailment, question answering, and semantic similarity.
- Pretrained Transformers show smaller OOD performance declines than traditional models, while diverse pretraining improves robustness and larger models do not necessarily help.
- Many non-pretrained NLP models are near or worse than chance at OOD detection, whereas pretrained Transformers are more capable detectors.
2 How We Test Robustness
The paper constructs realistic distribution shifts from seven datasets and evaluates thirteen NLP models spanning simple representations, neural encoders, and pretrained Transformers.
- Seven datasets induce distribution shifts through metadata-based splits or pairing datasets from distinct data-generating processes.
- Sentiment evaluation uses cross-dataset SST-2 and IMDb testing plus food-type groups from Yelp.
- STS-B uses four news and caption sources and reports Pearson’s correlation coefficient for semantic similarity.
- ReCoRD separates CNN and Daily Mail sources for reading comprehension, while MNLI separates Telephone, Face-to-Face, and Letters genres for entailment.
- The study compares thirteen models across three categories with different input representations and encoders.
- Baselines include BoW, word embeddings with averages, LSTMs, and ConvNets, while Transformer models include BERT, RoBERTa, ALBERT, and DistilBERT.
3 Out-of-Distribution Generalization
Pretrained Transformers generally retain performance better across NLP distribution shifts than traditional models, but scaling model size does not reliably narrow the IID/OOD gap.
- Pretrained Transformers often have smaller IID/OOD generalization gaps than traditional NLP models across the evaluated tasks.
- Over 35%: the LSTM semantic similarity model’s performance decline on OOD examples, while RoBERTa’s performance slightly increases.
- BERT Base MNLI accuracy only slightly fluctuates when trained on Telephone data and tested across three distributions.
- Larger BERT and ALBERT models do not reduce the IID/OOD generalization gap in NLP.
- Distillation can reduce robustness, so evaluating compressed BERT models only on IID examples may mask downstream costs.
- Pretraining on larger and more diverse datasets can improve robustness, with RoBERTa more robust than BERT Large.
4 Out-of-Distribution Detection
The paper evaluates OOD detection by using classifier confidence as an anomaly score and finds pretrained Transformers substantially outperform earlier models, though detection remains imperfect.
- Detection setup: Models use negative maximum prediction confidence as the anomaly score for separating in-distribution from OOD examples.The evaluation includes SST-2 test examples and OOD examples from five other datasets.
- Detection setup: FAR95 measures the probability that an in-distribution example raises a false alarm when 95% of OOD examples are detected, so lower is better.
- Results: Models without pretraining often detect OOD examples unreliably, with FAR95 sometimes worse than chance and false alarm rates approximately 100% on 20 Newsgroups.
- Results: Pretrained Transformers have FAR95 scores always better than chance, but they still do not cleanly separate in-distribution and OOD examples.The paper identifies imperfect detection as an avenue for future work.
- Results: Figure 5 overlays confidence histograms for SST-2 test examples and WMT16 English-German examples, with minimum prediction confidence 0.5.
5 Discussion and Related Work
The discussion considers why pretrained Transformers are more robust and distinguishes this benchmark from domain adaptation and adversarial-example research. It also connects OOD brittleness to annotation artifacts.
- Discussion: Model accuracy alone does not explain reduced IID/OOD generalization gaps, while diverse pretraining data and self-supervised training are offered as partial explanations.The paper notes that these explanations are incomplete and that BERT uses roughly 3 billion tokens versus GloVe’s roughly 840 billion.
- Related work: The benchmark tests generalization without adaptation to unforeseen shifts, unlike domain adaptation, which learns representations of source and target distributions.
- Related work: The paper studies realistic distribution shifts rather than adversarial examples, which prior work characterizes as potentially disconnected from real-world performance concerns.
- Related work: Annotation artifacts can create shortcuts that support high in-distribution accuracy but make models brittle when OOD examples lack the same spurious patterns.
6 Conclusion
The paper introduces a broad benchmark for realistic OOD shifts and finds that pretrained Transformers generally improve both OOD generalization and detection, while remaining only moderately robust.
- 6 Conclusion: The benchmark restructures and matches previous datasets across several NLP tasks to induce numerous realistic distribution shifts.
- 6 Conclusion: Pretrained Transformers generalize to OOD examples far better than previous models, often markedly reducing the IID/OOD generalization gap.
- 6 Conclusion: Pretrained Transformers detect OOD examples surprisingly well compared with previous models.
- 6 Conclusion: The evaluation concludes that pretrained Transformers are moderately robust, leaving room for future research on robustness.
A.1 Significant OOD Accuracy Drops
OOD evaluation reveals noticeable accuracy drops for STS-B, ReCoRD, and SST-2/IMDb compared with in-distribution testing.
- A.1 Significant OOD Accuracy Drops: STS-B, ReCoRD, and SST-2/IMDb show a noticeable drop in accuracy when evaluated on OOD examples.
A.2 Minor OOD Accuracy Drops
OOD accuracy declines are relatively minor for Amazon reviews, although BERT remains vulnerable to extreme shifts between unrelated categories.
- BERT shows minor performance declines on the Amazon dataset under distribution shift.The Amazon results are reported for BERT Base in Figure 6.
- BERT is robust across closely related clothing categories, including men’s, women’s, and baby clothing.
- BERT struggles under an extreme shift such as training on Baby Clothing and testing on Music.This shift is closer to a domain adaptation setting.
A.3 OOD Detection
The paper evaluates OOD detection using confidence-based anomaly scores and reports FAR95 and AUROC. Pretrained Transformers generally detect OOD examples more reliably than earlier models, though robustness remains imperfect.
- OOD detection uses the negative maximum prediction confidence as an anomaly score.The paper reports FAR95 and AUROC for evaluating detection performance.
- OOD accuracy declines vary across tasks: STS-B, ReCoRD, and SST-2/IMDb show noticeable drops, whereas MNLI and Yelp show smaller differences.The reported comparisons include Pearson correlation for STS-B, exact match for ReCoRD, and accuracy for the classification datasets.
- 50% AUROC represents random-chance OOD detection, while 100% represents perfect detection.