PaperlayerView exact revision on arXiv

Source-linked AI summary

Revisiting Out-of-distribution Robustness in NLP: Benchmark, Analysis, and LLMs Evaluations

Lifan Yuan, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Fangyuan Zou, Xingyi Cheng, Heng Ji, Zhiyuan Liu, Maosong Sun

arXiv:2306.04618v2cs.CLcs.CRcs.LG

TL;DR

The paper addresses weak and insufficiently challenging OOD benchmark design in NLP. It proposes a dataset-selection protocol and the BOSS suite, then finds distinct ID-OOD correlation patterns, limited gains from robustness-enhanced methods, and differing advantages for domain-specific fine-tuning versus LLM in-context learning across ID and OOD data.

  • Problem

    Previous NLP OOD evaluations often use insufficiently challenging or insufficiently distinct distribution shifts, limiting rigorous robustness assessment.

  • Method

    The paper proposes a protocol for selecting distinct and challenging ID-OOD datasets, introduces BOSS across five tasks and 20 datasets, and evaluates PLMs, robustness methods, and LLM adaptation paradigms.

  • Results

    Domain-specific fine-tuning is preferable on ID examples with sufficient data, whereas LLMs using in-context learning perform better on OOD instances; robustness-enhanced methods provide no significant overall gain over vanilla fine-tuning.

  • Takeaways & Limitations

    OOD robustness evaluation should use systematically distinct and challenging shifts, while model choice and adaptation strategy should distinguish ID from OOD settings.

  • Takeaways & Limitations

    Some LLM evaluation datasets may overlap with pretraining data, and BOSS covers only five NLP tasks rather than the full NLP literature.

Abstract

from arXiv · show

This paper reexamines the research on out-of-distribution (OOD) robustness in the field of NLP. We find that the distribution shift settings in previous studies commonly lack adequate challenges, hindering the accurate evaluation of OOD robustness. To address these issues, we propose a benchmark construction protocol that ensures clear differentiation and challenging distribution shifts. Then we introduce BOSS, a Benchmark suite for Out-of-distribution robustneSS evaluation covering 5 tasks and 20 datasets. Based on BOSS, we conduct a series of experiments on pre-trained language models for analysis and evaluation of OOD robustness. First, for vanilla fine-tuning, we examine the relationship between in-distribution (ID) and OOD performance. We identify three typical types that unveil the inner learning mechanism, which could potentially facilitate the forecasting of OOD robustness, correlating with the advancements on ID datasets. Then, we evaluate 5 classic methods on BOSS and find that, despite exhibiting some effectiveness in specific cases, they do not offer significant improvement compared to vanilla fine-tuning. Further, we evaluate 5 LLMs with various adaptation paradigms and find that when sufficient ID data is available, fine-tuning domain-specific models outperform LLMs on ID examples significantly. However, in the case of OOD instances, prioritizing LLMs with in-context learning yields better results. We identify that both fine-tuned small models and LLMs face challenges in effectively addressing downstream tasks. The code is public at \url{https://github.com/lifan-yuan/OOD_NLP}.

1 Introduction

The paper argues that NLP OOD evaluation is undermined by weak benchmark shifts and investigates how PLM ID performance relates to OOD generalization. It introduces a benchmark protocol and evaluates robustness-enhanced methods and LLM adaptation paradigms.

  • Previous OOD benchmarks often select datasets heuristically or by popularity, allowing similar distributions that weaken rigorous robustness evaluation.
  • The proposed protocol prioritizes large, diverse ID data, distinct OOD distributions, and challenging shifts selected by performance degradation.
  • BOSS-based analysis identifies three ID-OOD correlation types by varying model scale, training steps, sample availability, and tunable parameters.
  • Five robustness-enhanced methods remain generally comparable to vanilla fine-tuning, with only case-specific OOD improvements.
  • With sufficient training data, domain-specific fine-tuning is preferable for ID examples, whereas LLM in-context learning performs better on OOD instances.

2 BOSS Benchmark

BOSS constructs challenging OOD evaluations by selecting large, diverse ID datasets and distinct, dissimilar OOD datasets, then applies this protocol across five NLP tasks. Its dataset-selection examples illustrate how semantic similarity and cross-distribution performance degradation guide benchmark construction.

  • 2.2 Protocol to Construct OOD benchmark.: The protocol selects large, diverse ID datasets that provide comprehensive task knowledge rather than encouraging reliance on biased features.
  • 2.2 Protocol to Construct OOD benchmark.: OOD candidates are filtered for semantic dissimilarity and prioritized when an ID-trained model suffers the greatest performance drop.
  • 2.3 Dataset Selection: For sentiment analysis, Amazon is selected as ID because it contains reviews from 29 product categories, while DynaSent, SemEval, and SST form distinct challenging shifts.
  • 2.4 BOSS: BOSS covers five NLP tasks with one ID dataset and three associated OOD datasets per task, spanning classification, structured prediction, and extraction.
  • 2.4 BOSS: The benchmark’s task coverage includes sentiment analysis, toxic detection, natural language inference, named entity recognition, and extractive question answering.

3 Analysis of OOD Robustness

The analysis examines how ID performance relates to OOD generalization under varied model and training conditions, revealing three recurring correlation patterns plus an adversarial outlier.

  • Results: Three ID–OOD correlation types emerge across tasks: monotonic linear positive, monotonic piecewise linear positive, and non-monotonic V-shaped, alongside one exceptional case.The full results are organized around these patterns, with the exception occurring for Civil Comments to AdvCivil.
  • Type I: Type I shows that OOD performance rises linearly with ID performance, but its shallower-than-diagonal slope indicates increasing degradation as ID performance improves.This pattern is prevalent in sentiment analysis, named entity recognition, and most toxic detection pairs.
  • Type II: Type II, observed in extractive question answering, has a turning point after which small ID gains produce substantial OOD improvements.Before the turning point, OOD performance grows slowly; afterward, its increase accelerates.
  • Type III: Type III, mainly found in NLI, is V-shaped: OOD degradation worsens initially, then improves as models achieve higher ID performance.The proposed explanation is that medium-capacity models learn distractor-task signals such as word overlap before stronger models focus on the underlying task.
  • Outlier: The Civil Comments-to-AdvCivil shift is an outlier with monotonic negative correlation, consistent with greater reliance on spurious features that adversarial samples remove.As training increases dependence on spurious correlations, performance degradation on adversarial samples intensifies.

4 Evaluation of OOD Robustness

The evaluation finds that robustness-enhanced methods rarely improve consistently over vanilla fine-tuning, while adaptation choice creates a trade-off between ID and OOD performance for small models and LLMs.

  • Robustness-enhanced Methods: Vanilla fine-tuning remains a strong baseline, while no robustness method performs consistently well across all datasets within a task.FreeLB helps on more than half of the datasets, whereas model ensembling generally degrades performance.
  • Large Language Models: Given enough ID data, fine-tuned domain-specific models perform best on ID examples, whereas LLMs perform better on most OOD datasets.The benchmark-level comparison favors supervised fine-tuned small models and full-data T5-3B for ID evaluation, but LLMs for OOD evaluation.
  • Large Language Models: In-context learning does not improve small models but generally helps Turbo and significantly improves the LLaMA-series and Davinci3.For Turbo, ID demonstrations help on nearly two-thirds of datasets; for LLaMA-series and Davinci3, gains appear on most datasets.
  • Large Language Models: ID examples are generally more effective ICL contexts than examples from the evaluated OOD dataset.OOD contexts can help EQA, but they are otherwise less effective for identifying the target label space.
  • Large Language Models: Both paradigms remain limited: LLMs underperform fine-tuned models on ID data, while fine-tuned models lag LLMs on OOD performance.The discussion also highlights LLM inference costs and continuing difficulty on challenging downstream tasks such as EQA.

5 Related Work

Related work studies OOD robustness through diverse distribution shifts and automatic or static evaluation approaches.

  • Distribution Shifts: NLP distribution shifts include domain, temporal, spurious-correlation, adversarial, and backdoor shifts.These settings target changes in domains, time, learned shortcuts, perturbed inputs, or intentionally introduced correlations.
  • OOD Evaluation: Automatic OOD evaluation applies textual transformations and requires suitable transformations, metrics, and validity checks.Static evaluation is presented as a contrasting approach to automated transformations.

6 Conclusion

The paper identifies benchmark and evaluation deficiencies in NLP OOD robustness research and responds with a benchmark protocol, BOSS, and empirical analyses of models and methods.

  • Conclusion: The paper proposes a benchmark construction protocol and BOSS to support standardized OOD robustness evaluation in NLP.It also studies ID-OOD correlations, existing robustness methods, and LLM challenges.

Limitation

The work is limited by possible LLM pretraining contamination and by BOSS covering only five NLP tasks.

  • Limitations: Some benchmark datasets may have appeared in LLM pretraining, limiting their suitability for testing generalizability.The authors note that the protocol allows the benchmark to be updated as new datasets become available.
  • Limitations: BOSS covers only five tasks and therefore is not a comprehensive collection of current NLP literature.The authors explain the task selection in Appendix A.1.

A.1 What is the rationale for current task selection and why not include more difficult tasks?

BOSS focuses on five tasks whose distribution shifts can be evaluated more reliably, while excluding NLG and commonsense reasoning because their OOD evaluation presents unresolved difficulties.

  • Current task selection: The benchmark covers natural language understanding, structured data prediction, and question answering as core language-model competencies.The selected tasks span sentiment analysis, toxic detection, natural language inference, named entity recognition, and extractive question answering.
  • Why not NLG: NLG is excluded because reference-based metrics may not accurately assess generated-text quality when output styles and possible responses vary across domains.An in-domain model may produce stylistically different but potentially valid responses on OOD datasets, while current metrics emphasize resemblance to references.
  • Why not commonsense reasoning: Commonsense reasoning is excluded because datasets can require substantially different abilities, making knowledge transfer from an ID dataset difficult to assess cleanly.HellaSwag emphasizes world knowledge and logical reasoning, whereas StepGame relies on spatial imagination without requiring world knowledge.
  • Evaluation design: The benchmark’s distribution-shift analysis uses SimCSE representations, selected over alternatives because it is widely used and reported as more advanced than sentence-BERT.The authors also compare SimCSE variants and a Sentence Transformers model, observing variation in magnitude but similar trends.

C Datasets

The dataset collection spans five NLP tasks and uses diverse sources and task-specific dataset families to support OOD robustness evaluation.

  • Dataset coverage: The dataset suite covers sentiment analysis, toxic detection, natural language inference, named entity recognition, and extractive question answering.The passages identify representative datasets within each task, including review, social-media, biomedical, Wikipedia, and question-answering collections.
  • Sentiment Analysis: Sentiment-analysis datasets include product and movie reviews from Amazon, IMDb, and Yelp.These datasets use binary or review-rating sentiment labels, depending on the collection.
  • Toxic Detection: Toxic-detection datasets combine abusive, hateful, offensive, and adversarial content from forums, social media, and constructed attacks.AdvCivil introduces typos and distracting sentences and is validated by human annotators.
  • Natural Language Inference: NLI datasets draw from biomedical abstracts, web text, contracts, and other natural-language sources, with entailment labels defined over premise–hypothesis relations.BioNLI uses experimental evidence from PubMed abstracts, while other datasets include clause-embedded or general NLI texts.
  • NER and EQA: NER and extractive question answering datasets include diverse entity domains and question-answering sources such as Wikipedia, Google search results, and long documents.The NER collection spans topics including artificial intelligence, literature, music, natural science, and politics.

C.2.1 Sentiment Analysis

Sentiment-analysis data are harmonized through label balancing, task-specific relabeling, dataset merging or sampling, and conversion to a common classification setup.

  • Label balancing: Class-imbalanced datasets are balanced by retaining the same number of samples per class, equal to the smallest class size.The procedure discards redundant samples from larger classes.
  • Dataset adaptation: Amazon, SST, and Yelp are adapted to ternary sentiment classification by retaining or mapping selected rating levels.Amazon and Yelp retain ratings 1, 3, and 5, while SST scores are binned into five labels before labels 1 and 3 are dropped.
  • Dataset construction: Review datasets are consolidated or sampled to manage scale and preserve representation across product categories and sentiment labels.Amazon subsets are reduced to 20k samples before merging, while Yelp contributes 10k training reviews per retained label.
  • Dataset adaptation: IMDb, SemEval, and Dynasent receive simpler processing through label balancing or merging their available rounds.IMDb and SemEval require only balancing, whereas Dynasent combines two training and test rounds.
  • Toxic Detection: Toxic-detection data are cleaned of usernames, emojis, tags, and URLs, with additional label and split processing where required.AbuseAnalyzer is split 8:2, Civil Comments uses a toxicity threshold of 0.5, and several multiclass datasets are mapped to binary toxicity labels.
  • Natural Language Inference: NLI datasets generally require no label-imbalance correction, but ANLI rounds are merged, ANLI duplicates are removed from DocNLI, and MNLI uses its matched validation split.BioNLI, CB, ContractNLI, MNLI, SNLI, and WANLI otherwise receive no specific processing.
  • NER and EQA: NER datasets are aligned to common label schemes and the BIO tagging format, while extractive QA datasets are normalized to the SQuAD format.NER mappings support either Few-NERD or CoNLL alignment, and QA preprocessing follows MRQA.

D Dataset Selection for Other Tasks

For each task, the benchmark selects ID data for size and diversity, filters OOD candidates by similarity, and retains shifts that produce substantial performance degradation.

  • D.1 Toxic Detection: Toxic detection uses Civil Comments as ID and AdvCivil, Implicit Hate, and ToxiGen as OOD after similarity and degradation screening.Hate Speech is discarded because the ID model retains accuracy above 80, whereas the retained shifts produce performance drops exceeding 20 points.
  • D.2 Natural Language Inference: NLI uses MNLI as ID and ANLI, ContractNLI, and WANLI as OOD after excluding incompatible label spaces, small datasets, similar candidates, and weak shifts.BioNLI and DocNLI cause little degradation, while the retained alternatives produce significant degradation.
  • D.3 Name Entity Recognition: NER uses Few-NERD as ID and CoNLL, E-NER, and WNUT as OOD, excluding CrossNER because it shares the ID dataset’s source.Label sets are aligned to Few-NERD, and DeBERTa is used because standard prompt-based tuning for NER is lacking.
  • D.4 Extractive Question Answering: Extractive QA uses SQuAD as ID and AdvQA, NewsQA, and SearchQA as OOD because the ID model performs poorly on these shifts but well on SQuAD Shifts.The final shift is SQuAD → (AdvQA, NewsQA, SearchQA).

E.1.2 Full Results

Across tasks, ID–OOD relationships follow distinct patterns rather than a single universal trend: sentiment analysis and NER are linear, toxic detection mixes cases, NLI is V-shaped, and extractive QA is piecewise linear.

  • Sentiment Analysis: Sentiment-analysis results contain two positive linear trends, with flatter T5-small and steeper T5-base/T5-large fits beginning near ID accuracies of 60 and 85.
  • Toxic Detection: Toxic detection combines an AdvCivil outlier with Type I curves for Implicit Hate and ToxiGen, where the ID–OOD gap eventually reverses during training.
  • Natural Language Inference: NLI shows non-monotonic V-shaped ID–OOD curves, with the slope ordering differing across ANLI, ContractNLI, and WANLI.
  • Named Entity Recognition: NER exhibits Type I relationships below the diagonal, with CoNLL closest to linear parity and WNUT showing the weakest correlation.
  • Extractive Question Answering: Extractive QA follows Type II piecewise curves, with the sharpest change in SearchQA and the smoothest change in NewsQA.
Loading 2306.04618v2…