PaperlayerView exact revision on arXiv

Source-linked AI summary

Types of Out-of-Distribution Texts and How to Detect Them

Udit Arora, William Huang, He He

arXiv:2109.06827v2cs.CLcs.LG

TL;DR

The paper addresses limited consensus on what counts as OOD data and how detection methods should be evaluated. It categorizes shifts as background or semantic and compares calibration with density estimation, finding that their relative performance depends on shift type and that both encounter challenge-data failures.

  • Problem

    OOD detection lacks a rigorous definition, and prior NLP work often relies on arbitrary dataset pairs, despite distribution shifts taking varied forms.

    Source: §1 · p. 1

  • Method

    The paper categorizes OOD examples as background or semantic shifts and studies calibration and density estimation on natural text and simulated data.

    Source: §7 · p. 9

  • Results

    Density estimation outperforms calibration under background shifts, while calibration performs better under semantic shifts; challenge examples expose failure cases for both.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    Explicitly defining the targeted OOD shift can inform method choice and guide benchmarks that reflect realistic distribution shifts.

    Source: §7 · p. 9

    The authors identify a need for better frameworks and an explicit OOD definition to support progress in detection and realistic benchmarks.

    Source: §7 · p. 9

Abstract

from arXiv · show

Despite agreement on the importance of detecting out-of-distribution (OOD) examples, there is little consensus on the formal definition of OOD examples and how to best detect them. We categorize these examples by whether they exhibit a background shift or a semantic shift, and find that the two major approaches to OOD detection, model calibration and density estimation (language modeling for text), have distinct behavior on these types of OOD data. Across 14 pairs of in-distribution and OOD English natural language understanding datasets, we find that density estimation methods consistently beat calibration methods in background shift settings, while performing worse in semantic shift settings. In addition, we find that both methods generally fail to detect examples from challenge data, highlighting a weak spot for current methods. Since no single method works well across all settings, our results call for an explicit definition of OOD examples when evaluating different detection methods.

1 Introduction

The paper addresses the lack of consensus on OOD examples by separating background and semantic shifts, then compares calibration and density estimation across those settings. The methods favor different shift types, and both face failures on challenge data.

1 Introduction

  • Identifying OOD inputs matters because distribution changes can make predictions unreliable, and detection can support abstention, human feedback, or information gathering.Source: §1 · p. 1
  • OOD examples can arise from different shifts, so detection methods cannot be expected to work well for arbitrary OOD data without assumptions about the test distribution.The paper excludes task shift because users are often aware of a model’s intended task.Source: §1 · pp. 1–2
  • The paper distinguishes background shifts, such as domain or style changes, from semantic shifts, such as unseen classes.It evaluates calibration methods using prediction confidence and density estimation methods using the distribution of training inputs.Source: §1 · pp. 1–2
  • Density estimation consistently outperforms calibration on background shifts, while calibration performs better on semantic shifts.The comparison uses simulated and real ID/OOD pairs.Source: §1 · p. 2
  • Both methods have challenge-data failure cases: calibration can fail under spurious semantic confidence, while repetition can fool language-model detectors.The paper calls for more explicit OOD definitions and corresponding evaluation datasets.Source: §1 · p. 2

2 Categorization of OOD Examples

The paper defines OOD detection as identifying whether an inference-time input comes from the training distribution. It categorizes shifts by whether background or semantic features change, while allowing both to occur together.

2.1 Problem Statement

  • OOD detection asks whether an inference-time input was sampled from the training distribution.Source: §2.1 · p. 2

2.2 Types of Distribution Shifts

  • A representation is decomposed into background features independent of the label and semantic features that are not label-independent.Background features capture population-level statistics; semantic features correlate with the label.Source: §2.2 · p. 2
  • Background shift changes background-feature distributions, whereas semantic shift changes semantic-feature distributions; both may occur, and categorization follows the dominant shift.Examples include reviews changing from IMDB to GoodReads and a flight-booking system receiving a meal-voucher request.Source: §2.2 · pp. 2–3

3 OOD Detection Methods

The paper describes OOD detection through a thresholded score and contrasts confidence-based calibration with input-density estimation. These approaches attend to different aspects of the input.

3 OOD Detection Methods

  • An input is classified as OOD when its score falls below a predefined threshold.Source: §3 · p. 3
  • Calibration uses classifier confidence, while text density estimation uses language-model likelihood, scored by token perplexity.The maximum softmax probability is a simple confidence baseline; perplexity avoids sequence-length bias.Source: §3 · p. 3
  • Calibration relies on p(y | x) and is more influenced by label-correlated semantic features, whereas density estimation is sensitive to both background and semantic features.Source: §3 · p. 3

4 Simulation of Distribution Shifts

The simulation uses a controlled binary-classification setup to compare calibration and density estimation under semantic and background shifts. Calibration performs better as semantic shift grows, while density estimation performs better under larger background shifts.

4.1 Data Generation

  • The toy binary-classification experiment provides true calibrated scores and input densities, letting the authors study optimal detectors without estimation errors.ID examples come from a Gaussian mixture whose class centroids have opposite semantic components and a shared background component; the LDA classifier uses zero weights for background features.Source: §4 · p. 3, §4.1 · pp. 3–4

4.2 Semantic Shift

  • Semantic shift is simulated by varying overlap between the ID and OOD semantic features, with larger overlap rate r treated as stronger shift.The experiment uses 40 semantic and 160 background dimensions, varies r in 10% increments, and reports means across 20 trials with 95% confidence bands.Source: §4.2 · p. 4

4.3 Background Shift

  • Background shift is simulated by translating both class means along the background-feature direction while leaving semantic coordinates unchanged.The experiment varies the semantic/background dimension split while keeping their total at 200.Source: §4.3 · p. 4

4.4 Simulation Results

  • Calibration outperforms density estimation under larger semantic shifts, while density estimation performs better under larger background shifts.Under semantic shifts, calibration ranges from chance with completely overlapping semantic features to stronger performance when there is no overlap; it cannot improve under background shifts because those features do not affect p(y | x).Source: Figure 2 · p. 4, §4.4 · p. 4

5 Experiments and Analysis

Across semantic and background shifts, calibration performs better on semantic-shift pairs, while density estimation generally performs better on background-shift pairs. Controlled experiments and challenge data further expose different sensitivities and failure cases, motivating shift-aware evaluation and clearer definitions of OOD.

5.1 Setup

  • The study compares calibration and density-estimation methods on 14 ID/OOD pairs categorized as background shift or semantic shift, plus eight challenge-dataset pairs, evaluating detectors by AUROC and FAR95.Sentence probability p(x) is also used as a score, but is highly sensitive to sequence lengths.Source: §5 · p. 4, §5.1 · pp. 4–5

5.2 Semantic Shift; 5.3 Background Shift

  • MSP consistently outperforms PPL on semantic-shift pairs, while PPL outperforms MSP on background-shift pairs except MNLI versus RTE.The semantic-shift comparison is reported in Table 1; the background-shift comparison is reported for all pairs except MNLI versus RTE.Source: Table 1 · p. 5, §5.2 · p. 5, Table 2 · p. 6

5.3 Background Shift

  • Calibration performs close to random when discriminative models generalize well on background-shift pairs, as their confidence can remain high on correct OOD predictions.The authors suggest domain adaptation may be more appropriate when the shift is predominantly background-related.Source: §5.3 · p. 6

5.4 Analysis

  • For controlled shifts, PPL is more sensitive to added background text, while MSP is more sensitive to the number of ID/OOD classes.Background shift was varied by appending longer unrelated text; semantic shift was varied by moving classes from ID to OOD.Source: §5.4 · p. 6, Figure 3 · p. 6
  • Larger discriminative models narrow the background-shift performance gap with language models, while performance on the semantic-shift pair remains largely unchanged.The authors speculate that the larger model may learn some background features in its representation.Source: §5.4 · p. 6, Table 3 · p. 6

5.5 Challenge Data

  • Both methods lose performance on challenge data; MSP struggles with most examples, PPL struggles on small shifts, and PPL performs worse than random on several repetitive challenge categories.The authors also identify human-generated counterfactual data as particularly difficult to detect.Source: §5.5 · pp. 7–8, Table 4 · p. 7

5.6 Discussion

  • The authors recommend shift-specific evaluation and choosing calibration for anticipated semantic shifts or density estimation for background shifts, while emphasizing that challenge data exposes failures in both.They call for a more explicit definition of OOD to support methods and benchmarks that reflect realistic distribution shifts.Source: §5.6 · p. 8

6 Related Work

Related work considers distribution shifts in real-world benchmarks, calibration-based OOD detection, and methods that combine discriminative and generative signals. This paper frames method effectiveness as dependent on whether a detector’s assumptions match the test-time shift.

Distribution shift in the wild

  • Real-world benchmarks include diverse shifts, and no single detection method wins across the board.The paper explores semantic and background shift as two axes for characterizing distribution shifts.Source: §6 · p. 8

OOD detection in NLP; Inductive bias in OOD detection

  • Prior work studies calibration improvements and task-specific methods, while this analysis identifies a calibration limitation under background shift.Other approaches use outlier exposure or hybrid discriminative and generative objectives to balance semantic and background features.Source: §6 · p. 8

Domain adaptation versus OOD detection

  • OOD detection can permit abstention without access to OOD data, whereas domain adaptation aims to perform well across domains and usually relies on OOD data.The distinction is presented as especially relevant for safety-critical settings where incorrect predictions can be costly.Source: §6 · pp. 8–9

7 Conclusion

The paper categorizes OOD examples as background or semantic shifts and finds that calibration and density estimation perform differently across them. Challenge examples expose failures in both approaches, motivating clearer OOD definitions and benchmarks for realistic shifts.

7 Conclusion

  • Density estimation outperforms calibration under background shifts, while calibration performs better under semantic shifts.The paper studies these patterns in natural text and simulated data.Source: §7 · p. 9
  • Challenge-example failures show that the simplified framework leaves room for better frameworks and a more explicit definition of OOD.Such a definition can inform benchmarks that reflect realistic distribution shifts.Source: §7 · p. 9
  • Explicitly stating which OOD types a predictor targets can guide method choice and future modeling decisions.The paper suggests directing test-time fine-tuning or data augmentation toward a specific shift type.Source: §7 · p. 9
  • Hybrid models using calibration and density estimation are motivated when background and semantic shifts occur together.The paper notes that both shift types can happen simultaneously.Source: §7 · p. 9

Ethical Considerations

OOD detection may help identify distribution shifts affecting minority groups, but domain-based interventions also carry a risk of demographic discrimination. The cited example illustrates both the potential relevance and ethical risk of such interventions.

Ethical Considerations

  • A BERT-based model trained with ERM on CivilComments had 34.2% lower accuracy on its worst group than on average.The domains are annotated by whether comments mention any of eight demographic identities.Source: Ethical Considerations · p. 9
  • The authors suggest detection could alter model behavior on comments from domains associated with demographic groups.The example concerns toxic-comment classification and domain annotations based on demographic identities.Source: Ethical Considerations · p. 9
  • The same pipeline could be used to identify and discriminate against demographics displaying background shifts.The paper explicitly flags this as a risk of domain-triggered behavior changes.Source: Ethical Considerations · p. 9

A Example Probability

The appendix compares token perplexity with sequence log likelihood as density-estimation detection scores. It reports that sequence length can substantially affect performance, including a near-total failure for one IMDB-to-SST-2 setting.

A Example Probability

  • For IMDB as ID data, log p(x) fails for SST-2, with close to 100 FAR95 and near 0 AUROC.The authors attribute the suspected failure to paragraph-length IMDB examples versus one- to two-sentence SST-2 examples.Source: A Example Probability · p. 14
  • Unlike PPL, log p(x) sums token log likelihoods without averaging for sequence length.The passage defines log p(x) as the sum of conditional token log probabilities.Source: A Example Probability · p. 14
  • The authors suspect longer IMDB inputs naturally receive smaller log p(x) than SST-2 inputs, causing threshold-based AUROC failure.The passage contrasts full-paragraph IMDB reviews with one- to two-sentence SST-2 examples.Source: A Example Probability · p. 14

B FAR95 Results

The appendix reports FAR95 results for density-estimation and calibration scores across several ID/OOD pair groups. These results generally align with AUROC, with a noted DBPedia exception.

B FAR95 Results

  • FAR95 measures the false-positive rate at 95% recall; lower values indicate better performance.For OOD detection, it measures ID-data misclassification at 95% OOD-classification recall.Source: B FAR95 Results · p. 14
  • FAR95 results are in line with AUROC except for DBPedia, where density estimation performs better.The authors suggest the difference may reflect AUROC's cumulative nature versus FAR95 as a point measurement.Source: B FAR95 Results · pp. 14–15
  • The reported tables cover domain-driven background shifts, MNLI-genre shifts, and challenge data, comparing PPL, MSP, and Oracle.The challenge-data table also identifies the primary shift type; its caption specifies that lower MSP/PPL scores are bolded.Source: Table 6 · p. 14, Table 7 · p. 14, Table 8 · p. 15

C Background shift in MNLI Genres

Across five MNLI genres, density estimation with PPL significantly outperforms calibration for background-shift detection. Calibration performance is close to random because the discriminative model is more confident on correct OOD predictions.

C Background shift in MNLI Genres

Related papers

Loading 2109.06827v2…