Source-linked AI summary
Unlearn Dataset Bias in Natural Language Inference by Fitting the Residual
He He, Sheng Zha, Haohan Wang
TL;DR
NLI models can learn superficial dataset cues whose label associations change across distributions, undermining performance on challenging examples. The paper formalizes this shift and proposes DRiFt, which fits residuals after a biased model; across SNLI and MNLI, it improves challenge-set performance with limited degradation on original tests.
Problem
NLI models can learn superficial cues whose association with labels changes across training and test distributions, causing brittle performance on challenging examples.
Method
DRiFt trains a biased model on insufficient features, then trains a debiased model by fitting the biased model’s residuals on hard examples.
Results
DRiFt improves performance on challenge datasets across three NLI models trained on SNLI and MNLI, with only slight degradation on original test sets.
Takeaways & Limitations
The method improves challenge-data performance when dataset bias is known, while current improvements rely largely on task-specific prior knowledge.
Takeaways & Limitations
DRiFt can discard useful information when its insufficient feature representation encodes possible meanings, and model capacity limits what is learned beyond that representation.
Abstract
from arXiv · showhide
Statistical natural language inference (NLI) models are susceptible to learning dataset bias: superficial cues that happen to associate with the label on a particular dataset, but are not useful in general, e.g., negation words indicate contradiction. As exposed by several recent challenge datasets, these models perform poorly when such association is absent, e.g., predicting that "I love dogs" contradicts "I don't love cats". Our goal is to design learning algorithms that guard against known dataset bias. We formalize the concept of dataset bias under the framework of distribution shift and present a simple debiasing algorithm based on residual fitting, which we call DRiFt. We first learn a biased model that only uses features that are known to relate to dataset bias. Then, we train a debiased model that fits to the residual of the biased model, focusing on examples that cannot be predicted well by biased features only. We use DRiFt to train three high-performing NLI models on two benchmark datasets, SNLI and MNLI. Our debiased models achieve significant gains over baseline models on two challenge test sets, while maintaining reasonable performance on the original test sets.
1 Introduction
NLI models can exploit superficial dataset cues that fail under distribution shift. DRiFt addresses this by fitting residuals after learning from known biased features, improving challenge-set performance while largely preserving original-test performance.
- Negation words can become associated with contradiction because crowd workers mechanically negate premises when creating NLI examples.That association may disappear at test time, causing models that rely on “not” to fail.
- Distribution shift makes models rely on cues such as lexical overlap that work on training examples but fail on challenging test examples.NLI models may predict entailment from high word overlap despite different sentence meanings.
- Removing biased textual features can hurt because they may conflate superficial cues with useful semantic information.Existing bias-removal methods can also remain insufficient even when the bias is believed irrelevant to prediction.
- DRiFt first trains a biased model on insufficient features, then trains a debiased model to fit its residuals on examples those features cannot predict well.At test time, only the debiased model is used for prediction.
- DRiFt-trained models improve performance on several challenge datasets with only slight degradation on the original test sets across SNLI and MNLI.The method is evaluated with three NLI models trained on the two benchmark datasets.
2 Problem Statement
The paper frames dataset bias as label shift in the relationship between biased input features and labels across training and test distributions. It seeks to correct models using known bias information without removing features that may retain necessary meaning.
- Good performance from insufficient input, such as a hypothesis-only classifier, flags that dataset bias may be present.Such performance indicates examples can be solved without substantial reasoning about complete sentence meanings.
- Dataset bias is defined as a partial representation of inputs whose label relationship shifts between training distribution P and test distribution Q.The formulation decomposes input x into biased features b(x) and stable semantic features g(x), assumed conditionally independent given y.
- Stable features satisfy p(g(x) | y) = q(g(x) | y), whereas biased features are predictive on P but may lose their association with labels on Q.In NLI, sentence semantics represent g(x), while source-dependent word choices represent b(x).
- Without examples from Q, good performance there is impossible without additional information about potential biased features.The paper uses domain-specific knowledge, such as word-overlap heuristics in NLI, to correct models trained on P.
- Removing biased features is infeasible when they also contain necessary predictive information, so the paper targets their relation with labels instead.The proposed approach therefore mitigates biased relations rather than deleting the features themselves.
3 Approach
DRiFt debiases NLI by first modeling potential dataset bias with insufficient features, then training a classifier to learn residual information beyond that biased model. Its residual-fitting objective adjusts learning toward examples the biased classifier predicts poorly and away from information it already captures.
- Overview: DRiFt first detects potentially biased examples using prior knowledge and then focuses learning on unbiased, hard examples.The approach uses insufficient features I(x) to approximate known biased features.
- Overview: The biased-feature representation I(x) may miss dataset bias or include useful information, so it is only an approximation of the unknown bias.The paper notes that bag-of-words features can encode some distribution of possible meanings rather than only superficial cues.
- Overview: The biased classifier fs is trained on insufficient features I(x), while the debiased classifier fd learns information beyond fs.Standard empirical risk minimization can rely on biased features when they predict the majority of examples well.
- Overview: fd fits the residual of fs relative to the optimal predictor, with fs fixed while fd parameters are learned by minimizing the loss.At test time, only the debiased classifier fd is used.
- Cross-Entropy Analysis: DRiFt uses the biased classifier's predictions to adjust each example's gradient, reducing learning where biased features already predict the label well.When the biased classifier predicts perfectly, the gradient contribution becomes zero; when its output is uniform, the gradient reduces to the MLE gradient.
- Cross-Entropy Analysis: Under cross-entropy, DRiFt learns fd with p_a(y | x), whereas standard maximum likelihood uses p_d(y | x) with pretrained fs fixed.Its objective adds an example-wise regularizer that encourages pd to have minimal projection on predictions from ps.
4 Experiments
Experiments evaluate DRiFt across SNLI, MNLI, synthetic cheating bias, HANS, and STRESS, comparing it with MLE and RM across several NLI architectures. DRiFt generally improves challenge-set robustness while retaining reasonable in-distribution performance, though its benefits vary with the bias and model.
- Experimental Setup: DRiFt consistently outperforms MLE on challenge datasets across different NLI models, especially when the biased features match the challenge-set bias.Evaluation covers synthetic bias and two challenge datasets using different biased classifiers.
- Experimental Setup: DRiFt trains three NLI architectures—DA, ESIM, and BERT—on SNLI and MNLI, while comparing MLE and RM baselines.The biased models include HYPO, CBOW, and HAND; the debiased models use different capability levels.
- Synthetic Bias: At pcheat = 0.9, MLE accuracy drops about 20% from the no-cheating baseline, whereas DRiFt-HYPO’s maximum drop is about 5%.MLE degrades when most training examples contain cheating features; DRiFt-HYPO maintains similar accuracy across cheating rates.
- Synthetic Bias: In the ideal known-bias case, RM-cheat completely dominates MLE, while DRiFt-HYPO trails it because HYPO captures additional unbiased features.RM-cheat still loses accuracy at high cheating rates because fewer useful examples remain for learning.
- Word Overlap Bias: On HANS, DRiFt improves non-entailment F1 across lexical, subsequence, and constituent subsets with little degradation on entailment, unlike RM.RM improves non-entailment performance but significantly degrades entailment performance.
- Stress Tests: STRESS yields smaller overall debiasing gains because distractor phrases create both label shift on biased features and covariate shift in the input.DA and ESIM can degrade on other classes, particularly with DRiFt-CBOW, because CBOW retains possible semantic information and model capacity is limited.
5 Related Work and Discussion
Related work frames dataset bias as a distribution-shift problem and contrasts representation debiasing, data augmentation, and DRiFt’s use of known bias features.
- Representation-based debiasing removes biased information, but textual cues may mix superficial bias with useful semantics, making complete removal harmful.
- Dataset bias arises when training and real-world data follow different data-generating processes, making shortcuts such as lexical overlap unreliable under shift.
- DRiFt instead uses prior knowledge about potential biased features without requiring target-distribution data, focusing learning on examples those features cannot solve.
- Data augmentation corrects training distributions with challenge-like examples, but often depends on handcrafted rules targeting specific biases.
6 Conclusion
The paper treats nonrepresentative training examples as the central problem and adapts learning to emphasize examples that superficial biased features cannot solve.
- Most training examples may not represent real-world or challenge distributions, so minimizing average training loss no longer matches the objective.
- The proposed debiasing method improves challenge-data performance when the dataset bias is known, but current gains rely largely on task-specific prior knowledge.
- A key next step is developing methods that address different bias types more generally.
A Results on MNLI Stress Test
The MNLI Stress Test evaluates debiased models on tests targeting dataset bias and minority examples, revealing gains on Antonym but not Length.
- DRiFt improves performance on Antonym, possibly because it focuses on challenge examples that superficial cues cannot solve.
- DRiFt does not improve performance on Length in the MNLI Stress Test.
- Table 6 reports complete MNLI Stress Test results, including Overlap and Negation tests for dataset bias and tests of minority examples.