PaperlayerView exact revision on arXiv

Source-linked AI summary

End-to-End Bias Mitigation by Modelling Biases in Corpora

Rabeeh Karimi Mahabadi, Yonatan Belinkov, James Henderson

arXiv:1909.06321v3cs.CL

TL;DR

Strong NLU models can rely on dataset biases rather than learn underlying tasks, limiting robustness to out-of-domain data. The paper trains base models with bias-only models that adjust the loss to down-weight biased examples, and reports substantial robustness and transfer gains across NLI and fact verification evaluations.

  • Problem

    Neural NLU models exploit dataset biases and superficial shortcuts without learning the underlying task, motivating methods that reduce reliance on those biases.

  • Method

    The paper proposes Product of Experts and Debiased Focal Loss, using one or more bias-only models to adjust the base model’s loss and focus learning on hard examples.

  • Results

    7.4 points on HANS, 4.8 points on the SNLI hard set, and 9.8 points on the FEVER symmetric test set were reported as out-of-domain gains, with improved transfer across 12 NLI datasets.

  • Takeaways & Limitations

    The methods provide simple, model-agnostic debiasing that improves robustness to domain shift and can address multiple bias patterns simultaneously.

  • Takeaways & Limitations

    Transfer requires adjusting the PoE debiasing strength for each target dataset, with model selection performed on each target dataset’s development set.

Abstract

from arXiv · show

Several recent studies have shown that strong natural language understanding (NLU) models are prone to relying on unwanted dataset biases without learning the underlying task, resulting in models that fail to generalize to out-of-domain datasets and are likely to perform poorly in real-world scenarios. We propose two learning strategies to train neural models, which are more robust to such biases and transfer better to out-of-domain datasets. The biases are specified in terms of one or more bias-only models, which learn to leverage the dataset biases. During training, the bias-only models' predictions are used to adjust the loss of the base model to reduce its reliance on biases by down-weighting the biased examples and focusing the training on the hard examples. We experiment on large-scale natural language inference and fact verification benchmarks, evaluating on out-of-domain datasets that are specifically designed to assess the robustness of models against known biases in the training data. Results show that our debiasing methods greatly improve robustness in all settings and better transfer to other textual entailment datasets. Our code and data are publicly available in \url{https://github.com/rabeehk/robust-nli}.

1 Introduction

The paper addresses dataset biases that let neural NLU models exploit superficial shortcuts instead of learning underlying tasks, harming out-of-domain performance. It proposes end-to-end debiasing strategies using bias-only models and reports substantial robustness gains across challenging evaluation settings.

  • Motivation: Neural NLU models can exploit dataset shortcuts and annotation artifacts without learning the underlying task, leading to poor out-of-domain performance.In NLI, hypothesis words can correlate strongly with inference labels, allowing models to perform without considering the premise.
  • Approach: The paper proposes two end-to-end debiasing techniques that adjust cross-entropy loss using identified bias-only models.The methods down-weight examples classified correctly by the bias-only model and focus learning on harder examples.
  • Approach: The strategies are model agnostic and designed to address common biases across datasets and domains.The authors describe them as simple, highly effective, and requiring only a simple model on top of the base model.
  • Evaluation: The evaluation covers NLI and fact verification benchmarks with challenging out-of-domain datasets designed to limit spurious cues.The experiments include HANS, hard SNLI and MNLI sets, and the FEVER Symmetric test set.
  • Results: 7.4 points on HANS, 4.8 points on the SNLI hard set, and 9.8 points on the FEVER symmetric test set were reported as gains over challenging out-of-domain data.The contribution summary states that these results set a new state-of-the-art and that transfer improved across 12 NLI datasets.

2 Related Work

Prior work addresses dataset bias through balanced data, adversarial datasets, sample reweighting, and adversarial representation learning. The paper instead emphasizes end-to-end debiasing, evaluates new hard domains, and studies transfer and multiple bias patterns.

  • Prior approaches: Dataset augmentation and adversarial datasets have been proposed to balance cues or expose models to harder examples, but collecting large new datasets is costly.This motivates training strategies that operate on existing biased datasets.
  • Prior approaches: A prior fact-verification method identifies label-associated n-grams and solves an optimization problem to assign balancing weights to training samples.The paper contrasts this sample-weighting approach with its end-to-end debiasing strategies.
  • Prior approaches: Related adversarial work removes features enabling a hypothesis-only model to succeed in NLI sentence encoders.The cited approach targets representation-level bias removal rather than the paper’s loss-adjustment strategies.
  • Positioning: Concurrent PoE methods use two-stage training, whereas this paper trains its methods end to end and additionally studies DFL, multiple biases, new hard domains, bias correlations, and transfer.The paper reports that DFL is sometimes superior to PoE.

3 Reducing Biases

The paper reduces dataset-bias reliance by training a base model alongside bias-only models and modifying the loss to emphasize examples those models fail. It presents Product of Experts and Debiased Focal Loss, extends both to multiple biases, and removes the bias-only branch after training.

  • 3.1 Bias-only Branch: The approach first identifies shortcuts and trains bias-only models using only biased features, then incorporates their predictions into base-model training.The bias-only branch is discarded after training, leaving the base model for prediction.
  • 3.1 Bias-only Branch: The methods require prior knowledge of possible shortcuts because no out-of-domain data is available during training.Bias patterns must therefore be specified in advance through one or more bias-only models.
  • 3.2.1 Method 1: Product of Experts: Product of Experts combines bias-only and base-model probability distributions so the base model learns from characteristics beyond the identified biases.Its loss decreases updates on examples that the bias-only model predicts accurately; unbiased examples retain the usual cross-entropy behavior.
  • 3.2.2 Method 2: Debiased Focal Loss: Debiased Focal Loss directly uses bias-only predictions to down-weight biased examples and focus training on hard examples.The focusing parameter γ controls down-weighting: γ=0 gives cross-entropy, while larger γ increases the effect.
  • 3.4 Joint Debiasing Strategies: Both strategies are extended to multiple bias patterns by combining several bias-only models without training a classifier on their concatenated features.Joint Product of Experts multiplies their predictions with the base model, whereas Joint Debiased Focal Loss averages the bias-only predictions before applying DFL.

4 Evaluation on Unbiased Datasets

The evaluation tests debiasing methods on challenging, bias-controlled datasets for fact verification and natural language inference. Across these settings, the methods improve out-of-domain or hard-set performance, including when addressing multiple bias patterns.

  • Evaluation setup: The evaluation uses FEVER, SNLI, MNLI, HANS, and hard or symmetric splits designed to limit known dataset biases.FEVER’s symmetric set balances artifacts across labels, while SNLI and MNLI hard sets select examples a hypothesis-only classifier fails to predict.
  • 4.2 Natural Language Inference: 7.4 points: PoE improves over AdvCls on the SNLI hard set, setting a new state-of-the-art.With InferSent, DFL and PoE gain 4.1 and 4.8 points; with BERT, they gain 2.5 and 1.6 absolute points.
  • 4.2 Natural Language Inference: 1.1 points: PoE improves MNLI mismatched hard-test performance while retaining in-domain accuracy.On the hard development set, DFL and PoE gain 1.4 and 1.7 points with BERT, and 2.5 and 2.6 points with InferSent.
  • 4.3 Syntactic Bias in NLI: 7.4 and 4.4 points: DFL and PoE improve performance on HANS, respectively.Results are averaged across four runs with standard deviations reported in Table 4.
  • 4.4 Jointly Debiasing Multiple Bias Patterns: Joint debiasing improves both MNLI mismatched hard development performance and HANS, with gains close to those from individual debiasing.Models debiased only for hypothesis biases do not generalize to HANS, while jointly debiased models improve on both datasets.

5 Transfer Performance

The study tests trained NLI models on 12 target datasets, including cross-domain entailment and semantic matching tasks. On datasets beyond MNLI and MNLI-M, debiasing improves generalization over the baseline.

  • The transfer evaluation covers 12 NLI-related datasets, including MNLI, SNLI, SciTail, ADD1, JOCI, MPE, SICK, SPR, DPR, FN+, GLUE diagnostics, and QQP.
  • Models are trained on SNLI and tested on new target datasets, with labels mapped to each target dataset’s label space.
  • On MNLI and MNLI-M, the authors do not expect relative improvements because these datasets share biases with the SNLI training data.
  • On all remaining target datasets, the debiased models outperform the baseline, showing substantial improvement in generalization.

6 Discussion

The discussion examines how Debiased Focal Loss trades in-domain accuracy for robustness and how loss correlation reveals reduced reliance on bias-only predictions. The preferred method varies by dataset.

  • Analysis of Debiased Focal Loss: Increasing γ strengthens debiasing, reducing SNLI in-domain accuracy while improving accuracy on the SNLI hard set across a wide range of values.
  • Analysis of Debiased Focal Loss: The in-domain versus out-of-domain trade-off is especially pronounced for DFL when the baseline model is less powerful, such as InferSent.
  • Correlation Analysis: DFL and PoE reduce correlation between debiased-model losses and bias-only-model losses compared with baselines.
  • Correlation Analysis: PoE has lower bias-only correlation and better unbiased-split performance on MNLI, whereas DFL has lower correlation and higher performance on HANS.

7 Conclusion

The paper concludes that Product of Experts and Debiased Focal Loss are simple, model-agnostic strategies for reducing dataset bias through bias-only models. Experiments show substantial robustness gains and better transfer to other NLI datasets.

  • The paper proposes Product of Experts and Debiased Focal Loss to reduce biases learned by neural models.
  • The methods adjust cross-entropy loss using one or more bias-only models, down-weighting biased examples and focusing learning on hard examples.
  • The techniques are model agnostic, simple, and applicable whenever the biases can be specified through one or more bias-only models.
  • 9.8 points on FEVER symmetric, 7.4 on HANS, and 4.8 on SNLI hard measure substantial gains on challenging out-of-domain data.
  • The methods also improve generalization to other NLI datasets, while future work may identify bias patterns automatically.

B Natural Language Inference

For NLI experiments, InferSent uses separate BiLSTM encoders for premises and hypotheses, while BERT models are fine-tuned for three epochs. Results are reported on MNLI matched development and hard test sets.

  • InferSent learns premise and hypothesis representations with separate BiLSTM encoders before combining them in a nonlinear classifier.
  • InferSent models are trained for 20 epochs with 512-dimensional BiLSTMs, while BERT models are fine-tuned for 3 epochs.
  • InferSent debiasing models use a shallow linear classifier with 512 and 512 hidden units, whereas BERT debiasing models use the shallow nonlinear classifier described elsewhere.
  • Table 7 reports results on the MNLI matched development set and matched hard test set, with deltas measured against cross-entropy loss.

C Syntactic Bias in NLI

The section evaluates model accuracy on HANS syntactic heuristics, reporting separate entailment and non-entailment performance after training with specified configurations.

  • The base model is fine-tuned for 3 epochs.
  • The bias-only model uses a nonlinear classifier with two hidden layers of 6 units each and Tanh nonlinearities.
  • Accuracy is reported separately for entailment and non-entailment across individual HANS heuristics.

D Transfer Performance

The transfer evaluation trains models on SNLI and tests them on target datasets, applying label mappings where target datasets use fewer classes. Debiasing methods improve transfer under distribution shift.

  • Models are trained on SNLI and evaluated on other target datasets, with task-specific mappings from three SNLI labels to two target labels.Contradiction and neutral map to not-entailed for DPR, while contradiction maps to neutral for SciTail.
  • The evaluation corrects reported implementation and preprocessing issues before comparing models on the same corpora.These corrections include loss aggregation, out-of-vocabulary handling, and omitted samples during preprocessing.
  • Table 9 reports InferSent transfer accuracy and relative percentage differences from cross-entropy loss for models trained on SNLI.
  • The experiments sweep γ and α over seven values and β over five values.β modulates the bias-only model's loss during ensembling.
  • 3.39% and 2.57% are the relative generalization improvements of PoE and DFL over the baseline on MNLI-M.MNLI-M differs from the training distribution, whereas MNLI shares similar biases with SNLI.
  • PoE and DFL outperform M1 and M2 on 9 datasets, while the comparison methods perform better on SPR, FN+, and slightly on DPR.DPR is very small, and all models perform close to random chance there.

E Analysis of Debiased Focal Loss

This section examines how γ affects BERT trained with Debiased Focal Loss on SNLI and SNLI hard sets.

  • Figure 4 shows the impact of γ on BERT trained with DFL.
  • Accuracy is compared on SNLI and SNLI hard sets for different γ values.
Loading 1909.06321v3…