PaperlayerView exact revision on arXiv

Source-linked AI summary

Mind the Trade-off: Debiasing NLU Models without Degrading the In-distribution Performance

Prasetya Ajie Utama, Nafise Sadat Moosavi, Iryna Gurevych

arXiv:2005.00315v1cs.CL

TL;DR

NLU models can rely on dataset biases, while existing debiasing methods may improve out-of-distribution performance at the cost of in-distribution accuracy. The paper introduces confidence regularization to reduce reliance on biased examples while retaining their training signal. Across three NLU tasks, it reports competitive out-of-distribution performance while preserving original accuracy.

  • Problem

    Existing debiasing methods improve out-of-distribution performance but can reduce in-distribution accuracy on more diverse examples.

    Source: §1 · p. 1

  • Method

    Confidence regularization lowers models' confidence on biased examples so they are less likely to exploit their cues while still learning from all training examples.

    Source: §1 · p. 2, §7 · p. 9

  • Results

    Across four out-of-distribution datasets and three NLU tasks, the method provides competitive out-of-distribution performance while preserving original accuracy; it gains 7 points on an out-of-distribution NLI evaluation set.

    Source: §1 · p. 2, §7 · p. 9

  • Takeaways & Limitations

    The method allows models to learn from all training examples without exploiting dataset biases.

    Source: §7 · p. 9

    The framework assumes the relevant biases are correctly identified, and the paper identifies overcoming multiple biases at once or automatically identifying them as challenges.

    Source: §7 · p. 9

Abstract

from arXiv · show

Models for natural language understanding (NLU) tasks often rely on the idiosyncratic biases of the dataset, which make them brittle against test cases outside the training distribution. Recently, several proposed debiasing methods are shown to be very effective in improving out-of-distribution performance. However, their improvements come at the expense of performance drop when models are evaluated on the in-distribution data, which contain examples with higher diversity. This seemingly inevitable trade-off may not tell us much about the changes in the reasoning and understanding capabilities of the resulting models on broader types of examples beyond the small subset represented in the out-of-distribution data. In this paper, we address this trade-off by introducing a novel debiasing method, called confidence regularization, which discourage models from exploiting biases while enabling them to receive enough incentive to learn from all the training examples. We evaluate our method on three NLU tasks and show that, in contrast to its predecessors, it improves the performance on out-of-distribution datasets (e.g., 7pp gain on HANS dataset) while maintaining the original in-distribution accuracy.

1 Introduction

Dataset biases can make NLU models brittle on out-of-distribution examples, while existing debiasing methods can reduce in-distribution accuracy. The paper introduces confidence regularization to improve out-of-distribution performance while preserving in-distribution accuracy.

1 Introduction

  • Dataset biases are surface features associated with target labels, and models that rely on them can break when those associations are absent out of distribution.The introduction notes that this brittleness has limited practical applicability in some extrinsic use cases.Source: §1 · p. 1
  • Confidence regularization discourages models from exploiting biased features by encouraging lower-confidence predictions on examples containing them.The paper links this approach to the goal of preserving performance on previously unambiguous instances, even when they contain biases.Source: §1 · p. 2
  • 7 points gain on an out-of-distribution NLI evaluation set, with a slight in-distribution accuracy improvement; results also show competitive out-of-distribution improvement across three NLU tasks while retaining original in-distribution performance.The comparison is against product-of-expert and learned-mixin.Source: §1 · p. 2, Table 1 · p. 2
  • The paper reports that its method is more optimal when the dataset contains enough bias-free examples.This is presented as an insight into how debiasing methods behave across datasets with varying degrees of bias.Source: §1 · p. 2

2 Related Work

Prior work has studied dataset artifacts and methods for reducing their influence, but both data collection and debiasing approaches have limitations. This section situates confidence regularization among those approaches.

Biases in Datasets

  • Prior studies describe models leveraging dataset-specific patterns, including word overlap for MNLI entailment and indicative claim words for FEVER fact verification.These examples illustrate bias patterns across different NLU tasks.Source: §2 · p. 2
  • Data collection can reduce dataset artifacts, but human annotation may limit scale and diversity, while adversarial filtering is not guaranteed to remove artifacts.The paper presents learning methods that overcome biases as a complement to data collection.Source: §2 · pp. 2–3

Debiasing Models

  • Product-of-expert and learned-mixin use a biased model to identify examples it can solve, then down-weight those examples, which can reduce accuracy on the original training distribution.The paper's method also uses a biased model, but scales prediction confidence instead of diminishing those examples' training signal.Source: §2 · p. 3

Confidence Regularization

  • Feng et al. (2018) encourage uncertainty on nonsensical examples for interpretability, whereas this paper applies varying-strength confidence regularization across training examples to improve out-of-distribution performance.The paper specifies higher uncertainty for more biased examples.Source: §2 · p. 3

3 Method

Confidence regularization scales teacher predictions to reduce confidence on examples predictable from biased features, then trains the main model on those softened targets. This lets the model learn from all training examples while discouraging reliance on simple biased cues.

3 Method

  • Confidence regularization uses soft targets to teach lower confidence on biased examples and convey information about the task’s label distribution.Source: §3 · pp. 3–4
  • The main model is distilled from teacher output distributions scaled to lower confidence on examples predictable from biased features.The scaled teacher distributions serve as soft target labels for training.Source: §3 · p. 4
  • A biased model assigns each example a bias weight based on how well its biased features predict the ground-truth label.The weight is the biased model’s probability for the ground-truth class.Source: §3 · p. 4
  • The scaling strength β_i controls the teacher distribution: as β_i approaches 1, label probabilities approach minimum confidence, while β_i = 0 leaves them unchanged.Source: §3 · p. 4
  • Scaled targets are computed once as preprocessing, and the method does not require hyperparameters.Source: §3 · p. 4

4 Experimental Setup

Experiments evaluate confidence regularization and comparison methods on NLI, fact verification, and paraphrase identification, using task-specific training data and out-of-distribution challenge sets. The setup includes biased-feature models, BERT-based debiasing baselines, and repeated runs to address variance.

4 Experimental Setup

  • The experiments cover natural language inference, fact verification, and paraphrase identification, with datasets, models, and training details specified for each task.Source: §4 · p. 5

4.1 Natural Language Inference

  • MNLI supplies NLI training data and matched/mismatched in-distribution sets; HANS and MNLI-hard provide targeted out-of-distribution evaluations.HANS tests cases where word-overlap cues do not entail the hypothesis, while MNLI-hard examples resist a hypothesis-only model.Source: §4.1 · p. 5

4.2 Fact Verification

  • FEVER is used for training and Fever-Symmetric versions 1 and 2 for out-of-distribution evaluation.Fever-Symmetric examples are constructed so claim-only cues can lead to incorrect predictions.Source: §4.2 · p. 5

4.3 Paraphrase Identification

  • QQP supplies paraphrase-identification training data, and the QQP subset of PAWS tests out-of-distribution performance with duplicate and non-duplicate examples reported separately.Source: §4.3 · p. 5

4.4 Models

  • All debiasing methods use BERT-base and the same biased model; experiments run five times, compare product-of-experts and learned-mixin, and include FEVER example reweighting.Learned-mixin uses its default entropy-penalty weight because challenge datasets lack development sets for tuning.Source: §4.4 · pp. 5–6
  • For HANS and PAWS, biased models use hand-crafted sentence-overlap features; for MNLI-hard and FEVER, they use hypothesis-only or claim-only inputs.The latter models use an LSTM sentence encoder for MNLI and max-pooled word embeddings for FEVER.Source: §4.4 · pp. 5–6

5 Results

Confidence regularization limits the in-/out-of-distribution trade-off: it achieves a 7-point HANS gain without reducing MNLI-m test accuracy, while preserving in-distribution performance across bias types except for a QQP decline.

5.1 In-distribution Performance

  • Confidence regularization preserves in-distribution performance across bias types, while product-of-experts and learned-mixin reduce BERT accuracy on most tasks; all methods drop on QQP, especially duplicate examples.The method gains 0.6pp over BERT on FEVER and 0.1 points on both MNLI test sets in some cases.Source: §5.1 · pp. 6–7

5.2 Out-of-distribution Performance

  • 7 points of HANS improvement are achieved without reducing MNLI-m test accuracy, unlike product-of-experts, which gains 7 points but reduces MNLI-m accuracy by 1.6 points.Learned-mixin maintains in-distribution performance but raises average HANS accuracy only from 61.1% to 64.9%.Source: §5.2 · p. 7
  • On PAWS, the method finds a balance between non-duplicate accuracy and duplicate detection, whereas the compared methods improve the former while reducing the latter.Source: §5.2 · p. 7
  • On FEVER-symmetricv2, learned-mixin gains 0.5 points while losing 3 points on the FEVER development set.Source: §5.2 · p. 7

6 Discussions and Analysis

The analysis shows that confidence regularization's components are both needed for the overall improvements, while its effects include better calibration and a strong result when training includes all PAWS examples. It also examines why product-of-expert can lose in-distribution accuracy and why QQP provides too few unbiased examples for a strong training signal.

Impact on Models’ Calibration

  • ECE falls by 3.0 to 3.6 on MNLI-m and MNLI-mm dev sets, and lexical-overlap HANS examples receive lower confidence in the debiased model.On MNLI-m, confidence is more smoothly distributed; on HANS, higher-confidence predictions are more likely to be correct.Source: Figure 2 · p. 8, §6 · p. 8, Table 6 · p. 8

Ablation studies

  • Both self-distillation and bias-weighted scaling are necessary: each ablation improves HANS, but neither achieves the full method's overall improvements.Self-distillation suggests part of the gain comes from distillation regularization, while example-reweighting has a greater impact than self-distillation on both in- and out-of-distribution evaluations.Source: §6 · pp. 7–8

In-distribution performance drop of product-of-expert

  • Product-of-expert scales down gradients on biased examples, reducing training signal; restoring in-distribution accuracy by ensembling the biased model brings back poor HANS performance.The authors attribute much of product-of-expert's in-distribution degradation to those examples.Source: §6 · p. 8

Impact of adding bias-free examples

  • 77.05% overall accuracy with all 2,500 PAWS examples, versus 71.63% for baseline BERT.Adding PAWS examples gradually improves BERT's non-duplicate accuracy while lowering its duplicate accuracy; product-of-expert exaggerates this shift, reaching 93% non-duplicate and 40% duplicate accuracy.Source: Figure 3 · p. 9, §6 · p. 9

Impact of biased examples ratio

  • Almost 80% of QQP training examples are biased; the biased model is correct with confidence above 0.8 on 53% of QQP examples versus 12% of MNLI examples.The authors say this leaves too few unbiased QQP examples, so soft targets are mostly near-uniform and may provide insufficient training signal for training-distribution accuracy.Source: §6 · pp. 8–9

7 Conclusion

Confidence regularization addresses the training-signal loss associated with existing debiasing methods by regularizing confidence on biased examples while allowing learning from all training examples. Across four out-of-distribution datasets and three NLU tasks, it provides competitive out-of-distribution performance while preserving original accuracy.

7 Conclusion

  • Across four out-of-distribution datasets and three NLU tasks, confidence regularization provides competitive out-of-distribution performance while preserving original accuracy.The method regularizes confidence on biased examples so models can learn from all training examples without exploiting biases.Source: §7 · p. 9
  • The framework can extend to other task setups when the biases leveraged by models are correctly identified.Challenges include addressing multiple biases at once and automatically identifying bias formats when prior knowledge is unavailable.Source: §7 · p. 9

A Ablation Details

The second ablation setting reweights examples using the teacher model’s scaled probability for the ground-truth label when computing cross-entropy loss.

A Ablation Details

  • The second ablation setting reweights examples using the teacher model’s scaled probability for the ground-truth label.The probability assigned by the teacher to the correct label and the main model’s corresponding probability are denoted separately.Source: A Ablation Details · p. 12
  • The loss is computed as cross-entropy for each batch of size m.Source: A Ablation Details · p. 12
  • The ground-truth label is treated as the cth label when defining the teacher and main model probabilities.Source: A Ablation Details · p. 12

B Bias Weights Distribution

The biased models use lexical-overlap features for QQP and MNLI, and claim-only partial input for FEVER; on PAWS, many examples receive very high confidence from the biased model.

B Bias Weights Distribution

C HANS Biased Model

The HANS-based features comprise lexical-overlap and embedding-distance signals, following features proposed by Clark et al. (2019a).

C HANS Biased Model

  • The HANS-based features were proposed by Clark et al. (2019a).Source: C HANS Biased Model · p. 12
  • The feature set includes whether hypothesis words occur in the premise and whether the hypothesis is a contiguous premise subsequence.Source: C HANS Biased Model · p. 12
  • It also includes the fraction of hypothesis words found in the premise and average and maximum cosine distances between premise and hypothesis word vectors.Source: C HANS Biased Model · p. 12
  • Figure 4 describes the distribution of biased model confidence across QQP, MNLI, and FEVER training datasets.Source: Figure 4 · p. 13

Related papers

Loading 2005.00315v1…