PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning from Failure: Training Debiased Classifier from Biased Classifier

Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, Jinwoo Shin

arXiv:2007.02561v2cs.LGstat.ML

TL;DR

Spurious correlations can cause neural networks to learn unintended rules, while prior debiasing methods often require explicit bias supervision or domain-specific assumptions. LfF trains a deliberately biased network alongside a debiased network that emphasizes samples the biased model finds difficult, improving performance across biased datasets. The paper also reports gains on unbiased evaluation sets even when nearly all training samples are bias-aligned.

  • Problem

    Debiasing models trained on biased datasets commonly relies on costly explicit labels of misleading attributes or bias-specific expert supervision.

  • Method

    LfF simultaneously amplifies easy, bias-aligned examples in one network and reweights the second network toward samples the biased network struggles to learn.

  • Results

    35.34% →63.39% and 17.93% →31.66% accuracy were reported on unbiased evaluation sets for Colored MNIST and Corrupted CIFAR-10^1 with 99.5% bias-aligned training samples.

  • Takeaways & Limitations

    LfF successfully trains debiased classifiers across color, texture, and gender biases using minimal human supervision.

  • Takeaways & Limitations

    Assessing risk reduction is challenging without specifically identifying the biases, and the setup assumes a highly correlated bias attribute with an unintended decision rule.

Abstract

from arXiv · show

Neural networks often learn to make predictions that overly rely on spurious correlation existing in the dataset, which causes the model to be biased. While previous work tackles this issue by using explicit labeling on the spuriously correlated attributes or presuming a particular bias type, we instead utilize a cheaper, yet generic form of human knowledge, which can be widely applicable to various types of bias. We first observe that neural networks learn to rely on the spurious correlation only when it is "easier" to learn than the desired knowledge, and such reliance is most prominent during the early phase of training. Based on the observations, we propose a failure-based debiasing scheme by training a pair of neural networks simultaneously. Our main idea is twofold; (a) we intentionally train the first network to be biased by repeatedly amplifying its "prejudice", and (b) we debias the training of the second network by focusing on samples that go against the prejudice of the biased network in (a). Extensive experiments demonstrate that our method significantly improves the training of the network against various types of biases in both synthetic and real-world datasets. Surprisingly, our framework even occasionally outperforms the debiasing methods requiring explicit supervision of the spuriously correlated attributes.

1 Introduction

Biased datasets can make neural networks rely on spurious correlations, while existing debiasing methods often require costly bias-specific supervision. Learning from Failure (LfF) avoids that supervision by exploiting training dynamics to train a biased model and use its failures to guide a debiased model.

  • Spurious correlations can make classifiers learn unintended decision rules, such as predicting boats from water backgrounds rather than object content.
  • Existing debiasing approaches commonly require explicit labels for misleading attributes or bias-specific methods based on expert diagnosis.
  • Human supervision on bias is costly because misleading attributes require manual labeling or expert analyses to identify their existence and characteristics.
  • LfF uses a cheaper, generic form of human knowledge and does not require explicit bias labels or bias-tailored training techniques.
  • LfF trains one network to amplify easy, bias-aligned samples and trains another by emphasizing samples the biased network struggles to learn.
  • 35.34% →63.39% and 17.93% →31.66% accuracy improvements were reported on unbiased evaluation sets for Colored MNIST and Corrupted CIFAR-10^1 with 99.5% bias-aligned training samples.

2 A closer look at training deep neural networks on biased datasets

Bias does not always harm classifier performance: its effect depends on whether the bias attribute is easier to learn than the target. Under malignant bias, bias-aligned samples are learned earlier than bias-conflicting samples, motivating the paper’s distinction between malignant and benign bias.

  • 2.1 Setup: Unbiased evaluation sets decorrelate target and bias attributes by balancing every possible target–bias value pair.
  • 2.1 Setup: In Colored MNIST and Corrupted CIFAR-10, bias can involve color or corruption type instead of the intended digit or object category.
  • 2.2 Not all biases are malignant: A biased dataset harms predictions only when its bias attribute is easier for the classifier to capture than the target attribute.
  • 2.2 Not all biases are malignant: Malignant bias is defined by performance degradation on an unbiased evaluation set, whereas benign bias causes no such degradation compared with an unbiased training dataset.
  • 2.3 Malignant bias is learned first: For malignant bias, bias-aligned-sample loss quickly reaches zero, while bias-conflicting-sample loss initially rises and decreases later.
  • 2.3 Malignant bias is learned first: Malignant-bias training shows a larger early loss gap between bias-conflicting and bias-aligned samples, unlike benign-bias training where losses are nearly indistinguishable.

3 Debiasing by learning from failure (LfF)

Learning from Failure (LfF) trains biased and debiased networks simultaneously: one amplifies easy, bias-aligned samples, while the other emphasizes samples the biased model struggles to learn. The method uses relative difficulty to reweight cross-entropy training toward bias-conflicting samples.

  • Overview: LfF simultaneously trains a biased network fB and a debiased network fD, using the former’s failures to guide the latter.The biased model follows an intentionally amplified unintended decision rule, while the debiased model focuses on difficult samples.
  • Training procedure: Algorithm 1 initializes fB and fD and updates both networks over mini-batches from the training set.The procedure takes model parameters, a dataset, learning rate, and iteration count as inputs.
  • Training the biased model: Generalized cross entropy (GCE) amplifies the biased model’s prejudice by emphasizing easier samples with strong agreement between predictions and targets.Its parameter q controls amplification, and at q = −log p it becomes equivalent to standard cross entropy.
  • Training the debiased model: The debiased model trains with cross entropy reweighted by relative difficulty, assigning larger weights to samples the biased model struggles to learn.Bias-conflicting samples receive weights close to 1 when the biased model’s loss exceeds the debiased model’s loss.
  • Evaluation: Tables 2 and 3 evaluate accuracy on unbiased and bias-conflicting samples across Colored MNIST and Corrupted CIFAR-10 under varying bias-aligned ratios and supervision settings.The tables compare no supervision, bias-tailored supervision, and explicit bias supervision.

4 Experiments

Experiments across controlled and real-world biased datasets evaluate LfF against vanilla and other debiasing baselines. LfF improves performance on bias-conflicting or unbiased samples, with analyses linking this to its intentionally biased model and focus on difficult samples.

  • Experimental setup: LfF is evaluated on Colored MNIST, Corrupted CIFAR-10, CelebA, and the newly constructed BAR action-recognition dataset.The experiments use MLP, ResNet-20, and ResNet-18 models, with results averaged over three independent trials.
  • Experimental setup: Controlled experiments compare LfF with methods that presume a bias type or require explicit bias-attribute labels.Baselines include HEX, REPAIR, and Group DRO.
  • Controlled experiments: 41.37% versus 22.72%: LfF versus vanilla accuracy on the Corrupted CIFAR-10 unbiased evaluation set with 99% bias-aligned training samples.LfF significantly outperforms the baseline across all tested bias-aligned ratios.
  • Controlled experiments: The debiased model reaches about 80% accuracy on both bias-aligned and bias-conflicting samples, unlike vanilla, which reaches 100% only on aligned samples.The intentionally biased model performs close to random guessing on bias-conflicting samples, supporting its reliance on the bias attribute.
  • Controlled experiments: Replacing GCE with standard cross entropy for the biased model does not help debiasing.The CE-trained biased model partially learns the target attribute, weakening its usefulness for computing relative difficulty.
  • Real-world experiments: On CelebA, LfF consistently outperforms vanilla and is comparable to Group DRO, while outperforming it on the HeavyMakeup unbiased evaluation set.CelebA experiments use Gender as the bias attribute for HairColor and HeavyMakeup targets.
  • Real-world experiments: On BAR, LfF outperforms vanilla for every action class and outperforms ReBias for most classes except Diving.BAR uses action-place correlations to provide a realistic evaluation setting.

5 Related work

Related work addresses dataset bias through explicit bias labels, bias-specific expert knowledge, or intentionally biased auxiliary models. LfF instead uses general training-dynamics observations and only requires affordable yes/no knowledge about whether those observations apply.

  • Debiasing without explicit supervision: Some debiasing methods use explicit labels for misleadingly correlated attributes, while others rely on expert knowledge about particular biases such as texture bias.These approaches can require manual labeling or bias-discovery studies.
  • Debiasing from the biased model: Other methods use intentionally biased models to debias another model, typically for biases that are already known and easily characterized.Examples include question-only models for VQA and bias-only models for VQA, reading comprehension, and natural-language tasks.
  • LfF relative to prior work: LfF uses general properties of bias-aligned and bias-conflicting training dynamics rather than expert knowledge tailored to a particular bias type.Its human knowledge requirement is whether the bias follows the observed training behavior.
  • LfF relative to prior work: LfF assumes that bias-conflicting samples exist and presents a scheme free from choosing the form and amount of bias supervision.The paper characterizes this knowledge as more affordable and applicable across bias types.

6 Conclusion

The paper proposes Learning from Failure (LfF), a debiasing scheme grounded in the relationship between neural-network training and the easiness of biased attributes. Extensive experiments report successful debiased training with minimal human supervision.

  • Conclusion: Learning from Failure (LfF) trains neural networks on biased datasets using observations about the easiness of biased attributes.The paper positions LfF as a failure-based debiasing scheme.
  • Conclusion: Extensive experiments show successful debiased training with minimal human supervision.The authors suggest these results may clarify the nature of neural-network debiasing.

Broader Impact

Biased datasets can create social risks when deployed AI systems learn discriminatory shortcuts. The paper recommends general debiasing behaviors to address diverse and underexplored biases, while acknowledging challenges in assessing risk reduction.

  • Biased models can produce discriminatory outcomes, including over-predicting Asians as blinking in facial-recognition software trained on Caucasian faces.
  • Bias-specific debiasing schemes may fail to address underexplored biases such as gender or race.
  • The proposed scheme leverages general neural-network behavior on biased datasets for debiasing across diverse applications.
  • Assessing potential risk reduction is challenging without specifically identifying the biases, although the approach may help analyze underexplored bias types.

A.1 Controlled experiments

Controlled experiments use equal numbers of target and bias categories with a prescribed biased training distribution. The paper evaluates this setup on Colored MNIST, Corrupted CIFAR-10, and CelebA variants.

  • Experiments use the same number of categories for target and bias attributes, satisfying |At| = |Ab|.
  • The training distribution is biased by assigning each target value a corresponding bias value through a bijection.
  • Colored MNIST: Colored MNIST introduces Digit and Color attributes using ten randomly selected RGB values reused across experiments.
  • Corrupted CIFAR-10: Corrupted CIFAR-10 uses Object category and Corruption type as two attributes, with separate datasets built from two corruption protocols.
  • Corrupted CIFAR-10: Corruption severity controls difficulty, with more severe corruption making images less distinguishable; each dataset uses 50,000 training and 10,000 test samples.
  • CelebA: CelebA experiments use BlondHair or HeavyMakeup as target attributes and Male as the bias attribute.

B Biased action recognition dataset

BAR is a real-world six-class action-recognition dataset designed around action–place correlations. Its construction separates typical action–place images for training from atypical cases for evaluation using worker validation and bounding-box tasks.

  • Dataset design: BAR contains six action classes whose images are biased toward distinct places, enabling action classification from place alone.
  • Source of dataset: Images come from imSitu, Stanford 40 Actions, and Google Image Search, with visually similar action classes merged into single BAR classes.
  • Construction process: Typical six action–place pairs form the training set, whereas atypical pairings form the evaluation set after inappropriate images are excluded.
  • Dataset size: The finalized BAR dataset contains 2,595 images, each larger than 400px wide and 300px high.
  • Dataset statistics: Table 7 reports the per-class counts of the BAR dataset.
  • Worker annotation: Workers validate images using three binary questions and draw bounding boxes around relevant contextual regions.

C Experimental details

The experiments use dataset-specific neural architectures and training settings, and compare LfF with debiasing baselines including HEX, REPAIR, and Group DRO. These baselines differ in whether they require bias information or explicit bias labels.

  • Architecture details: Colored MNIST uses a three-hidden-layer multilayer perceptron, while Corrupted CIFAR-10 uses ResNet-20.
  • Architecture details: CelebA and BAR use pretrained ResNet-18 models from torchvision.
  • Training details: All experiments use Adam; learning rates are 0.001 for Colored MNIST and Corrupted CIFAR-10 and 0.0001 for CelebA and BAR.
  • Baselines: HEX projects representations orthogonal to texture statistics captured with GLCM or NGLCM when texture domain identifiers are unavailable.
  • Baselines: REPAIR reweights samples to reduce mutual information between bias labels and intermediate target-classifier representations.
  • Baselines: Group DRO minimizes worst-case loss over predefined target–bias groups and requires bias-attribute labels to construct those groups.

E Additional experiments

Additional experiments vary bias difficulty and sample composition across Colored MNIST and Corrupted CIFAR-10, evaluating unbiased and bias-conflicting samples. LfF consistently outperforms baseline methods and often surpasses explicit bias-supervision methods.

  • Difficulty of the bias attribute: Tables 8 and 9 evaluate accuracy on unbiased and bias-conflicting samples while varying the difficulty of biased attributes.The experiments use Colored MNIST and Corrupted CIFAR-10, with comparisons across no supervision, bias-tailored supervision, and explicit bias supervision.
  • Difficulty of the bias attribute: LfF consistently outperforms the baseline by a large margin regardless of the difficulty of the biased attribute.
  • Difficulty of the bias attribute: Both baseline and LfF classifiers generally become more biased as the difficulty of the bias attribute increases.This pattern supports the paper’s claim that lower bias difficulty makes classifiers more likely to suffer from bias.
  • Difficulty of the bias attribute: LfF outperforms methods using explicit bias labels in most cases.
  • Sample composition: Table 10 evaluates accuracy on unbiased and bias-conflicting Colored MNIST samples while varying the ratio of bias-aligned samples.
  • Comparison to other combination rule: Other combination-rule approaches use prediction masking, logit ensembling, or residual learning to combine biased and debiased models.The cited methods are RUBi, LearnedMixin, and DRiFt.
Loading 2007.02561v2…