Source-linked AI summary
Learning Not to Learn: Training Deep Neural Networks with Biased Data
Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, Junmo Kim
TL;DR
Biased training data can cause neural networks to use irrelevant attributes, harming performance when those attributes differ at test time. The paper introduces mutual-information regularization with adversarial bias prediction to remove target-bias information from feature embeddings. Across experiments, the approach extracts bias-independent features and achieves the best performance in most experiments, though benefits are limited when attributes such as age and gender are correlated.
Problem
Training on biased data can make networks learn irrelevant bias information, causing poor performance on test samples when that bias does not determine the label.
Method
The method minimizes mutual information between feature embeddings and a known target bias using adversarial training between label and bias prediction networks.
Results
The proposed regularization produced bias-independent feature embeddings and achieved the best performance in most experiments, including performance better than the Gray model.
Takeaways & Limitations
Removing target-bias information can encourage more informative classification features and improve performance when the bias is independent of the categories.
Takeaways & Limitations
Bias removal is not completely beneficial when target attributes such as age and gender are not completely independent, so understanding the specific bias must precede removal.
Abstract
from arXiv · showhide
We propose a novel regularization algorithm to train deep neural networks, in which data at training time is severely biased. Since a neural network efficiently learns data distribution, a network is likely to learn the bias information to categorize input data. It leads to poor performance at test time, if the bias is, in fact, irrelevant to the categorization. In this paper, we formulate a regularization loss based on mutual information between feature embedding and bias. Based on the idea of minimizing this mutual information, we propose an iterative algorithm to unlearn the bias information. We employ an additional network to predict the bias distribution and train the network adversarially against the feature embedding network. At the end of learning, the bias prediction network is not able to predict the bias not because it is poorly trained, but because the feature embedding network successfully unlearns the bias information. We also demonstrate quantitative and qualitative experimental results which show that our algorithm effectively removes the bias information from feature embedding.
1. Introduction
Biased training data can make neural networks rely on irrelevant attributes, producing fragile and potentially unjust predictions. The paper proposes mutual-information regularization and adversarial training to remove target-bias information from feature embeddings.
- Biased training distributions can be learned as meaningful information, weakening robustness and introducing unjust discrimination.
- The paper treats attributes such as color, race, and gender as bias information rather than as data points themselves.
- Figure 1 illustrates that multiple decision boundaries can perfectly classify biased training samples yet perform poorly on unbiased test samples.The example separates digit shape from color, with only high-saturation samples available during training.
- The proposed regularization minimizes mutual information between extracted features and a known target bias through an adversarial prediction setup.One network predicts the target bias while another predicts labels and minimizes shared information between the embedding and bias.
- Experiments report that removing target-bias information improves classification and produces the best performance in most experiments.The study intentionally planted bias in training sets while keeping test sets unbiased to evaluate bias removal.
2. Related Works
The related work connects bias removal to unknown unknowns, unsupervised domain adaptation, and disentangled representations. This paper instead minimizes mutual information to remove specified target information from learned features.
- Unknown unknowns are high-confidence mispredictions, whereas known unknowns are mispredictions made with low confidence.
- Prior bias-removal work used joint learning and unlearning with confusion loss, while this paper proposes a different regularization strategy.
- Unlike unsupervised domain adaptation, this problem does not assume access to target images and instead uses a description of the target bias.
- Disentangled representation methods seek to separate feature information, including through generative adversarial approaches such as InfoGAN.
- The paper contrasts its approach with InfoGAN by minimizing mutual information so the network does not learn selected target information.
- The overall neural architecture uses a feature network followed by label and bias prediction components, implemented with ResNet-18 for real images and four convolution layers for MNIST.
3. Problem Statement
The paper targets networks that rely on biased training distributions and proposes minimizing mutual information between feature embeddings and a specified target bias. An auxiliary bias predictor and adversarial training procedure operationalize this regularization while preserving label classification.
- The method defines target bias as information to remove from the feature embedding, such as color when color is irrelevant to digit categorization.
- The feature extractor feeds both a label prediction network and a bias prediction network, and the formulation is applicable to arbitrary network architectures.
- The objective is robust test-time performance on unbiased data despite training on biased data.
- Formulation: Mutual information is rewritten using conditional entropy and an auxiliary distribution Q because the posterior bias distribution is not tractable directly.
- Training Procedure: The training procedure minimizes bias-prediction loss for h while training f adversarially to make predicting the target bias difficult.
- Training Procedure: At the end of training, poor bias prediction by h is attributed to f unlearning bias information rather than to divergence of h.
4. Dataset
The experiments plant controlled biases in public benchmarks so models can be evaluated when training and test distributions differ. The datasets cover colored digits, color-correlated dogs and cats, and age–gender correlations in faces.
- The study intentionally plants bias in balanced public benchmarks because conventional train/test splits can obscure biased-data effects.
- Colored MNIST: Colored MNIST assigns each digit class a distinct mean color and samples training-image colors around that class-specific mean.
- Colored MNIST: Test-σ2 images use random mean colors and are unbiased, while smaller σ2 values indicate stronger bias in the corresponding training set.
- Colored MNIST: In colored MNIST, color predicts digits in training but is insufficient and disruptive for test images, motivating removal of color from feature embeddings.
- Dogs and Cats: The dogs-and-cats experiment uses TB1 with bright dogs and dark cats, TB2 with dark dogs and bright cats, and an unbiased original test set.
- Dogs and Cats: Because test labels are unavailable for dogs and cats, performance is measured against predictions from an oracle ResNet-18 trained on all 25K training images.
- IMDB Face: The cleaned IMDB face dataset contains 112,340 images, with EB1 and EB2 reversing age–gender correlations across training subsets.
- Colored MNIST Evaluation: Across the colored-MNIST evaluation, the proposed model outperforms the baseline and BlindEye and performs comparably to the grayscale model.
5. Experiments
Experiments evaluate bias removal across colored MNIST, dogs and cats, and IMDB face tasks using quantitative, qualitative, and ablation analyses. The proposed method removes target-bias information and improves performance, though gains are marginal when age and gender correlate.
- Experimental setup: The experiments remove color, age, and gender as target biases across colored MNIST, dogs and cats, and IMDB face datasets.Age and gender labels were supervised, while color bias was removed through self-supervision.
- Colored MNIST: Across colored MNIST settings, the proposed algorithm outperformed BlindEye and the baseline for every σ2 value and usually slightly exceeded grayscale training.Smaller σ2 indicates more severe color bias.
- Colored MNIST: Re-colored-test confusion matrices showed vertical class patterns for baseline models, while the proposed algorithm removed the color bias.The baseline patterns reflected digit-associated mean colors in the biased training data.
- Ablation: Ablations found that gradient reversal improved performance alone and together with confusion loss or the proposed regularization.The authors interpret this as evidence that gradient reversal can itself remove bias.
- Dogs and cats: On dogs and cats, baseline predictions depended on image color, whereas the proposed networks classified query images independently of color.The oracle model supplied reference labels, and stacked bar charts visualized predictions by model.
- IMDB face: On IMDB face, removing target bias improved overall performance, but gains were marginal because age and gender are correlated rather than independent.The gender task remained difficult despite improvement, with random guessing achieving 50% accuracy.
6. Conclusion
The paper proposes mutual-information-based regularization with a minimax training strategy to categorize inputs while unlearning bias. Experiments show bias-independent feature embeddings and improved performance in most experiments, including against a model trained with almost unbiased data.
- The method minimizes mutual information so networks categorize inputs while unlearning the target bias.
- The proposed regularization extracts bias-independent feature embeddings and achieves the best performance in most experiments.
- The model outperforms the “Gray” model trained with almost unbiased data, indicating more informative feature embeddings.