Source-linked AI summary
Turning a Blind Eye: Explicit Removal of Biases and Variation from Deep Neural Network Embeddings
Mohsan Alvi, Andrew Zisserman, Christoffer Nellaker
TL;DR
Neural networks can encode dataset biases and spurious variations, making decisions for unreliable reasons. The paper introduces an algorithm for removing multiple such variations from feature representations and creates the 14,000-image LAOFIW dataset. Across facial classification tasks, the representations remain informative for one task while becoming uninformative for others, and gender-classification accuracy improves by up to 20% on unbiased data after training with extreme age bias.
Problem
Training and fine-tuning datasets can encode biases and spurious variations that make neural-network decisions unreliable or unfair.
Method
The paper alternates primary-task learning with auxiliary spurious-variation classification and confusion objectives, using LAOFIW to study ancestral-origin variation.
Results
Up to 20%: classification accuracy improved on an unbiased test dataset when gender networks were trained on extremely age-biased data, while representations stayed informative for one task and uninformative for others.
Takeaways & Limitations
The approach supports better generalization from biased training data and can remove undesirable ancestral-origin information from feature representations.
Takeaways & Limitations
Some spurious variations are harder to remove than others, and different objective weights or dynamic weighting may be needed to improve convergence.
Abstract
from arXiv · showhide
Neural networks achieve the state-of-the-art in image classification tasks. However, they can encode spurious variations or biases that may be present in the training data. For example, training an age predictor on a dataset that is not balanced for gender can lead to gender biased predicitons (e.g. wrongly predicting that males are older if only elderly males are in the training set). We present two distinct contributions: 1) An algorithm that can remove multiple sources of variation from the feature representation of a network. We demonstrate that this algorithm can be used to remove biases from the feature representation, and thereby improve classification accuracies, when training networks on extremely biased datasets. 2) An ancestral origin database of 14,000 images of individuals from East Asia, the Indian subcontinent, sub-Saharan Africa, and Western Europe. We demonstrate on this dataset, for a number of facial attribute classification tasks, that we are able to remove racial biases from the network feature representation.
1 Introduction
Deep neural networks learn feature representations from labeled image datasets, but those representations can encode biases and spurious variations that undermine reliable decisions. The paper introduces an algorithm to remove known biases and multiple spurious variations, and creates a 14,000-image ancestral-origin dataset for evaluation.
- 1 Introduction: Large image datasets can contain biases or spurious variations that are irrelevant or problematic for the target task.Celebrity face datasets may reflect age, gender, and ancestral-origin imbalances rather than broader real-world distributions.
- 1 Introduction: Fine-tuning pretrained networks can transmit biases from the original large dataset and learn spurious variations from a smaller task dataset.
- 1 Introduction: Bias-aware evaluation is needed because accuracy alone may not reveal decisions based on gender, ancestral origin, or other inappropriate cues.Prior work found that networks can amplify training-data associations, such as linking kitchens with women.
- 1 Introduction: The proposed algorithm aims to make networks blind to known biases, improve classification under extreme bias, and remove multiple spurious variations from feature representations.The framework is demonstrated using age, gender, ancestral origin, and pose information in facial images.
- 1 Introduction: LAOFIW contributes 14,000 publicly sourced images spanning sub-Saharan Africa, the Indian Subcontinent, Europe, and East Asia, with varied poses, illumination, and expressions.
2 Related Work
The paper builds on domain adaptation and invariant-representation methods to make networks agnostic to dataset biases and other spurious variations. Related approaches use multi-task weighting, data rebalancing, or adversarial gradient reversal.
- 2 Related Work: Domain adaptation improves classifier generalizability across domains by minimizing differences between source and target domains.
- 2 Related Work: Multi-task learning can address pose variation by automatically weighting spurious-variation objectives during training.
- 2 Related Work: Adjusting the training-data distribution can reduce learned biases, but this approach requires labels for each spurious variation in every training example.
- 2 Related Work: Gradient-reversal methods remove variation by updating representations against an auxiliary task, while uniform-distribution cross-entropy keeps classifiers equally uninformative across tasks.
3 Datasets
The paper combines several face datasets to study ancestral-origin, age, gender, and pose variation. LAOFIW provides balanced, varied ancestral-origin images, while cleaned IMDB data and an unbiased test set support age–gender bias analysis.
- 3 Datasets: LAOFIW was assembled from Bing Image Search using origin-related terms combined with gender and age descriptors, then filtered for photographic images.
- 3 Datasets: LAOFIW contains roughly equal numbers of male and female individuals, and more than one-third of its images are non-frontal.
- 3 Datasets: The IMDB dataset contains images from profiles of the 100,000 most popular actors, with gender and date-of-birth labels used to study celebrity-dataset age and gender bias.
- 3 Datasets: IMDB labels are noisy because profile images may show co-stars, while age calculations can rely on incorrect or unreliable timestamps.
- 3 Datasets: Azure-based filtering reduced an identity-balanced IMDB subset from 150,000 images to 60,000 by rejecting gender disagreements and age discrepancies exceeding 10 years.
- 3 Datasets: 75% to 99%: gender-classification accuracy improved after cleaning the IMDB data, which still showed younger women and older men.
- 3 Datasets: The unbiased IMDB test set contained equal numbers of men and women in each age category and was withheld from training.
- 3 Datasets: AFLW pose data were divided into five yaw categories, with 24,000 images for training and 6,000 for testing after augmentation and class balancing.
4 Methods
The JLU algorithm learns features that remain informative for a primary task while becoming invariant to multiple specified spurious variations. It alternates optimization of primary-task learning, spurious-variation classification, and confusion objectives.
- JLU learns a single feature representation that is informative for a primary classification task and uninformative for multiple spurious variations.
- The experiments use M secondary datasets, each describing a single spurious variation, alongside the primary dataset.
- Each secondary branch uses classification and confusion losses to measure and remove spurious-variation information from the shared feature representation.
- The confusion loss changes the representation so classifiers for spurious variations become uninformative, promoting invariance to those variations.
- The joint objective combines primary classification with weighted confusion losses, while α controls confusion strength and β_m weights the m-th spurious variation.
- Because spurious classification and confusion objectives oppose one another, training alternates between optimizing the spurious classifiers and the primary-plus-confusion loss.
5 Experiments
The experiments evaluate bias removal for age and gender classification, including artificially extreme age–gender biases, and test simultaneous removal of multiple variations across facial tasks. They use cleaned IMDB, LAOFIW, and adapted AFLW data with an adapted VGG-M base network.
- The experiments cover bias removal, extreme-bias removal, and simultaneous removal of multiple spurious variations from primary-task representations.
- The age-bias experiment compares a baseline age network with a blind network trained while removing gender-specific information, using an unbiased test set.
- Extreme-bias subsets pair women aged 0–30 with men aged 40+ or women aged 40+ with men aged 0–30, separated by a 10-year buffer.
- For multiple-variation experiments, one of age, gender, ancestral origin, or pose is primary while the others are designated spurious variations.
- The study uses cleaned IMDB for age and gender, LAOFIW for ancestral origin, and adapted AFLW for pose, with an adapted VGG-M base network.
- Age classification uses 5-year bins, with predictions within one class of the true age counted as positive classifications.
6 Results
The results show that JLU removes gender and other spurious information from learned feature representations, while generally preserving or improving primary classification performance. On strongly biased data, unlearning reduces gender-dependent age predictions and improves gender classification accuracy.
- 6.1 Removal of a bias from a dataset: Gender unlearning removes gender separability from age-classification embeddings trained on gender-biased data.The baseline representation is separable by gender, whereas the blind network’s representation is no longer separable.
- 6.1 Removal of a bias from a dataset: 0.049 and 0.027: KL-divergence between men’s and women’s age-prediction distributions for biased and unbiased networks, respectively.The lower value indicates more similar prediction distributions after gender unlearning.
- 6.1 Removal of a bias from a dataset: 78.9% and 78.1%: average age-classification accuracy for baseline and unbiased networks, respectively, a reduction of 0.8%.The reduction was 1.2% for females and 0.5% for males.
- 6.2 Removal of an extreme bias from a network: 70% to 86%: gender-classification accuracy on EB1, a 16% improvement after simultaneously unlearning age information.On EB2, accuracy increased from 62% to 82%, amounting to a 20% increase.
- 6.2 Removal of an extreme bias from a network: JLU shifts age distributions for predicted male and female groups closer to the true distribution on both EB1 and EB2.Baseline models associate age with gender in opposite directions across the two biased datasets.
- 6.3 Simultaneous removal of multiple spurious variations: For age classification, ancestral-origin information was removed completely, while gender and pose information were removed by 88% and 91%, respectively.Primary classification accuracy was 5% lower for the blind network than for the baseline.
- 6.3 Simultaneous removal of multiple spurious variations: For ancestral-origin classification, blind-network accuracy improved by 1%, while age, gender, and pose information were unlearned by 92%, 98%, and 86%, respectively.These results quantify simultaneous removal of multiple spurious attributes while evaluating the primary task.
- 6.3 Simultaneous removal of multiple spurious variations: Across multiple-variation experiments, blind networks usually retain or improve primary accuracy, while spurious-variation accuracies approach random chance in 9/12 cases.The pose experiment was least effective, and ancestral-origin information was most difficult to remove.
7 Conclusion
The paper presents JLU as a way to make feature representations informative for primary tasks while removing multiple spurious variations. Experiments show improved generalization from biased training data, alongside a weighting limitation and a public ancestral-origin dataset for bias removal.
- JLU compares spurious-variation classifier outputs with a uniform distribution to produce representations informative for the primary task but blind to selected variations.
- The algorithm remained informative for one facial classification task while simultaneously becoming uninformative for age, ancestral origin, pose, or gender variations.
- Up to 20% higher classification accuracy on an unbiased test dataset was achieved when gender networks were trained on extremely age-biased data.
- The method supports better generalization from biased training data to unbiased settings by removing known biases from the feature representation.
- The authors created a 14,000-image ancestral-origin dataset that can support detection and removal of racial biases in network representations.
- Some spurious variations are harder to remove than others, motivating variation-specific weights or dynamic weighting during training.