Source-linked AI summary
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation
Zeyu Wang, Klint Qinami, Ioannis Christos Karakozis, Kyle Genova, Prem Nair, Kenji Hata, Olga Russakovsky
TL;DR
The paper addresses the limited systematic comparison of techniques for mitigating spurious correlations in visual recognition. It introduces a controlled benchmark and evaluates mitigation strategies, finding that domain-independent training outperforms alternatives and mitigates gender bias on CelebA.
Problem
Existing bias-mitigation techniques for visual recognition have not been systematically compared, despite models learning spurious correlations from training data.
Method
The paper introduces a controlled visual-recognition benchmark and evaluates adversarial, inference-time, domain-conditional, and domain-independent mitigation strategies.
Results
Domain-independent training significantly outperforms other techniques on the benchmark and successfully mitigates real-world gender bias in CelebA attribute recognition.
Takeaways & Limitations
Controlled evaluation identifies a simpler prior-shift alternative to Reducing Bias Amplification and supports domain-independent training as an effective bias-mitigation approach.
Takeaways & Limitations
The paper leaves open cases involving non-discrete domains, unknown training-time domain distributions, downstream prediction decisions, and broader ethical fairness questions.
Abstract
from arXiv · showhide
Computer vision models learn to perform a task by capturing relevant statistics from training data. It has been shown that models learn spurious age, gender, and race correlations when trained for seemingly unrelated tasks like activity recognition or image captioning. Various mitigation techniques have been presented to prevent models from utilizing or learning such biases. However, there has been little systematic comparison between these techniques. We design a simple but surprisingly effective visual recognition benchmark for studying bias mitigation. Using this benchmark, we provide a thorough analysis of a wide range of techniques. We highlight the shortcomings of popular adversarial training approaches for bias mitigation, propose a simple but similarly effective alternative to the inference-time Reducing Bias Amplification method of Zhao et al., and design a domain-independent training technique that outperforms all other methods. Finally, we validate our findings on the attribute classification task in the CelebA dataset, where attribute presence is known to be correlated with the gender of people in the image, and demonstrate that the proposed technique is effective at mitigating real-world gender bias.
1. Introduction
The paper studies how spurious correlations in visual training data propagate into recognition models and evaluates strategies for mitigating them. It introduces a controlled benchmark, compares existing methods, and validates a domain-independent approach on CelebA.
- Spurious correlations, including gender cues associated with activities, can become discriminative features and may be amplified by visual models.
- The paper introduces a simple benchmark using controlled dataset augmentation to study bias mitigation in CNN-based visual recognition.The benchmark provides precise evaluation targets in an easier-to-analyze setting than a fully fledged black-box system.
- The benchmark enables direct comparison of domain adversarial training, Reducing Bias Amplification, and domain-conditional training under common conditions.The paper notes that these methods had previously been evaluated on different benchmarks and varying conditions.
- A domain-independent approach outperforms more complex competing methods on the benchmark.
- The proposed domain-independent training model successfully mitigates real-world gender bias on CelebA attribute recognition.
2. Related Work
Related work spans methods for identifying dataset bias, mitigating bias in linear and deep models, and evaluating fairness criteria. The paper contributes a bias-controlled dataset for directly comparing mitigation strategies in modern visual recognition models.
- Prior studies examine class imbalance, bias amplification, and model bias toward legally protected classes.
- The paper’s constructed dataset isolates and controls bias while retaining a modern discriminative architecture, enabling direct strategy comparisons.
- Earlier bias-mitigation methods primarily targeted simpler linear models, while newer work addresses sophisticated visual recognition models.
- Existing approaches include inference updates, subgroup-aware attribute detection, classifier decoupling, and adversarial mitigation.
- Fairness criteria such as demographic parity and predictive parity require careful selection, while ignoring protected attributes can remain ineffective because of redundant encoding.
3. A Simple Setting for Studying Bias
The paper constructs CIFAR-10S, a controlled benchmark that correlates object classes with color or grayscale domains. This setup shows that spurious correlations can reduce out-of-domain recognition even when biased data contains more information.
- CIFAR-10S Setup: CIFAR-10S correlates object-class decisions with an auxiliary color-versus-grayscale attribute.
- CIFAR-10S Setup: Each CIFAR-10 class is split 95% to 5% across color and grayscale domains, with five classes predominantly color and five predominantly grayscale.The total number of training images assigned to each domain remains balanced.
- The paper adopts this simplified color/grayscale framework before evaluating more complex gender-related settings because the transformation is easier to construct than gender recognition.
- CIFAR-10S Setup: The framework allows synthetic control of domain correlation, guarantees that color contains more information than grayscale, and avoids a fairness-accuracy trade-off.
- 89.0 ± 0.5% accuracy is achieved on COLOR test images by a model trained with the skewed setup, versus 93.0% by a model trained entirely on grayscale.
- The performance disparity arises because the skewed model associates color with object classes and uses color-domain membership when predicting on all-color images.
4. Benchmarking Bias Mitigation Methods
The benchmark compares bias-mitigation strategies for visual recognition under class-domain correlations, exposing weaknesses in adversarial and resampling approaches. Domain-independent training, with explicit combination of per-domain predictions, performs best among the evaluated methods and substantially improves color-image accuracy.
- Benchmark setup: CIFAR-10S benchmarks bias mitigation by correlating object classes with color and grayscale domains while evaluating multiple strategies under controlled conditions.The evaluation uses ResNet-18 models and reports mean per-class per-domain accuracy as its primary metric.
- Strategic Sampling: Oversampling raises baseline accuracy only from 88.5 ± 0.3% to 89.1 ± 0.4%, while removing augmentation drops oversampling accuracy to 79.2 ± 0.8%.Both models drive training loss to zero, indicating that augmentation is critical for avoiding overfitting in this setting.
- Domain Discriminative Training: Adversarial training enforces domain confusion but also introduces undesirable class confusion, reaching 84.1% accuracy in the reported evaluation.The t-SNE comparison links domain confusion with degraded separation of class boundaries.
- Reducing Bias Amplification: Applying RBA after prior shift changes accuracy only from 90.3% to 90.4%, whereas RBA without prior shift performs significantly worse than prior-shift inference.The authors therefore regard RBA after prior shift as extraneous in this setting.
- Domain Independent Training: Domain-independent training combines per-domain decision boundaries and achieves 92.0% accuracy, significantly exceeding the 88.5 ± 0.3% baseline.On color images, the same model reaches 92.4%, versus 89.0 ± 0.5% for BASELINE and 93.0 ± 0.2% for grayscale-only training.
5. Real World Experiments
The authors test whether bias-mitigation findings generalize beyond the original synthetic CIFAR-10S setting, using varied domain shifts and CelebA gender-correlated attributes. DOMAININDEPENDENT remains effective under stronger skew and realistic shifts, while oversampling performs best in a lower-skew CelebA setting.
- 5.1. CIFAR Extensions: DOMAININDEPENDENT consistently outperforms BASELINE as color/grayscale skew varies, with larger gains at higher skew levels.CelebA’s average gender skew is reported as 80.0%.
- 5.1. CIFAR Extensions: 83.5 ± 0.3% mean per-class per-domain accuracy makes DOMAININDEPENDENT best on the subtle CIFAR-to-32x32 ImageNet shift.BASELINE achieves 79.4 ± 0.4%, while DOMAINDIS-CRIMINATIVE achieves 81.5 ± 0.7%.
- 5.1. CIFAR Extensions: The CIFAR-10S conclusions hold across cropping, resolution changes, and CIFAR-to-ImageNet replacements.Table 2 reports mean per-class per-domain accuracy across these transformations.
- 5.2. CelebA Attribute Recognition: On CelebA, DOMAININDEPENDENT improves over BASELINE on 24 of 34 attributes, with improvement correlated with gender skew (ρ = 0.709).Attributes with skew above 80% always benefit from DOMAININDEPENDENT.
- 5.2. CelebA Attribute Recognition: Oversampling achieves 77.6% mAP and -0.061 bias amplification on CelebA, outperforming other techniques in this setting.The paper relates this result to more similar domains, lower skew, and a large dataset size.
6. Conclusions
The paper contributes a benchmark and systematic analysis of visual bias mitigation, while leaving several fairness and deployment questions for future work.
- 6. Conclusions: The study does not address non-discrete domains, unknown imbalanced-domain distributions, or downstream prediction decisions.These questions are explicitly left for future work.