Source-linked AI summary
Mitigating Unwanted Biases with Adversarial Learning
Brian Hu Zhang, Blake Lemoine, Margaret Mitchell
TL;DR
The paper addresses how machine-learning models can perpetuate undesirable demographic biases present in training data. It proposes adversarial learning to preserve predictive performance while reducing information about a protected variable, achieving less biased analogy predictions and equality of odds within 1% on both protected groups in UCI Adult results.
Problem
Training data can encode undesirable demographic biases, causing models to reproduce discrimination in tasks such as creditworthiness prediction and analogy completion.
Method
The method jointly trains a predictor for Y and an adversary for protected variable Z, using the adversary's gradient to reduce information about Z in predictions while supporting multiple fairness definitions.
Results
The method produces less biased word-analogy models and achieves equality of odds to within 1% on both protected groups in the UCI Adult Dataset.
Takeaways & Limitations
Adversarial learning can train models that remain useful for analogy tasks while becoming less biased, and can enforce fairness constraints across different predictor and variable types under stated assumptions.
Takeaways & Limitations
Adversarial training is sensitive to hyperparameter choices and can quickly diverge, while the theoretical guarantees rely on assumptions not satisfied in most neural-network use cases.
Abstract
from arXiv · showhide
Machine learning is a tool for building models that accurately represent input training data. When undesired biases concerning demographic groups are in the training data, well-trained models will reflect those biases. We present a framework for mitigating such biases by including a variable for the group of interest and simultaneously learning a predictor and an adversary. The input to the network X, here text or census data, produces a prediction Y, such as an analogy completion or income bracket, while the adversary tries to model a protected variable Z, here gender or zip code. The objective is to maximize the predictor's ability to predict Y while minimizing the adversary's ability to predict Z. Applied to analogy completion, this method results in accurate predictions that exhibit less evidence of stereotyping Z. When applied to a classification task using the UCI Adult (Census) Dataset, it results in a predictive model that does not lose much accuracy while achieving very close to equality of odds (Hardt, et al., 2016). The method is flexible and applicable to multiple definitions of fairness as well as a wide range of gradient-based learning models, including both regression and classification tasks.
1 Introduction
The paper frames biased training data as a source of discriminatory model behavior and introduces adversarial learning to preserve task accuracy while enforcing selected fairness constraints. It applies the approach to latent gender semantics and supervised classification, reporting close equality of odds on UCI Adult data.
- Training data can encode racial and gender biases that faithful machine-learning models perpetuate in decisions and analogy completion.
- The paper defines demographic parity, equality of odds, and equality of opportunity as alternative fairness constraints involving predictions, protected variables, and true labels.
- The adversarial method trains predictor f for accurate Y prediction while using adversary g to reduce information about protected variable Z in the prediction.
- For word embeddings, the method targets latent gender semantics while retaining analogy performance and reducing problematic gender-biased examples.
- Within 1% equality of odds on both protected groups was achieved on the UCI Adult Dataset.
- The approach better equalized group differences in False Positive Rate and False Negative Rate than Beutel et al. (2017), although that work had better overall False Negative Rate.
2 Related Work
Related work includes fairness definitions, post-processing and simple-model approaches, debiased word embeddings, and adversarial training. The paper situates its method among efforts to mitigate bias across data types and predictor settings.
- Bolukbasi et al. (2016) remove gender bias from embeddings using a human-supplied training set of gender-specific words.
- Removing a protected variable may fail to debias models because other features can remain highly correlated with it.
- Equality of odds and demographic parity are generally incongruent goals.
- Prior work defines fairness measures including Demographic Parity, Equality of Odds, and Equality of Opportunity.
- Adversarial training uses competing networks, and Beutel et al. (2017) applied it to Equality of Opportunity with discrete output variables.
3 Adversarial Debiasing
The adversarial debiasing architecture combines a task predictor with an adversary that estimates the protected variable from predictor outputs. Gradient updates are designed to reduce protected information without sacrificing the target objective.
- The predictor learns Y from X by minimizing prediction loss with gradient-based weight updates.
- The adversary receives the predictor’s output and attempts to predict protected variable Z.
- For Demographic Parity, the adversary uses predicted label Ŷ; for Equality of Odds, it uses Ŷ and true label Y.
- For Equality of Opportunity on class y, adversary training is restricted to examples whose true label satisfies Y = y.
- The projection term prevents predictor updates from helping the adversary, while the final adversarial term increases the adversary’s loss.
- The method supports regression and classification with discrete or continuous output and protected variables.
4 Properties
The method is presented as general across fairness definitions and variable types, model-agnostic for gradient-trained predictors, and theoretically optimal under convergence conditions. Its guarantees depend on stated assumptions.
- The approach can enforce Demographic Parity, Equality of Odds, or Equality of Opportunity, including when output or protected variables are continuous.
- The adversarial method is model-agnostic for predictors trained with gradient-based methods, regardless of predictor complexity.
- Under certain conditions, a converged predictor satisfies the desired fairness definition while also reducing prediction loss.
5 Theoretical Guarantees
Under stated convergence and adversary assumptions, the adversarial framework theoretically enforces demographic parity and equality of odds. The guarantees extend to continuous variables and restricted adversaries, while practical convergence remains difficult.
- Theoretical Guarantees: Under the stated assumptions, the adversary gains no advantage from access to the predictor’s output weights.This equality at convergence links the adversary’s inability to exploit the prediction to the fairness guarantees.
- Theoretical Guarantees: The theoretical claims require assumptions that are generally unmet by neural networks, and guaranteed convergence is difficult in theory and practice.The practical discussion therefore focuses on encouraging convergence and choosing adversaries that are powerful yet trainable.
- Theoretical Guarantees: The predictor satisfies demographic parity when a sufficiently strong adversary is trained on the prediction and is completely fooled at convergence.The proof uses the adversary’s loss reaching H(Z), which rules out dependence between the prediction and protected variable.
- Theoretical Guarantees: The predictor satisfies equality of odds when the adversary receives both the prediction and true label and cannot improve beyond H(Z|Y).The corresponding proof excludes conditional dependence between the prediction and protected variable given the true label.
- Theoretical Guarantees: The guarantees also apply analogously to continuous variables and to adversaries restricted to limited predictor classes.For continuous variables, differential entropy replaces discrete entropy; a linear adversary can yield no linear relationship after conditioning on Y.
6 Experiments
The experiments evaluate adversarial debiasing on word analogies and UCI Adult income prediction, measuring changes in bias, fairness, and predictive performance. The method preserves useful prediction while reducing protected-variable information and nearly equalizing subgroup error rates.
- Toy Scenario: The toy logistic model changes from y = σ(0.7u+0.7r) to y = σ(0.6u −0.6r + 0.6), making predictions independent of protected variable r.The debiased model uses u −r, which is distributed independently of r.
- Word Embeddings: The analogy model trains on word vectors while using a gender subspace defined from 10 male–female word pairs and its top k principal components.The experiments use k = 1 and do not update the original embeddings or project them during input construction.
- Word Embeddings: After debiasing, w correlates more strongly with the gender direction, increasing wTg from 0.08 to 0.55 while preserving gendered analogies and removing many biased analogies.The reported vector norms are 0.82 without debiasing and 0.96 with debiasing.
- UCI Adult Dataset: The UCI Adult experiment predicts whether income exceeds $50k from categorical and continuous attributes, with sex as the binary protected variable.The model discards fnlwgt, bucketizes age, and represents categorical features as sparse tensors.
- UCI Adult Dataset: 86.0% vs 84.5% overall accuracy accompanies near-equality of odds after debiasing, with FNR 0.0647 ≈0.0701 and FPR 0.4458 ≈0.4349 across sex subgroups.The experiment compares debiased and non-debiased models using test-set confusion matrices and subgroup FPR/FNR.
7 Conclusion
The paper presents a general adversarial method for training models that satisfy multiple fairness definitions while retaining strong task performance. It provides theoretical guarantees and demonstrates reduced bias in supervised learning and word-embedding analogy tasks.
- The method theoretically enforces multiple fairness constraints across predictor complexities and discrete or continuous predicted and protected variables.The guarantees hold under stated reasonable assumptions.
- In practice, the method produces models that are less biased while still performing extremely well on two different tasks.The applications are a standard supervised learning task and debiased word embeddings for analogies.
- A simple adversary can be used for discrete output and protected variables regardless of the underlying model's complexity.
- The paper also discusses difficulties in getting adversarial models to converge.
8 Future Work
The paper identifies several directions for extending and stabilizing adversarial debiasing. These include testing debiased embeddings on harder tasks, improving convergence, applying multiple adversaries to image recognition, and studying adversary complexity.
- Future work should test whether debiased word embeddings remain useful in more complex tasks beyond analogies.
- Adversarial training is sensitive to hyperparameter choices, which can cause rapid algorithmic divergence and motivates methods for stabilizing convergence.
- Multiple adversaries could be investigated for accurate image-recognition systems that avoid racial performance biases.
- Further study is needed to determine whether complex predictors require more complex adversaries, especially for continuous variables.The paper raises whether simple adversaries remain sufficient or become too easy to deceive.