Source-linked AI summary
Achieving Fairness through Adversarial Learning: an Application to Recidivism Prediction
Christina Wadsworth, Francesca Vera, Chris Piech
TL;DR
The paper addresses racial bias in recidivism prediction, where COMPAS and learned criminal-history correlations can disadvantage black inmates. It trains an adversarial neural network to reduce demographic information in recidivism predictions while retaining accuracy. Compared with COMPAS, the model improves accuracy and approaches parity and equality of odds, though it is slightly more biased on calibration.
Problem
Recidivism prediction can perpetuate racial bias because features correlated with race remain informative even when race is excluded.
Method
The paper adds an adversary that predicts demographic membership from the recidivism predictor’s output and penalizes demographic predictability during training.
Results
The adversarial model outperforms COMPAS in accuracy and comes closer to parity and equality of odds, while showing slightly greater calibration bias.
Takeaways & Limitations
Adversarial learning can reduce bias in models trained on demographically biased data while enforcing multiple fairness definitions in recidivism prediction.
Abstract
from arXiv · showhide
Recidivism prediction scores are used across the USA to determine sentencing and supervision for hundreds of thousands of inmates. One such generator of recidivism prediction scores is Northpointe's Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) score, used in states like California and Florida, which past research has shown to be biased against black inmates according to certain measures of fairness. To counteract this racial bias, we present an adversarially-trained neural network that predicts recidivism and is trained to remove racial bias. When comparing the results of our model to COMPAS, we gain predictive accuracy and get closer to achieving two out of three measures of fairness: parity and equality of odds. Our model can be generalized to any prediction and demographic. This piece of research contributes an example of scientific replication and simplification in a high-stakes real-world application like recidivism prediction.
1. Introduction
Machine-learning systems used in high-stakes decisions can perpetuate societal biases, making fairness an additional concern beyond accuracy. The paper targets racial bias in recidivism prediction by adding an adversary to the prediction network.
- High-stakes algorithms can perpetuate existing gender- and race-based societal biases, so fairness must be considered alongside accuracy.
- COMPAS is a widely used, proprietary recidivism score whose algorithm Northpointe has not publicly released.COMPAS uses part of a 137-question survey covering areas including criminal history, social environment, criminal personality, and criminal attitudes.
- Prior research reported that COMPAS was biased against black inmates based on Florida inmate data, while another analysis argued that it satisfies calibration.
- Neural networks can learn racial bias indirectly because criminal-history features such as priors and previous jail time correlate with race.
- The proposed adversarial network penalizes the recidivism predictor when race can be inferred from its output, counteracting racial bias in criminal-history datasets.The authors state that the approach generalizes to almost any prediction and demographic.
2. Related Work
Related work frames fairness through parity, equality of odds, and calibration, while prior adversarial methods and COMPAS debiasing approaches target different fairness constraints. The paper positions its approach within ongoing debate about which fairness definitions should govern recidivism prediction.
- Fairness Definitions: The paper uses demographic parity, equality of odds, and calibration as its three fairness definitions.
- Fairness Definitions: Parity requires equal high-risk classification proportions across demographics.
- Fairness Definitions: Equality of odds requires equal high-risk classification proportions across demographics when true future recidivism is held constant.
- Fairness Definitions: Calibration requires high-risk scores to represent the same recidivism likelihood irrespective of demographic group.
- Adversarial Fairness: Prior adversarial methods used shared hidden layers, domain classifiers, reverse gradients, or projection terms to pursue parity or equality of odds.
- Fairness and COMPAS: Researchers disagree about which fairness definitions recidivism predictions should satisfy, including whether racial disparities may reflect differing arrest rates and priors.
- Methods to Improve Fairness on COMPAS: Earlier COMPAS debiasing methods used group-specific thresholds, randomization, misclassification penalties, or decision-boundary-distance penalties.
3. Adversarial Model
The model combines a recidivism predictor with an adversary that tries to infer demographic membership from the predictor’s output. Joint training preserves prediction accuracy while discouraging demographic information in that output.
- 3.1. Model Structure: A multilayer network N predicts recidivism probability ˆY, while an adversary A receives N’s logit and predicts demographic D.
- 3.1. Model Structure: The adversary tests whether the predictor’s logit contains demographic information, even when demographic D is not an input feature.
- 3.2. Model Training: Training aims for N to predict ˆY accurately while A predicts D poorly, producing an accurate output closer to parity.
- 3.2. Model Training: The predictor and adversary use binary cross-entropy losses Ly and Ld, respectively, with the predictor trained using Ly − αLd.
- 3.2. Model Training: Subtracting the adversary loss encourages N to maximize Ld, making its logit less useful for predicting race.
4. Experiments and Results
The experiments compare regular and adversarial recidivism predictors with COMPAS on accuracy and fairness metrics. The adversarial model improves accuracy, comes closer to parity and equality of odds, and is slightly less calibrated than COMPAS.
- Experiments: The study uses Broward County criminal-record data containing 8,230 training cases and 2,213 test cases, restricted to white and black inmates.The dataset includes COMPAS scores, enabling direct comparison with the model.
- Experiments: Accuracy is evaluated with area under the ROC curve, while High Risk Gap, False Positive Gap, False Negative Gap, and calibration assess fairness.Zero High Risk Gap indicates parity; zero False Positive and False Negative Gaps indicate equality of odds.
- Experiments: The experiments compare a regular recidivism predictor with adversarial variants whose adversaries receive the predictor logit, the logit plus true recidivism, or a hidden layer.The reported adversarial model uses only the logit as adversary input.
- Results: The chosen adversarial model is much closer than COMPAS to satisfying parity and equality of odds, while also improving accuracy over COMPAS.These comparisons are reported using Tables 1 and 2.
- Results: The adversarial model relies more heavily on 6 of the top 10 important features and appears to use more holistic information than the baseline.The baseline relies mostly on charge degree, age, and priors, features discussed as potentially contributing to racial bias.
- Results: The model matches state-of-the-art COMPAS debiasing work on racial fairness, but is slightly more biased than COMPAS on calibration.Calibration bias is described as slight and especially evident at the 0.5 recidivism threshold.
5. Case Study
The case study compares COMPAS with the adversarial model on two Broward County inmates and examines model behavior through conditional-probability and feature-importance analyses. The examples show substantial disagreement between COMPAS and the adversarial predictions for both inmates.
- Case study: The case study compares COMPAS predictions with the adversarial model's results for two Broward County inmates.The analysis investigates the individual stories of Joe and Bob.
- Joe: Joe received a COMPAS score of 8 out of 10 despite having 1 prior and 2 charges, while the adversarial model predicted 0.05.Joe was a 55-year-old black inmate.
- Joe: Joe has not recidivated since receiving his COMPAS score.
- Bob: Bob received a COMPAS score of 5 out of 10 despite 13 priors and 24 charges, while the adversarial model predicted 0.84.Bob was a 27-year-old white inmate.
- Bob: Since receiving his COMPAS score, Bob has been charged 6 times, including for aggravated battery.The passage identifies aggravated battery as an act of violent recidivism.
6. Conclusion
The paper concludes that adversarial learning can reduce bias in models trained on demographically biased data while enforcing multiple fairness definitions. Its adversarial models are less biased than COMPAS and the baseline, yet outperform COMPAS in accuracy.
- Conclusion: Adversarial learning reduced bias in a machine-learning model trained on demographically biased data.
- Conclusion: The method can enforce constraints for multiple definitions of fairness.
- Conclusion: The adversarial models were less biased than COMPAS scores and the recidivism-prediction baseline.
- Conclusion: The adversarial models outperformed COMPAS in accuracy while achieving lower bias.