Source-linked AI summary
A statistical framework for fair predictive algorithms
Kristian Lum, James Johndrow
TL;DR
Biased outcome records can lead predictive algorithms to produce predictions biased against groups, and omitting a protected variable may leave correlated information in the predictors. This paper transforms predictors to remove protected-variable information; on recidivism data, the adjustment nearly eliminated racial prediction disparities with little change in AUC.
Problem
Predictive models trained on outcomes recorded through biased processes can produce predictions biased against groups disproportionately represented in the training data.
Source: §1 · p. 1Method
The framework transforms predictors using conditional distributions to make them independent of protected variables while retaining information in the predictors.
Source: §2.1 · p. 2, §2.2 · p. 2Results
AUC was 0.71 for unadjusted data and 0.72 for adjusted data, whose prediction distributions nearly eliminated racial disparities.
Source: §3 · pp. 4–5Takeaways & Limitations
Abstract
from arXiv · showhide
Predictive modeling is increasingly being employed to assist human decision-makers. One purported advantage of replacing human judgment with computer models in high stakes settings-- such as sentencing, hiring, policing, college admissions, and parole decisions-- is the perceived "neutrality" of computers. It is argued that because computer models do not hold personal prejudice, the predictions they produce will be equally free from prejudice. There is growing recognition that employing algorithms does not remove the potential for bias, and can even amplify it, since training data were inevitably generated by a process that is itself biased. In this paper, we provide a probabilistic definition of algorithmic bias. We propose a method to remove bias from predictive models by removing all information regarding protected variables from the permitted training data. Unlike previous work in this area, our framework is general enough to accommodate arbitrary data types, e.g. binary, continuous, etc. Motivated by models currently in use in the criminal justice system that inform decisions on pre-trial release and paroling, we apply our proposed method to a dataset on the criminal histories of individuals at the time of sentencing to produce "race-neutral" predictions of re-arrest. In the process, we demonstrate that the most common approach to creating "race-neutral" models-- omitting race as a covariate-- still results in racially disparate predictions. We then demonstrate that the application of our proposed method to these data removes racial disparities from predictions with minimal impact on predictive accuracy.
1 Introduction
Predictive algorithms can inherit bias from outcomes recorded through biased processes, even when they do not use protected characteristics directly. The paper proposes chained conditional models to remove information about protected variables from predictors, extending adjustment to arbitrary numbers and types of variables.
1 Introduction
- Biased outcome records can produce predictions biased against groups disproportionately represented in training data.The paper cites arrest-rate disparities for Black and white drug users despite roughly similar estimated drug-use rates.Source: §1 · p. 1
- Omitting protected variables is insufficient when included predictors remain correlated with those characteristics.The paper describes this as omitted variable bias in linear models.Source: §1 · p. 1
- The proposed procedure chains conditional models to remove protected-variable information from predictors, building on [7] from a statistical modeling perspective.Source: §1 · p. 1
- Unlike prior approaches limited to binary or categorical protected variables and few adjusted covariates, the framework supports arbitrary numbers of continuous or discrete variables.Likelihood-based regression methods, including Bayesian hierarchical models for sparse classes, can be used within the framework.Source: §1 · pp. 1–2
2 Methods
The paper defines fairness as prediction independence from protected characteristics and seeks adjusted predictors independent of those characteristics while retaining information from the original predictors. It uses conditional-distribution transformations to construct such predictors, with multivariate extensions and caveats about when independence is guaranteed.
2.1 Setup
- A prediction is defined as fair when it is independent of protected characteristics, and adjusted predictors independent of them suffice for fair models fit to those predictors.Source: §2.1 · p. 2
2.2 Transformations to independence
- For each predictor, the proposed transformation maps its conditional cumulative probability given protected variables through the inverse of a chosen distribution function.The transformed variable is independent of the protected variables under an adequate conditional model.Source: §2.2 · p. 2
- Univariate transformations can achieve pairwise independence but do not generally guarantee fair predictions; conditional models may use nonparametric or generalized linear methods, and discrete predictors can use randomized maps.The paper notes univariate transformations may suffice depending on the prediction model and data dependence structure.Source: §2.2 · p. 2
- The multivariate transformation applies conditional transformations sequentially, conditioning each variable on protected variables and previously transformed predictors, yielding mutual independence.Source: §2.2 · p. 3
- The transformation preserves conditional ranks, while the choice of marginal distribution for transformed values does not change predictor ranks.The paper suggests using the original marginal distribution so researchers retain access to it.Source: §2.2 · p. 3
3 Application: Removing racial bias in the recidivism risk assessment
On Broward County recidivism data, adjusting predictors produced nearly identical prediction distributions by race, unlike unadjusted random forests that omitted race. The adjusted and unadjusted models had similar AUC values.
3 Application: Removing racial bias in the recidivism risk assessment
- The analysis used Broward County defendants’ criminal-history measures, race, sex, age, and two-year re-arrest outcomes to predict re-arrest within two years.The dataset was compiled by ProPublica to investigate risk-assessment recommendations for Black defendants.Source: §3 · p. 3
- Conditional models matched predictor types: zero-inflated Poisson or negative binomial models for counts, logistic regression for binary sex, and linear regression with bootstrapped residual distributions for continuous age.Source: §3 · pp. 3–4
- Because discrete-variable transformations are stochastic, predictions and parameter uncertainty used averages over M fair datasets; conditional-model checks failed to reject uniformity in all but the prior juvenile count.Source: §3 · p. 4
- AUC was 0.71 for unadjusted data and 0.72 for adjusted data, while adjusted predictions nearly eliminated racial disparities in the distributions.The unadjusted random forest omitted race and assigned Black individuals more probability mass above about 0.5; adjusted distributions were nearly identical by race.Source: Figure 1 · p. 4, §3 · pp. 4–5, Figure 2 · p. 5
4 Discussion
The paper concludes that its statistical framework removed protected-variable information in the recidivism application, producing predictions independent of the protected variable with minimal loss in predictive accuracy.
4 Discussion
- In the recidivism application, the framework produced predictions independent of the protected variable with minimal loss in predictive accuracy, comparable to commercially available algorithms.Source: §4 · p. 5