Source-linked AI summary
On the relation between accuracy and fairness in binary classification
Indre Zliobaite
TL;DR
The paper addresses misleading comparisons of non-discriminatory classifiers when their positive-output rates differ. It analyzes threshold-dependent accuracy and discrimination, proposes normalized comparison measures, and studies theoretical and empirical tradeoffs under discrimination constraints.
Problem
Comparisons of non-discriminatory classifiers can be misleading when classifiers produce different acceptance rates.
Method
The paper varies classification thresholds and introduces normalization of accuracy and discrimination with respect to acceptance rates.
Results
The analysis shows that changing acceptance rates changes baseline accuracy and discrimination, while normalized measures preserve linear upper bounds under discrimination constraints.
Takeaways & Limitations
When acceptance rates are fixed, classifiers can be compared using accuracy and discrimination; otherwise, comparison should use κ and normalized discrimination δ.
Takeaways & Limitations
The paper notes that acceptance rates may be constrained by available resources and that massaging and other removal techniques require revision for differing rates and normalized discrimination measures.
Abstract
from arXiv · showhide
Our study revisits the problem of accuracy-fairness tradeoff in binary classification. We argue that comparison of non-discriminatory classifiers needs to account for different rates of positive predictions, otherwise conclusions about performance may be misleading, because accuracy and discrimination of naive baselines on the same dataset vary with different rates of positive predictions. We provide methodological recommendations for sound comparison of non-discriminatory classifiers, and present a brief theoretical and empirical analysis of tradeoffs between accuracy and non-discrimination.
1. Introduction
Discrimination-aware machine learning seeks predictive models that satisfy non-discrimination constraints despite potentially biased historical data. This paper revisits classifier comparisons, warning that differing positive-prediction rates can make performance comparisons misleading.
- Discrimination-aware machine learning develops predictive models that are free from discrimination despite biased, incomplete, or historically discriminatory data.
- Existing work often targets equal acceptance rates between favored and protected groups, which can constitute affirmative action and may be legally or politically desirable.
- The paper revisits discrimination-aware classification by identifying pitfalls in comparing classifiers with different proportions of positive predictions.
- It recommends sound comparison methods and analyzes tradeoffs between accuracy and non-discrimination theoretically and empirically.
2. Problem setting and assumptions
The setting considers binary decisions under non-discrimination constraints, with protected characteristics potentially recoverable from correlated input variables. Classifier thresholds control acceptance rates while accuracy and discrimination are defined relative to group and dataset rates.
- The objective is to maximize classifier accuracy while obeying non-discrimination constraints for polar binary outcomes such as loan acceptance or rejection.
- A classifier maps input variables X to target y, even though correlated inputs can indirectly encode the protected characteristic s.
- Discrimination is defined as the difference in acceptance rates between favored and protected groups, while π denotes the classifier’s overall acceptance rate.
- Probability scores are converted into positive decisions using a threshold, so changing the threshold changes the acceptance rate π.
3. Accuracy and fairness
Comparing binary classifiers on accuracy and discrimination can mislead when their acceptance rates differ, because changing that rate changes both baseline accuracy and baseline discrimination. The paper recommends normalizing these measures by acceptance-rate-dependent baselines and illustrates the resulting tradeoffs theoretically and empirically.
- Different acceptance rates make accuracy and discrimination comparisons misleading because they change baseline accuracy and baseline discrimination.
- An Adult-dataset experiment varies the classification threshold from 0 to 1, thereby changing the acceptance rate π.The experiment trains logistic regression on one half of the data and evaluates it on the other half.
- As acceptance rates become extreme, logistic-regression accuracy approaches random-classifier accuracy, so higher observed accuracy need not indicate better classification ability.
- Cohen’s Kappa normalizes accuracy against a random classifier whose accuracy is R = π0π + (1 −π0)(1 −π), with κ = 1 for ideal accuracy and κ = 0 for random results.
- Discrimination also shrinks toward zero at extreme acceptance rates without any change in the classifier, motivating normalization by the maximum possible discrimination dmax at each π.
- The maximum-discrimination benchmark ranks all favored-community candidates before protected-community candidates, with dmax determined by favored-group proportion α and acceptance rate π.
4. Baselines and tradeoffs
The paper uses oracle and random classifiers as baselines for analyzing accuracy–non-discrimination tradeoffs. It shows that normalized accuracy bounds remain linear, while unconstrained acceptance rates make the optimal strategy depend on additional dataset and target parameters.
- Baselines: The oracle is a maximally informed classifier satisfying non-discrimination constraints, while the random classifier predicts acceptance with probability π.The oracle has perfect accuracy before imposing discrimination constraints; the random classifier has κ = 0 and becomes a majority-class classifier when π = 0 or π = 1.
- Fixed discrimination: Removing discrimination can reduce maximum accuracy linearly with the difference between historical and desired discrimination.The decrease is (A0 − A) = min(α, 1 − α)(d0 − d).
- Normalized tradeoff: With fixed acceptance rate π = π0, normalized oracle accuracy decreases linearly as normalized discrimination decreases.
- Flexible acceptance rates: When acceptance rates are unconstrained, the oracle either decreases favored-group acceptance or increases protected-group acceptance.Which strategy is optimal depends on α, π0, and the target discrimination δ⋆.
- Flexible acceptance rates: Figure 3 presents simulated Adult-dataset oracle results, showing linear upper bounds for normalized accuracy and discrimination.The “Change both” solution keeps the acceptance rate equal to that in the original data.
5. Interesting cases
The experiment compares three classifiers trained with different protected-attribute and label-massaging strategies, showing that acceptance rates materially affect observed accuracy and discrimination. Massaging generally improves discrimination relative to removing the protected characteristic, but its effects vary across acceptance rates and can reverse discrimination.
- Experimental setup: The experiment compares logistic regression, Naive Bayes, and decision tree J48 under three strategies involving protected-characteristic removal and label massaging.Testing is performed on unmodified data after training labels are massaged to preserve the original positive rate while setting discrimination to zero.
- Observed results: All classifiers tend to produce lower acceptance rates than the original data, making nominal discrimination and accuracy comparisons difficult to interpret.Using the protected characteristic can reduce nominal discrimination while normalized discrimination becomes higher than in the data.
- Observed results: Removing the protected characteristic yields little discrimination improvement because correlated features continue to capture it through redlining.For logistic regression and decision tree, discrimination remains higher than in the original dataset.
- Observed results: Massaging produces higher acceptance rates and lower discrimination than protected-characteristic removal, with rates closer to those in the original data.This indicates that lower nominal discrimination can accompany reduced acceptance rates while normalized discrimination remains.
- Observed results: Across acceptance rates, massaging removes some discrimination but is often imprecise and can overshoot into reverse discrimination.The figure reports normalized accuracy and discrimination at different acceptance rates.
6. Conclusion
The conclusion argues that fair-classifier evaluation must control for positive output rates. It introduces normalized discrimination and specifies when raw versus normalized metrics support valid comparisons.
- Evaluation of non-discriminatory classifiers can be misleading when their positive output rates differ, invalidating comparative performance conclusions.
- The paper normalizes discrimination using the maximum possible discrimination at a given acceptance rate.That maximum occurs when protected individuals are accepted only after everyone in the favored community has been accepted.
- When data and classifier acceptance rates are fixed, classifiers are comparable using accuracy A and discrimination d; otherwise use normalized measures κ and δ.