Source-linked AI summary
A comparative study of fairness-enhancing interventions in machine learning
Sorelle A. Friedler, Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P. Hamilton, Derek Roth
TL;DR
Fairness-enhancing algorithms are difficult to compare consistently across datasets, measures, and preprocessing choices, despite their importance for high-impact decisions. The paper develops an open benchmark for such comparisons and finds correlated fairness measures, distinct algorithmic tradeoffs, and sensitivity to training-test splits. It focuses on group fairness in batch-learning settings and recommends evaluating performance and stability across multiple randomized splits.
Problem
The paper addresses limited effective comparison of fairness-enhancing algorithms across evaluation measures and datasets, a gap relevant to decisions affecting population subgroups.
Method
The paper develops an open benchmark that compares algorithms across datasets, fairness and accuracy measures, consistent preprocessing, and training-test splits.
Results
Many fairness measures strongly correlate, algorithms make significantly different accuracy-fairness tradeoffs without a dominant performer, preprocessing affects reported measures, and results vary across training-test splits.
Takeaways & Limitations
Fairness evaluations should compare methods across measures, preprocessing choices, and multiple training-test splits rather than relying on a single reported configuration.
Takeaways & Limitations
The study focuses on group fairness approaches and does not study algorithms that seek to optimize individual fairness.
Abstract
from arXiv · showhide
Computers are increasingly used to make decisions that have significant impact in people's lives. Often, these predictions can affect different population subgroups disproportionately. As a result, the issue of fairness has received much recent interest, and a number of fairness-enhanced classifiers and predictors have appeared in the literature. This paper seeks to study the following questions: how do these different techniques fundamentally compare to one another, and what accounts for the differences? Specifically, we seek to bring attention to many under-appreciated aspects of such fairness-enhancing interventions. Concretely, we present the results of an open benchmark we have developed that lets us compare a number of different algorithms under a variety of fairness measures, and a large number of existing datasets. We find that although different algorithms tend to prefer specific formulations of fairness preservations, many of these measures strongly correlate with one another. In addition, we find that fairness-preserving algorithms tend to be sensitive to fluctuations in dataset composition (simulated in our benchmark by varying training-test splits), indicating that fairness interventions might be more brittle than previously thought.
1 Introduction
The paper addresses the lack of consistent comparisons among fairness-enhancing algorithms across datasets, preprocessing choices, and evaluation measures. Its benchmark finds correlated fairness measures, distinct algorithmic tradeoffs, preprocessing effects, and sensitivity to training-test splits.
- The paper introduces an open test-bed for comparing fairness-aware algorithms across datasets, accuracy measures, and fairness measures.The framework also supports adding methods, measures, and data for comparative evaluation.
- Preprocessing choices can affect the accuracy and fairness measures reported for algorithms.Algorithms differ in how they encode sensitive attributes, support multiple sensitive attributes, and handle categorical attributes.
- Many fairness measures strongly correlate, so optimizing one measure often produces good or poor performance on another.
- No algorithm appears to dominate in both accuracy and fairness because different algorithms produce significantly different tradeoffs.
- Fairness measures vary substantially across training-test splits, suggesting that fairness-enhancing algorithms may be more brittle than previously recognized.
2 Background
Fairness-aware methods target non-discriminatory predictions across protected groups through preprocessing, algorithm modification, or postprocessing. The paper focuses on batch-learning group fairness and distinguishes its benchmark from related bias-auditing and individual-fairness approaches.
- Fairness-aware algorithms seek non-discriminatory predictions for people based on protected-class status, also called a sensitive attribute.Methods are broadly categorized as preprocessing, algorithm modification, or postprocessing interventions.
- Preprocessing methods modify training data, while algorithm modifications alter learning procedures and postprocessing methods modify trained-model outputs.
- The paper limits its fairness-method description to batch-learning interventions and excludes sequential and reinforcement-learning interventions.
- The study focuses on group fairness across protected groups and does not study algorithms designed to optimize individual fairness.The stated goal is to examine group-based discrimination, including metrics such as disparate impact.
- Related efforts audit potential biases, generate test cases for group-based or causal discrimination, or evaluate one algorithm across datasets and fairness measures.
3 Benchmark Structure
The benchmark separates data input, preprocessing, benchmarking, and analysis so algorithms can be compared under consistent datasets, preprocessing, splits, and measures. Its modular pipeline is designed for reproducibility and extension.
- The pipeline separates data input, preprocessing, benchmarking, and analysis, saving intermediate files to support reproducibility.
- Each algorithm is evaluated using the same dataset, preprocessing, training-test splits, and fairness and accuracy measures.This structure addresses comparison difficulties caused when prior work combines dataset preprocessing with algorithm code.
- Datasets, measures, and algorithms are modularized so new objects can be added and evaluated across the existing pipeline.
4 Data
The experiments use five real-world datasets from fairness-relevant domains, with preprocessing applied consistently according to algorithm requirements. The datasets cover promotion, income, credit, and recidivism prediction tasks.
- The benchmark uses five real-world datasets previously studied in fairness-aware machine learning and preprocesses them consistently according to algorithm needs.The domains include hiring and promotion, credit-worthiness and loans, and recidivism prediction.
- The Ricci dataset contains 118 firefighter-promotion cases with Race as a sensitive attribute and predicts the original promotion decision.
- The Adult Income dataset contains 32,561 census instances and predicts whether income is above or below $50,000 using race and sex as sensitive attributes.The benchmark removes 2,399 instances with missing data during preprocessing.
- The German Credit dataset contains 1,000 instances for good-versus-bad credit-risk classification, with sex and age used as sensitive attributes.Age is discretized into adult and youth categories.
- The ProPublica recidivism dataset contains 6,167 individuals and predicts rearrest within two years using race and sex as sensitive attributes.
- The violent-recidivism variant predicts rearrest for a violent crime within two years and includes 4,010 individuals with race and sex as sensitive attributes.
5 Preprocessing
The benchmark standardizes algorithm inputs through multiple preprocessing versions, enabling comparisons across methods and representations. In the Feldman analysis, representation affects accuracy for SVM variants more than fairness, while fairness varies across data splits.
- 5 Preprocessing: The benchmark creates compatible input versions so algorithms with different data requirements can be compared consistently.Preprocessing includes dataset-specific modifications, categorical one-hot encoding, and binary sensitive-attribute encodings.
- 5 Preprocessing: For most Feldman variants, accuracy is independent of representation, but all three SVM variants perform consistently better with numerical+binary inputs.The authors speculate that grouping unprivileged categories provides more people per sensitive group during preprocessing; one-hot encoding may also affect SVM effectiveness.
- 5 Preprocessing: Fairness varies across runs, while representation differences are smaller than for accuracy, although SVMs remain representation-sensitive.For this algorithm, classifier and representation affect accuracy, whereas the achieved fairness measure is comparatively less affected by representation.
6 Measures
The paper evaluates models using conditional-probability measures of accuracy and fairness. It considers standard accuracy metrics and normalized fairness measures whose perfect-fairness value is 1.
- 6 Measures: Accuracy and fairness are defined from conditional probabilities involving true outcomes, predictions, and the sensitive attribute.The dataset is represented as D = (X, S, Y), with S denoting the sensitive attribute and Y the binary label.
- 6 Measures: The accuracy measures include uniform accuracy, true positive rate, true negative rate, and balanced classification rate.Balanced classification rate is an accuracy variant that is unweighted per class.
- 6 Measures: All listed accuracy measures lie in the range [0, 1].
- 6 Measures: Fairness measures are grouped into three broad categories and normalized for comparison, with perfect fairness achieved at 1.Depending on the measure, normalized values lie in [0, ∞) or [0, 2].
6 An upcoming tutorial puts the number of fairness measures at 21 Narayanan (2018)!
The paper organizes fairness measures into base-rate, group-conditioned accuracy, and calibration families, then examines how they relate and trade off across algorithms and data splits.
- Base-rate measures: Base-rate measures compare groups’ predicted positive rates using ratios such as DI or differences such as CV.DI groups unprivileged classes or compares them pairwise, while CV uses the corresponding difference without taking its absolute value.
- Group-conditioned accuracy: Group-conditioned fairness measures assess whether prediction error rates are similar across sensitive groups.Related formulations include error rate balance and equalized odds, which target different combinations of group-conditioned rates.
- Group-conditioned calibration: Calibration-based measures condition calibration functions on sensitive groups to assess whether predicted probabilities remain aligned with outcomes across groups.The paper discusses separate calibration measures for positive and negative outcomes.
- Relationships among measures: Many fairness measures are correlated, with group-conditioned measures especially closely related and DI and CV closely related by definition.An exception is s-Calibration-, which is more closely associated with base-rate measures than with other group-conditioned measures.
- Relationships among measures: Accuracy measures correlate with group-conditioned fairness measures, making fairness-accuracy tradeoffs more pertinent to base-rate measures.The paper reports this pattern from analyses across dataset-algorithm pairs and fairness measurements.
- Tradeoffs: There is a clear tradeoff between s-calibration- and s-TPR, with different algorithms occupying different positions along the tradeoff.The paper links this empirical pattern to the impossibility of simultaneously achieving calibration and error rate balance when base rates differ across populations.
7 Algorithms
The benchmark compares fairness-aware algorithms across datasets, metrics, preprocessing choices, and randomized splits. Results show substantial variation in fairness–accuracy tradeoffs and instability across splits and sensitive attributes.
- Benchmark design: The benchmark runs selected fairness-aware and baseline algorithms across datasets, metrics, preprocessing choices, and randomized train-test splits.Algorithms include SVM, decision trees, Gaussian naive Bayes, and logistic regression baselines.
- Overall performance: No algorithm is consistently both more fair and more accurate across all datasets.Performance varies substantially even within a single algorithm across random splits.
- Overall performance: Algorithms occupy different positions on the fairness–accuracy tradeoff, and some favorable balances have high variability across splits.On Adult Income, Zafar et al. may balance fairness and accuracy well for sex, but its DI has a large standard deviation.
- Parameter tuning: Parameter tuning exposes additional fairness–accuracy tradeoffs for algorithms such as Zafar et al. and Feldman et al.The benchmark varies parameters across train-test splits and records results for each parameter value.
- Preprocessing and sensitive attributes: Fairness and accuracy outcomes depend on preprocessing requirements and the sensitive attribute being considered.The study examines race, sex, and composite attributes; correcting for both can produce intermediate DI values and higher variance for Zafar et al.
8 Discussion
The discussion identifies preprocessing, metric selection, and training instability as important considerations for comparing fairness interventions. It recommends consistent preprocessing, a compact metric set, and evaluation over multiple randomized splits.
- Preprocessing: Different preprocessing choices can change the accuracy and fairness measures reported for an algorithm.The authors recommend reporting multiple plausible preprocessing choices and using the same preprocessing when comparing algorithms.
- Algorithm comparisons: Fairness interventions can make significantly different tradeoffs, and no algorithm dominates in both accuracy and fairness across the benchmark.Different mechanisms produce substantially different predictions and performance profiles.
- Fairness measures: Many fairness measures strongly correlate, so measures should be added only when they behave fundamentally differently from existing metrics.The study identifies class-sensitive error rates combined with DI or CV as a minimal working set.
- Training stability: A single training-test split is insufficient for evaluating algorithm success because fairness measures vary across randomized splits.The authors recommend reporting both performance and stability over a moderate number of randomized splits.