PaperlayerView exact revision on arXiv

Source-linked AI summary

Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey

Max Hort, Zhenpeng Chen, Jie M. Zhang, Mark Harman, Federica Sarro

arXiv:2207.07068v4cs.LG

TL;DR

Practitioners lack clear guidance on evaluating bias mitigation methods, including which measures, datasets, and comparison mechanisms to use. This survey analyzes 341 publications, categorizes mitigation methods and evaluation practices, and reports broad dataset and benchmarking patterns.

  • Problem

    Practitioners still lack clear guidance on which measures, datasets, and comparison mechanisms to use when evaluating new bias mitigation methods.

  • Method

    The survey analyzes 341 publications, categorizes methods by intervention type and approach, and examines evaluation through datasets, metrics, and benchmarking.

  • Results

    In-processing is most common; 81 datasets were identified, the Adult dataset appeared in 77% of publications, and methods usually used the original model as a baseline.

  • Takeaways & Limitations

    The survey identifies challenges and opportunities to support more informed development and evaluation of bias mitigation methods.

  • Takeaways & Limitations

    Findings based on proposed methods and publicly available data may not transfer to real-world applications.

Abstract

from arXiv · show

This paper provides a comprehensive survey of bias mitigation methods for achieving fairness in Machine Learning (ML) models. We collect a total of 341 publications concerning bias mitigation for ML classifiers. These methods can be distinguished based on their intervention procedure (i.e., pre-processing, in-processing, post-processing) and the technique they apply. We investigate how existing bias mitigation methods are evaluated in the literature. In particular, we consider datasets, metrics and benchmarking. Based on the gathered insights (e.g., What is the most popular fairness metric? How many datasets are used for evaluating bias mitigation methods?), we hope to support practitioners in making informed choices when developing and evaluating new bias mitigation methods.

1 INTRODUCTION

ML systems can reproduce or amplify social bias, creating unfair and potentially unlawful outcomes in high-stakes applications. This survey addresses practitioners’ evaluation challenges by analyzing 341 publications and their mitigation methods, datasets, metrics, and benchmarks.

  • Motivation: ML models can replicate or amplify bias in training data, producing unequal treatment across groups.The paper frames fairness as decisions independent of sensitive attributes such as gender or race.
  • Motivation: Bias in applications such as recruitment, admissions, advertising, and criminal justice can affect rights and trigger anti-discrimination concerns.The paper cites COMPAS as an example where non-white defendants received higher risk scores than white defendants.
  • Mitigation: Practitioners mitigate bias through pre-processing training data, in-processing model training, or post-processing trained models.The survey treats bias mitigation and fairness improvement interchangeably, defining fairness as the absence of bias.
  • Survey scope: 341 publications are analyzed to identify datasets, fairness metrics, and benchmarking practices used to evaluate bias mitigation methods.The survey aims to reduce the experimental-design effort required when developing new methods.
  • Contributions: The survey contributes a comprehensive overview, experimental-design details, future research challenges, and a public repository for replication and manual investigation.The paper organizes its broader discussion across related surveys, methodology, mitigation methods, challenges, recommendations, and conclusions.

2 RELATED SURVEYS

Earlier surveys covered fairness concepts, metrics, datasets, and mitigation methods, but did not systematically evaluate how mitigation methods are tested. This survey combines method coverage with evaluation analysis and recommendations for developers.

  • Prior coverage: Earlier surveys reviewed bias types, fairness definitions and metrics, mitigation methods, and datasets across classification research.Some surveys also catalogued datasets and metrics used by individual bias mitigation methods.
  • Prior coverage: Other surveys focused on specialized perspectives, including sequential decisions, criminal justice, graph representations, ML testing, software engineering, and NLP.These perspectives extend beyond the survey’s central focus on bias mitigation for ML classification.
  • Knowledge gap: Previous surveys did not systematically cover how bias mitigation methods are evaluated, including dataset selection and benchmarking.The gap concerns evaluation practices rather than only the existence or implementation of mitigation methods.
  • Survey contribution: This survey combines extensive mitigation-method coverage with information on datasets and metrics, then analyzes the literature to devise actionable recommendations.The approach responds to limitations identified in closely related surveys by Dunkelau and Leuschel and Pessach and Shmueli.

3 SURVEY METHODOLOGY

The survey systematically searches and filters literature on bias mitigation for tabular classification, then analyzes publication trends, visibility, and scope. It gathers 341 publications through multiple search stages and focuses on algorithmic fairness in ML models.

  • Search methodology: The search combines a preliminary search, repository searches across six established databases, manual screening, and backward snowballing.
  • Search methodology: 100 unique publications were collected from four existing surveys during the preliminary search.
  • The survey targets bias mitigation for classification problems using tabular data and human-bias criteria.
  • Selected publications: 341 publications were gathered across the search procedure, including a second repository search for publications from 2022.
  • Publication trends: Publication activity increased substantially, with 2018 publications more than doubling from 20 in 2017 to 46.
  • Limitations: The survey is limited to algorithmic fairness in ML models, tabular-data mitigation, and methods evaluated mainly on publicly available data whose results may not transfer to real-world applications.

4 ALGORITHMS

The survey organizes bias mitigation algorithms by intervention stage and technique, covering pre-processing, in-processing, and post-processing methods. Its pre-processing review includes data relabelling, perturbation, sampling, augmentation, latent variables, and representation changes.

  • Bias mitigation methods are grouped by intervention stage into pre-processing, in-processing, and post-processing.The 341 publications include 123 pre-processing, 212 in-processing, and 56 post-processing uses, with 70 publications counted across multiple types.
  • Pre-processing: Pre-processing methods change training data, including ground-truth labels through relabelling or remaining features through perturbation.
  • Relabelling and perturbation: Massaging selects instances near the decision boundary and switches equal numbers of positive and negative labels to limit accuracy loss.
  • Relabelling and perturbation: Perturbation adjusts non-protected attributes to bring privileged and unprivileged distributions closer while preserving within-group ranks.
  • Sampling: Sampling methods alter training-data distributions through reweighing, downsampling, oversampling, preferential sampling, or synthetic examples such as SMOTE.
  • Data generation and representation: Other pre-processing approaches generate or augment data, use latent variables or proxy group membership, and learn representations that reduce protected-attribute information while retaining non-sensitive information.
  • Representation: Representation fairness can also be improved by removing discriminating features or adding features through nonlinear transformations.

4.2 In-processing Bias Mitigation Methods

In-processing methods mitigate bias during model training, with 212 publications identified in the survey. The reviewed approaches modify loss functions, use adversaries, train multiple models, or alter learning procedures.

  • Scope: 212 publications applied in-processing bias mitigation methods during model training.The survey refers readers to prior work describing 38 in-processing approaches for various ML tasks.
  • Regularization and Constraints: Regularization penalizes discrimination through an added loss term, whereas constraints enforce specified bias levels during training.Both approaches modify the learning algorithm’s loss function.
  • Adversarial Learning: Adversarial learning jointly trains a classifier and an adversary that exploits fairness issues, creating competition between their objectives.Applications include preventing sensitive-attribute prediction, improving individual fairness, robustness to data poisoning, and reweighting training data.
  • Compositional Approaches: Compositional approaches train separate or ensemble classifiers, allowing group-specific predictions or combined model votes.Separate models may improve group accuracy but reduce the training data available to each classifier; transfer learning can reduce this impact.
  • Compositional Approaches: Multiple models with different fairness goals can form a Pareto front from which practitioners select a fairness-accuracy trade-off.Some approaches formulate bias mitigation as multi-objective optimization over multiple fairness metrics.
  • Adjusted Learning: Adjusted learning methods change existing training procedures or create new algorithms across models including Bayesian models, neural networks, trees, boosting, and Logistic Regression.Examples include active learning, rejection learning, representation learning, multicalibration, and distributionally robust optimization.

4.3 Post-processing Bias Mitigation Methods

Post-processing methods are applied after model training and were the least frequently used category, with 56 publications. They modify test inputs, trained classifiers, or predicted outputs to address fairness.

  • Scope: 56 publications applied post-processing methods, making this the least frequently used mitigation category in the survey.
  • Input Correction: Only two publications applied perturbation-based input corrections to testing data.One used perturbation post-training, while another used the same procedure in both pre-processing and post-processing.
  • Classifier Modification: Post-processing can modify trained classifiers directly, including Naive Bayes, Logistic Regression, Decision Trees, Neural Networks, and Regression Models.Savani et al. referred to this direct model modification as intra-processing.
  • Classifier Modification: Hardt et al.’s method derives a fairer classifier by optimizing fairness loss terms for Equalized Odds and Equality of Opportunity.The method starts from an unfair classifier and has been adapted by subsequent publications.
  • Classifier Modification: Classifier modification can produce poor accuracy when the loss function is not strictly convex.Woodworth et al. proposed an adapted approach alongside training-time constraints.
  • Output Correction: Output correction changes predicted labels through rule-confidence adjustment, thresholding, reject options, or counterfactual correction.These methods target discriminatory rule outputs, biased decisions near boundaries, or protected-attribute effects in causal models.

4.4 Combined Approaches

The survey identified 70 publications combining multiple bias mitigation techniques. In-processing appeared most often in these combinations, and methods were used both across different stages and within the same stage.

  • Overview: 70 publications applied multiple bias mitigation techniques simultaneously.These combinations could involve multiple methods at one stage or methods spanning pre-, in-, and post-processing.
  • Across Categories: 86% of combined-approach publications used in-processing, compared with 54% using pre-processing and 31% using post-processing.
  • Within-Stage Combinations: 26 of 70 publications combined multiple mitigation methods at the same development stage.Seven used multiple pre-processing methods, including sampling and relabeling in five publications; 19 used multiple in-processing methods, 17 involving regularization or constraints.
  • Cross-Stage Combinations: 47 publications combined at least two methods across different development stages, while three addressed pre-, in-, and post-processing together.The survey presents this as evidence that mitigation methods can be used in conjunction.
  • Examples: Calders and Verwer combined latent unbiased labels, sensitive-attribute-specific models, and output probability modification for Naive Bayes classification.
  • Examples: Gupta et al. used a proxy group variable and training constraints when group labels were missing, then adapted an existing classifier.
  • Examples: Iosifidis et al. combined sampled AdaBoost classifiers trained with balanced group representation and group-dependent prediction thresholds.

4.5 Classification Models

Logistic Regression was the most frequently used classification model for evaluating all three mitigation categories, followed by Neural Networks. Most publications evaluated a method on only one model, especially for in-processing methods.

  • Model Usage: 140 unique publications used Logistic Regression, making it the most frequent model for pre-, in-, and post-processing evaluations.
  • Model Usage: 102 publications used Neural Networks, with most of those evaluations involving in-processing methods.Linear Regression models appeared in 22 publications.
  • Model Usage: Decision Trees appeared in 36 publications, while Random Forests appeared in 45.Decision Tree variants included Hoeffding trees, C4.5, J48, and Bayesian random forests.
  • Evaluation Breadth: 70% of publications applied their bias mitigation method to only one classification model.Figure 4 summarizes the number of classification models considered during experiments.
  • Evaluation Breadth: 24% of in-processing publications evaluated more than one model, compared with 35% of pre-processing and 43% of post-processing publications.In-processing methods are model-specific, whereas pre-processing and most post-processing methods can be developed independently of the classifier.

5 DATASETS

The survey examines dataset selection, usage frequency, diversity, synthetic-data construction, and data-splitting practices across empirical bias-mitigation studies. Dataset usage is concentrated in a few benchmarks, while most publications evaluate methods on relatively few datasets and provide incomplete split information.

  • Methodological scope: The survey treats subsets from the same source as one dataset and examines split repetition because training instability can affect conclusion validity.This analysis excludes splits used before evaluation for hyperparameter tuning.
  • Dataset inventory: 83 unique datasets were identified, including synthetic and semi-synthetic data used in bias-mitigation experiments.56 publications created synthetic or semi-synthetic datasets; the survey separately reports 54 synthetic-data and 3 semi-synthetic-data publications.
  • Dataset usage: 59 of 81 non-synthetic datasets were used by only one or two publications, indicating that most datasets are rarely reused.48 datasets were used once and 11 twice.
  • Dataset usage: The Adult dataset was used by 77% of empirical publications, while COMPAS, German Communities and Crime, Bank, and Law School each appeared in at least 10%.The survey suggests considering at least Adult and COMPAS for comparison, while warning that both have received criticism as generic benchmarks.
  • Dataset diversity: Two datasets were the most common experimental choice, and 90% of publications used four or fewer datasets.Two datasets were used in 104 of 324 publications; the survey recommends evaluating methods on three datasets or more when possible.
  • Data-split practices: Among 324 empirical publications, 232 reported data-split information and 143 reported the number of runs.Common non-k-fold splits were 80%-20% and 70%-30%, while repeated experiments most often used 10 or 5 runs.

6 FAIRNESS METRICS

The survey organizes fairness metrics by what information they use, spanning dataset labels, predictions, actual outcomes, probabilities, similarity, and causal reasoning. Across 324 empirical publications, metric usage is broad but typically limited in number, with Equality of Opportunity the most frequently reported metric among predicted-and-actual-outcome measures.

  • Metric taxonomy: The survey categorizes fairness metrics into six groups based on labels, predicted outcomes, predicted and actual outcomes, probabilities and actual outcomes, similarity, and causal reasoning.The categories structure the analysis of metrics used across the surveyed publications.
  • Metric usage: 109 unique fairness metrics were identified across the 324 publications that performed experiments.Most studies used binary sensitive attributes and binary predictions, although non-binary attributes and multi-class predictions also appeared.
  • Metric usage: Publications used two fairness metrics on average, while 45% used only one.The survey notes that different metrics emphasize different aspects of classification performance.
  • Dataset-label metrics: Mean Difference was the most popular dataset-label metric, appearing in six publications.elift and slift use ε to compare bias magnitudes, with ε = 0 representing perfect fairness.
  • Predicted-outcome metrics: Parity-based metrics compare positive-label outcomes between groups, using differences such as Statistical Parity Difference or ratios such as Disparate Impact.The survey notes that ratios can be more challenging to interpret than differences.
  • Outcome-based metrics: Equality of Opportunity was the most popular predicted-and-actual-outcome metric, used 90 times, followed by Equalized Odds, used 52 times.Equality of Opportunity requires equal TPR, whereas Equalized Odds requires equal TPR and FPR across population groups.
  • Outcome-based metrics: Other outcome-based evaluations included FPR 38 times, TNR 8 times, and accuracy differences 29 times.Precision, recall, F1, and Area Under Curve were also used to compare group performance.

7 BENCHMARKING

The survey finds substantial variation in how bias mitigation methods are benchmarked, with original-model comparisons most common and source-code availability improving in recent years.

  • Benchmarking coverage: 308 of 324 experiments perform benchmarking, while 15 experiments report results without any benchmark comparison.The survey analyzes benchmarking practices among publications that both conduct experiments and apply benchmarking.
  • Baselines: 254 of 308 experiments compare mitigation methods against a fairness-agnostic Original Model, making it the most frequently used baseline.This comparison is treated as the minimum requirement for determining whether a method improves fairness over an unmitigated model.
  • Baselines: 30 of 308 experiments use suppression and 13 use random baselines; four publications use constant classifiers as fairness-aware baselines.Suppressing only protected attributes may leave unfairness because other features can correlate with them.
  • Comparisons with existing methods: 44% of pre-processing, 36% of in-processing, and 31% of post-processing methods do not benchmark against other mitigation methods, with same-type comparisons otherwise more likely.The percentages correspond to 50/114, 66/184, and 16/52 publications, respectively.
  • Comparisons with existing methods: 137 mitigation methods are benchmarked by at least one other publication; among the 18 most frequently benchmarked methods, eight are pre-processing, nine in-processing, and four post-processing.The listed methods are each used as benchmarks by at least eight other publications.
  • Fairness-unaware benchmarks: 51 publications benchmark fairness-aware methods against fairness-unaware methods, including 23 pre-processing and 27 in-processing publications, but only six post-processing publications.These comparisons include general data-augmentation methods used to benchmark fairness-aware resampling.
  • Source-code availability: Source-code sharing reached 71% of publications in 2020, while the proportion was 46% in 2018 and 49% in 2019.The survey identified 192 publications with available source-code implementations, including resources found through an additional search.
  • Source-code availability: AIF 360 implements 13 surveyed mitigation methods, compared with four in Fairlearn and three in Themis-ML.These are the three frameworks identified as providing implementations of surveyed bias mitigation methods.

8 CHALLENGES AND OPPORTUNITIES

The survey identifies challenges in evaluating and deploying bias mitigation methods, especially metric selection, data limitations, real-world transfer, and underexplored method combinations. It also points to broader experiments, toolkits, and user studies as opportunities.

  • Metrics: 109 fairness metrics make it difficult to consolidate evaluation, while many studies focus on binary classification and only 15 publications address non-binary sensitive attributes.The survey also identifies multi-class problems and representative metrics as open evaluation needs.
  • Metrics: Accuracy–fairness trade-offs and conflicts among multiple fairness metrics complicate method evaluation.The literature also considers allowing explainable discrimination when enforcing one fairness criterion could produce unfairness elsewhere.
  • Guarantees: Fairness guarantees matter in regulated domains because test-set performance alone may not establish effectiveness or bound bias for a specific fairness metric.Guarantees can provide performance assurances with respect to a defined fairness metric.
  • Datasets: Protected attributes are often unavailable in practice, creating a pressing dataset-access and reliability challenge.This concern was mentioned in nine publications.
  • Datasets: Prior experiments mainly consider race and gender, leaving the diversity of protected attributes underexplored.The survey notes that targeted collection of additional subgroup data may itself be biased.
  • Real-world deployment: Existing public-dataset experiments may not transfer directly to real-world applications because data distributions can change over time.Changing distributions may require continuous implementation of bias mitigation methods.
  • Real-world deployment: Binarized protected attributes can simplify population structure and make it harder to identify relevant groups for bias measurement and mitigation.The COMPAS example transforms six demographic categories into “Caucasian” and “not Caucasian.”
  • Research opportunities: 40 publications call for additional metrics, 12 for additional classification models, and only eight for additional datasets in extended experiments.The survey also identifies recommendations to study recommendations, ranking, and clustering beyond classification.

9 CURRENT BEST PRACTICES / RECOMMENDATIONS

The survey recommends broader, more transparent empirical evaluations that report protected attributes, use multiple fairness and performance metrics, and benchmark against original and comparable models.

  • Evaluation design: Apply each bias mitigation method to at least three datasets while accounting for dataset diversity and criticism when making real-world claims.The recommendation is intended to strengthen the basis for claims about real-world impact.
  • Evaluation design: State the protected attributes used for each dataset.This makes the evaluated fairness context explicit.
  • Evaluation design: Evaluate methods with at least two fairness metrics and one performance metric, such as accuracy.Using different metric types is suggested to reduce correlation among individual fairness metrics.
  • Benchmarking: Benchmark at least against the original model and consider similar existing methods.These comparisons align evaluation with established benchmarking practices.

10 CONCLUSION

The survey synthesizes 341 publications on classification bias mitigation, covering method types, techniques, evaluation datasets and metrics, benchmarking practices, and current challenges. Its findings show uneven method and evaluation practices, while highlighting unresolved issues for researchers and practitioners.

  • Bias mitigation methods: 341 publications were categorized into 123 pre-processing, 212 in-processing, and 56 post-processing methods, making in-processing the most common type.The survey defined 13 technique categories; loss-function changes were especially frequent, including 51 regularization and 74 constraint publications.
  • Evaluation practices: 81 datasets were identified, but publications used only 2.7 datasets on average, with Adult appearing in 77% of publications.The survey assessed datasets as one of three evaluation aspects.
  • Evaluation practices: 109 unique fairness metrics were grouped into six categories, with predicted-outcome and predicted-and-actual-outcome definitions used most frequently.The survey reports substantial metric variety across the literature.
  • Benchmarking: 82% of bias mitigation methods used the original model as a baseline, while 56% of publications made source code available to support replicability and benchmarking.The survey also identified comparisons with other mitigation methods, fairness-unaware methods, and three available implementation frameworks.
  • Open challenges: The literature identifies unresolved opportunities around metric synthesis, fairness guarantees, population-group identification, and dataset reliability in practical applications.The survey notes that no consensus exists on which fairness metrics to use and that developers may struggle to identify relevant groups or assess whether prior observations are biased.
  • Implications: The survey aims to help researchers and practitioners understand existing approaches and support development of new bias mitigation methods.Its synthesis is intended to inform method creation and evaluation decisions.
Loading 2207.07068v4…