PaperlayerView exact revision on arXiv

Source-linked AI summary

A Comprehensive Empirical Study of Bias Mitigation Methods for Machine Learning Classifiers

Zhenpeng Chen, Jie M. Zhang, Federica Sarro, Mark Harman

arXiv:2207.03277v3cs.SEcs.AI

TL;DR

Existing evaluations of bias mitigation methods offer limited coverage of metrics and methods, despite the need to choose methods for different software decision scenarios. This study compares 17 methods across eight tasks and multiple performance, fairness, and trade-off measures. Performance often decreases, fairness improvement is inconsistent, and no method is best in every scenario.

  • Problem

    Prior evaluations cover limited metrics and methods, leaving incomplete evidence for choosing bias mitigation methods across application scenarios.

    Source: §1 · p. 2, §2.2 · p. 4

  • Method

    The study evaluates 17 bias mitigation methods across eight benchmark tasks using 11 ML performance metrics, 4 fairness metrics, and 20 trade-off measures.

    Source: §1 · p. 3

  • Results

    42%–66% of scenarios show ML performance decreases, fairness improves in 46%, and both fairness and performance decrease in 25%.

    Source: §7 · p. 27

  • Takeaways & Limitations

    Method effectiveness varies by task, model, protected attribute, and metrics, so practitioners must choose methods for their intended scenarios.

    Source: §1 · p. 3

    The study considers each protected attribute individually because most existing mitigation methods do not support multiple attributes at once.

    Source: §5 · p. 26

Abstract

from arXiv · show

Software bias is an increasingly important operational concern for software engineers. We present a large-scale, comprehensive empirical study of 17 representative bias mitigation methods for Machine Learning (ML) classifiers, evaluated with 11 ML performance metrics (e.g., accuracy), 4 fairness metrics, and 20 types of fairness-performance trade-off assessment, applied to 8 widely-adopted software decision tasks. The empirical coverage is much more comprehensive, covering the largest numbers of bias mitigation methods, evaluation metrics, and fairness-performance trade-off measures compared to previous work on this important software property. We find that (1) the bias mitigation methods significantly decrease ML performance in 53% of the studied scenarios (ranging between 42%~66% according to different ML performance metrics); (2) the bias mitigation methods significantly improve fairness measured by the 4 used metrics in 46% of all the scenarios (ranging between 24%~59% according to different fairness metrics); (3) the bias mitigation methods even lead to decrease in both fairness and ML performance in 25% of the scenarios; (4) the effectiveness of the bias mitigation methods depends on tasks, models, the choice of protected attributes, and the set of metrics used to assess fairness and ML performance; (5) there is no bias mitigation method that can achieve the best trade-off in all the scenarios. The best method that we find outperforms other methods in 30% of the scenarios. Researchers and practitioners need to choose the bias mitigation method best suited to their intended application scenario(s).

1 INTRODUCTION

The introduction argues that existing evaluations cover too few metrics and methods to guide scenario-specific choices, then presents a broad empirical study of bias mitigation. Its findings show performance losses and inconsistent fairness gains, with no method best across all scenarios.

1 INTRODUCTION

  • ML systems are used in consequential decisions, where unfair behavior can harm disadvantaged groups and create legal risks for engineers.The introduction frames fairness as a software quality concern alongside ML performance.Source: §1 · p. 2
  • Prior evaluations often use only one or two ML performance metrics, leaving out measures that may matter in different application situations.For example, different scenarios can require metrics beyond accuracy.Source: §1 · p. 2
  • The study evaluates 17 representative methods using 11 ML performance metrics, 4 fairness metrics, and 20 trade-off measures across 8 benchmark tasks.The authors describe this as broader coverage than previous software-fairness studies.Source: §1 · p. 3
  • After applying bias mitigation methods, all ML performance metrics used decrease in 42%–66% of scenarios; fairness improves in 46%, while both fairness and ML performance decrease in 25%.Source: §1 · p. 3
  • Method effectiveness depends on tasks, models, protected attributes, and evaluation metrics, and no method achieves the best trade-off in every scenario.Researchers and practitioners therefore need to choose methods suited to their intended applications.Source: §1 · p. 3

2 PRELIMINARIES

The preliminaries define the paper’s focus on group fairness and motivate a broader empirical comparison of bias mitigation methods. Existing evaluations leave gaps in metric and method coverage, limiting guidance for scenario-specific choices.

2.1 Definition of Fairness

  • Group fairness concerns treating different groups equally, with protected attributes such as sex or race defining privileged and unprivileged groups.The paper focuses on group fairness because existing mitigation methods primarily address it.Source: §2.1 · p. 3

2.2 Motivation

  • The Adult dataset example illustrates a sex-related disparity: 31% of men and 11% of women are labeled high income, and the trained model tends to favor men.Simply omitting protected attributes may leave similar unfairness because other training attributes can correlate with them.Source: §2.2 · pp. 3–4
  • Existing evaluations use limited metrics and methods, while engineers need evidence to choose a mitigation method for application-specific requirements.The paper motivates a study spanning methods from both ML and SE communities to support those choices.Source: §2.2 · pp. 4–5
  • The authors state that current mitigation approaches lacked scientific measurement of their effect size, and position their study as a baseline for future improvement.Source: §2.2 · p. 5

3 EXPERIMENTAL SETUP

This section introduces the paper’s research questions and experimental design.

3 EXPERIMENTAL SETUP

  • The section sets out the research questions and experimental design.Source: §3 · p. 5

3.1 Overview of Experimental Design

The experimental design applies mitigation methods to models trained on benchmark tasks and evaluates performance, fairness, and their trade-off. Three research questions organize the assessment around these outcome dimensions.

3.1 Overview of Experimental Design

  • Five benchmark datasets covering eight software decision tasks are used to train models, then 17 bias mitigation methods from ML and SE are applied.Source: §3.1 · p. 5
  • The design evaluates methods with 11 ML performance metrics and 4 fairness metrics, addressing how mitigation changes each outcome.Source: §3.1 · p. 5
  • Fairness and ML performance are assessed together using 20 fairness-performance metric pairs to examine trade-offs between the outcomes.Source: §3.1 · pp. 5–6

3.2 Bias Mitigation Methods

The study evaluates 17 bias mitigation methods from the ML and SE communities, covering pre-processing, in-processing, and post-processing approaches. It also includes Fairway and Fair-SMOTE, two methods developed in the SE community.

Pre-processing

  • Pre-processing methods transform features, labels, representations, or sample weights to address bias.The methods include OP, LFR, RW, and DIR.Source: §3.2 · p. 6

In-processing

  • In-processing methods incorporate fairness into model training through regularization, adversarial techniques, or a fairness-metric input.The methods include PR, AD, and MFC.Source: §3.2 · p. 6

Post-processing

  • Post-processing methods alter predictions, including uncertain predictions or classifier outputs, to target fairness objectives.The methods include ROC, CEO, and EOP.Source: §3.2 · p. 6

SE methods

  • Fairway combines removal of ambiguous training points with multi-objective optimization for model performance and fairness.Fair-SMOTE balances subgroup training counts, then removes ambiguous points using Fairway's approach.Source: §3.2 · p. 6
  • The study evaluates 17 methods, spanning ML methods implemented in AIF360 and two methods from the SE community.The 17 include alternative metric-guided settings for MFC, ROC, and CEO.Source: §3.2 · p. 6

3.3 Benchmark Datasets

The study uses five benchmark datasets across social, financial, and medical domains. It defines eight binary-classification tasks by pairing datasets with provider-specified protected attributes.

  • Five benchmark datasets cover social, financial, and medical domains, and are widely used in fairness research.The datasets are implemented in IBM AIF360.Source: §3.3 · p. 7
  • The benchmarks address income prediction, recidivism, credit risk, bank-term-deposit subscription, and health-care utilization.These are the prediction tasks represented by Adult, Compas, German Credit, Bank Marketing, and Medical Survey 2015 datasets, respectively.Source: §3.3 · p. 7
  • The study uses provider-specified protected attributes, one attribute per task, yielding eight dataset-attribute pairs treated as binary-classification tasks.Identifying all potentially protected attributes is beyond the paper's scope and may require requirements engineers, legal practitioners, compliance officers, and policy makers.Source: §3.3 · p. 7

3.4 Evaluation Metrics and Measures

The evaluation combines ML performance and group-fairness metrics with Fairea-based trade-off assessment. It uses broad metric coverage while noting that metric choice depends on the application.

3.4 Evaluation Metrics and Measures

  • The study evaluates 11 ML performance metrics, 4 fairness metrics, and 20 fairness-performance trade-off combinations.The trade-off combinations use five overall-performance metrics paired with four fairness metrics.Source: §3.4 · p. 7, §3.4.3 · p. 11

3.4.1 ML Performance Metrics.

  • Performance measures include class-specific precision, recall, and F1, alongside accuracy, macro-averages, and MCC.The full metric set is used across datasets for comprehensive, unified comparisons, though application requirements may call for only selected metrics.Source: §3.4.1 · pp. 8–9

3.4.2 Fairness Metrics.

  • The four group-fairness metrics are SPD, AOD, EOD, and ERD; absolute values are used, with zero indicating greatest fairness.The study selects SPD rather than DI, while noting that SPD may not suit scenarios where different group acceptance rates are allowed.Source: §3.4.2 · p. 9

3.4.3 Fairness-performance Trade-off Measures.

  • Fairea benchmarks methods against a baseline connecting the original model to prediction-mutated models, then classifies and quantifies trade-off effectiveness.The mutation replaces selected predictions with the data's majority class at degrees from 10% to 100%; a larger area indicates a better trade-off. The study assesses 20 pairs because class-specific metrics may not yield a baseline.Source: §3.4.3 · pp. 10–11, Figure 2 · p. 10

3.5 Experimental Settings

The experiments use benchmark datasets, seven model architectures, repeated train-test splits, and statistical tests to compare bias mitigation methods. The reported figure indicates significant decreases in ML performance metrics in an average of 53% of applications.

  • The experiments train seven original models: LR, SVM, RF, and four fully connected neural networks.The neural networks use five hidden layers, with architectures drawn from prior fairness research.Source: §3.5 · p. 11
  • Each of 16 methods is applied across all eight tasks, while OP is applied to six; each application is repeated 50 times.Runs use shuffled data split into 70% training and 30% test sets.Source: §3.5 · p. 11
  • 53% of applications show significant decreases across the 11 ML performance metrics on average, with metric-specific rates from 42% to 66%.The figure summarizes the effects of bias mitigation methods on ML performance metrics.Source: Figure 3 · p. 12
  • The study uses the Mann–Whitney U-test at a default 95% confidence level, Cohen's d for effect size, and Spearman correlation for metric-change similarity.Statistical significance uses p < 0.05; effect sizes are categorized as small, medium, or large.Source: §3.5 · p. 12

4 RESULTS

Across the studied applications, bias mitigation often reduced ML performance but improved fairness less consistently. Method rankings and metric changes also varied across the metrics assessed.

4.1.1 RQ1.1: Effects on ML Performance Metric Values.

  • 53% of applications showed significant ML-performance decreases on average, while fairness improved significantly in 46% on average.Performance decreases ranged from 42% to 66% across metrics; fairness improvements ranged from 24% to 59%.Source: §4.1.1 · p. 13, §4.2.1 · p. 16
  • Changes in newly considered ML-performance metrics do not consistently correlate with previously used metrics, so the latter cannot serve as reliable proxies.Of 55 metric pairs, 41 had significant positive correlations, 11 negative correlations, and 3 no significant correlation; 52 pairs lacked a consistent pattern across task-model pairs.Source: §4.1.1 · p. 14, Table 2 · p. 14

4.1.2 RQ1.2: ML Performance Comparison among Methods.

  • RW best retained ML performance, with significant decreases in 27% of scenarios, while LFR performed worst, with large-effect decreases in 89%.The Fig. 4 comparison covers scenarios combining task-model pairs and ML-performance metrics.Source: §4.1.2 · pp. 14–15, Figure 4 · p. 15

4.2.1 RQ2.1: Effects on Fairness Metric Values.

  • Fairness improvements varied by metric: AOD and EOD were positively correlated across all task-model pairs, while ERD changes lacked consistent correlation with other metrics.AOD and EOD had a correlation coefficient of 0.906; ERD was negatively correlated with other fairness metrics overall.Source: §4.2.1 · p. 17, Table 4 · p. 18

4.2.2 RQ2.2: Fairness Comparison among Methods.

  • LFR significantly improved fairness in 71% of scenarios, whereas CEO-W did so in 23%; only 7 of 17 methods exceeded 50%.Fig. 6 compares methods across task-model-pair and fairness-metric scenarios.Source: Figure 6 · p. 18, §4.2.2 · pp. 18–19
  • Fairness rankings differed by metric: LFR ranked first for SPD but 15th for ERD, and methods targeting specific metrics tended to have poor overall fairness.These differences point to trade-offs among fairness metrics as well as between fairness and performance.Source: §4.2.2 · p. 19, Table 5 · p. 19

4.3 RQ3: Fairness-performance Trade-off

Across Fairea’s effectiveness-region analysis and quantitative assessment, mitigation outcomes vary by method and scenario, and no method leads in every setting. Fairea’s trade-off baseline cannot be applied to ERD, which is excluded from the region and quantitative analyses.

4.3 RQ3: Fairness-performance Trade-off

  • Fairea’s baseline is constructed separately for each task-model and metric-pair combination, yielding 1,120 baselines before ERD is excluded.The study uses 840 baselines for the remaining 15 fairness-performance trade-offs because Fairea cannot construct an ERD baseline.Source: §4.3 · pp. 20–21
  • ERD does not meet Fairea’s baseline hypothesis: increased model mutation can worsen both ML performance and ERD fairness.With all samples assigned the same label, SPD, AOD, and EOD reach their minima, whereas ERD does not.Source: §4.3 · pp. 20–21

4.3.1 RQ3.1: Effectiveness Region Distribution.

  • 42% of cases achieve win-win or good trade-offs for AOD and Acc, compared with 59% for SPD and Mac-R.Region distributions also vary across fairness-performance metric pairs.Source: Figure 8 · p. 22, §4.3.1 · p. 23
  • 25% of mitigation cases fall into a lose-lose region, while RW achieves a win-win or good trade-off in 77% and CEO-W in 13%.Across methods, RW has a win-win result in 27% and a good trade-off in 50% of cases; 12% of RW cases are lose-lose.Source: §4.3.1 · pp. 21, 24, Figure 8 · p. 22
  • Mitigation effectiveness depends on models, tasks, protected attributes, and metric pairs; imbalanced datasets have worse trade-offs than balanced datasets.Good-trade-off proportions differ between LR and RF (39% versus 27%); poor-trade-off proportions are 15% for Adult-Sex and 8% for Adult-Race.Source: §4.3.1 · pp. 23–24

4.3.2 RQ3.2: Quantitative Assessment of Trade-off.

  • Fairea’s quantitative assessment compares methods using the area for good-trade-off cases and ranks win-win above good, poor, inverted, and lose-lose.For each scenario, the study averages ML performance and fairness across 50 runs before quantifying trade-off effectiveness against the baselines.Source: §4.3.2 · p. 24
  • Fairway combines pre-processing and in-processing methods; the authors suggest combining strategies across training phases or adapting them to scenarios.Fairway’s overall quantitative result is best in 30% of scenarios, not all scenarios.Source: §4.3.2 · p. 25
  • Fairway achieves the best quantitative trade-off in 30% of scenarios, and no method achieves the best trade-off in all studied scenarios.The task-level results vary: OP is best in 77% of Compas-Sex scenarios but 0% in Adult-Sex, Adult-Race, German-Sex, and German-Age.Source: §4.3.2 · pp. 24–25, Table 6 · p. 24

5 THREATS TO VALIDITY

The study addresses validity threats through broad metric, method, model, and task coverage, along with established frameworks and statistical tests. Its results remain bounded by library-based experiments, individually considered protected attributes, and the study’s selected methods, metrics, measures, and datasets.

5 THREATS TO VALIDITY

  • Construct validity depends on the selected indicators: 11 ML performance metrics cover class-level and overall performance, while four fairness metrics are widely adopted.The authors describe metric definitions and indicator choice as the primary construct-validity threat.Source: §5 · p. 25
  • The study uses IBM AIF360 and released code for Fairway, Fair-SMOTE, and Fairea, and applies Mann–Whitney U and Spearman rank tests.Each mitigation method is run 50 times on different tasks; scripts and data are made publicly available for reproduction and replication.Source: §5 · pp. 25–26
  • Fairea’s hypothesis limits its use to metrics that fit its assumed fairness-performance trade-off, so ERD is excluded from Section 4.3.The study uses 11 ML performance metrics, four fairness metrics, and 20 trade-off types overall.Source: §5 · p. 25
  • The authors identify a scope boundary: future replications could include more methods, metrics, measures, and datasets.They note that these are increasing as attention to software fairness grows.Source: §5 · p. 26
  • The study evaluates eight tasks spanning social, financial, and medical domains, but experiments on libraries rather than real deployed software.The authors report that public real-software code and data are unavailable because privacy and legal concerns often restrict access.Source: §5 · pp. 25–26
  • Protected attributes are considered individually because most existing mitigation methods do not support handling multiple attributes simultaneously.The authors identify this choice as a possible threat to validity and suggest future evaluation when suitable methods become available.Source: §5 · p. 26

6 RELATED WORK

Related work covers fairness testing, fairness-improvement techniques, and empirical comparisons of mitigation methods. The authors position their study as a broader evaluation addressing limitations in prior evaluations’ measurements and method coverage.

6 RELATED WORK

  • Fairness-testing work includes test-suite generation for causal discrimination, robustness-guided testing, symbolic execution, gradient-based generation, and surveys.The cited examples are Themis, Aequitas, SG, ADF, and a survey by Chen et al.Source: §6 · p. 26
  • Fairness-improvement approaches include changing feature sets, removing ambiguous data, balancing training data, and toolkits integrating mitigation methods.Examples include work by Zhang and Harman, Chakraborty et al., and IBM AIF360.Source: §6 · pp. 26–27
  • Prior empirical studies compare mitigation methods or quantify fairness-performance trade-offs, but use limited measurements and methods.This paper presents a comprehensive evaluation of existing bias mitigation methods.Source: §6 · p. 27

7 CONCLUSIONS AND FUTURE WORK

Across the studied scenarios, bias mitigation often reduced ML performance, improved fairness in fewer than half, and sometimes decreased both. No method performed best in every scenario, motivating scenario-specific method selection.

7 CONCLUSIONS AND FUTURE WORK

  • 42%–66% of scenarios showed significant decreases in ML performance, depending on the metric, while significant fairness improvement occurred in 24%–59%.Across the four fairness metrics, significant improvement occurred in 46% of scenarios.Source: §7 · p. 27
  • 25% of scenarios saw decreases in both fairness and ML performance.Source: §7 · p. 27
  • Effectiveness depended on task, model, protected-attribute choice, and the metrics used to assess fairness and ML performance.Source: §7 · p. 27
  • No method achieved the best trade-off in every studied scenario; the best method outperformed others in 30% of scenarios.Source: §7 · p. 27
  • Future work plans to develop methods addressing the identified weaknesses, build a scenario-specific selection framework, and replicate the study as methods, metrics, and datasets expand.Source: §7 · p. 27

Related papers

Loading 2207.03277v3…