Source-linked AI summary
Fairway: A Way to Build Fair ML Software
Joymallya Chakraborty, Suvodeep Majumder, Zhe Yu, Tim Menzies
TL;DR
Machine-learning software can make high-stakes decisions, and bias in these systems makes fairness a software-engineering concern. Fairway identifies biased training data and combines preprocessing with in-processing optimization; the authors report improved fairness with little damage to predictive performance.
Problem
Machine-learning models used in high-stakes software can behave in biased ways, making fairness a concern for software engineering.
Source: §1 · p. 1Method
Fairway identifies biased training points and combines preprocessing with multiobjective in-processing optimization to improve fairness while maintaining model performance.
Source: §1 · p. 2, §6.2 · p. 5Results
Fairway is statistically significantly better in 2/5 AOD cases and 3/5 EOD cases, wins on false alarm in all cases, and keeps the same recall in 3/5 cases.
Source: §7 · p. 9Takeaways & Limitations
The authors argue that bias testing and mitigation should be primary concerns in software-engineering tasks that use AI.
Source: §1 · p. 1The evaluation uses EOD and AOD, which consider TPR and FPR but neglect class distribution.
Source: §8 · p. 10
Abstract
from arXiv · showhide
Machine learning software is increasingly being used to make decisions that affect people's lives. But sometimes, the core part of this software (the learned model), behaves in a biased manner that gives undue advantages to a specific group of people (where those groups are determined by sex, race, etc.). This "algorithmic discrimination" in the AI software systems has become a matter of serious concern in the machine learning and software engineering community. There have been works done to find "algorithmic bias" or "ethical bias" in the software system. Once the bias is detected in the AI software system, the mitigation of bias is extremely important. In this work, we a)explain how ground-truth bias in training data affects machine learning model fairness and how to find that bias in AI software,b)propose a methodFairwaywhich combines pre-processing and in-processing approach to remove ethical bias from training data and trained model. Our results show that we can find bias and mitigate bias in a learned model, without much damaging the predictive performance of that model. We propose that (1) test-ing for bias and (2) bias mitigation should be a routine part of the machine learning software development life cycle. Fairway offers much support for these two purposes.
1 INTRODUCTION
The paper argues that bias can enter AI/ML software through training data and other development stages, making fairness a software-engineering concern. Fairway combines training-data bias detection with pre-processing and in-processing mitigation, aiming to improve fairness without much damaging predictive performance.
1 INTRODUCTION
- Because software supports high-stakes decisions, the authors frame fairness as a non-functional requirement and bias testing and mitigation as software-engineering concerns.Examples include hospital release, loan approval, and criminal sentencing.Source: §1 · p. 1
- Prior software-engineering bias detection work did not explain bias origins or remove bias, while existing mitigation methods trusted original training labels.The authors argue that human or algorithmic decisions can make those labels biased.Source: §1 · p. 2
- Fairway evaluates original training labels to identify biased data points, then uses multiobjective optimization to keep model performance similar while improving fairness.The approach draws on prior work on training-data bias and the fairness–performance trade-off.Source: §1 · p. 2
- Fairway combines training-data bias detection with pre-processing and in-processing mitigation, and the authors report fairness without much damage to predictive performance.They also report that the combined method performs better than either individual approach.Source: §1 · p. 2
- The paper discusses shortcomings of fairness metrics and argues that fairness definitions should vary across applications.The authors also identify a publicly available Fairway replication package as a systems contribution that supports confirmation or further improvement of results.Source: §1 · p. 2
2 BACKGROUND
The background reviews examples of unfair software outcomes and prior software-engineering work on fairness. It distinguishes bias testing from mitigation and outlines the main mitigation families, explaining Fairway’s choice to combine two of them.
2.1 About Software Fairness
- Examples of software bias include disparate facial-recognition error rates and recidivism assessments that more often falsely label Black defendants as future criminals.The section defines fair software as software that gives no undue advantage to a group or individual and describes Fairway for binary classification.Source: §2.1 · p. 2
2.2 Previous Work
- Prior work spans fairness principles, toolkits, and research groups, while software-engineering research has focused on testing models for bias and making predictions fairer.The section names IEEE, the European Union, and Microsoft among organizations publishing AI ethical principles.Source: §2.2 · pp. 2–3
2.3 Finding Ethical Bias
- THEMIS, AEQUITAS, and related methods generate discriminatory test inputs for trained models, whereas Fairway also seeks bias in training data using situation testing.Situation testing checks whether a prediction changes when the protected attribute changes for the same data point.Source: §2.3 · p. 3
2.4 Removing Ethical Bias
- Mitigation methods are grouped as pre-processing, in-processing, and post-processing; Fairway combines the first two and omits post-processing because correcting labels requires application-specific domain knowledge.The section describes post-processing as changing predicted class labels to reduce discrimination.Source: §2.4 · p. 3
3 FAIRNESS TERMINOLOGY
The paper limits its discussion to binary classifiers and tabular data, then defines labels and fairness in terms of outcomes and protected groups. It distinguishes fairness between groups from fairness between similar individuals.
3 FAIRNESS TERMINOLOGY
- The paper’s scope is binary classification with tabular data containing attribute columns and a class-label column.A favorable label denotes an outcome that benefits its recipient, such as hiring or receiving a loan.Source: §3 · p. 3
- A protected attribute divides a population into privileged and unprivileged groups; group fairness seeks similar treatment across those groups, while individual fairness seeks similar outcomes for similar individuals.Protected attributes such as sex or race are specific to the application.Source: §3 · p. 3
4 FAIRNESS MEASURES
The section defines EOD and AOD as group differences in classification rates and explains how to interpret their values. It also notes that common fairness metrics omit class distribution, sampling, and misclassification costs, and describes steps taken to address some of those shortcomings.
4 FAIRNESS MEASURES
- EOD is the difference in true positive rates between unprivileged and privileged groups, while AOD averages their false- and true-positive-rate differences.The study computes both metrics from classifier input and output datasets and considers their absolute values.Source: §4 · pp. 3–4
- A metric value of 0 indicates equal benefit for both groups; a negative value indicates greater privileged-group benefit, and a positive value indicates greater unprivileged-group benefit.The study uses absolute values of EOD and AOD.Source: §4 · p. 4
- The described fairness metrics can ignore group class distributions, data sampling, and the differing costs of misclassification.Figure 1 illustrates uneven positive and negative class distributions across privileged and unprivileged groups in the Adult dataset.Source: §4 · p. 4, Figure 1 · p. 4
- The study uses both EOD and AOD, situation testing, and ten random data samples, but leaves misclassification costs unresolved because they require application-specific domain knowledge.Situation testing checks whether a prediction changes when the protected attribute changes for the same data point.Source: §4 · p. 4
5 DATASET DESCRIPTION
The study evaluates Fairway on five UCI binary-classification datasets spanning income, criminal-risk, credit, and heart-health tasks. It uses logistic regression, while noting that the approach applies to any classification model.
5 DATASET DESCRIPTION
- The experiments use five popular fairness-domain datasets from the UCI Machine Learning Repository, all framed as binary-classification tasks.The dataset descriptions cover Adult Census Income, COMPAS, German Credit, Default Credit, and Heart Health.Source: §5 · pp. 4–5
- The prediction tasks include income above $50K, COMPAS risk-score data, good-versus-bad credit, default payment, and heart-condition prediction.The passages give dataset sizes of 48,842 for Adult, 7,214 for COMPAS, 1,000 for German Credit, 30,000 for Default Credit, and 297 adults for Heart Health.Source: §5 · p. 4
- The datasets have differing protected-attribute counts: Adult and COMPAS have two each, while German Credit and Default Credit have one each.The passages do not specify a protected-attribute count for Heart Health.Source: §5 · p. 4
- The experiments use logistic regression, and the authors state that Fairway applies to any classification model.The passages describe the datasets as binary classification tasks.Source: §5 · p. 5
6 THE “FAIRWAY” METHOD
Fairway combines training-data bias removal with multiobjective model optimization. The method tests whether protected-attribute-group models disagree on training points, then optimizes model parameters for predictive and fairness metrics.
6.1 Why not Remove the Protected Attributes?
- Removing protected-attribute columns barely changed bias metrics; for these datasets, the authors instead identify improper historical labels as the main bias source.They report that checking attribute correlations did not explain the bias in their datasets.Source: §6.1 · p. 5
6.2 Removal of Ambiguous (Biased) Data Points
- The method trains separate models for protected-attribute groups and removes points where their predictions contradict, retaining points without such disagreement.The authors call these disagreement cases ambiguous data points; the models are used to find them, not for prediction.Source: §6.2 · pp. 5–6
- At most 15% of training data was removed across the five datasets; if removal damages prediction, the authors suggest relabeling, which they did not test.They recommend relabeling only when prediction is affected, noting its extra distance-computation cost.Source: §6.2 · p. 6
6.3 What if there are two protected attributes?
- For datasets with two protected attributes, the method divides data into four groups, trains four models, and checks whether their predictions match.The authors report losing no more than 16% of training data with this approach; they did not explore more than two protected attributes because their datasets would split into unmanageably small regions.Source: §6.3 · pp. 6–7
6.4 Model Optimization
- FAIR_FLASH tunes logistic-regression parameters on validation data for higher recall and lower false alarm, AOD, and EOD.The workflow uses training, validation, and test splits; FLASH models each objective with a separate CART model and uses surrogate evaluations to rank parameter guesses.Source: §6.4 · p. 7
6 THE “FAIRWAY” METHOD
- Fairway combines preprocessing that removes biased training points with in-processing optimization to make the trained model fairer.The authors describe this combination as the method’s two parts.Source: §6 · p. 5, §6.4 · p. 7
7 RESULTS
Fairway combines training-data pre-processing with learner optimization, improving fairness across the reported evaluations while generally limiting predictive-performance damage. Its evaluation also uses situation testing to identify protected-attribute-sensitive predictions and biased training labels.
RQ1: Standard learners
- The default logistic-regression learner had high AOD and EOD across all five datasets, indicating unfair predictions and providing the experiment’s baseline.Recall and false alarm measured predictive performance; higher recall and lower values for the other metrics were preferred.Source: §7 · pp. 7–8, Figure 4 · p. 8
RQ2: Pre-processing
- Pre-processing significantly improved fairness metrics, with only minor recall damage in some cases, after removing ambiguous training points.That cleaning step removed at most 15% of the training data.Source: §7 · p. 8
RQ3: Optimization
- Optimization alone produced similar or better recall and false alarm than pre-processing, while pre-processing did better on AOD and EOD.FAIR_FLASH tuned for higher recall and lower false alarm, AOD, and EOD.Source: §7 · p. 8
RQ5: Comparison with prior algorithms
- Across three datasets, Fairway matched or outperformed prior algorithms on AOD/EOD in most cases and won on false alarm in all cases.The comparison covers Adult, Compas, and German; Table 4 marks improvement, damage, or no change.Source: Table 4 · p. 9, §7 · p. 9
RQ4: Fairway’s combined approach
- Fairway performed better than pre-processing and optimization in most cases; it was best on all four metrics for Default Credit/sex and nearly matched optimization's recall for Adult/race.For Adult/race, it also did better on the other three metrics.Source: Figure 4 · p. 8, §7 · p. 9
RQ5: Comparison with prior algorithms
- Fairway kept recall unchanged in 3/5 comparison cases and reduced it by 10%-12% when it lost; prior algorithms also slightly damaged recall.Fairway won on false alarm in all cases, while its AOD and EOD results were not uniformly better.Source: §7 · p. 9
RQ6: Detection and mitigation
- Situation testing switches protected-attribute values to check whether predictions change; after pre-processing, the reported failure percentages were far lower in every case.Fairway identifies training points with unfair labels and removes them before testing predictions again.Source: §7 · p. 10
- Fairway reports the percentage of predictions sensitive to the protected attribute, letting users set an application-specific tolerance; it is model-agnostic and operates on model outputs.The authors describe it as usable with black-box models when the underlying algorithm cannot be revealed.Source: §7 · p. 10
8 THREATS TO VALIDITY
The study’s validity is bounded by its datasets, fairness metrics, protected-attribute settings, and classification focus. The authors identify broader datasets, metrics, attributes, and task types for future exploration.
Sampling Bias
- Fairway was evaluated on five UCI datasets, but whether they represent a broad range of fairness issues remains open.The authors plan to explore more datasets.Source: §8 · p. 10
Evaluation Bias
- Using EOD and AOD leaves open whether other fairness metrics, including Conditional Equality of Opportunity, would change the evaluation.The authors note that some metrics neglect class distribution and plan to explore more performance criteria.Source: §8 · p. 10
Construct Validity
- The study addresses up to two protected attributes, while evaluation on larger datasets with more protected attributes remains future work.The authors report that their method is feasible for the datasets’ one or two protected attributes.Source: §8 · p. 10
External Validity
- Fairway is limited to classification models; extending it to regression and other data-mining problems remains future work.The authors specifically mention text mining and video processing as other possible problem types.Source: §8 · p. 10
9 CONCLUSION
The conclusion presents Fairway as a tool for addressing bias from improper training labels while optimizing a trained model for fairness and performance. It also describes the tool’s detection and mitigation role and identifies broader data availability as a future concern.
9 CONCLUSION
- Fairway addresses bias from improper training labels by removing ethical bias from training data and optimizing a trained model for fairness and performance.The authors describe it as a combined tool for detecting and mitigating bias.Source: §9 · p. 10
- Fairway validates ground-truth labels, finds and mitigates bias, and is presented as the first SE-domain work focused on ethical-bias mitigation in software.The authors report that its source code is publicly available to software researchers and practitioners.Source: §9 · p. 10
- The authors call for more software researchers to work on this area and for industries to publish datasets, after which the study could be rerun.Source: §9 · p. 11