Source-linked AI summary
Bias in Machine Learning Software: Why? How? What to do?
Joymallya Chakraborty, Suvodeep Majumder, Tim Menzies
TL;DR
Biased machine-learning decisions may reflect prior choices about collected data and assigned labels, while existing mitigation approaches can compromise performance. The paper introduces Fair-SMOTE to remove biased labels and rebalance distributions across sensitive attributes and classes. It reports comparable bias mitigation, higher recall and F1, and a 220% speed advantage over Chakraborty et al.’s method.
Problem
Machine-learning software can make biased decisions affecting protected groups, while prior bias-mitigation work often changes data or learners without grounding the intervention in postulated root causes.
Method
Fair-SMOTE uses situation testing to remove biased labels and oversampling to equalize distributions across sensitive attributes and positive and negative classes.
Results
Fair-SMOTE was as effective at bias mitigation as two state-of-the-art algorithms, achieved higher recall and F1, and ran 220% faster than Chakraborty et al.’s method.
Takeaways & Limitations
Fair-SMOTE combines finding, explaining, and removing bias while improving fairness without compromising predictive performance in the reported study.
Takeaways & Limitations
The evaluation used 10 real-world datasets, three classification models, and four fairness metrics, so conclusions may change with other datasets, models, or evaluation criteria.
Abstract
from arXiv · showhide
Increasingly, software is making autonomous decisions in case of criminal sentencing, approving credit cards, hiring employees, and so on. Some of these decisions show bias and adversely affect certain social groups (e.g. those defined by sex, race, age, marital status). Many prior works on bias mitigation take the following form: change the data or learners in multiple ways, then see if any of that improves fairness. Perhaps a better approach is to postulate root causes of bias and then applying some resolution strategy. This paper postulates that the root causes of bias are the prior decisions that affect- (a) what data was selected and (b) the labels assigned to those examples. Our Fair-SMOTE algorithm removes biased labels; and rebalances internal distributions such that based on sensitive attribute, examples are equal in both positive and negative classes. On testing, it was seen that this method was just as effective at reducing bias as prior approaches. Further, models generated via Fair-SMOTE achieve higher performance (measured in terms of recall and F1) than other state-of-the-art fairness improvement algorithms. To the best of our knowledge, measured in terms of number of analyzed learners and datasets, this study is one of the largest studies on bias mitigation yet presented in the literature.
1 INTRODUCTION
Machine-learning software can make biased decisions affecting protected social groups, motivating approaches that address bias at its data and labeling roots. Fair-SMOTE removes biased labels, rebalances sensitive-attribute and class distributions, and reports competitive fairness and performance results.
- Biased machine-learning decisions have affected groups defined by attributes such as sex, race, age, and marital status.
- The paper attributes bias to prior decisions affecting which data was collected and how labels were assigned.
- Fair-SMOTE removes biased labels and balances distributions across sensitive attributes and positive and negative classes.
- 220% faster than Chakraborty et al.’s method, Fair-SMOTE also produces models with higher recall and F1 than compared fairness-mitigation algorithms.
- The study combines finding, explaining, and removing bias, while reporting fairness improvements with better recall and F1.
- Fair-SMOTE mutates training data while leaving test data in its original form and preserves associations by extrapolating neighboring examples consistently.
2 RELATED WORK
Prior fairness research includes bias detection, mitigation, optimization, and data transformation, but often separates these tasks or reports degraded learner performance. This paper positions Fair-SMOTE as a root-cause-oriented approach that combines bias identification and mitigation.
- Fairness research includes toolkits, workshops, and industry systems for evaluating or mitigating bias in machine-learning models.
- Chakraborty et al. framed bias mitigation as stochastic multi-goal hyperparameter optimization, but did not guide its search with domain knowledge.
- Several mitigation methods improve fairness while degrading learner performance, motivating the paper’s challenge to the fairness–performance trade-off.
- Related work studies biased labels, underrepresentation, feature-level causes, and bias testing, whereas this paper emphasizes data distributions and labels and proposes a fix.
- The study’s experimental context comprises 10 binary-classification datasets and performance and fairness metrics defined in the accompanying tables.
- Optimized Pre-processing treats bias mitigation as convex optimization that probabilistically transforms data to preserve performance and achieve fairness.
3 WHAT ARE THE ROOT CAUSES OF BIAS?
The paper identifies data imbalance and improper labeling as root causes of bias in binary classification datasets. Traditional class balancing can improve performance while worsening fairness, motivating Fair-SMOTE’s joint treatment of labels, class distributions, and protected attributes.
- The study analyzes 10 datasets, all framed as binary classification problems with two target-class values.
- A protected attribute divides the population into privileged and unprivileged groups that differ in access to benefits.
- Historical decisions can introduce bias through both selection of data and assignment of labels.
- 3.1 Data Imbalance: Most datasets exhibit class imbalance alongside imbalance based on protected attributes.
- 3.1 Data Imbalance: On Adult, 24% of examples have high income and 76% have low income; among high-income examples, 86% are male and 14% female.Among low-income examples, 60% are male and 40% female.
- 3.1 Data Imbalance: Traditional balancing methods usually improve model performance but damage fairness by ignoring protected-attribute distributions.SMOTE balances majority and minority classes but can further damage protected-attribute balance.
- 3.2 Improper Data Labeling: Fair-SMOTE balances positive and negative examples across privileged and unprivileged groups while removing biased labels identified through situation testing.Its pipeline first oversamples to balance training data, then detects and removes biased points.
4 METHODOLOGY
Fair-SMOTE addresses training-data bias by correcting both subgroup imbalance and biased labels before model training. It generates nearby synthetic examples, removes biased points through situation testing, and evaluates models on unchanged test data.
- Data imbalance and improper labeling are proposed as the two main reasons for model bias.
- Fair-SMOTE partitions training data by class and protected attribute, then expands smaller subgroups until all match the largest subgroup.For binary class and protected attributes, this produces four subgroups.
- Fair-SMOTE extrapolates variables by the same amount between neighboring examples to preserve average associations among variables.
- Synthetic examples are generated from a parent point and its two nearest neighbors, with separate handling for boolean, symbolic, and numeric columns.The process repeats until subgroup sizes become similar.
- The pipeline balances training data, uses fair situation testing to identify and remove biased points, trains on the remainder, and predicts on original test data.The test data remains unchanged, while only a small percentage of training data is reported as removed.
- Experiments use 10 datasets, three classifiers, repeated 5-fold cross-validation, and Scott-Knott ranking for distribution comparisons.The classifiers are logistic regression, random forest, and support vector machine.
5 RESULTS
The results show that standard SMOTE can worsen fairness, whereas Fair-SMOTE significantly reduces bias and generally preserves or improves predictive performance. Across comparisons, Fair-SMOTE also performs strongly against state-of-the-art mitigation methods and can address multiple protected attributes simultaneously.
- RQ2: SMOTE consistently increases AOD, EOD, SPD, and DI bias scores, despite similar or better performance than the default learner.
- RQ3: Fair-SMOTE significantly reduces all reported bias scores, including AOD, EOD, SPD, and DI.
- RQ3: Fair-SMOTE and SMOTE often tie on recall, false alarm, precision, accuracy, and F1, but Fair-SMOTE performs clearly better on fairness metrics.
- RQ4: Against two state-of-the-art mitigation algorithms, Fair-SMOTE performs similarly or better on fairness and consistently achieves higher recall and F1.
- RQ4: Fair-SMOTE is reported to achieve better fairness and performance without the performance compromise associated with prior fairness methods.
- RQ5: On the Adult dataset, Fair-SMOTE simultaneously reduces bias for sex and race while producing higher recall and F1.The method balances training data across class and two protected attributes.
6 DISCUSSION: WHY FAIR-SMOTE?
Fair-SMOTE combines bias detection, explanation, and removal through data-focused preprocessing. It is presented as broadly applicable, performance-preserving, and effective across comparative evaluations, while its demonstrated use remains limited to classification data.
- Combination: Fair-SMOTE combines finding, explaining, and removing bias by detecting imbalance and improperly labeled data before oversampling and relabeling.
- Fair-SMOTE is compared with SMOTE, Fairway, and Optimized Preprocessing across 10 datasets and 3 learners, including simultaneous sex- and race-bias reduction on Adult.
- Uncompromising: Fair-SMOTE improves fairness alongside recall and F1 without substantially damaging accuracy or precision.
- Group & Individual: Fair-SMOTE addresses group fairness through protected-group balancing and individual fairness through situation testing for similar individuals.
- Generality: Fair-SMOTE uses only data and does not require access to model internals, supporting use with different model types.
- Versatility: The study used Fair-SMOTE only on classification datasets, while the authors identify regression and image data as future applications.
7 THREATS TO VALIDITY
The study reports broad evaluation coverage but identifies limits in sampling, metrics, causal explanations of bias, and generalization beyond binary tabular classification.
- Sampling Bias: The evaluation used 10 real-world datasets and 3 classification models, but conclusions may change with other datasets or models.
- Evaluation Bias: The study used four fairness metrics, although IBM AIF360 contains more than 50, leaving additional evaluation criteria for future work.
- Internal Validity: The analysis focused on data distribution and labels and did not validate other possible causes, including objective-function and homogenization bias.
- External Validity: The work is based on binary classification and tabular data, with extensions to regression, text mining, and image processing left for future work.
8 CONCLUSION
Fair-SMOTE targets bias by removing biased labels and balancing distributions across class and sensitive attributes. Across the reported evaluation, it matched state-of-the-art bias mitigation while achieving higher recall and F1 and faster execution, motivating domain-informed rather than blind optimization.
- Fair-SMOTE removes biased labels and rebalances internal distributions equally across class and sensitive attributes.
- Fair-SMOTE was as effective as two state-of-the-art algorithms for bias mitigation while achieving higher recall and F1.
- 220% faster median runtime across ten datasets was reported for Fair-SMOTE compared with Chakraborty et al.'s method.
- The authors recommend Fair-SMOTE for bias mitigation and reject the view that fairness necessarily reduces learner performance.
- The paper advocates reflecting on the domain and using those insights to guide improvements instead of blindly applying optimization methods.