Source-linked AI summary
AdaFair: Cumulative Fairness Adaptive Boosting
Vasileios Iosifidis, Eirini Ntoutsi
TL;DR
Fairness-aware methods can achieve group parity while leaving minority-class performance weak when class imbalance makes overall error misleading. AdaFair adapts AdaBoost with cumulative ensemble fairness and balanced-error optimization. Across four real-world datasets, it reports parity in severe-imbalance cases alongside stronger classification performance than other approaches.
Problem
Method
AdaFair updates AdaBoost instance weights using cumulative fairness across the current ensemble and selects the ensemble size for balanced error.
Source: §1 · p. 1, §6 · p. 10Results
Across four real-world datasets, AdaFair reports parity in severe and extreme imbalance cases while maintaining significantly better classification performance than other fairness-aware approaches; it outperforms state-of-the-art methods by 8% to 25%.
Source: §1 · p. 2, §6 · p. 10Takeaways & Limitations
AdaFair’s reported results indicate that parity and stronger classification performance can be achieved together in cases of severe class imbalance.
Source: §6 · p. 10The method currently searches after training for the optimal ensemble-round parameter θ; online selection and convergence properties remain future work.
Source: §6 · p. 10
Abstract
from arXiv · showhide
The widespread use of ML-based decision making in domains with high societal impact such as recidivism, job hiring and loan credit has raised a lot of concerns regarding potential discrimination. In particular, in certain cases it has been observed that ML algorithms can provide different decisions based on sensitive attributes such as gender or race and therefore can lead to discrimination. Although, several fairness-aware ML approaches have been proposed, their focus has been largely on preserving the overall classification accuracy while improving fairness in predictions for both protected and non-protected groups (defined based on the sensitive attribute(s)). The overall accuracy however is not a good indicator of performance in case of class imbalance, as it is biased towards the majority class. As we will see in our experiments, many of the fairness-related datasets suffer from class imbalance and therefore, tackling fairness requires also tackling the imbalance problem. To this end, we propose AdaFair, a fairness-aware classifier based on AdaBoost that further updates the weights of the instances in each boosting round taking into account a cumulative notion of fairness based upon all current ensemble members, while explicitly tackling class-imbalance by optimizing the number of ensemble members for balanced classification error. Our experiments show that our approach can achieve parity in true positive and true negative rates for both protected and non-protected groups, while it significantly outperforms existing fairness-aware methods up to 25% in terms of balanced error.
1 INTRODUCTION
Fairness-aware classifiers can achieve group parity yet perform poorly on minority classes when class imbalance is evaluated through overall error. AdaFair adapts AdaBoost using cumulative ensemble fairness and reports improved fairness and balanced performance.
1 INTRODUCTION
- Observed discrimination in high-impact AI applications, including racial and gender bias, motivates fairness-aware decision making.The cited examples include service areas largely inhabited by Black people being ignored and more highly paid job advertisements being shown to men.Source: §1 · p. 1
- Overall-error-focused fairness methods can yield low true positive rates on minority classes, even when protected and non-protected groups achieve parity.The authors argue that class imbalance must also be addressed to preserve predictive performance for both groups.Source: §1 · p. 1
- AdaFair updates instance weights each boosting round using cumulative fairness across the ensemble and the dynamically discriminated group.It also weights instances using prediction confidence and selects a sequence of weak learners for high performance and fairness.Source: §1 · pp. 1–2
- AdaFair outperforms current state-of-the-art methods by 8% to 25% in performance, depending on class-imbalance severity.The paper also reports parity between groups while maintaining high true positive and true negative rates.Source: §1 · pp. 1–2
2 RELATED WORK
Related work spans preprocessing, in-processing, and post-processing methods, with fairness measured in several ways. The paper selects in-processing competitors that share its equalized-odds measure.
2 RELATED WORK
- Preprocessing methods alter data through relabeling, reweighting, sampling, or augmentation, while in-processing methods incorporate fairness constraints or regularization during training.Examples include data transformations, decision-tree criteria [34], and equalized-odds constraints in logistic regression [33].Source: §2 · p. 2
- Post-processing methods modify a model’s decision boundary or prediction labels; examples include thresholding to equalize group error rates [15].Other cited approaches change AdaBoost boundaries [12] or calibrate classifiers under same error rates among groups [27].Source: §2 · p. 2
- Equalized odds compares errors across protected and non-protected groups and is the fairness measure shared with the selected competitors.The authors select [33] and exclude [10], citing concerns about their fairness notions.Source: §2 · p. 2
- The authors say selected competitors optimize error rate without addressing class imbalance.They identify [33] as in-processing competitors and use equalized odds as their common fairness measure.Source: §2 · p. 2
3 BASIC CONCEPTS AND DEFINITIONS
The paper formulates binary classification with one binary sensitive attribute and adopts equalized odds to measure group differences in false-positive and false-negative rates. It argues that ordinary error rate can obscure minority-class failures under imbalance, motivating balanced error and fairness-aware boosting.
3 BASIC CONCEPTS AND DEFINITIONS
- The setup assumes n i.i.d. samples from P(F,S,y), binary labels, and one binary sensitive attribute separating protected and non-protected groups.F denotes non-sensitive attributes and S denotes sensitive attributes.Source: §3 · p. 2
- Equalized odds sums the absolute differences between protected and non-protected groups’ false-positive and false-negative rates.The paper defines those rate differences as δFPR and δFNR and seeks to minimize both.Source: §3 · p. 3
- Under class imbalance, a low error rate can coexist with misclassification of most minority instances and near-zero equalized-odds disparity.The stated goal is to achieve equalized odds while maintaining low false-positive and false-negative rates for both groups.Source: §3 · p. 3
- Balanced error rate replaces ordinary error rate as the performance indicator for class-imbalanced data.The passage presents it as a combination of true-positive and true-negative rates.Source: §3 · p. 3
- AdaBoost iteratively reweights instances based on misclassification, combining weak learners into an overall model.For fairness-aware classification, the data distribution between rounds must account for predictive performance and fairness.Source: §3 · p. 3
4 CUMULATIVE FAIRNESS ADAPTIVE BOOSTING
AdaFair adapts AdaBoost by weighting instances according to the ensemble’s cumulative fairness and prediction confidence. It also selects the final ensemble using balanced error to address class imbalance.
4.1 Cumulative boosting fairness and fairness costs
- Cumulative fairness evaluates the current ensemble’s parity for protected and non-protected groups across positive and negative classes.It is assessed at each boosting round using differences in false-positive and false-negative rates.Source: §4.1 · p. 3
- When parity is lacking, AdaFair assigns extra next-round costs to misclassified instances in discriminated groups, with costs shared by group and class.The costs depend on the direction and size of the fairness gap, subject to a tolerance condition.Source: §4.1 · pp. 3–4
- Confidence scores increase the weight of confidently misclassified instances, helping the learner focus on harder cases.Vanilla AdaBoost’s global learner weight does not distinguish instances by prediction reliability.Source: §4.1 · p. 4
4.2 Te AdaFair Algorithm
- AdaFair trains a weighted weak learner each round, computes error and fairness quantities and costs, then updates and normalizes instance weights using confidence scores.The training procedure initializes instance weights and costs before iterating over boosting rounds.Source: §4.2 · p. 4, §4.1 · p. 4
4.3 Optimizing the number of weak learners θ
- The selected ensemble size θ minimizes an objective combining balanced error, ordinary error, and Eq.Odds, addressing imbalance while considering fairness.The parameter c controls the relative contribution of balanced and ordinary error.Source: §4.3 · p. 4
- For all tested c values, AdaFair achieves Eq.Odds ≈0; c = 1 yields the highest TPRs with only slight TNR decreases for both groups.The authors recommend c = 1, which optimizes for balanced error.Source: §4.3 · p. 4
5 EVALUATION
The evaluation is designed to compare AdaFair’s predictive performance and fairness with related approaches, and to examine how its design choices affect behavior.
5 EVALUATION
- The evaluation compares AdaFair with related approaches on predictive performance and fairness.Predictive performance is assessed using accuracy and balanced accuracy; fairness is assessed using Eq.Odds and group-specific TPR and TNR.Source: §5 · p. 4
- The experiments investigate the effects of cumulative versus non-cumulative fairness and balanced error versus ordinary error.Source: §5 · p. 4
- The evaluation also compares cumulative margin distributions to examine the impact of confidence scores on instance weighting.Source: §5 · p. 4
5.1 Experimental setup
The evaluation uses four real-world datasets with varied size, dimensionality, and class imbalance, comparing AdaFair with fairness-aware and fairness-agnostic baselines. Results are averaged over ten random train/test splits.
5.1.1 Datasets.
- The four benchmarks span income prediction, bank-term-deposit subscription, recidivism, and census income, with positive-to-negative ratios ranging from 46%:54% to 1:15.The protected group is female for Adult, Compass, and KDD, and married for Bank.Source: §5.1.1 · p. 5
5.1.2 Baselines.
- Comparators include equalized-odds-focused methods by Zafar et al. and Krasanakis et al., vanilla AdaBoost, and SMOTEBoost, plus AdaFair ablations.The ablations remove cumulative fairness or confidence scores; SMOTEBoost targets class imbalance without fairness or confidence scores.Source: §5.1.2 · p. 5
5.1.3 Parameter selection and evaluation.
- Each dataset is randomly split 50:50 into training and test sets, and results average ten splits, with Eq.Odds tolerance set to zero.AdaFair uses 200 boosting rounds; the reported experiments set c = 1, optimizing balanced error rate.Source: §5.1.3 · p. 5
5.2 Predictive and fairness performance
Across four datasets, AdaFair combines strong balanced accuracy with high true-positive rates for both groups, while fairness and accuracy comparisons vary by dataset. On KDD, it leads the reported balanced-accuracy comparisons and maintains low Eq.Odds.
5.2.2 Bank.
- On Bank, AdaFair achieves better balanced accuracy than SMOTEBoost and the other approaches, with TPRs almost 33%↑ above Zafar et al. for each group.Its TNRs are 6%↓ and 7%↓ for protected and non-protected groups, respectively, compared with Zafar et al.Source: Figure 2 · p. 6, §5.2.2 · p. 6
5.2.1 Adult census income.
- On Adult, SMOTEBoost has the best balanced accuracy, followed by AdaFair (2%↓); AdaFair's TPR is 29%↑ and 22%↑ over Krasanakis et al. for protected and non-protected groups.Krasanakis et al. achieve the best Eq.Odds, followed by AdaFair (3%↑).Source: §5.2.1 · p. 6
5.2.3 Compass.
- On Compass, Krasanakis et al. edge AdaFair by 0.6%↓ in Eq.Odds and 0.2%↑ in balanced accuracy, while AdaFair has better TPRs for both groups.AdaFair's TNRs are lowest at 61%–65%, consistent with optimizing balanced error on a nearly balanced dataset.Source: Figure 3 · p. 6, §5.2.3 · p. 6
5.2.4 KDD census income.
- 25%↑ over Krasanakis et al., 18%↑ over AdaBoost, and 8%↑ over SMOTEBoost in balanced accuracy on KDD.AdaFair also records TPR above 80% for both groups; AdaFair and Krasanakis et al. reduce discrimination to 2% Eq.Odds.Source: §5.2.4 · pp. 6–7, Figure 4 · p. 7
5.3 Cumulative vs non-cumulative fairness
AdaFair's cumulative fairness measure evaluates the ensemble built through the current boosting round, rather than only the latest weak learner. Compared with this cumulative approach, the non-cumulative variant has worse fairness results and more fluctuating fairness costs.
5.3 Cumulative vs non-cumulative fairness
- The comparison tests ensemble fairness across rounds against fairness measured only for the current weak learner.AdaFair's cumulative notion uses the ensemble members built through the current round.Source: §5.3 · p. 7
- Without cumulative fairness, Eq.Odds increases 52%↑ on Adult, 15%↑ on Bank and Compass, and 45%↑ on KDD compared with AdaFair.The non-cumulative variant also has higher standard deviation and lower protected-group TPRs alongside higher protected-group TNRs.Source: §5.3 · p. 7
- AdaFair's per-round fairness costs smooth out and converge to [−0.05, 0.05], while costs for AdaFair NoCumul fluctuate more.The authors report that AdaFair mitigates discrimination over the early rounds.Source: §5.3 · p. 7
5.4 Te effect of confidence scores
AdaFair weights instances using prediction confidence, and its positive-class results generally improve over AdaFair NoConf and AdaBoost, especially on imbalanced datasets. This comes with a small increase in negative-class errors on Bank and KDD.
5.4 Te effect of confidence scores
- For positive examples, AdaFair is more confident than the other approaches on every dataset except COMPAS, where AdaFair and AdaFair NoConf have no clear winner.On the negative class, AdaBoost performs best, consistent with its focus on overall accuracy and the majority class.Source: §5.4 · p. 7
- On Bank, AdaFair has 15% fewer positive-class misclassifications than AdaFair NoConf and 36% fewer than AdaBoost, with negative-class error increases of 5% and 7%, respectively.The comparison reports misclassifications by class.Source: §5.4 · p. 7
- On KDD, AdaFair has 23% and 46% fewer positive-class misclassifications than AdaFair NoConf and AdaBoost, with negative-class error increases of 12% and 14%, respectively.AdaBoost classifies almost all negative majority instances correctly but performs worst on the minority class.Source: §5.4 · p. 8
- Confidence-based weighting gives more weight to confidently misclassified instances, differentiating them from other errors and focusing the classifier on harder cases.The paper concludes that confidence scores increase ensemble margins, especially under class imbalance.Source: §5.4 · p. 8
5.5 Te effect of balanced error
As the balanced-error objective parameter c increases, balanced accuracy and TPRs rise on the imbalanced datasets, while TNRs decline less. The paper concludes that directly addressing imbalance through balanced error is more effective for fairness-aware classification.
5.5 Te effect of balanced error
- On all imbalanced datasets, balanced accuracy and TPRs increase with c, while TNRs decrease less; COMPAS shows no significant differences as c varies.For c = 0, only error rate is considered; c = 1 optimizes balanced error.Source: §5.5 · p. 8, Figure 8 · p. 9
- For Adult, moving from c = 0 to c = 1 raises TPR by 18% for s and 15% for s̄, while reducing TNR by 5% for both groups.The comparison is between error-rate and balanced-error objectives.Source: §5.5 · p. 8
- For the bank dataset, TPR increases by 56% for both groups, while TNR decreases by 8% and 9% for s and s̄, respectively.Source: §5.5 · p. 10
- For the KDD dataset, TPR grew by 20% for both groups, while TNR decreased by 5% for s and 6% for s̄, respectively.Source: §5.5 · p. 10
- Directly tackling imbalance through balanced error is more effective for fairness-aware classification.Source: §5.5 · p. 10
6 CONCLUSIONS AND FUTURE WORK
AdaFair adapts AdaBoost by adjusting instance weights using cumulative ensemble fairness. Experiments report parity and stronger classification performance in severely imbalanced settings, while future work includes online ensemble-size selection and theoretical convergence analysis.
6 CONCLUSIONS AND FUTURE WORK
- AdaFair adjusts per-round weights using the ensemble’s cumulative fairness behavior to account for the group discriminated against in that round.The method adapts AdaBoost by changing the data distribution during training.Source: §6 · p. 10
- Across four real-world datasets, AdaFair achieves parity and stronger classification performance in severe or extreme imbalance, and outperforms imbalance-focused methods in extreme imbalance.On balanced COMPAS data, its parity is similar to other fairness-aware methods.Source: §6 · p. 10
- Parity can hold without optimizing balanced error, but this is associated with low TPRs, high TNRs, and low Eq.Odds.The paper also reports cumulative fairness as superior to non-cumulative fairness and confidence weighting as increasing confidence relative to vanilla AdaBoost.Source: §6 · p. 10
- Online selection of ensemble rounds θ is a proposed extension to replace the current post-training search; convergence properties are also future work.Source: §6 · p. 10