Source-linked AI summary
Mitigating Bias in Federated Learning
Annie Abay, Yi Zhou, Nathalie Baracaldo, Shashank Rajamoni, Ebube Chuba, Heiko Ludwig
TL;DR
Bias-mitigation methods have largely focused on centralized ML, leaving open how to measure and reduce bias in federated learning without full data access. This paper adapts three mitigation methods for FL and evaluates them across data distributions and privacy designs. The proposed methods are reported effective with skewed data and when only 20% of parties employ them.
Problem
Bias measurement and reduction methods often require full data access, which conflicts with federated learning's privacy constraints.
Source: §1 · p. 1Method
The paper adapts local reweighing, global reweighing with differential privacy, and federated prejudice removal for FL, with different privacy designs.
Source: §1 · p. 2Results
The proposed methods remain effective with skewed data distributions and when as little as 20% of parties employ them.
Source: §1 · p. 2Takeaways & Limitations
Local reweighing produces fair models without sacrificing privacy or model accuracy, according to the authors' experiments.
Source: §6 · p. 12Disparate impact is unreliable in FL experiments, with high variability across runs.
Source: §5.2 · p. 11
Abstract
from arXiv · showhide
As methods to create discrimination-aware models develop, they focus on centralized ML, leaving federated learning (FL) unexplored. FL is a rising approach for collaborative ML, in which an aggregator orchestrates multiple parties to train a global model without sharing their training data. In this paper, we discuss causes of bias in FL and propose three pre-processing and in-processing methods to mitigate bias, without compromising data privacy, a key FL requirement. As data heterogeneity among parties is one of the challenging characteristics of FL, we conduct experiments over several data distributions to analyze their effects on model performance, fairness metrics, and bias learning patterns. We conduct a comprehensive analysis of our proposed techniques, the results demonstrating that these methods are effective even when parties have skewed data distributions or as little as 20% of parties employ the methods.
1 Introduction
Bias mitigation methods have largely assumed centralized access to training data, while federated learning (FL) keeps raw data with participating parties. This paper adapts three mitigation methods for FL and examines their privacy, performance, and fairness properties under heterogeneous data and participation.
1 Introduction
- FL complicates bias measurement and mitigation because parties hold heterogeneous private datasets, share updates rather than raw data, and may participate dynamically.The aggregator and parties have only a partial view of the data and how updates affect the global model.Source: §1 · pp. 1–2
- The paper proposes local reweighing, global reweighing with differential privacy, and federated prejudice removal, adapting centralized fairness techniques to FL.The methods are designed to train models without compromising data privacy or prediction accuracy.Source: §1 · p. 2
- The authors assess four fairness metrics in FL and report that one is unreliable in this setting.They also study the effect of parties refusing to participate in mitigation procedures.Source: §1 · p. 2
- The authors examine fairness and privacy trade-offs across three approaches with different privacy designs, including their effects on model performance and simplicity.The proposed designs avoid transferring sensitive information or add differential privacy.Source: §1 · p. 2
- The paper studies skewed data distributions and reports that the proposed methods remain effective when as little as 20% of parties employ them.The experiments consider disproportionate data and multiple sensitive attributes.Source: §1 · p. 2
2 Related work
Centralized bias-mitigation research includes pre-processing, in-processing, and post-processing approaches, but these methods assume access to raw data. Prior FL fairness work has focused on contribution fairness rather than sensitive-attribute fairness.
2 Related work
- Pre-processing changes data or assigns sample weights; in-processing modifies the classifier objective; post-processing adjusts predictions after training.Pre-processing may destabilize performance, while in-processing is tied to a model or training algorithm.Source: §2 · pp. 2–3
- These centralized approaches cannot be applied as-is in FL because they rely on raw-data sharing.The paper's Table 1 summarizes its proposed approaches.Source: §2 · p. 3, Table 1 · p. 3
- A few prior FL studies address contribution fairness, but not sensitive-attribute fairness, and none measure resulting models with well-known fairness metrics.Contribution fairness concerns whether the global model learns equally from parties.Source: §2 · p. 3
3 Fairness Analysis in FL: Causes
Bias in FL can come from familiar sources within parties as well as from interactions among parties and the aggregator. Party selection, data heterogeneity, and update fusion can each shape the fairness of the final model.
3 Fairness Analysis in FL: Causes
- Traditional sources of unfairness, including prejudice, underestimation, and negative legacy, remain relevant in FL, with parties introducing biases through shared model updates.These sources had previously been identified in centralized ML.Source: §3 · p. 3
- Party selection, subsampling, or dropouts may correlate with sensitive attributes and introduce bias into training.The passage gives network speed and device charge as factors that can affect whose data is collected or queried.Source: §3 · p. 3
- Different party-level data distributions can make local bias difficult to identify and affect the global model, even when all parties are queried.A branch near a women's college may have a customer dataset comprising mostly women.Source: §3 · p. 3
- Fusion algorithms may amplify group representation differences when they weight contributions from larger datasets more heavily.The paper focuses on FL with fewer organizational parties, such as companies, subsidiaries, or data centers.Source: §3 · p. 3
4 Proposed Approaches
The paper adapts Reweighing and Prejudice Remover for federated learning using two privacy designs: keeping sensitive information local or adding differential privacy. The approaches modify either sample weights before training or local training objectives, with trade-offs that include dynamic participation and tuning the fairness–accuracy balance.
4 Proposed Approaches
- The three approaches adapt Reweighing and Prejudice Remover for federated learning through designs that avoid transferring sensitive information or add differential privacy.The methods are intended to mitigate bias without compromising data privacy.Source: §4 · p. 4
4.1 Reweighing for FL
- Local reweighing computes weights on each party’s data without communicating sensitive attributes or sample information; global reweighing instead uses noisy counts to calculate and share weights.Both approaches adjust sample weights during preprocessing, without affecting the convergence behavior of the FL algorithm.Source: §4.1 · pp. 4–5
- Global reweighing does not apply to systems with dynamic participation because changing party numbers or dataset sizes require recalibrating its weights.Its weights are based on the participating parties’ training-set statistics.Source: §4.1 · pp. 4–5
4.2 Prejudice Remover for FL
- Federated prejudice removal adds a fairness-aware regularizer to each party’s local training while sharing only model parameters, leaving model fusion unchanged.The regularizer targets learned prejudice, but choosing η is difficult: increasing it can improve fairness while worsening accuracy, and tuning is complicated by parties’ differing data sizes and distributions.Source: §4.2 · p. 5
5 Experimental Results
The experiments compare federated models with a centralized baseline across IID and highly imbalanced party data, assessing prediction performance and multiple fairness metrics.
5.1 Performance under similar data distributions (IID)
- Under IID data, local reweighing brought Adult fairness metrics within acceptable margins for both sensitive attributes while accuracy and F1 remained consistent.For Compas, most fairness metrics fell within the margin, with accuracy and F1 also consistent.Source: §5.1 · p. 7
- Federated prejudice removal reduced bias in some metrics but did not bring results within fairness margins, and better fairness often coincided with lower accuracy and F1.Its metrics were more stable than those of local reweighing in the reported IID experiments.Source: §5.1 · pp. 7–8
- With global reweighing and differential privacy, F1 decreased slightly as ε decreased, remaining consistent until ε became very small.Most fairness metrics remained within predefined bounds until ε = 0.01, while F1 dropped significantly below ε = 0.4.Source: §5.1 · p. 8
5.2 Performance under highly imbalanced data distributions (non-IID)
- Under highly imbalanced data, local reweighing was ineffective for parties with only one group, but could mitigate bias when both groups were present, even at a 1% share.Most fairness metrics were within the fairness margin in the reported cases with both groups.Source: §5.2 · p. 10
- With highly heterogeneous party data, the global model’s bias often aligned with Party 2’s local model, indicating uneven bias learning across parties.The underestimation index was higher for Party 1 than Party 2 across most ratio combinations.Source: §5.2 · pp. 10–11
- Disparate impact was unreliable across FL experiments: in one highly imbalanced case, its standard deviation was 0.55 against a 0.4 fairness-margin width.The other three fairness metrics had approximate standard deviations of 0.2.Source: §5.2 · p. 11
- Local reweighing more effectively reduced FL bias across settings, including with reduced mitigation participation or disproportionate party data.The experiments also found it effective when parties had different data distributions and sample sizes.Source: §5.2 · pp. 10–11
6 Conclusion
The paper concludes that local reweighing produces fair models without sacrificing privacy or accuracy, even when few parties use it. It also reports instability in some fairness metrics and says UEI better reflects how local datasets influence the global model.
6 Conclusion
- Local reweighing produces fair models without sacrificing privacy or accuracy, even when only a small fraction of parties use the procedure.The paper also reports contrasting three methods for producing fair federated models.Source: §6 · p. 12
- Some fairness metrics lack stability in the federated setting, while UEI provides a better picture of how local datasets influence the global model.Source: §6 · p. 12
- Figure 8 presents performance metrics for models trained on the highly imbalanced Adult dataset.Source: Figure 8 · p. 13