PaperlayerView exact revision on arXiv

Source-linked AI summary

FairFed: Enabling Group Fairness in Federated Learning

Yahya H. Ezzeldin, Shen Yan, Chaoyang He, Emilio Ferrara, Salman Avestimehr

arXiv:2110.00857v3cs.LGcs.CY

TL;DR

Federated learning preserves client-data privacy but makes group-fair model training difficult because fairness methods often require centralized sensitive information or struggle with heterogeneous client distributions. FairFed combines local debiasing with server-side fairness-aware aggregation, producing fairer models particularly under high heterogeneity while supporting diverse client debiasing strategies.

  • Problem

    Federated learning can produce models that discriminate against demographic groups, while many fairness approaches require centralized sensitive information or perform poorly across heterogeneous clients.

  • Method

    FairFed combines local client debiasing with server-side aggregation-weight adjustments based on local and global fairness metrics.

  • Results

    FairFed improves group fairness particularly under heterogeneous data distributions, including 93% and 50% EOD improvements on Adult and COMPAS at α = 0.1.

  • Takeaways & Limitations

    FairFed supports privacy-preserving fair federated learning while allowing different clients to use different local debiasing methods.

  • Takeaways & Limitations

    The reported empirical evaluation is limited to binary classification tasks, with extensions to regression and NLP identified as future work.

Abstract

from arXiv · show

Training ML models which are fair across different demographic groups is of critical importance due to the increased integration of ML in crucial decision-making scenarios such as healthcare and recruitment. Federated learning has been viewed as a promising solution for collaboratively training machine learning models among multiple parties while maintaining the privacy of their local data. However, federated learning also poses new challenges in mitigating the potential bias against certain populations (e.g., demographic groups), as this typically requires centralized access to the sensitive information (e.g., race, gender) of each datapoint. Motivated by the importance and challenges of group fairness in federated learning, in this work, we propose FairFed, a novel algorithm for fairness-aware aggregation to enhance group fairness in federated learning. Our proposed approach is server-side and agnostic to the applied local debiasing thus allowing for flexible use of different local debiasing methods across clients. We evaluate FairFed empirically versus common baselines for fair ML and federated learning, and demonstrate that it provides fairer models particularly under highly heterogeneous data distributions across clients. We also demonstrate the benefits of FairFed in scenarios involving naturally distributed real-life data collected from different geographical locations or departments within an organization.

1 Introduction

Group fairness methods developed for centralized machine learning are difficult to apply in federated learning because data and sensitive-group information remain decentralized. FairFed addresses this challenge through server-side fairness-aware aggregation combined with local debiasing.

  • Centralized group-fairness methods generally require access to the entire dataset and are therefore unsuitable for federated learning.
  • Federated learning preserves data privacy through decentralized training, but this decentralization complicates fair model training across demographic groups.
  • Local debiasing with standard aggregation can perform poorly when client data distributions are highly heterogeneous.
  • Global debiasing may achieve reasonable fairness but can require sharing detailed information about local group composition, risking subgroup privacy.
  • FairFed combines local debiasing with server-side adjustment of aggregation weights based on clients’ fairness evaluations of the global model.
  • FairFed is designed to improve fairness under heterogeneous client distributions while supporting different local debiasing strategies across clients.

2 Background and Related Work

Fair federated learning builds on centralized fairness techniques but must address decentralized data and distinguish group fairness from client-based and collaborative fairness. Prior approaches include fairness-constrained optimization, global reweighting, and adaptations of centralized debiasing algorithms.

  • Centralized fairness methods include pre-processing, in-processing, and post-processing, but many require centralized access to datapoint-level sensitive information.
  • Federated learning research also studies client-based fairness and collaborative fairness, which target performance equality or reward allocation across clients rather than demographic groups.
  • This paper focuses on group fairness, aiming to prevent discrimination against datapoints belonging to different sensitive groups.
  • Existing group-fairness approaches in federated learning include distributed fairness-constrained optimization, global reweighting, and federated adaptations of FairBatch.

3 Preliminaries

Federated learning trains models across decentralized, privacy-preserving clients, but non-IID data complicates group-fairness evaluation and can produce globally discriminatory models. The section distinguishes global fairness over all clients from local fairness at individual clients.

  • Federated learning setup: Federated learning averages locally trained model updates at the server, weighting them by clients’ dataset sizes while using Secure Aggregation to protect individual updates.FedAvg samples clients for local training and aggregates their updates without requiring direct access to client data.
  • Motivation: Standard federated training can yield a global model that discriminates against an underlying demographic group.This motivates evaluating group fairness alongside predictive performance in federated learning.
  • Group-fairness metrics: EOD and SPD measure group disparities, with values closer to zero indicating better fairness.EOD conditions positive prediction rates on Y = 1, while SPD compares positive prediction rates directly.
  • Global and local fairness: Non-IID client distributions create distinct global and local fairness notions, whereas IID client distributions make the two coincide.The distinction can become substantial when client distributions differ.
  • Global and local fairness: Global fairness evaluates whether a model is nondiscriminatory across the union of all client datasets.The full dataset is represented as D̄ = ∪kDk across K clients.
  • Global and local fairness: Local fairness evaluates the same fairness criteria using only client k’s local dataset and distribution.Local performance can be measured by applying the fairness metrics to Dk.

4 FairFed: Fairness-aware aggregation in FL

FairFed combines local debiasing with fairness-aware server aggregation, adjusting client weights according to local and global fairness metrics. Secure aggregation enables the server to compute global fairness information without accessing individual client statistics or datasets.

  • FairFed approach: FairFed uses local debiasing and adjusts how the server aggregates client model updates to improve global group fairness.The approach keeps debiasing local while making aggregation fairness-aware.
  • Motivation: Fairness-oblivious FedAvg can over-weight clients with more data, allowing locally biased updates to disproportionately influence the global model.Dataset-size weighting may amplify the contribution of biased clients.
  • Aggregation weights: FairFed adaptively changes client aggregation weights according to the mismatch between each local fairness metric and the global fairness metric.Clients whose local fairness is closer to the global metric receive higher weights.
  • Aggregation weights: At β = 0, FairFed is equivalent to FedAvg; increasing β gives fairness metrics greater influence over aggregation weights.β controls the fairness budget and the accuracy–fairness trade-off.
  • Training procedure: At each round, clients train locally, the server obtains global fairness and accuracy through secure aggregation, clients compute metric gaps and weights, and the server aggregates weighted updates.The resulting global model is broadcast back to clients for the next round.
  • Flexibility: FairFed’s weights depend on local and global fairness metrics rather than a specific debiasing method, allowing different clients to use different local debiasing methods.The server incorporates these methods through metric-based client reweighting.
  • Secure computation: FairFed computes the global EOD metric from client-computed components using Secure Aggregation, without revealing individual components to the server.Clients locally compute conditional-probability terms, while shared full-dataset statistics are aggregated securely.

5 Experimental Evaluation

The evaluation examines FairFed across datasets, baselines, heterogeneity levels, and fairness budgets. FairFed improves fairness most under heterogeneous client distributions, while larger fairness budgets trade accuracy for fairness.

  • Datasets: Experiments use Adult and COMPAS binary classification datasets, with gender and race as the respective sensitive attributes.
  • Data partitioning: A smaller Dirichlet parameter α creates more heterogeneous client distributions, while α approaching infinity yields IID distributions.
  • Experimental setup: The study compares FairFed with FedAvg, local debiasing baselines, and global reweighting, using logistic regression and results averaged over 20 random seeds.
  • Heterogeneity results: At α = 0.1, FairFed improves EOD by 93% on Adult and 50% on COMPAS, with only a 0.3% accuracy decrease on each dataset.
  • Heterogeneity results: At highly homogeneous distributions, FairFed provides no significant fairness gains over local debiasing except when using FairBatch.
  • Fairness-budget analysis: Increasing fairness budget β makes fairness constraints more strongly affect aggregation weights, improving EOD toward zero while reducing accuracy.

6 Cases Studies for Fair Training in FL

The case studies evaluate FairFed on naturally heterogeneous data from U.S. states and hospital occupation groups, showing improved fairness in both settings, including a small accuracy trade-off on TILES.

  • 6.1 Case Study 1: FairFed is evaluated on ACSIncome, treating each U.S. state as an FL participant with heterogeneous dataset sizes and race distributions.The dataset contains 1,664,500 datapoints across 51 participants; state sizes range from 3,064 users in Wyoming to 195,665 in California.
  • 6.1 Case Study 1: 20% EOD improvement results from FairFed with reweighting on ACSIncome, changing EOD from -0.062 to -0.050.Local reweighting alone performs worse than FedAvg under cross-state heterogeneity, whereas FairFed with reweighting overcomes this issue.
  • 6.2 Case Study 2: TILES models daily stress for hospital workers using wearable-sensor signals, with clients representing three nurse occupation groups.The clients differ in data size, gender distribution, and target stress distribution; day-shift registered nurses have the most datapoints, more female participants, and higher stress levels.
  • 6.2 Case Study 2: FairFed improves TILES EOD from -0.199 to 0.004 with only a 2.6% accuracy decrease, from 0.567 to 0.556.Both FairFed and local reweighting improve EOD relative to FedAvg, but the reported FairFed result combines fairness improvement with a small accuracy reduction.

7 Conclusion and Future Works

The paper concludes that FairFed enhances group fairness through fairness-aware aggregation while maintaining data decentralization and utility. Future work targets broader task types, heterogeneous local debiasing, and additional fairness notions.

  • Conclusion: FairFed uses fairness-aware aggregation to improve performance across sensitive groups while maintaining high utility in federated learning.The conclusion reports that FairFed outperforms state-of-the-art fair federated learning frameworks under high data heterogeneity.
  • Future Works: Future work will extend evaluation beyond binary classification to regression and NLP tasks, support heterogeneous local debiasing, and integrate other fairness notions.The proposed extensions include collaborative fairness and client-based fairness.

A Changes to FairFed when tracking the Statistical Parity Difference metric

When tracking Statistical Parity Difference, FairFed decomposes the global metric into securely aggregated dataset statistics and client-side evaluations of the global model.

  • Metric decomposition: The decomposition substitutes each client’s datapoint-sampling probability with that client’s ratio of points to the total dataset size.This weighting connects local client information to the global SPD calculation.
  • Server-side changes: The server securely aggregates S = {Pr(A = 0), Pr(A = 1)} during the dataset-statistics step.These statistics support computation of the global SPD metric from client-shared evaluation information.
  • Client-side changes: Each client computes m(SPD)_global,k using the dataset statistics S and its local evaluation of the global model.The computation follows the SPD decomposition defined in equation (8).

B Heterogeneity of sensitive attribute distribution in our experiments

The experiments use a Dirichlet-based non-IID synthesis method to control heterogeneity in sensitive-attribute distributions across clients.

  • Sensitive-attribute heterogeneity: A Dirichlet distribution controls heterogeneity in clients’ sensitive-attribute distributions for the Adult and COMPAS datasets.Table 4 gives example heterogeneous distributions for α = 0.1 and α = 10.

C Additional experimental results

Additional experiments show that FairFed benefits from broader participation in local debiasing, supports mixed client strategies, and remains effective when clients contain only one sensitive group. Its revised objective also trades group fairness against uniform accuracy through η.

  • Subset participation: Group fairness improves as more clients adopt local debiasing, while the minimum accuracy across ratios and α is 0.826.FairFed needs 40% participation to outperform local reweighting without fairness-aware aggregation and more than 60% to outperform both comparison methods.
  • Mixed local debiasing: FairFed improves fairness metrics when clients mix reweighting and FairBatch, with minimum accuracy 0.829 and accuracy loss at most 1.1%.Applying alternative debiasing methods at clients can reduce the fairness degradation associated with fewer reweighting clients.
  • Single-group clients: With each client containing only one sensitive group, FairFed improves EOD over FedAvg by 27%, compared with 15% for global reweighting.Local reweighting is ineffective in this setting because each client lacks both groups.
  • Uniform accuracy: The revised FairFed objective introduces η to trade off global fairness against uniform accuracy across clients.Larger η gives the global fairness constraint greater influence on aggregation-weight updates.
  • Uniform accuracy: FairFed improves EOD and Std-Accuracy simultaneously on Adult and COMPAS, especially under highly heterogeneous distributions.The revised method provides greater uniform-accuracy improvement when client performance variances are large at smaller α.
  • Uniform accuracy: As η increases, group fairness improves at the expense of accuracy uniformity in the α = 0.5 experiment.The figure evaluates uniform accuracy using Std-Acc.

D.2 Experimental results FL case studies

Experiments on ACSIncome and TILES show that η controls a trade-off between accuracy uniformity across clients and group fairness metrics. Lower η improves accuracy uniformity while sacrificing group fairness.

  • FL case studies: On ACSIncome and TILES, decreasing η improves accuracy uniformity across clients with a trade-off in group fairness metrics.Uniform accuracy is controlled through the uniform accuracy constraint parameter η.
Loading 2110.00857v3…