PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Fair Federated Learning with Zero-Shot Data Augmentation

Weituo Hao, Mostafa El-Khamy, Jungwon Lee, Jianyi Zhang, Kevin J Liang, Changyou Chen, Lawrence Carin

arXiv:2104.13417v1cs.CVcs.LGstat.ML

TL;DR

Federated learning can perform unevenly across clients when their data distributions differ, leaving clients with less common classes at a disadvantage. This paper proposes Fed-ZDA, which generates synthetic data from shared model information for use at clients or the server. Experiments report better performance and fairness than commonly used federated-learning baselines.

  • Problem

    Client data heterogeneity can make federated models perform worse for some clients, with clients holding less common classes tending to see worse performance.

    Source: §1 · p. 1

  • Method

    Fed-ZDA generates synthetic examples for underrepresented data from shared models, with zero-shot augmentation deployed at clients or the server.

    Source: §3 · p. 3, §3.2 · p. 4

  • Results

    Fed-ZDA achieves better performance and fairness than commonly used federated-learning baselines.

    Source: §5 · p. 8

  • Takeaways & Limitations

    The methods use shared-model statistics to generate fake data rather than user data.

    Source: §5 · p. 8

    Fed-ZDAS assumes the server has no data.

    Source: §3.4 · p. 5

Abstract

from arXiv · show

Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data. Although it is recognized that statistical heterogeneity of the client local data yields slower global model convergence, it is less commonly recognized that it also yields a biased federated global model with a high variance of accuracy across clients. In this work, we aim to provide federated learning schemes with improved fairness. To tackle this challenge, we propose a novel federated learning system that employs zero-shot data augmentation on under-represented data to mitigate statistical heterogeneity and encourage more uniform accuracy performance across clients in federated networks. We study two variants of this scheme, Fed-ZDAC (federated learning with zero-shot data augmentation at the clients) and Fed-ZDAS (federated learning with zero-shot data augmentation at the server). Empirical results on a suite of datasets demonstrate the effectiveness of our methods on simultaneously improving the test accuracy and fairness.

1. Introduction

Federated learning keeps data on client devices, but client-level statistical heterogeneity can impair a single global model and distribute accuracy losses unevenly. Fed-ZDA uses zero-shot augmentation at clients or the server to generate pseudo-exemplars of unseen classes and promote more equitable performance.

1. Introduction

  • Statistical heterogeneity can make local models diverge, while a one-size-fits-all global model may perform worse for some clients.Federated learning methods tend to perform more poorly when client data are not i.i.d.Source: §1 · p. 1
  • Clients with less common classes may see worse performance, as their learned features can be drowned out during model-weight aggregation.Replay is difficult because clients lack access to data from parts of the distribution underrepresented in their own data, which remains private and local.Source: §1 · pp. 1–2, Figure 1 · p. 1
  • Fed-ZDA generates pseudo-exemplars of unseen classes without private data access, with augmentation at either clients (Fed-ZDAC) or the server (Fed-ZDAS).Experiments on MNIST, FMNIST, and CIFAR-10 report more equitable model performance; the approach also has a differential privacy analysis showing (0, δ) differential privacy.Source: §1 · p. 2

2. Related Work

Related work addresses client heterogeneity, fairness, data-free augmentation, and privacy through distinct strategies, each with trade-offs in model scope, performance, or data access.

2.1. Federated Learning

  • Federated-learning approaches address heterogeneity with multi-task models [29] [7], FedProx [20], or shared data [40], while many methods retain a single global model [26].The shared-data strategy may compromise privacy.Source: §2.1 · p. 2
  • Prior fairness approaches include uniformity weighting [21] and client-specific factorization [12], but may reduce some clients’ performance or increase training time.The paper focuses instead on a single global model.Source: §2.1 · p. 2

2.2. Zero-Shot Data Augmentation

  • Most augmentation transforms existing data, whereas data-free methods such as DeepInversion [36] synthesize it; Fed-ZDA generates samples from model information without sharing client data.The paper distinguishes its augmentation goal from synthesis for communication compression [9].Source: §2.2 · p. 2

2.3. Differentially Private Federated Learning

  • Differentially private federated learning has used Gaussian noise on updates [8] or secure multiparty computation [31], while Fed-ZDA is presented as a different randomization mechanism.The passage says prior noise-based approaches struggle to balance performance and privacy budget.Source: §2.3 · p. 3

3. Federated Learning with Zero-Shot Data Augmentation

Fed-ZDA adds labeled synthetic examples generated without access to client data to address under-representation and improve federated learning fairness. It has client-side and server-side augmentation variants.

3.1. Federated Learning

  • Heterogeneous client data can leave under-represented clients with poor accuracy; FedAvg aggregates client models, with weights that may depend on client data volume.When that information is unavailable, the paper says a simple arithmetic mean is used.Source: §3.1 · p. 3

3.2. Zero-Shot Data Generation

  • Zero-shot data generation creates labeled synthetic data without training-data access by matching generated inputs to pretrained-model batch-normalization statistics and target-class predictions.The generated input is initialized randomly and updated by gradient descent while model parameters remain fixed.Source: §3.2 · p. 4

3.3. Zero-Shot Data Augmentation at Clients

  • In Fed-ZDAC, clients use ZSDG on the received pre-update global model, then train on both synthetic and real local data before sending updated models for aggregation.The synthetic data is generated across classes from the received global model.Source: §3.3 · p. 5

3.4. Zero-Shot Data Augmentation at Server

  • In Fed-ZDAS, the server creates equal amounts of fake data from received client models, combines it into a balanced dataset, aggregates the models, then trains the global model on that dataset.The server-side variant is proposed partly because client devices may have limited compute and storage, and clients may prioritize their own classes.Source: §3.4 · p. 5

4. Experiments

The experiments evaluate Fed-ZDA on three datasets under non-i.i.d. client distributions, assessing local and global accuracy and fairness as well as synthetic-data quality. Results vary by dataset and setting, and the analyses show that augmentation quality depends on the model being inverted.

4.1. Datasets and Settings

  • Experiments use MNIST, FMNIST, and CIFAR-10 with 100 clients, 100 communication rounds, and non-i.i.d. splits that are either unimodal or multimodal.The comparison includes FedAvg, FedProx, and q-FFL.Source: §4.1 · pp. 5–6

4.4. The Analysis of Augmented Data

  • For each class, the study generates 64 label-conditioned images and evaluates their ability to fool separately trained oracle classifiers.The oracle classifiers reach 99.06% on MNIST, 89.79% on FMNIST, and 67.32% on CIFAR-10.Source: §4.4 · p. 6, Figure 3 · p. 8

4.2. Local Test and Client-Level Fairness

  • MNIST and FMNIST: Fed-ZDA achieves the best mean local accuracy and improves fairness over all baselines; on CIFAR-10, it beats q-FFL in accuracy and FedAvg and FedProx in fairness.Local fairness is measured by variance across clients.Source: §4.2 · p. 6, Table 1 · p. 7

4.3. Global Test and Class-Level Fairness

  • On MNIST and FMNIST, Fed-ZDA shows better global-test performance; on CIFAR-10, results are comparable.Global fairness is measured by variance of accuracy across classes.Source: §4.3 · p. 6, Table 2 · p. 7

4.5. The Influence of Client Data Distribution

  • 95.20%, 73.96%, and 58.10% are the test accuracies of fa, fb, and fc, respectively; recovered-image quality decreases across these models.The models are trained on aggregated data, federated i.i.d. data, and federated non-i.i.d. data, respectively.Source: §4.5 · p. 7, Figure 4 · p. 8

4.6. When to start data augmentation

  • Fed-ZDAC is compared with augmentation starting at global epochs 80, 90, and 95, with variance used to assess fairness.The authors state that higher-quality augmented data make client distributions more similar and contribute to reducing variance.Source: Figure 5 · p. 8, §4.6 · p. 8

5. Conclusions and Future Work

The paper proposes server-side or client-side zero-shot data augmentation for federated learning and reports improved performance and fairness over commonly used baselines. It identifies combining the placements and extending beyond FedAvg aggregation as future work.

  • Fed-ZDA generates fake data from shared-model statistics, with server-side or client-side deployment; the experiments assume FedAvg arithmetic-mean aggregation.Combining the deployment locations and studying more complex aggregation schemes are left for future research.Source: §5 · p. 8

A. Differential Privacy Analysis

The appendix establishes client-level (0, δ)-DP for Fed-ZDA using a total-variation bound on augmented client datasets and the data-processing inequality.

A. Differential Privacy Analysis

Related papers

Loading 2104.13417v1…