PaperlayerView exact revision on arXiv

Source-linked AI summary

Advances and Open Problems in Federated Learning

Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G. L. D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B. Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchinson, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmus Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Sebastian U. Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X. Yu, Han Yu, Sen Zhao

arXiv:1912.04977v3cs.LGcs.CRstat.ML

TL;DR

As federated learning research grows, a broad synthesis of its open challenges is needed. This paper surveys the setting, practical constraints, and research directions, highlighting model-update poisoning and fairness concerns related to client sampling.

  • Problem

    Despite significant recent interest in federated learning, a broad account of its open challenges was needed.

    Source: §1 · p. 4

  • Method

    The paper describes federated learning’s defining characteristics and challenges, practical constraints, and research directions.

    Source: §1 · p. 4

  • Results

    The paper identifies model-update poisoning as a new FL attack and training-data bias shaped by federation’s sampling steps as a key fairness consideration.

    Source: §5.4 · p. 73, §6.6 · p. 79

  • Takeaways & Limitations

    Practical federated learning faces efficiency requirements from system constraints, while privacy presents distinct challenges across threat models and assumptions.

    Source: §8 · p. 89

    Secure aggregation assumes a semi-honest server, exposes per-round aggregates, handles sparse vectors inefficiently, and cannot enforce well-formed client inputs.

    Source: §4.4.2 · p. 55

Abstract

from arXiv · show

Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g. service provider), while keeping the training data decentralized. FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches. Motivated by the explosive growth in FL research, this paper discusses recent advances and presents an extensive collection of open problems and challenges.

1 Introduction

Federated learning keeps client data local while enabling server-coordinated training through focused updates. The paper characterizes the setting and its practical constraints, then surveys open problems and interdisciplinary research directions.

1 Introduction

  • FL keeps clients’ raw data local while they collaborate through a central server and send focused updates for aggregation; the paper focuses mainly on cross-device settings while recognizing cross-silo variants.The paper aims to highlight theoretically and practically important research problems with potential real-world impact.Source: §1 · pp. 4–5

1.1 The Cross-Device Federated Learning Setting

  • Cross-device and cross-silo FL differ from datacenter learning across practical axes that constrain applicable techniques and motivate the paper’s open challenges.The paper presents these as representative examples, while noting that FL settings can combine characteristics differently.Source: §1 · p. 5, §1.1 · p. 5, Table 1 · p. 6

1.1.1 The Lifecycle of a Model in Federated Learning

  • A model lifecycle spans problem identification and client instrumentation through federated training, evaluation, and deployment, and a primary challenge is making this workflow straightforward to use.Evaluation can use standard datacenter datasets or held-out clients’ local data; deployment includes quality assurance, live A/B testing, and staged rollout.Source: §1.1.1 · pp. 7–8

1.1.2 A Typical Federated Training Process

  • Training typically repeats client selection, model broadcast, local computation, aggregation, and server model updates, with eligibility constraints intended to avoid disrupting users.Aggregation can incorporate secure aggregation, lossy compression, clipping, and noise for privacy and communication efficiency; training models remain separate from live predictions.Source: §1.1.2 · pp. 8–9

1.2 Federated Learning Research

  • Because researchers often test approaches in simulation rather than production-scale deployments, FL research should specify its setting, simulation scope, privacy and communication assumptions, and evaluation benchmarks.The paper calls for reproducible simulations that explain which real-world characteristics they capture, and identifies standard metrics and benchmark datasets for cross-device and cross-silo settings as important research directions.Source: §1.2 · pp. 9–10

1.3 Organization

  • The paper proceeds from other FL settings to efficiency and effectiveness, privacy threats and protections, model manipulation and failures, and fairness and bias.These topics are addressed in Sections 2 through 6, respectively.Source: §1.3 · p. 10

2 Relaxing the Core FL Assumptions: Applications to Emerging Settings and Scenarios

The section surveys alternatives and extensions to standard server-coordinated FL, focusing on decentralized learning, cross-silo settings, and split learning. Each changes coordination or data-partitioning assumptions while raising distinct algorithmic, practical, or privacy challenges.

2.1 Fully Decentralized / Peer-to-Peer Distributed Learning

  • Peer-to-peer FL replaces server coordination with sparse connected client graphs, averaging neighbors’ models toward consensus, though a trusted authority may still define the task and hyperparameters.The server can otherwise be a single point of failure or bottleneck.Source: §2.1 · p. 11

2.1.1 Algorithmic Challenges

  • Deep-learning decentralized SGD still faces open robustness questions; MATCHA reduces communication delays through selected parallel graph matchings, while asynchronous variants avoid global synchronization.The passage notes that fixed-probability message drops on a complete graph can still yield convergence rates comparable to a reliable network [498].Source: §2.1.1 · pp. 12–13
  • Open problems include convergence with multiple local steps on non-IID data, personalized models and incentives against malicious contributors, communication compression, and privacy-utility trade-offs.Existing work studies personalized models and similarity graphs, but convergence policies, reliable incentives, and privacy methods with better utility remain challenges.Source: §2.1.1 · p. 13

2.1.2 Practical Challenges

  • Blockchain smart contracts could coordinate decentralized model aggregation, but public ledgers raise privacy concerns addressed through secure aggregation, deposits or confidential enclaves, and client-level noise.Deposits could penalize dropouts and simplify secure aggregation; confidential smart contracts can aggregate encrypted updates inside secure hardware.Source: §2.1.2 · p. 14

2.2 Cross-Silo Federated Learning

  • Cross-silo FL supports collaboration among organizations that cannot share data directly; feature-partitioned training exchanges intermediate results, while incentive design and privacy guarantees require further attention.Participants may be business competitors, and local differential privacy may be needed when other clients are a relevant threat.Source: §2.2 · pp. 14–16

2.3 Split Learning

  • Split learning divides model execution between clients and a server, exchanging cut-layer activations and gradients rather than raw data, but communicated values can leak information and formal privacy analysis remains open.NoPeek SplitNN reduces activation leakage by reducing distance correlation with raw data; parallelizing split learning on edge devices also remains unexplored.Source: §2.3 · pp. 16–17

3 Improving Efficiency and Effectiveness · 3.1 Non-IID Data in Federated Learning · 3.2 Optimization Algorithms for Federated Learning

This section examines how non-IID data, heterogeneous client behavior, and practical participation costs shape federated learning’s efficiency, effectiveness, and optimization. It surveys strategies and analyses addressing these challenges while identifying unresolved questions around convergence, local updates, acceleration, and deployment.

3.1 Non-IID Data in Federated Learning

  • Client data can differ in feature, label, conditional, and quantity distributions, and real-world datasets likely mix these regimes, making realistic heterogeneity characterization an open question.Such characterization could enable controlled, realistic test datasets for assessing algorithms under different degrees of client heterogeneity.Source: §3.1 · pp. 18–19

3.1.1 Strategies for Dealing with Non-IID Data

  • When client labelings differ, personalization may be essential; client-specific models can also make some degenerate non-IID distributions trivial to learn.A shared proxy dataset may reduce client differences, while heterogeneous objectives raise questions about example weighting and fairness among clients.Source: §3.1 · p. 19, §3.1.1 · p. 20

3.1 Non-IID Data in Federated Learning

  • Client availability and distributions can change over time: nighttime eligibility creates diurnal and geographic sampling bias, while training and deployment client differences can introduce dataset shift.Mitigating availability effects and adapting dataset-shift techniques to FL remain open questions; Eichner et al. [171] describe the availability issue and mitigation strategies.Source: §3.1 · p. 19

3.2 Optimization Algorithms for Federated Learning

  • Federated optimization must support sampling a handful of clients per round and stateless devices; Federated Averaging [337] has clients run local SGD steps before the server averages their models.Local updates reduce communication and accommodate data locality, but developing algorithms tailored to federated settings remains an open problem.Source: §3.2 · pp. 20–21

3.2.1 Optimization Algorithms and Convergence Rates for IID Datasets

  • For IID convex objectives, minibatch SGD attains the optimal statistical term and single-device SGD the optimal optimization term, but federated averaging still cannot match accelerated minibatch SGD’s rate.Whether federated averaging can close this gap remains open; the usual weighted-average aggregation method’s effect on fastest error convergence is also unclear.Source: §3.2.1 · pp. 22, 24
  • For non-convex IID objectives, local SGD achieves an error bound of 1/TKM when K is below T^(1/3)/M; later analyses extend local updates and give an O(1/TKM) rate under the PL condition.Wang and Joshi [467] remove the bounded-gradient assumption and show K can reach T/M^3; for PL functions, Haddadpour et al. [216] show T^2/M local updates per round yield O(1/TKM).Source: §3.2.1 · pp. 23–24
  • The best number of local updates K balances slower training-loss convergence against lower communication cost and iteration time; adaptive communication can tune K to training loss [468].Wang and Joshi [468] propose adjusting K at regular intervals during training.Source: §3.2.1 · p. 24
  • Stragglers motivate overprovisioning or time-windowed local work, but optimal handling, convergence analysis for asynchronous methods, and compatibility with privacy and secure aggregation remain challenging.Bonawitz et al. [81] propose requesting extra client updates and accepting the first M; asynchronous approaches may also be difficult to combine with differential privacy or secure aggregation.Source: §3.2.1 · p. 24
  • A larger client set per round improves convergence but increases vulnerability to unpredictable computation and communication tail delays.Source: §3.2.1 · pp. 24–25

3.2.2 Optimization Algorithms and Convergence Rates for Non-IID Datasets

  • Under non-IID data, local SGD has a worse error bound, while FedProx adds a proximal term to improve robustness to heterogeneous local objectives and empirically improve FedAvg [501, 300].Khaled et al. [268] assume all clients participate and use client-side batch gradient descent, which may converge faster than stochastic client updates.Source: §3.2.2 · p. 25
  • Partial-participation convergence analysis assumes uniform or dataset-size-proportional client sampling [303], though such ideal sampling may be infeasible in cross-device settings.The cited work considers only subsets of clients per round; device eligibility requirements can constrain which clients participate.Source: §3.2.2 · p. 25
  • Heterogeneous local step counts can make FedAvg converge to a mismatched objective; Wang et al. call this objective inconsistency and propose a correction.Eligibility constraints and changing client characteristics motivate semi-cyclic SGD, which samples blocks of clients in a regular pattern [171].Source: §3.2.2 · p. 27
  • Convergence analyses generally assume smooth or Lipschitz local functions; FedAvg results suggest K > 1 can slow convergence, leaving when local updates help or hurt as an open question [303].The surveyed rates measure optimal objective error for convex functions and gradient norm for nonconvex functions; strong convexity can improve rates, and bounded gradient variance is often assumed for stochastic updates.Source: §3.2.2 · p. 27
  • PD-SGD enables periodic local updates in decentralized SGD [467], MATCHA samples clients for computation and communication, and SCAFFOLD uses control variates without bounding client heterogeneity [265].PD-SGD was extended to non-IID data [304], and MATCHA's analysis shows local updates can accelerate convergence.Source: §3.2.2 · p. 27
  • Client momentum can improve final accuracy but doubles per-round communication [501], whereas server adaptive optimization retains FedAvg's communication cost [389]; proving momentum acceleration remains challenging [264].MIME provides a general approach to adapting centralized optimization algorithms to heterogeneous federated settings [264].Source: §3.2.2 · pp. 27–28

3.3 Multi-Task Learning, Personalization, and Meta-Learning

The section surveys ways to move beyond one shared model—through context features, task-specific models, or local adaptation—and identifies when personalization helps and how to evaluate it as open problems.

3.3.1 Personalization via Featurization

  • A shared model supplied with user and context features could make personalized predictions, but suitable architectures remain an open problem, in part because public datasets rarely contain these features.On-device personalization has improved mobile keyboard next-word prediction, while context features offer a complementary approach.Source: §3.3.1 · p. 28

3.3.2 Multi-Task Learning

  • Multi-task learning produces one model per client task, but often requires every client to participate each round and retain state, making it more suited to cross-silo than cross-device FL.An open direction is assigning tasks to client subsets, for example by geography, device or user characteristics, clustering, or learned-graph components; sparse problems also raise the challenge of hiding subset membership.Source: §3.3.2 · pp. 28–29

3.3.3 Local Fine Tuning and Meta-Learning

  • Personalization can follow global training with local fine-tuning, transfer learning, domain adaptation, or interpolation, but the best method and its federated learning guarantees remain open questions.These approaches may assume only source and target domains, potentially missing the richer structure of federated learning.Source: §3.3.3 · p. 29
  • MAML frames personalization as learning a global starting point adaptable with a few local gradient steps; Reptile closely resembles Federated Averaging, and Personalized FedAvg connects MAML to federated learning.The section also notes applications of local fine-tuning to federated speech recognition and study of differential privacy connections.Source: §3.3.3 · p. 29
  • Comprehensive personalized-performance metrics remain undefined: the section asks whether small gains for every client are preferable to larger gains for a subset, despite the observed global-to-personalized accuracy gap [250].MAML evaluation also needs realistic FL benchmarks beyond synthetic image-classification tasks and theoretical guarantees clarifying when such methods succeed.Source: §3.3.3 · pp. 29–30
  • Models with similar structure and performance can differ in personalization capacity, and maximizing global performance may hurt later personalization [250].The section also raises whether this multi-task and learning-to-learn framework can address concept drift or use non-parameter-transfer methods such as ProtoNets.Source: §3.3.3 · p. 30

3.3.4 When is a Global FL-trained Model Better?

  • Federated learning can outperform local training with small IID datasets, while local models can do better under pathologically non-IID distributions; when each is preferable remains an open theoretical question.The section asks whether personalization can guarantee performance at least matching both local and federated models, and how much improvement to expect. The answer may depend on data distribution and, for vertically partitioned data, noisy record linkage.Source: §3.3.4 · p. 30

3.4 Adapting ML Workflows for Federated Learning

Adapting machine-learning workflows to federated learning raises challenges because data are decentralized and devices are resource-constrained. Key open areas include hyperparameter tuning, architecture design, and privacy-preserving debugging and interpretability.

3.4.1 Hyperparameter Tuning

  • Federated learning expands the tuning space and training constraints, motivating robust, adaptive, or self-tuning hyperparameter optimization.Besides standard settings, practitioners may need to tune aggregation, client optimizers, clients selected per round, local steps, and update compression; these choices can affect accuracy, convergence, and model quality. Repeated evaluations can overuse mobile compute and communication resources, while existing AutoML HPO mainly targets accuracy.Source: §3.4.1 · p. 31

3.4.2 Neural Architecture Design

  • Gradient-based neural architecture search with weight sharing reduces search from over 3000 GPU days to 1 GPU day, and federated NAS has seen early exploration.Predefined architectures may be suboptimal when user data are invisible to developers, including redundant components or poor fit to non-IID data; NAS approaches include evolutionary algorithms, reinforcement learning, and gradient descent. Existing methods have not been developed for distributed settings, but are considered feasible to transfer.Source: §3.4.2 · p. 31

3.4.3 Debugging and Interpretability for FL

  • Privacy-preserving techniques for inspecting decentralized data remain an open problem, including improving the fidelity of FL-trained differentially private generative models proposed by Augenstein et al. [31].Such tools could support sanity checks, debugging misclassifications, discovering outliers, manual labeling, and bias detection; the proposed models include GANs.Source: §3.4.3 · p. 32

3.5 Communication and Compression · 3.6 Application To More Types of Machine Learning Problems and Models · 3.7 Executive summary

Communication is a major practical bottleneck in federated learning, while extending FL to additional learning paradigms and models raises unresolved challenges. These sections highlight open questions in compression, privacy compatibility, wireless co-design, Bayesian scalability, and the broader practice of FL.

3.5 Communication and Compression

  • Communication is a primary bottleneck; Federated Averaging with sparsification or quantization can reduce communication cost with minimal accuracy impact, and client-to-server compression offers the greatest practical runtime potential.The latter benefit reflects slower client upload than download and the possibility of more aggressive lossy compression through averaging across clients.Source: §3.5 · p. 32
  • Open questions include translating communication-constrained statistical theory into practical optimization methods and obtaining instance-specific or data-dependent gradient compression results in federated settings.Existing theoretical work often characterizes minimax rates while ignoring optimization algorithms; data-dependent schemes have shown promise for compression, motivating evaluation in FL.Source: §3.5 · pp. 32–33
  • Compression objectives include reducing client updates, server broadcasts, and local computation, while research combining these aims with a compact final model could have substantial practical impact.Existing compact-model methods may not map straightforwardly to FL or can complicate training; the three compression objectives are usually complementary.Source: §3.5 · pp. 32–33
  • Secure aggregation, can add O(log M) bits per scalar, while standard quantization is incompatible with real-valued Gaussian or Laplacian noise, making jointly privacy-compatible compression an open problem.The extra communication may undermine aggressive quantization when M is large; discrete-noise methods exist, but their optimality is unclear.Source: §3.5 · p. 33
  • Wireless interference, noisy channels, and channel fluctuations threaten latency and reliability; analog over-the-air aggregation can speed server aggregation and training by up to the number of workers.Other proposed approaches include federated distillation and scheduling workers with suitable communication and computing resources; the analog approach uses wireless broadcast and superposition.Source: §3.5 · pp. 33–34

3.6 Application To More Types of Machine Learning Problems and Models

  • Extending FL beyond supervised learning remains open, while Bayesian methods may support aggregation across non-IID data and heterogeneous platforms but still face scalability and computational-feasibility questions.Bayesian neural networks provide uncertainty estimates and can learn from small datasets; preliminary aggregation work by Lalitha et al. motivates further study.Source: §3.6 · p. 34

3.7 Executive summary

  • Non-IID data and client heterogeneity remain central open problems, and current convergence theory does not fully explain Federated Averaging’s empirical effectiveness over mini-batch SGD [481].The executive summary also identifies open questions about when global models outperform personalization, how many models are needed, and which optimization methods work best with local fine-tuning.Source: §3.7 · pp. 34–35
  • Adapting centralized workflows to FL remains a practical roadblock, while automated compression and extensions to reinforcement, semi-supervised, unsupervised, active, and online learning remain important open directions.The executive summary also points to theoretical study of communication, privacy compatibility, and co-design with wireless infrastructure.Source: §3.7 · p. 35

4 Preserving the Privacy of User Data

The section frames user-data privacy as a system-wide goal: participants should learn only what they need and understand privacy risks, while the system must also meet other operational and model-quality goals. FL’s data minimization offers practical privacy benefits, but does not formally guarantee privacy, motivating modular study and rigorous protections.

4 Preserving the Privacy of User Data

  • Ideally, each actor would learn only what their role requires, and participants could understand privacy disclosures well enough to make informed choices about whether to participate.For example, an analyst might need only a quality-threshold result, while end users might need only model predictions.Source: §4 · p. 36
  • The section advocates studying and improving modular system units independently, while ultimately measuring the complete system against ideal privacy goals.The paper notes that achieving privacy alongside usability, model quality and fairness, resource efficiency, and resilience remains an open challenge.Source: §4 · pp. 36–37
  • FL keeps raw user data on devices and sends task-focused model updates to the server, which may be held only ephemerally, offering practical privacy improvements over centralized training.The paper says updates contain no additional information about users beyond the raw data and typically significantly less.Source: §4 · p. 37
  • The baseline FL model has no formal privacy guarantee: a server with a previous model and a user’s gradient update may infer that user’s training example.The section surveys results and open challenges for rigorous privacy guarantees specific to FL and analytics, leaving broader machine-learning questions to [344].Source: §4 · p. 37

4.1 Actors, Threat Models, and Privacy in Depth

Meaningful FL privacy guarantees require specifying whose view and which adversary capabilities they cover, then composing protections across the system. The paper calls for “Privacy in Depth”: designs should preserve privacy as gracefully as possible when individual techniques or assumptions fail.

4.1 Actors, Threat Models, and Privacy in Depth

  • Formalizing FL privacy requires a holistic, interdisciplinary approach that combines technical protections and cross-disciplinary efforts.End-to-end systems may layer techniques such as MPC within a TEE and use MPC for aggregation followed by Private Disclosure before sharing aggregate updates beyond the server.Source: §4.1 · pp. 37–38
  • Characterizing actors and their views is necessary because protection against a malicious analyst may not protect against a malicious server administrator.The example assigns analysts a stronger differential privacy guarantee, ε′ < ε, than the server administrator’s view.Source: §4.1 · p. 37
  • Privacy guarantees depend on adversary limits, including control of up to γ clients and inability to break cryptography at security level σ; stronger adversaries may receive weaker ε0 > ε protection.Precisely specifying privacy goals and assumptions can implicate parameters such as ε, ε′, ε0, γ, and σ, as well as concepts such as differential privacy and honest-but-curious security.Source: §4.1 · p. 38
  • “Privacy in Depth” combines protections so privacy degrades gracefully if a technique or security assumption fails.For example, running an MPC server component inside a TEE can preserve privacy if either the TEE or MPC security assumption fails, but not both; sending gradient updates to a TEE is preferred to sending raw examples because failure degrades more gracefully.Source: §4.1 · p. 38

4.2 Tools and Technologies

This section surveys tools for privacy engineering in federated learning, focusing on controlling information flow during computation, limiting disclosure from outputs, and verifying faithful execution. It covers secure computation, differential privacy, and verifiability approaches, including their trade-offs, open challenges, and the use of TEEs and remote attestation to verify server computations.

4.2.1 Secure Computations

  • Secure computation—including MPC, homomorphic encryption, and TEEs—can protect inputs or intermediate results, but TEEs face memory, GPU-access, and side-channel limitations, and enclave partitioning remains open.Secure enclaves may also require data-oblivious code, and attestation proves which binary ran but not that it has the desired privacy properties.Source: §4.2.1 · pp. 40, 42
  • Secure aggregation lets a server learn only an aggregate of client values, while secure shuffling gives it an unordered message collection without linking messages to senders.These are specialized secure-computation functionalities used in FL.Source: §4.2.1 · p. 43

4.2.2 Privacy-Preserving Disclosures

  • Distributed differential privacy can yield more accurate statistics than local DP at the same privacy level, but relies on different setups and typically stronger assumptions, such as MPC.Secure aggregation can combine client perturbations to match centrally added noise; shuffling can achieve far smaller error than the local model while relying on weaker trust assumptions than central DP. Hybrid DP partitions users by trust preference and can, in some circumstances, outperform local DP across all users and central DP on opt-in users.Source: §4.2.2 · pp. 45–46

4.2.3 Verifiability

  • Verifiability lets servers prove faithful processing to clients and clients prove protocol-compliant inputs to servers without exposing private data; ZKPs provide mathematical security guarantees, unlike hardware-based trust assumptions.Relevant techniques include zero-knowledge proofs, TEEs, and remote attestation.Source: §4.2.3 · pp. 46–47
  • ZKP choices trade off trusted setup, interaction, proof length, and prover and verifier computation; relaxing succinctness or non-interactivity can avoid setup and use more standard assumptions [92, 464, 20, 63].Prover complexity ideally scales linearly with evaluated functionality, while verification requires at least linear computation in input size, which is significant for FL server proofs.Source: §4.2.3 · pp. 47–48
  • SNARKs offer constant proof size and verification linear in input size, but require stronger assumptions and trusted setup, with superlinear prover cost; Libra achieves linear prover complexity at increased proof size and verification time.Most existing SNARK constructions use quadratic arithmetic programs.Source: §4.2.3 · p. 48
  • Efficient use of interactive ZKPs and NIZKs in FL remains an open question: NIZKs can prove client-input properties to servers, while client verification faces statements involving other clients’ inputs; shared statement shares have been addressed.Converting interactive proofs to non-interactive ones may require the stronger random-oracle assumption for hash functions.Source: §4.2.3 · pp. 47–48
  • TEEs can attest to the binary and starting state and provide execution integrity; in FL, enclaves could run secure aggregation or shuffling, but clients must trust the exact code.Remote attestation can also let servers attest to client requirements; software-based and hybrid designs trade hardware requirements for verifiability.Source: §4.2.3 · p. 48

4.3 Protections Against External Malicious Actors

Assuming a trusted server, the section examines how to protect against external actors who may inspect intermediate model iterates or the final model, and identifies open problems in auditing, privacy, concealment, repeated analyses, and model protection.

4.3.1 Auditing the Iterates and Final Model

  • Federated learning’s control over model access and influence could enable tractable attack-susceptibility auditing and adaptive defenses, but reliable auditing remains challenging.Simulated attacks on proxy datasets may not reflect real users; theoretical bounds can be vacuous, while empirical bounds may be intractable. The section calls for methods that quantify average- or worst-case susceptibility and support on-the-fly defenses.Source: §4.3.1 · p. 49

4.3.2 Training with Central Differential Privacy

  • User-level DP clips and aggregates updates, then adds Gaussian noise, but privacy–accuracy trade-offs, variable user contributions, non-uniform sampling, and implementation correctness remain open challenges.The section highlights production-ready, self-accounting privacy guarantees robust to user availability and dropout, and notes that iterates-visible clients may receive weaker guarantees than final-model-only analysts; existing privacy-amplification analysis may not extend beyond convex optimization.Source: §4.3.2 · pp. 49–51

4.3.3 Concealing the Iterates

  • TEEs or MPC could conceal training iterates from clients, but device availability, computation and memory demands, aggregation-before-decryption, and server visibility pose challenges.Concealing iterates from the server as well requires trusted execution or controlled decryption so the final model is released only after training criteria are met.Source: §4.3.3 · pp. 51–52

4.3.4 Repeated Analyses over Evolving Data

  • Repeated releases for evolving data consume privacy budget, so federated methods must balance timely model updates with privacy and accuracy.Open questions include private updates or transfer to similar new datasets, federated extensions of dynamic-database methods, repeated use of synthetic data, and privacy-preserving collection of within-subject time-series data.Source: §4.3.4 · p. 52

4.3.5 Preventing Model Theft and Misuse

  • TEEs can protect deployed model parameters during inference, but MPC may impose latency, bandwidth, and connectivity costs, while prediction APIs can still expose parameters.Prior work shows that an adversary may reconstruct parameters from an inference or prediction API, leaving open how to protect models deployed across millions or billions of devices.Source: §4.3.5 · p. 53

4.4 Protections Against an Adversarial Server

Protecting FL from an adversarial server requires more than private aggregation: client selection, privacy mechanisms, communication constraints, and sparse sub-model updates all create unresolved challenges. The section surveys weaknesses of existing protections and open problems for distributed differential privacy and privacy-preserving sub-model training.

4.4.1 Challenges: Communication Channels, Sybil Attacks, and Selection

  • A malicious server can use Sybil clients or selective cohort formation to control more participants than expected, undermining MPC honesty assumptions and potentially exposing honest clients’ updates.A TEE-based shuffler can also be vulnerable if an honest user’s input is mixed with known inputs from fake users.Source: §4.4.1 · pp. 53–54

4.4.2 Limitations of Existing Solutions

  • LDP can impair utility in high-dimensional data, hybrid DP relies on uncertain trust-preference and data-distribution assumptions, and shuffle privacy degrades with adversarial users and requires a trusted intermediary.Secure aggregation is robust to dropouts and large numbers of parties, but assumes a semi-honest server during key setup, reveals per-round aggregates, handles sparse vectors inefficiently, and cannot enforce well-formed inputs.Source: §4.4.2 · pp. 54–55

4.4.3 Training with Distributed Differential Privacy

  • Distributed DP raises open questions about jointly optimizing privacy, accuracy, and communication, including optimal communication-accuracy trade-offs for shuffled federated optimization and secure aggregation.For secure aggregation, the proposed design problem is to minimize estimation error under per-user communication and aggregate-DP constraints; candidate methods [9] and leave optimality unresolved.Source: §4.4.3 · pp. 55–57
  • Distributed DP also needs privacy accounting suited to discrete noise, dropout-robust noise generation without excess noise, and scalable protocols for intermittently available clients.The section also asks what adversarial capabilities are realistic and whether a long-lived server can be trusted-but-verified.Source: §4.4.3 · pp. 57–58

4.4.4 Preserving Privacy While Training Sub-Models

  • Communication-efficient sub-model training can reveal a client’s sub-model choice; private retrieval paired with sparse secure aggregation is promising, but their joint efficiency and useful sparsity regimes remain open.Local or distributed DP noise added to zero entries can densify sparse updates, leaving open whether privacy can be achieved while preserving sparsity.Source: §4.4.4 · pp. 58–59

4.5 User Perception

The section shifts from formal privacy guarantees to users’ perceptions and needs, highlighting open questions about understanding FL protections, expressing privacy preferences, and governing tradeoffs.

4.5 User Perception

  • FL privacy choices raise open questions about communicating protections and limits, avoiding false confidence, modeling differing preferences, and deciding who controls privacy-utility-security tradeoffs.The section also asks whether users should choose their desired privacy level and discusses “Privacy for the Protected (Only)” [267] as a possible way to permit targeted surveillance for societal priorities.Source: §4.5 · p. 59

4.5.1 Understanding Privacy Needs for Particular Analysis Tasks

  • Task-specific preferences could protect sensitive inferences while permitting less noise, using Pufferfish’s protected predicates and user restrictions applied on-device or during aggregation.This requires understanding users’ needs for a particular task and data collection procedure; further work is needed on technical tools and meaningful preference elicitation.Source: §4.5.1 · pp. 59–60

4.5.2 Behavioral Research to Elicit Privacy Preferences

  • Preference elicitation should follow education that verifies users understand privacy guarantees and FL data sharing, then use informed, representative studies conducted ethically without deception.Studies may use behavioral labs, field experiments, or focus groups; little behavioral work has elicited differential-privacy preferences [6].Source: §4.5.2 · p. 60

4.6 Executive Summary

Privacy in federated learning depends on both what is computed and how computation is executed; combining privacy techniques can provide layered protection, while strong protection against an adversarial server remains open.

Privacy

  • Adapting differential-privacy accounting and privatization to overlapping populations, evolving data, multiple actors, and client availability while preserving utility and privacy budget remains an open challenge.Data minimization and differential privacy address what is computed.Source: §4.6 · p. 60
  • MPC, homomorphic encryption, and TEEs address how computation is executed, but some federation-critical MPC functionalities remain costly, and exploit-immune TEEs and attestation infrastructure remain open challenges.Practical MPC techniques for some federation-critical functionalities have been deployed at scale; connecting attested binaries to specific privacy properties remains immature.Source: §4.6 · p. 60
  • Distributed differential privacy combines “what” and “how” techniques to offer high accuracy and privacy under an honest-but-curious server, a trusted third party, or a trusted execution environment.Privacy in Depth composes techniques so privacy expectations degrade gracefully if one system component is compromised.Source: §4.6 · pp. 60–61

Verifiability

  • Verifiability lets parties prove faithful execution of their computation, using zero-knowledge proofs or TEEs, but strong protection against an adversarial server remains a significant federation challenge.Privacy assessment must consider both the function computed and how computation is executed, including who can see or influence intermediate results.Source: §4.6 · pp. 60–61

5 Defending Against Attacks and Failures

This section examines attacks and failures in federated learning, including threats to model performance, inference-time risks, unreliable clients, and corrupted data. It surveys defenses such as differential privacy and highlights open challenges in detection, robustness, and practical privacy-compatible protection.

5.1.1 Goals and Capabilities of an Adversary

  • The authors propose capability axes for comparing defenses; adversary impact depends on FL context, while untargeted attacks seek global degradation and targeted attacks alter minority cases without sacrificing overall accuracy.Untargeted and targeted goals form a continuum, and a single-client adversary may have very different impact in cross-device and cross-silo settings.Source: §5.1.1 · pp. 63, 65

5.1.2 Model Update Poisoning

  • Byzantine clients can send arbitrary updates that cause suboptimal convergence or divergence, while white-box knowledge can make malicious updates hard to detect.Robust aggregators have theoretical guarantees under assumptions that may not hold in FL, and empirical work found several defenses ineffective against model poisoning. Redundancy-based defenses may require server access to data or global shuffling, conflicting with FL’s goal of reducing communication; collusion can further strengthen attacks and hinder detection.Source: §5.1.2 · pp. 66–67
  • 10% of participating devices compromised sufficed to insert a backdoor despite server anomaly detectors, and poisoned updates could resemble benign ones, making targeted attacks hard to detect [67] [44].A single-shot attack may suffice, and backdoors inserted through federated meta-learning have been shown to persist for tens of training rounds [109].Source: §5.1.2 · p. 67

5.1.3 Data Poisoning Attacks

  • Data poisoning is a constrained special case of model-update poisoning, but its relative strength under FL participation limits is unknown and merits quantification.Data sanitization and network pruning often require client data or representative clean data, and heuristic sanitization defenses can remain vulnerable to adaptive poisoning.Source: §5.1.3 · p. 68

5.1.4 Inference-Time Evasion Attacks

  • Adversarial training can improve white-box evasion robustness, but gains are attack-specific; federated model broadcasting exposes parameters to clients and makes white-box defenses especially relevant.Limited sample reuse, non-IID data, unavailable data for setting perturbation bounds, and client compute and power constraints complicate adaptation. Training-time attacks can also create inference vulnerabilities or weaken adversarial training [115] [214].Source: §5.1.4 · pp. 69–70
  • Detecting backdoors alone may not ensure inference-time robustness, and how training-time attacks affect inference vulnerabilities remains an open question.The paper suggests combining training-time defenses such as robust aggregation or differential privacy with adversarial training, while noting that this area remains speculative and unexplored.Source: §5.1.4 · p. 70, §5.4 · p. 74

5.1.5 Defensive Capabilities from Privacy Guarantees

  • Differential privacy can defend against poisoning by limiting client updates and adding Gaussian noise, but edge-case backdoors can bypass it, and noise and attacker restrictions limit its protection [438].Experiments on DP defenses against targeted attacks remain preliminary [438]; data poisoning defenses have also been explored [326] [199]. The right DP bounds and noise levels for different models and malicious-client fractions remain unclear, and adversarial examples imply edge-case backdoors.Source: §5.1.5 · pp. 70–71

5.2 Non-Malicious Failure Modes

  • Federated systems face client-reporting, data-pipeline, and noisy-update failures; unreliable clients and restricted raw-data access make these harder, motivating dropout mitigation, privacy-preserving debugging, and noise-robust training.For secure aggregation, selecting extra clients, improving protocol efficiency, or developing asynchronous or multi-round variants may reduce dropout problems [81]. Federated differentially private generative models could help debug pipelines, while robust training under small to moderate noise remains open.Source: §5.2 · pp. 71–73

5.3 Exploring the Tension between Privacy and Robustness

  • Secure aggregation obscures individual updates, complicating adversarial defenses; efficient range proofs and compatibility with nonlinear Byzantine-robust aggregation remain open challenges.Approaches approximate the geometric median using several secure-aggregation calls [379], but it remains unclear which aggregators can be computed under secure aggregation.Source: §5.3 · p. 73

5.4 Executive Summary

  • The executive summary flags a tension between robustness and fairness, since long-tail users may be mistaken for noisy updates, and notes that privacy-preserving aggregation can be impractical.Protecting participant identity can conflict with detecting adversaries, while proposed integration techniques may require communication and computation beyond practical limits.Source: §5.4 · p. 74

6 Ensuring Fairness and Addressing Sources of Bias

Federated learning inherits fairness concerns from centralized machine learning while introducing new sources of bias and opportunities to improve representation. The section identifies open problems spanning training data, fairness without sensitive attributes, privacy, robustness, and federated fairness methods.

6.1 Bias in Training Data

  • Under-represented groups can receive worse predictions, while federated client selection based on timing, availability, or computation speed can skew which users’ data influence training.The section also identifies unequal data volumes, latency-dependent participation, lack of device access, and unweighted loss aggregation as potential sources of under-representation or unequal performance.Source: §6.1 · p. 75
  • Bias in user interactions and participation can be difficult to detect in privacy-preserving FL, making bias identification, correction of collected data, and post-hoc fairness correction open research problems.Existing federated work has explored adversarial correction [255]; further methods are needed, including for data shaped by uninclusive product design.Source: §6.1 · pp. 75–76

6.2 Fairness Without Access to Sensitive Attributes

  • When sensitive attributes are unavailable or devices do not map one-to-one to people, measuring and correcting unfairness is difficult; fairness may instead mean equal access to effective models.DRO and multicalibration have not been applied in FL, and their scaling to large n and p, as well as attribute-independent fairness methods, remain open problems.Source: §6.2 · pp. 76–77

6.3 Fairness, Privacy, and Robustness

  • Differential privacy may disproportionately affect minority subgroups [43] [246] [283], while fairness measurement often needs sensitive-group information; jointly achieving privacy, fairness, and robustness remains an open challenge.Personalization or hybrid differential privacy are possible ways to relax the tension, but adaptive privacy mechanisms must preserve privacy guarantees; evaluating fairness without sensitive attributes is also difficult.Source: §6.3 · pp. 77–78

6.4 Leveraging Federation to Improve Model Diversity

  • Federation can combine previously siloed institutional datasets to increase data diversity and potentially improve model generalizability and fairness, but the effects of differential privacy on those gains need empirical and theoretical study.Institutional membership can correlate with sensitive attributes, so pooling data across boundaries may improve representation; research should quantify the interplay among diversity, fairness, privacy, and performance.Source: §6.4 · p. 78

6.5 Federated Fairness: New Opportunities and Challenges

  • FL enables research with sensitive partitioned data, but client sampling can introduce bias, and adapting fairness criteria and centralized methods to decentralized, potentially unrepresentative data remains open.Agnostic federated learning offers an approach to controlling training-objective bias; adapting online fairness methods and determining assumptions for centralized techniques are further challenges.Source: §6.5 · p. 79

6.6 Executive Summary

  • The section’s summary highlights training-data bias, missing sensitive attributes, privacy-fairness tensions, diversity opportunities, and the distinct fairness challenges and solutions that federation presents.These issues are expected to grow in importance as FL deployment expands.Source: §6.6 · pp. 79–80

7 Addressing System Challenges

This section examines operational challenges in federated systems, spanning cross-device availability, capability, and on-device training limits, and cross-silo differences in coordination, data, software, governance, and partitioning. It also identifies open work in bias mitigation, system tuning, and efficient execution.

7.1 Platform Development and Deployment Challenges

  • End-user deployment is difficult because heterogeneous devices resist exhaustive testing and limited, delayed telemetry and lack of interactive access complicate debugging.Rollouts may take weeks or months and require compatibility safeguards; privacy limits on logs and anonymous, aggregate contributions make diagnosis harder.Source: §7.1 · pp. 81–82

7.2 System Induced Bias

  • Variable availability and device capabilities make participation uneven, and system-induced bias arises when participation correlates with data statistics.Unlike platform-independent raw-data bias, these effects can reflect device availability, compute and network limits, and differences in data volume or type.Source: §7.2 · p. 82

7.2.1 Device Availability Profiles

  • Mobile-device availability is governed by hard and soft constraints and tends to follow device-specific daily periodic patterns.Constraints include connectivity and operating-system permission as well as usability conditions such as idleness, charging, and unmetered networks.Source: §7.2.1 · pp. 82–83

7.2.2 Examples of System Induced Bias

  • Round selection and completion can under-represent devices active at fleet-wide availability peaks or troughs and favor faster, better-connected devices with less data.These imbalances can over-represent demographics with high-quality internet or high-end devices and under-represent high-engagement users whose larger datasets take longer to process.Source: §7.2.2 · pp. 83–84

7.2.3 Open Challenges in Quantifying and Mitigating System Induced Bias

  • Systematic quantification of system-induced bias in realistic settings remains an open research direction, with per-device contribution rates proposed as a proxy.Possible remedies include early stopping or smaller models, post-stratification, weighting by contribution history, and long rounds without straggler timeouts.Source: §7.2.3 · p. 84

7.3 System Parameter Tuning

  • System tuning balances model quality against throughput, fairness, privacy, robustness, and resource use, but composite objectives and automated tuning remain unexplored.For example, pace steering dynamically balances device connections, but stochastic availability, weak population prediction, and feedback loops make it difficult to develop.Source: §7.3 · pp. 85–86

7.4 On-Device Runtime

  • On-device runtimes must balance flexibility and efficiency: high-level TensorFlow can be too resource-intensive, while machine-code deployment can delay novel computations.The described TensorFlow-based runtime lacks easy support for alternative front ends, performs poorly on constrained devices, and uses an unstable, nonstandard intermediate representation that creates version-compatibility problems.Source: §7.4 · p. 86
  • No known solution meets on-device runtime needs for compact computation formats, portable open-source implementations, and common ML frameworks; ideally it also supports inference, as limited device training may hinder FL adoption.Source: §7.4 · p. 87

7.5 The Cross-Silo Setting

  • Cross-silo clients are fewer, more powerful, reliable, and addressable, yet differences in data and software plus involvement of many entities can drive coordination costs.Source: §7.5 · p. 87
  • Silos may struggle to normalize differently generated, preprocessed, labeled, and stored data; software-stack differences can be addressed by shared containers or standardized formats and protocols.FATE is given as an example of container-based software delivery, which may not be acceptable to all parties.Source: §7.5 · p. 87
  • Cross-silo data-use approvals may become harder across organizations and jurisdictions; infrastructure could encode access policies and enforce conditions such as minimum aggregation of M clients.Source: §7.5 · p. 87
  • Vertical feature partitioning across silos may require different communication patterns and extra processing for entity alignment and missing features.Source: §7.5 · p. 88

7.6 Executive Summary

  • Frequent large-scale updates, monitoring and debugging, availability bias, conflicting tuning objectives, and limited on-device runtimes remain cross-device system challenges.Source: §7.6 · p. 88

8 Concluding Remarks

Federated learning moves shared-model training onto decentralized clients, and this paper surveys its advances while identifying open problems shaped by practical system constraints and varied privacy assumptions.

8 Concluding Remarks

  • Federated learning trains a shared prediction model on client devices while keeping training data on device, separating machine learning from cloud data storage.Unlike local models that only make predictions on mobile devices, federated learning brings training to the device as well.Source: §8 · p. 89
  • Federated learning has attracted growing interest, with production deployments, startups, and research spanning machine learning, optimization, information theory, statistics, cryptography, fairness, and privacy.The surveyed work describes traction across industry and academia.Source: §8 · p. 89
  • The paper surveys recent advances and open problems, emphasizing practical algorithm efficiency and the distinct challenges posed by a range of privacy threat models.The authors argue that data privacy is not binary and that system constraints create efficiency requirements that may not arise in other settings.Source: §8 · p. 89
  • The open problems are not comprehensive; the paper excludes non-learning tasks on decentralized data, including descriptive statistics and histogram-head computation [510], as well as legal and business issues.The authors note that existing algorithms for such tasks may lack an efficient federated version under the motivating system assumptions or a useful notion of data protection.Source: §8 · p. 89

A Software and Datasets for Federated Learning

Federated learning research needs software that can simulate decentralized, heterogeneous clients and datasets that reflect real-world distributions and partitions. Existing platforms and benchmarks provide useful starting points, but important dataset gaps remain.

Software for simulation

  • Simulation frameworks must handle partitioned data, variable client workloads, communication metrics, time-varying arrivals, and client dropout to accelerate FL research.TensorFlow Federated supports large-scale simulation and flexible orchestration; FedML spans on-device, distributed, and single-machine settings.Source: A Software and Datasets for Federated… · p. 119

Production-oriented software

  • Production-oriented FL platforms must address authentication, communication protocols, encryption, and deployment to physical devices or silos.Examples include FATE, an open-source secure computing framework, and Flower, which supports experimentation on mobile and embedded devices under simulated real-world conditions.Source: A Software and Datasets for Federated… · p. 120

Datasets

  • TensorFlow Federated supports three recommended benchmarks—EMNIST, Stackoverflow, and Shakespeare—with client-level data that reflects heterogeneous federated settings.EMNIST has 671,585 images across 3,400 writer-indexed clients; Stackoverflow has 135,818,730 training examples from over 342,477 users and timestamps for modeling incoming data; Shakespeare has 715 character-clients, with 16,068 training and 2,356 test examples. Stackoverflow is the most realistic cross-device example at scale, while other datasets may be too small for scale-focused questions.Source: A Software and Datasets for Federated… · pp. 120–121

Cross-silo datasets

  • Cross-silo research can use iNaturalist partitions to study transfer and multi-task learning, while Google-Landmark-v2 offers greater scale and diversity.iNaturalist observations can be partitioned by location, author, or organism group; Google-Landmark-v2 contains over 5 million images spanning more than 200 thousand landmark types.Source: A Software and Datasets for Federated… · p. 121
  • A federated street-image dataset from 26 cameras has over 900 annotated images across 7 object categories, but may not represent a challenging realistic scenario.Its annotations include detailed bounding boxes.Source: A Software and Datasets for Federated… · p. 121

The need for more datasets

  • The community needs more representative datasets, including feature-partitioned data; existing open datasets can be split into unbalanced, non-IID clients while retaining metadata.A patient’s pathology and radiology records may be held by separate medical institutes, partitioning sample features across institutions regulated under HIPAA.Source: A Software and Datasets for Federated… · p. 121

Related papers

Loading 1912.04977v3…