Source-linked AI summary
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu, Naibo Wang, Yuan Li, Xu Liu, Bingsheng He
TL;DR
Federated learning needs systems and infrastructure for collaborative model training when organizations cannot share raw data and must address effectiveness, efficiency, privacy, and autonomy. The paper surveys FLSs from a systems perspective, defining their components and organizing them across six aspects. It synthesizes existing systems, design factors, case studies, benchmarks, and future research opportunities.
Problem
Organizations hold dispersed data that cannot be directly shared under privacy restrictions, while existing work lacks a comprehensive survey focused on FLS systems and infrastructure.
Method
The paper defines FLSs, analyzes their parties, manager, and computation-communication framework, and categorizes systems across six design aspects.
Results
The survey systematically summarizes existing FLSs and compares their features and designs, including systems such as FATE, PaddleFL, TFF, PySyft, and FedML.
Takeaways & Limitations
FLS design should address effectiveness, efficiency, privacy, autonomy, heterogeneity, and emerging needs such as benchmarks and decentralized platforms.
Takeaways & Limitations
Many existing studies assume a fixed number of parties, despite practical federations requiring dynamic scheduling for party entries and departures.
Abstract
from arXiv · showhide
Federated learning has been a hot research topic in enabling the collaborative training of machine learning models among different organizations under the privacy restrictions. As researchers try to support more machine learning models with different privacy-preserving approaches, there is a requirement in developing systems and infrastructures to ease the development of various federated learning algorithms. Similar to deep learning systems such as PyTorch and TensorFlow that boost the development of deep learning, federated learning systems (FLSs) are equivalently important, and face challenges from various aspects such as effectiveness, efficiency, and privacy. In this survey, we conduct a comprehensive review on federated learning systems. To achieve smooth flow and guide future research, we introduce the definition of federated learning systems and analyze the system components. Moreover, we provide a thorough categorization for federated learning systems according to six different aspects, including data distribution, machine learning model, privacy mechanism, communication architecture, scale of federation and motivation of federation. The categorization can help the design of federated learning systems as shown in our case studies. By systematically summarizing the existing federated learning systems, we present the design factors, case studies, and future research opportunities.
1 Introduction
Federated learning systems address privacy-constrained collaboration across dispersed data, but require reusable infrastructure that balances effectiveness, efficiency, privacy, and autonomy. This survey examines FLSs systematically from a systems perspective and organizes existing work into design dimensions and future directions.
- Motivation: Privacy restrictions prevent organizations such as hospitals from directly sharing dispersed data, limiting the quality of models trained independently.Collaborative training on the union of data could benefit the organizations, but policies and regulations restrict raw-data sharing.
- Existing Systems: Existing efforts support federated training for neural networks, gradient boosted decision trees, logistic regression, and support vector machines.The surveyed systems include a scalable production system enabling tens of millions of devices to train a neural network.
- Motivation: Reusable systems and infrastructures let developers combine common building blocks, including privacy mechanisms, instead of implementing each federated algorithm from scratch.The paper compares this role with that of PyTorch and TensorFlow in deep learning.
- Survey Scope: The survey analyzes FLS definitions and components, categorizes systems across six aspects, summarizes existing studies, and identifies design factors and future research directions.The six aspects are data distribution, machine learning model, privacy mechanism, communication architecture, federation scale, and federation motivation.
- Survey Scope: Prior FL surveys address foundational concepts, mobile and edge networks, or broad FL characteristics, whereas this work focuses on systems and infrastructure.The paper presents its survey as the first comprehensive analysis of FL from a systems perspective.
2 An Overview of Federated Learning Systems
Federated learning systems extend federated computing's cooperation among autonomous, heterogeneous, distributed parties to privacy-preserving model training. Their design depends on system roles, party characteristics, data distributions, communication, and the field's rapid growth.
- Definition: Federated learning enables multiple parties to train machine learning models collaboratively without exchanging their raw data.The system outputs a model for each party, which may be the same or different.
- Definition: A practical FLS should produce a model whose evaluation performance exceeds local training with the same model architecture.The paper gives test accuracy as an example of the evaluation metric.
- Federation Context: FLSs retain federation's cooperation among independent parties while adding distributed-training and privacy-protection challenges.The paper relates FLS design to autonomy, heterogeneity, distribution, and data sharing in conventional federated systems.
- Research Growth: Federated learning's related-paper count reached about 4,400 in the reported year, increasing more rapidly than the compared federated database and cloud areas.The comparison is based on Google Scholar keyword searches for the three research areas.
- System Components: An FLS comprises parties, a manager, and a communication-computation framework for training the machine learning model.Parties act as data owners and beneficiaries and may be organizations or mobile devices in cross-silo or cross-device settings.
- Party Characteristics: Party hardware, scale, stability, and data distribution constrain system design, especially when mobile devices have limited resources or intermittent connections.Large cross-device federations commonly select only a fraction of devices for computation in each round.
- Party Characteristics: Non-IID local data is a practical challenge because parties may hold different distributions across cross-device and cross-silo settings.Such heterogeneity can be particularly evident among organizations with different purposes or populations.
- System Challenges: A fully decentralized FLS remains challenging when sequential decision-tree training must be coordinated with reasonable communication overhead.In the described boosting framework, each party trains trees sequentially and the final model combines all trees.
3 Taxonomy
The taxonomy classifies federated learning systems by data partitioning, model, privacy mechanism, and threat model, while highlighting system-design choices and remaining support gaps.
- Taxonomy dimensions: FLSs are classified by data partitioning, machine learning model, privacy mechanism, communication architecture, federation scale, and federation motivation.These aspects provide common system abstractions and building blocks for FLS design.
- Data Partitioning: Data partitioning is typically horizontal, vertical, or hybrid, based on how data are distributed across sample and feature spaces.Horizontal partitions share features with limited sample overlap, vertical partitions share samples with different features, and hybrid partitions combine both.
- Machine Learning Models: FLSs support neural networks, decision trees, linear models, and ensemble methods, with ensembles allowing heterogeneous local models without parameter averaging.The survey notes a remaining gap in supporting state-of-the-art models beyond frameworks primarily based on stochastic gradient descent.
- Privacy Mechanisms: Privacy mechanisms include cryptographic methods, differential privacy, and trusted execution environments, and an FLS can combine multiple methods.Differential privacy protects individual records through noise but tends to reduce model accuracy.
- Threat Models: Threats can arise at inputs, during learning, or in the learned model, including data poisoning, model inversion, and membership inference attacks.The survey frames threat models as varying with the privacy level and process stage under consideration.
- Motivation of Federation: Fair incentive mechanisms remain challenging because participating parties may be both collaborators and competitors, requiring rewards aligned with contribution.The survey points to incentive designs and game-theoretic equilibrium models as relevant directions.
4 Summary of Existing Studies
This section states that the survey summarizes and compares existing federated learning systems according to the aspects introduced earlier.
- The survey summarizes and compares existing studies on federated learning systems according to the aspects considered in Section 3.
4.1 Methodology
The methodology searches Google Scholar for “Federated Learning,” restricts the corpus to published computer science studies, and defines conventions for comparison tables.
- Methodology: The study searches Google Scholar using the keyword “Federated Learning” and considers only published studies in computer science.
- Methodology: The comparison does not evaluate federation scale or motivation because these aspects are problem dependent.
- Methodology: The study uses NN, DT, LM, CM, and DP as abbreviations for neural networks, decision trees, linear models, cryptographic methods, and differential privacy.
- Methodology: Comparison tables report the models implemented in experiments, while the main-area column identifies each paper’s major research area.
4.2 Individual Studies
The surveyed studies reveal recurring gaps in federated learning systems, especially around data partitioning, privacy guarantees, communication architecture, and support for heterogeneous settings. They also develop optimization and model-specific techniques to improve training under non-IID data and diverse model architectures.
- Key findings: Most existing studies use horizontal data partitioning, while vertical federated learning remains underexplored despite its prevalence between organizations.The survey identifies vertical FL as a way to enable collaboration between diverse parties.
- Key findings: Most studies exchange raw model parameters without privacy guarantees, while differential privacy and cryptographic methods introduce quality, computation, or communication trade-offs.Differential privacy may affect model quality, whereas secure multi-party computation and homomorphic encryption can create substantial overhead.
- Key findings: Centralized communication is the mainstream design, but decentralized architectures remain important where trusted servers are difficult to identify.Removing the central server can increase communication and computation costs because parties share parameters and maintain global models locally.
- SGD-Based: Federated optimization methods include SGD-based and model-specialized algorithms, with FedAvg reducing communication rounds by averaging locally updated model weights.FedAvg performs multiple local SGD steps before averaging local weights into the global model.
- SGD-Based: Non-IID local data can substantially degrade performance, motivating FedProx, SCAFFOLD, and FedNova to limit, correct, or normalize heterogeneous updates.FedProx modifies local objectives, SCAFFOLD corrects local updates, and FedNova normalizes local models during aggregation.
- Model-Specialized: Model-specialized approaches exploit architecture or alternative training arrangements, including PFNM, FedMA, vertical FL methods, and SplitNN.FedMA uses layer-wise matching and can reduce communication size, while SplitNN divides a neural network between participating parties.
Linear/Logistic Regression
Federated learning research covers linear and logistic regression alongside hybrid learning methods, while addressing communication, privacy, security, and reliability challenges.
- Non-SGD federated studies specifically target linear and logistic regression models.
- Horizontal ridge regression systems combine homomorphic encryption with garbled circuits or secure summation to protect privacy.
- Vertical federated regression applies secret sharing or additively homomorphic encryption, with entity resolution used for two-party logistic regression.
- Federated learning has also been combined with multi-task, meta-, reinforcement, transfer learning, and Bayesian optimization.
- Communication remains a central efficiency concern, with structured or sketched updates reducing federated averaging communication costs by two orders of magnitude with slight degradation.
- Model updates can leak training information, motivating differential privacy, secure aggregation, and encrypted noisy updates, while poisoning attacks remain a security concern.
4.2.4 Fairness and Incentive Mechanisms
Federated learning research includes fairness, blockchain-based reliability, and applications in edge computing and recommendation, but fairness and incentives remain comparatively underdeveloped.
- q-FedAvg defines fairness through variance in model performance across parties and optimizes an α-fairness-inspired objective.
- Blockchain-based federated systems can exchange local updates through a network intended to improve stability and reward participating devices.
- Compared with effectiveness, efficiency, and privacy, fewer studies address fairness and incentive mechanisms.
- Communication overhead remains the main efficiency challenge, including both per-iteration size and the choice of communication rounds.
- Differential privacy may significantly affect model quality, while secure multi-party computation may be time-consuming; robust defenses against poisoning are not widely adopted.
- Federated learning has been applied to mobile edge computing, recommender systems, keyboard prediction, and credit-card fraud detection.
Transaction Fraud Detection
Federated learning has been extended to transaction fraud detection through a privacy-preserving meta-learning framework, while benchmarks and realistic data partitioning remain open system challenges.
- A deep K-tuplet network introduces a meta-learning-based federated framework for credit-card transaction fraud detection.
- The framework is reported to guarantee data privacy and achieve significantly higher performance than existing approaches.
- Benchmarks quantitatively characterize federated learning components, but security and privacy benchmarking remains at an early stage.
- Existing benchmark systems do not comprehensively cover all federated algorithms or application types and often lack certain metrics.
- Benchmarking lacks consensus on dataset splitting and non-IID metrics, while realistic partitioning becomes difficult at large scale.
4.3 Open Source Systems
Open-source federated learning systems differ in abstraction level, supported algorithms, privacy mechanisms, and machine-learning frameworks, with FATE offering the broadest algorithm coverage among compared systems.
- The survey introduces five open-source systems: FATE, TFF, PySyft, PaddleFL, and FedML.
- FATE: FATE combines distributed storage and computing, federated algorithms and protocols, workflow, serving, monitoring, and deployment modules.
- TFF: TFF provides high-level FL APIs and lower-level Federated Core APIs built around models, federated computation builders, and datasets.
- PySyft: PySyft supports PyTorch and TensorFlow and offers optional secure multi-party computation and differential privacy mechanisms.
- PaddleFL: PaddleFL separates compile-time strategy and job generation from runtime server, worker, and scheduler components, while vertical algorithms were planned for future integration.
- Comparison: FATE, PaddleFL, and FedML expose algorithm-level APIs, whereas TFF and PySyft provide finer-grained building blocks for implementing federated processes.
- Comparison: Among compared systems, FATE supports the most machine-learning models across both horizontal and vertical settings, while PySyft and PaddleFL provide more privacy mechanisms.
5 System Design
FLS design must jointly address effectiveness, privacy, efficiency, autonomy, and robustness across algorithm choice, system operation, and evaluation. The paper provides a design reference that links federation characteristics to algorithms, privacy mechanisms, incentives, and evaluation criteria.
- Design factors: Effectiveness depends on matching the FL algorithm to the parties’ data partitioning and the machine learning model.For example, FedAvg and SimFL fit same-feature or same-sample settings for neural networks and trees, respectively, while FedBCD and SecureBoost address complementary partitions.
- Design factors: Privacy design depends on the manager’s reliability and may require mechanisms such as differential privacy when inference attacks are possible.The FL framework does not exchange raw data, but model parameters can still expose training information.
- Design factors: Efficiency is constrained by multi-round computation and communication, with hardware acceleration and compression addressing the respective bottlenecks.Training time, time per round, or rounds to reach a target performance can be used for comparison.
- Design factors: Autonomy requires FLSs to tolerate unreliable parties, including device dropouts in large cross-device federations.Robustness can be evaluated by the maximum number of disconnected parties the system tolerates.
- The Design Reference: A design process starts by identifying participating entities and the task, then selects the data partition, model, FL algorithm, privacy mechanism, and incentives.The paper states that differential privacy is preferred over secure multiparty computation when efficiency matters more than model performance.
- Evaluation: FLS evaluation should cover model performance, system security, system efficiency, and system robustness.Model evaluation may target a global model or personalized local models, depending on the FL objective.
6 Case Study
The case studies show that FLS requirements vary with the application’s entities, data partitioning, resources, communication structure, privacy risks, and participation incentives. Healthcare and finance especially require privacy-preserving training over vertically or hybrid-partitioned data.
- Mobile Prediction: Mobile prediction services use centralized, cross-device FLSs for horizontally partitioned data distributed across millions of devices.Limited device computation and bandwidth, together with users joining or leaving at any time, make efficiency and robustness important.
- Mobile Prediction: Additional privacy mechanisms such as differential privacy may be needed for mobile prediction because the basic FL framework remains vulnerable to inference attacks.Secure multiparty computation may be unsuitable for weak devices because of expensive encryption operations.
- Healthcare: Healthcare federations combine horizontal and vertical data partitioning because parties hold different residents’ data and diverse feature sets.The setting typically involves few parties with substantial computational resources and may require private, decentralized, robust FLSs.
- Healthcare: Healthcare collaboration is largely motivated by regulations, while privacy can be strengthened with secure multiparty computation and differential privacy.The absence of a central server and the possibility of dishonest or malicious parties complicate system design.
- Finance: Financial participants may use theoretically proven privacy mechanisms when a trusted third party is absent or not trusted.Banks and insurers seek cooperation without revealing customers’ information.
- Finance: Financial federations often use vertically partitioned data linked by user ID and require privacy-preserving record linkage before vertical federated training.Training linked data without sharing original records remains a challenge in cross-silo, decentralized settings.
7 Vision
The paper identifies heterogeneity, system architecture, model markets, benchmarks, and domain constraints as future directions for practical FLS development. These directions reflect changing participants, broader system abstractions, evaluation needs, and resource or regulatory boundaries.
- Heterogeneity: FLSs must accommodate heterogeneous parties that differ in accessibility, privacy requirements, contributions, and reliability.Dynamic scheduling is needed because parties may join or leave during learning, although increasing party populations remain understudied.
- System Development: Future architectures should support multiple aggregation methods and learning algorithms because model averaging is limited when parties train heterogeneous models.The paper proposes investigating system-level abstractions analogous to parameter servers in deep learning.
- System Development: Model markets could let parties store, share, buy, sell, and locally aggregate models with task information, increasing federation flexibility.The paper also emphasizes that model evaluation and incentive mechanisms are important in such systems.
- Benchmark: A widely used benchmark with representative datasets and workloads is needed to evaluate existing FLSs and direct future development.Several benchmarks exist, but none has been widely used in experiments.
- FL in Domains: IoT and edge deployments must address computation and energy constraints because privacy and security mechanisms introduce runtime overhead.The paper also identifies uncertainty about how FL complies with existing regulations such as GDPR.
8 Conclusion
The survey synthesizes FLS design around heterogeneity, autonomy, and six categorization dimensions, then identifies system and evaluation opportunities. It frames FLSs as a research direction spanning machine learning, systems, and data privacy.
- Conclusion: Heterogeneity and autonomy are important factors in designing practical FLSs.The survey uses these factors alongside six aspects to organize existing systems and their designs.
- Conclusion: The six-aspect categorization supports comparison of existing FLS features and designs.The paper also highlights opportunities including more benchmarks and blockchain integration.
- Conclusion: Future FLS development calls for contributions from machine learning, systems, and data privacy communities.The conclusion presents FLSs as an active research direction.