PaperlayerView exact revision on arXiv

Source-linked AI summary

Federated Learning: Opportunities and Challenges

Priyanka Mary Mammen

arXiv:2101.05428v1cs.LGcs.DC

TL;DR

Centralized ML training requires aggregating potentially sensitive data, while FL seeks collaborative learning without sharing local data. The survey reviews FL’s applications, architectures, and security challenges, emphasizing that privacy is improved but attacks and deployment constraints remain.

  • Problem

    Centralized aggregation of sensitive data creates security and privacy risks, while FL remains vulnerable to inference and malicious-participant attacks.

  • Method

    The paper surveys FL applications, architectures, security concerns, training challenges, and developments including communication-efficient, incentive-based, and blockchain approaches.

  • Results

    The survey finds that FL offers better privacy guarantees than other ML frameworks but remains prone to several attacks and difficult to defend because of its distributed nature.

  • Takeaways & Limitations

    Future FL research should pursue Byzantine-tolerant models that preserve user privacy while requiring low computational cost.

  • Takeaways & Limitations

    FL faces unresolved constraints from malicious participants, heterogeneous systems and data, aggregator dependence, communication overhead, and incentive mechanisms lacking convergence modeling.

Abstract

from arXiv · show

Federated Learning (FL) is a concept first introduced by Google in 2016, in which multiple devices collaboratively learn a machine learning model without sharing their private data under the supervision of a central server. This offers ample opportunities in critical domains such as healthcare, finance etc, where it is risky to share private user information to other organisations or devices. While FL appears to be a promising Machine Learning (ML) technique to keep the local data private, it is also vulnerable to attacks like other ML models. Given the growing interest in the FL domain, this report discusses the opportunities and challenges in federated learning.

1 INTRODUCTION

Federated Learning addresses privacy risks from centralized ML training by aggregating model parameters rather than local data. The survey reviews its applications, architectures, security challenges, and research directions.

  • Motivation: Centralizing sensitive data for ML training can expose private information, especially in healthcare applications.The paper uses collaborative breast-cancer detection from MRI scans as an example.
  • Federated Learning: Federated Learning shares and aggregates local model parameters while keeping local data on participating devices or organizations.This collaborative approach is presented as an alternative to sharing private patient information with a central server.
  • Opportunities: Introduced by Google in 2016, Federated Learning has potential applications in healthcare, transportation, finance, and smart-home systems.The paper highlights collaborative COVID-19 diagnosis from chest scans as a healthcare example.
  • Challenges: FL challenges include communication overhead, device and data heterogeneity, privacy threats, and attacks by malicious participants.Adversaries may infer training-data membership or manipulate model behavior through targeted or non-targeted attacks.
  • Survey Scope: The survey covers FL applications, security concerns, and recent developments beyond new application domains and security mechanisms.It is motivated by the expanding research activity surrounding FL architectures, incentives, and cloud services.

2 OVERVIEW OF FEDERATED LEARNING

Federated Learning repeatedly selects clients, broadcasts the global model, trains locally, and aggregates returned parameters without centralizing local training data.

  • Client Selection: FL begins with server-side selection or sampling of participating clients from a device pool.Selection may be random or algorithmic.
  • Parameter Broadcasting: The server broadcasts global model parameters to the selected clients before local training.
  • Local Model Training: Selected clients retrain models in parallel using their local data.
  • Model Aggregation: Clients return local model parameters, which the server aggregates toward the global model.
  • Iteration: The four-step process repeats iteratively for n rounds as desired.

3 TYPES OF FEDERATED LEARNING

The paper introduces federated-learning frameworks based on how participating datasets align in feature space and sample instances.

  • Vertical Federated Learning: Vertical Federated Learning combines different features from the same sample instances across devices or organizations.For example, organizations holding different features about the same people can build a shared model.
  • Horizontal Federated Learning: Horizontal Federated Learning combines different sample instances that share the same feature space across devices.

4 APPLICATIONS

FL supports privacy-conscious collaboration across healthcare, transportation, finance, and language applications where centralized data sharing is limited or risky.

  • Healthcare: Healthcare models trained on one hospital’s limited EHR data may be biased, while cross-organization training can improve generalizability without directly sharing sensitive data.
  • Transportation: In vehicular networks, FL enables online training across geographically distributed vehicles for autonomous-driving and traffic-management tasks.The paper connects these applications to dynamic surroundings and the need for substantial data.
  • Finance: Banks can use FL for loan-risk assessment by collaborating with other financial institutions and e-commerce companies without sharing customers’ private information.
  • Natural Language Processing: FL can address privacy bottlenecks in language-model training from mobile and tablet data containing user information.NLP models require large amounts of data for high accuracy.

5 TRAINING BOTTLENECKS

Federated Learning faces training-time bottlenecks from communication overhead and heterogeneity across participating systems and their data.

  • 5.1 Communication Overheads: Communication overhead is a major bottleneck during federated learning training.Existing approaches reduce transmitted information through data compression or by sending only relevant client outputs.
  • 5.2 Systems and Data Heterogeneity: System and data heterogeneity affect the performance of federated learning models.FedAvg addresses heterogeneity but remains insufficiently robust to system heterogeneity, motivating modified aggregation methods.

6 PRIVACY AND SECURITY CONCERNS

Federated Learning remains vulnerable to attacks and privacy inference despite keeping raw user data on local devices. Defenses include secure computation, differential privacy, and trusted execution environments.

  • 6 PRIVACY AND SECURITY CONCERNS: Federated Learning models can be attacked through compromised servers, local devices, or other workflow participants.The paper treats these threats as security concerns affecting federated learning systems.
  • 6 PRIVACY AND SECURITY CONCERNS: Model updates can reveal information about local training data even when raw user data stays on the device.Proposed defenses seek differential privacy guarantees through secure computation, differential privacy schemes, or trusted execution environments.
  • 6 PRIVACY AND SECURITY CONCERNS: Secure Multiparty Computation reveals computed outputs only to selected participants after jointly processing participant inputs.SMC is one of the two secure-computation techniques discussed.
  • 6 PRIVACY AND SECURITY CONCERNS: Homomorphic encryption performs computations on encrypted inputs without decrypting them first.It is presented alongside Secure Multiparty Computation as a secure-computation technique.
  • 6 PRIVACY AND SECURITY CONCERNS: Trusted Execution Environments provide a secure platform for federated learning with lower computational overhead than secure computation techniques.The paper states that the current TEE environment is suitable only for CPU devices.

6.2 Data Poisoning Attacks

Data and model poisoning attacks compromise federated learning by manipulating training data or local model parameters, while defenses reject suspicious updates using error- or loss-based criteria.

  • 6.2 Data Poisoning Attacks: Data poisoning compromises global-model accuracy by injecting poisoned training data into participating devices.The attack may be targeted or non-targeted and can inject data directly or through other devices.
  • 6.2 Data Poisoning Attacks: Defenses identify malicious participants from model updates before model averaging in each learning round.This defense is described for data poisoning attacks and is also applicable to model poisoning attacks.
  • 6.2 Data Poisoning Attacks: Model poisoning modifies local model parameters on compromised devices to introduce errors into the global model.Unlike data poisoning, the adversary targets local models rather than local training data.
  • 6.2 Data Poisoning Attacks: Models can be rejected when they significantly worsen the global error rate or loss function.Some defenses combine error-based and loss-function-based rejection criteria.

6.4 Backdoor Attacks

Backdoor attacks exploit anonymous secure averaging to implant targeted misclassification behavior while preserving overall global-model accuracy.

  • 6.4 Backdoor Attacks: Anonymous model updating can allow compromised devices to introduce backdoor functionality into the global model.Backdoors can mislabel selected tasks without affecting the model’s overall accuracy and are also called targeted attacks.
  • 6.4 Backdoor Attacks: Backdoor attack intensity depends on the proportion of compromised devices and the model capacity.The attack can associate specific characteristics with an attacker-chosen label.
  • 6.4 Backdoor Attacks: Weak differential privacy and norm thresholding are defenses, but secure aggregation makes malicious participants difficult to identify.Participant-level differential privacy can defend against backdoors at the cost of global-model performance; norm thresholding removes updates with boosted parameters.

7 RECENT DEVELOPMENTS IN FL

Recent FL developments target communication overhead, participation incentives, cloud delivery, asynchronous training, and decentralized aggregation through blockchain.

  • 7.1 One-shot federated Learning: One-shot federated learning learns the global model in a single communication round to reduce repeated communication overhead.Distilled one-shot FL further sends fabricated data distilled from each device instead of bulky gradients.
  • 7.2 Incentive Mechanisms: Reputation-based incentive mechanisms reward devices according to model accuracy, data reliability, and contribution to the global model.These approaches do not model convergence or the additional communication overhead they introduce.
  • 7.3 FL as a Cloud Service: Federated learning has been developed as a cloud service so third-party applications can contribute to and collaborate on a machine learning model.The proposed framework is claimed to suit any operating environment.
  • 7.4 Asynchronous FL: Asynchronous FL addresses settings where training and model transfer occur asynchronously because of systems and data heterogeneity.Synchronized aggregation may not be feasible for scaling federated optimization under these conditions.
  • 7.5 Blockchain in FL: Blockchain-based FL enables collaborative learning without a central aggregator because blockchain is a decentralized network.The paper presents a sample architecture for federated learning over blockchain.

8 FINAL REMARKS

Federated learning supports collaborative learning without sharing private data and has applications including healthcare and transportation. However, its distributed structure still leaves it vulnerable to attacks and complicates defense deployment.

  • 8 FINAL REMARKS: Federated learning enables secure collaboration among devices without sharing their private data.Its privacy guarantee is described as better than that of other machine learning frameworks.
  • 8 FINAL REMARKS: Federated learning has been applied in domains such as healthcare and transportation.The paper describes extensive research and growing application of FL across domains.
  • 8 FINAL REMARKS: Federated learning remains vulnerable to several attacks, while its distributed nature makes defense measures harder to deploy.Gaussian noise added for differential privacy can confuse aggregation schemes.
Loading 2101.05428v1…