Source-linked AI summary
Achieving Security and Privacy in Federated Learning Systems: Survey, Research Challenges and Future Directions
Alberto Blanco-Justicia, Josep Domingo-Ferrer, Sergio Martínez, David Sánchez, Adrian Flanagan, Kuan Eeik Tan
TL;DR
Federated learning avoids sharing clients’ raw data but remains vulnerable to privacy leakage and malicious updates that undermine model security. This paper surveys and evaluates mitigation methods, then examines the difficulty of combining privacy with security and sketches possible solutions. Its conclusions report effective security countermeasures, with Krum offering the best performance among the tested methods, while current approaches remain constrained by incompatibilities and scalability concerns.
Problem
Federated learning faces privacy attacks from client updates and security attacks that can poison the model or prevent convergence.
Method
The paper surveys and critically reviews security and privacy mitigations, empirically evaluates security countermeasures, and analyzes simultaneous protection.
Results
Krum offered the best performance among the discussed and tested security countermeasures, while countermeasures were effective against security attacks.
Takeaways & Limitations
Achieving security, privacy, and accuracy together requires addressing the limitations of current privacy-preserving methods and breaking the link between client privacy and malicious-client detection.
Takeaways & Limitations
Current security countermeasures can be incompatible with privacy protection, and the approaches face scalability issues.
Abstract
from arXiv · showhide
Federated learning (FL) allows a server to learn a machine learning (ML) model across multiple decentralized clients that privately store their own training data. In contrast with centralized ML approaches, FL saves computation to the server and does not require the clients to outsource their private data to the server. However, FL is not free of issues. On the one hand, the model updates sent by the clients at each training epoch might leak information on the clients' private data. On the other hand, the model learnt by the server may be subjected to attacks by malicious clients; these security attacks might poison the model or prevent it from converging. In this paper, we first examine security and privacy attacks to FL and critically survey solutions proposed in the literature to mitigate each attack. Afterwards, we discuss the difficulty of simultaneously achieving security and privacy protection. Finally, we sketch ways to tackle this open problem and attain both security and privacy.
1. Introduction
Federated learning trains models across decentralized clients without sharing raw private data, but its updates and aggregated model remain exposed to privacy and security attacks. This survey reviews mitigations, evaluates security-enhancing methods, and examines the open challenge of achieving both protections simultaneously.
- Federated learning keeps clients’ raw private data on their devices while the server aggregates local model updates.This shifts computation toward client devices and avoids centralized data collection.
- Security attacks can prevent convergence through Byzantine behavior or drive convergence toward an incorrect model through poisoning.
- Model updates can leak private attributes, reveal training-set membership, or enable reconstruction of sampled data points.These privacy attacks may be orchestrated by malicious servers or participating clients.
- The paper critically surveys attack mitigations and empirically evaluates and compares security-enhancing methods.
- Achieving security on the server side and privacy on the client side simultaneously remains an open problem because current solutions have limitations.The paper identifies research challenges and sketches ways to attain both protections.
2. Federated learning
Federated learning coordinates decentralized client training through a central server that distributes a model, receives local updates, and aggregates them iteratively. The framework keeps client data local while relying on client computation and stochastic-gradient-based training.
- Federated learning typically uses stochastic gradient descent, with training hyperparameters such as epochs and learning rate.The paper situates SGD as a core component of many learning algorithms.
- Each client trains a local model from the global model using its locally stored data, while the server updates the global model by averaging client updates.
- Client participation is commonly randomized or occurs implicitly through devices that are not always online.
- Gradient clipping may precede local-contribution collection to bound possible changes to model parameters.
3. Security attacks and countermeasures
The section surveys how malicious clients can manipulate federated learning aggregation to poison models or prevent convergence, then evaluates countermeasures against these attacks. Experiments identify Krum and global model-accuracy detection as especially effective, while highlighting a fundamental tension: security defenses requiring individual updates can conflict with privacy protection.
- Security attacks: A single malicious client can exploit weighted-sum aggregation to induce an arbitrary aggregate update and potentially replace the global model.This vulnerability underlies both Byzantine attacks, which prevent convergence, and model-poisoning attacks, which induce targeted misclassification.
- Model poisoning attacks: A single, highly constrained malicious client can conduct poisoning attacks, while explicit boosting can introduce a backdoor but becomes effective only when its factor reaches the number of participants.Stealthy boosting instead makes the malicious update resemble a benign one in validation loss and accuracy, complicating anomaly detection.
- Empirical analysis: For coalitions up to 20% of clients, distance-based outlier detection performs worst, whereas median aggregation is low-complexity and performs similarly to model-accuracy detection and Krum.For the 40% coalition case, Krum offers the best results among the reported countermeasures.
- Conclusions on security attacks: On very large client communities, a single malicious client has little to no effect on global model metrics, but attacks succeed when at least 10% of clients collude, even with 1% participation per epoch.These findings may not generalize to tasks with many more labels than MNIST’s 10 labels.
- Conclusions on security attacks: Global model-accuracy detection and Krum provide the best protection, but accuracy detection requires test data and is computationally expensive.Krum is emphasized for effective malicious-update discarding, low complexity, and easy implementation.
- Security–privacy tension: All evaluated countermeasures require the server to access individual client updates, making them incompatible with privacy protection based on update aggregation.This requirement restricts the ability to defend clients against privacy attacks while applying these security mechanisms.
4. Privacy attacks and defenses
The paper surveys privacy attacks in federated learning, where model updates can reveal private records or properties of clients’ training data. It examines passive and active adversaries, including servers and clients, and reviews defenses against these inference risks.
- Privacy threats: Federated learning updates can reveal sensitive information about clients’ private data, including through gradients and successive model parameters.The paper connects gradient observations to private input features and describes adversaries using model updates or global parameters to infer client data.
- Privacy threats: Privacy attacks include membership inference, which detects whether a record appeared in training, and property inference, which identifies characteristics of a class or dataset.The survey treats these as two fundamental and related categories of privacy attack.
- Gradient leakage: Embedding gradients can expose words in text batches because only embeddings for appearing words receive non-zero updates.The embedding matrix is collaboratively optimized, and gradients for words absent from a batch are zero.
- Adversaries: Passive attacks infer properties from observed learning behavior, whereas active adversaries alter training updates to make the target model expose information about training data.Active attacks can involve a server or curious client crafting parameter updates and adding an auxiliary property classifier.
- GAN-based attacks: GAN-based attacks can infer targeted training-set information, and record-level differential privacy does not protect against the cited GAN-based attack.The survey reports that GANs can generate samples related to the targeted training set and characterizes record-level DP as ineffective against this attack.
- Defenses: The surveyed defenses include local update aggregation, differential privacy, and anonymization, but differential privacy can severely reduce model accuracy.The paper also notes that properly applied differential privacy offers a strong privacy guarantee.
5. Simultaneously achieving privacy and security in federated learning
The paper argues that jointly protecting client privacy and server-side model security while retaining low computation and accuracy is difficult. It proposes anonymous peer-to-peer forwarding to unlink clients from updates while preserving server access for malicious-update detection.
- Challenge: Achieving privacy, security, accuracy, and low computational overhead simultaneously remains a major challenge in federated learning.Heavy secure aggregation adds overhead and hides individual updates, while differential privacy preserves update visibility but can severely degrade accuracy.
- Proposed direction: Breaking the link between updates and their generators through anonymous communications is proposed as a privacy mechanism compatible with server-side security checks.Unlike encrypted aggregation, this approach aims to preserve access to individual updates while preventing the server from identifying their originators.
- Peer-to-peer forwarding: The single-hop design protects originators from the server but does not protect them from the forwarding client that directly receives their update.The multi-hop extension is introduced to address this limitation.
- Peer-to-peer forwarding: Multi-hop forwarding prevents both the server and forwardees from knowing whether they received an update from its originator or another forwardee.Successive updates from the same client are described as unlinkable under the multi-hop design.
- Incentives and security: Reputation mechanisms can incentivize clients to forward updates, support co-utility, and punish clients that submit malicious updates.Clients with good reputations gain greater chances that others will submit their updates, while privacy-preserving reward and punishment protocols remain necessary.
- Trade-offs: Peer-to-peer forwarding introduces communication overhead but has significantly lower computational cost than cryptographic secure aggregation based on SMC.The proposed solution also retains model accuracy because updates are neither perturbed nor aggregated, allowing the server to compute update statistics.
6. Conclusions
The conclusions identify Byzantine and poisoning attacks as persistent federated-learning vulnerabilities and report that tested countermeasures can thwart large malicious coalitions. They propose anonymous peer-to-peer update forwarding to combine privacy, security, and accuracy, while leaving protocol, incentive, and privacy-preserving reward mechanisms as implementation challenges.
- Security attacks: Countermeasures were effective against security attacks conducted by large coalitions of malicious clients.The conclusion reports experiments showing that countermeasures thwarted attacks by large malicious coalitions.
- Countermeasures: Krum offered the best tested results for discarding malicious updates while maintaining low overhead.The conclusion identifies Krum as the strongest among the discussed and tested countermeasures on these criteria.
- Privacy-security trade-off: Cryptographic aggregation can prevent detection of malicious clients, while update distortion can degrade global-model accuracy to unacceptable levels.The conclusion presents this as a central limitation of privacy-preserving methods based on cryptographic aggregation.
- Future direction: Anonymous peer-to-peer communication is proposed to break the client-update link while enabling malicious-client detection and preserving model accuracy.The proposed direction targets security, privacy, and accuracy together through decentralized update forwarding.
- Open challenges: Practical deployment requires update-forwarding protocols, peer incentives, and privacy-preserving mechanisms to reward good updates and punish bad ones.The conclusion lists these as open challenges for bringing the proposed solution into practice.