PaperlayerView exact revision on arXiv

Source-linked AI summary

Survey on Federated Learning Threats: concepts, taxonomy on attacks and defences, experimental study and challenges

Nuria Rodríguez-Barroso, Daniel Jiménez López, M. Victoria Luzón, Francisco Herrera, Eugenio Martínez-Cámara

arXiv:2201.08135v1cs.CRcs.AIcs.LG

TL;DR

Federated learning seeks privacy-preserving collaborative learning, but inaccessible distributed data complicates protection against attacks on model integrity and data privacy. This paper surveys those threats and defences, builds taxonomies, compares attacks and defences experimentally, and derives selection guidelines; it concludes that FL threats remain an ongoing research field with unresolved challenges.

  • Problem

    Federated learning remains vulnerable to attacks on model integrity and data privacy, while inaccessible data makes protection and common evaluation difficult.

  • Method

    The paper reviews FL threats and defences, develops taxonomies across FL categories, conducts an experimental comparison, and proposes defence-selection guidelines.

  • Results

    The study concludes that FL threats remain an ongoing research field with additional vulnerabilities and defence mechanisms still needing investigation.

  • Takeaways & Limitations

    Ensuring FL is a robust, privacy-safeguarding paradigm requires continued work on threats and mechanisms for resilience against them.

  • Takeaways & Limitations

    The field still contains many vulnerabilities that must be addressed before a truly secure and privacy-preserving learning environment is ensured.

Abstract

from arXiv · show

Federated learning is a machine learning paradigm that emerges as a solution to the privacy-preservation demands in artificial intelligence. As machine learning, federated learning is threatened by adversarial attacks against the integrity of the learning model and the privacy of data via a distributed approach to tackle local and global learning. This weak point is exacerbated by the inaccessibility of data in federated learning, which makes harder the protection against adversarial attacks and evidences the need to furtherance the research on defence methods to make federated learning a real solution for safeguarding data privacy. In this paper, we present an extensive review of the threats of federated learning, as well as as their corresponding countermeasures, attacks versus defences. This survey provides a taxonomy of adversarial attacks and a taxonomy of defence methods that depict a general picture of this vulnerability of federated learning and how to overcome it. Likewise, we expound guidelines for selecting the most adequate defence method according to the category of the adversarial attack. Besides, we carry out an extensive experimental study from which we draw further conclusions about the behaviour of attacks and defences and the guidelines for selecting the most adequate defence method according to the category of the adversarial attack. This study is finished leading to meditated learned lessons and challenges.

1. Introduction

Federated learning addresses privacy and distributed-data challenges by collaboratively training a global model without sharing local data, but remains vulnerable to attacks on model integrity and data privacy. This survey responds with broad taxonomies, comparative experiments, defence-selection guidelines, and lessons about unresolved challenges.

  • Motivation and federated learning: Federated learning collaboratively trains a global model without sharing data owned by distributed sources.A coordinator often manages information exchange during training.
  • Motivation and federated learning: Inaccessible training data prevents inspection-based detection of maliciously altered data, leaving FL exposed to integrity and privacy attacks.These vulnerabilities arise within FL’s distributed learning setting.
  • Research gap: Previous surveys often covered either model attacks or privacy attacks, lacked common experimental comparisons, and focused mainly on horizontal FL.The stated gaps motivate a broader survey scope.
  • Contributions: The paper reviews threats and defences across horizontal, vertical, and federated transfer learning and develops taxonomies for both.These taxonomies aim to provide a general picture of attacks and defensive countermeasures.
  • Contributions: The study compares prominent attacks and defences in a common evaluation framework and derives guidelines for selecting defence categories.It also reports lessons learned and challenges from the literature review and experiments.
  • Paper organization: The paper organizes its review into background concepts, attack and defence taxonomies, experiments, defence guidelines, lessons learned, and challenges.The experimental study appears in Section 5, followed by application guidelines and discussion sections.

2. Background concepts on Federated Learning threats

Federated learning distributes data across clients that collaboratively train and apply a model, with categories determined by overlaps in feature, label, and sample spaces. Its threat model includes privacy mechanisms and attacker knowledge, while non-IID data and scalable secure computation remain important constraints.

  • Federated learning operation: Federated learning separates model training, where clients exchange non-data information, from inference, where they jointly apply the trained model.Both processes may be synchronous or asynchronous.
  • Federated learning categories: Horizontal FL shares feature and label spaces but not sample space, whereas vertical FL shares sample space but neither feature nor label space.The corresponding formal conditions are given for each category.
  • Federated learning categories: Federated transfer learning shares neither feature, label, nor sample space, but transfers knowledge through partial overlap or similarity.The paper describes FTL as more difficult than HFL and VFL because privacy-preserving techniques are required.
  • Data and participants: In most HFL scenarios, client-held data is non-IID, and its distribution is unknown even when IID data might otherwise be present.The paper identifies this non-IID setting as a real challenge.
  • Data and participants: Clients act as data owners and distributed participants, while a coordinator orchestrates communication and may combine partial updates during training or inference.Clients can include servers, smartphones, IoT devices, hospitals, banks, and other organizations.
  • Privacy and threat models: Differential privacy adds calibrated random noise and measures privacy cost through ϵ or (ϵ, δ), while assuming the adversary has arbitrary external knowledge.Approximate DP is identified by the (ϵ, δ)-DP property.
  • Privacy and threat models: Outsider attackers come from outside the learning system, whereas insider attackers are participating clients or servers; attacker knowledge can also range from white-box to black-box.Knowledge categories may overlap, and complex attacks can require information from multiple participants.

3. Adversarial Attacks in Federated Learning: Taxonomies

Federated-learning attacks are difficult to taxonomize because they vary in objectives, targets, and execution methods. The paper first separates attacks on the federated model from privacy attacks, then classifies model attacks across four criteria.

  • Taxonomy overview: Adversarial attacks in FL are challenging because their variety of objectives and methods makes a common taxonomy difficult to establish.The distributed nature of FL increases exposure to diverse attacks.
  • Taxonomy overview: The first broad division distinguishes attacks that modify federated-model behaviour from privacy attacks that infer sensitive information.This two-category division is represented in Figure 2.
  • Taxonomy overview: The survey separately examines attacks to the federated model and privacy attacks after establishing the initial two-category classification.The categories are discussed in Sections 3.1 and 3.2.

3.1. Adversarial attacks to the federated model

Federated-model attacks exploit clients’ ability to send poisoned updates while the server cannot inspect their training data. The survey organizes these attacks by timing, objective, poisoned FL component, and frequency, with detailed treatment of backdoor strategies and empirical attack behavior.

  • Motivation: Federated-model attacks are challenging because clients can send poisoned updates while the server cannot inspect their stored training data.This limitation makes attacks against the federated model a significant FL challenge.
  • Attack taxonomy: The taxonomy classifies attacks by attack moment, objective, poisoned part of the FL scheme, and frequency.The four linked taxonomies cover when attacks occur, what they seek, what information they poison, and how often they act.
  • Attack timing: Training-time attacks can modify the still-training federated model or infer information, whereas inference-time attacks generally seek wrong predictions or model information.Training-time attacks may be continuous or single attacks; inference-time attacks occur after training.
  • Attack objectives: Backdoor attacks preserve original-task performance while adding a secondary task, making them stealthy and difficult to detect.The survey distinguishes input-instance-key strategies, which target specific examples, from pattern-key strategies, which associate a pattern with a target label.
  • Backdoor taxonomy: Backdoor taxonomies further classify injected patterns by design strategy and number of patterns, including blended, accessory, blended-accessory, single-pattern, and multi-backdoor attacks.Blended injection requires modifying the entire sample during training and testing, which may be infeasible.
  • Empirical findings: Backdoor attacks remain a real FL threat: tail-distribution poisoning can compromise the global model, and feature-partitioned attacks can work without training labels.Some naive backdoor approaches are mitigated by simple defences, but other results demonstrate substantial attack potential.
  • Attack behavior: Attack success increases faster with the number of poisoned samples than with the number of attackers, although more attackers can improve success without increasing total poisoned samples.Boosting techniques also depend on knowing the number of participating clients, a restrictive client-side knowledge condition often assumed in experiments.

3.2. Privacy attacks

Privacy attacks in federated learning target disclosure of participant information through a broad attack surface, and are organized by the attacker’s objective. The section covers feature, membership, and property inference attacks across federated settings.

  • Privacy attacks seek information about participants or their training data, despite federated learning’s privacy-preserving design.
  • Feature inference attacks: Feature inference attacks reconstruct client datasets, including images, text, or private features, from exchanged federated information.
  • Feature inference attacks: Gradient-based feature inference exploits shared client gradients, while parameter-based attacks exploit shared local model parameters.
  • Membership inference attacks: Membership inference attacks determine whether particular data was used to train a victim model, using client-side or outsider-side knowledge.Server-side knowledge can be more powerful than client-side knowledge in some proposed attacks.
  • Property inference attacks: Property inference attacks recover whether an individual or participant population has a sensitive property unrelated to the main learning task.Population-distribution attacks infer feature proportions, whereas individual-distribution attacks target a property of one client.

4. Defence methods against adversarial attacks: Taxonomy

The defence taxonomy groups countermeasures by where they are implemented—client, server, or communication channel—because individual defences may address multiple attack categories.

  • Defences are categorized into client, server, and communication-channel groups rather than being grouped solely by the attacks they mitigate.The taxonomy also identifies which attack types each defence can address.

4.1. Server defences

Server defences rely on the server’s usual trusted and accessible role, using robust aggregation, anomaly detection, differential privacy, learning-rate modification, and preprocessing methods.

  • Most server defences assume a reliable server because it is controlled and accessible, unlike independent and inaccessible clients.
  • Robust aggregation operators: Robust aggregation replaces or modifies mean-based aggregation to reduce the influence of extreme or malicious client updates.Examples include median, trimmed-mean, geometric-mean, norm thresholding, Krum, Multikrum, and Bulyan.
  • Server defences: AFA weights clients using cosine similarity and a Hidden Markov model, while Sageflow combines staleness-aware grouping, entropy filtering, and loss-weighted averaging.AFA is reported to discard poor and malicious clients; Sageflow addresses stragglers and adversaries together.
  • Server defences: The game-theory aggregation approach is limited to IID training-data distributions.
  • Anomaly detection: Anomaly-based defences identify suspicious clients from local-update distributions, but dimensionality reduction can lose information.PCA and spectral embeddings reduce dimensionality before detection, with information loss as a stated limitation.
  • Based on Differential Privacy: Differentially private aggregation clips update norms and adds calibrated Gaussian noise, but the noise required for guarantees can significantly reduce federated-task accuracy.The section also notes that data imbalance can worsen differential privacy’s performance effects.
  • Modification of the learning rate: Robust Learning Rate adjusts the server learning rate per dimension according to client agreement on update directions and can be combined with differential privacy.
  • Less is more: Weight-truncation preprocessing addresses unbalanced data and seeks a balance between model performance and Byzantine robustness.The choice of truncation parameters remains a disadvantage, although selection procedures are proposed.

4.2. Client defences

Client defences are needed when the server is not trusted, generally protecting against server-side privacy attacks through local differential privacy, perturbation, training adjustments, and adversarial training.

  • Client-side defences remove the trusted-server assumption and require at least a portion of clients to be benign.They protect individual clients rather than clients collectively through server-side processing.
  • Based on Differential Privacy: Local differential privacy protects client information and shared gradients, while relaxed variants seek tighter privacy-budget usage.The section also reports attacks against some Local DP protocols and performance reductions associated with attack-effectiveness reductions.
  • Based on Differential Privacy: In vertical federated learning, perturbing intermediate outputs can provide privacy while removing the need for a coordinator and costly homomorphic encryption.The surveyed work is described as limited to one publication in this area.
  • Non-differential-privacy defences add noise or perturb vulnerable model components, including shared parameters, datasets, or selected layers.Examples include gradient compression, multitask-based perturbation, and random sketching.
  • Client training can be optimized by fine-tuning benign clients or perturbing parameter subspaces affected by persistent model-poisoning attacks.FL-WBC perturbs the parameter space where long-lasting attack effects reside.
  • Adversarial training: Adversarial training transfers robustness into federated learning, while pivotal training aims to make predictions independent of sensitive attributes.

4.3. Communication channel defences

Communication-channel defences use secure computation to protect shared FL updates and predictions, but full-procedure protection is limited by party-dependent complexity. Secure aggregation is therefore favored, often with differential privacy or complementary approaches.

  • SFL preserves input privacy while potentially revealing the global computation output, so output privacy is not assured for all parties.
  • SMPC protocols provide a simulation-based security framework for Secure FL and use Homomorphic Encryption as a key component.The paper describes Homomorphic Encryption as a building block of SMPC protocols.
  • VFL uses SMPC for private entity alignment, shared partial updates and predictions, and computation of final updates and predictions.
  • SMPC complexity grows with the number of parties, reducing the feasibility of applying full-fledged SMPC throughout federated training.The paper reports a shift toward protocols focused on FL communication steps, especially aggregation.
  • Secure aggregation is a key HFL communication step where SMPC can provide security and efficiency.Bonawitz et al. developed an efficient and robust SMPC aggregation protocol and later studied its parameter selection.
  • Differential privacy can be added to SFL protocols and secure aggregation schemes to provide additional protection against adversarial and privacy attacks.
  • Blockchain-based FL offers decentralization, no single point of failure risks, and improved scalability, but inherits blockchain security issues and requires user incentives.

5. Experimental study

The experimental study finds that untargeted and targeted attacks can substantially compromise federated models, while defences reduce attack effects but may trade off task performance and depend strongly on configuration. Privacy attacks show varied effectiveness: membership inference is weak in the tested few-client setting, whereas feature inference can recover private information and remains difficult to block completely.

  • Untargeted attacks: Random-weights model-poisoning was the most damaging untargeted attack, reducing the federated model to most-frequent-label behaviour.Among data-poisoning attacks, label flipping was slightly more effective than out-of-distribution poisoning.
  • Untargeted attacks: A single adversarial client, 1-out-of-30, was more effective than multiple adversarial clients because the boosting factor was divided among attackers.Distributing the attack across several clients weakened each client’s contribution.
  • Untargeted attacks: MultiKrum (20) and Bulyan (f=5) generally mitigated attacks best, but simpler median and trimmed-mean defences sometimes outperformed them.Defence effectiveness depended substantially on input-parameter values, creating a practical obstacle to deployment.
  • Targeted attacks: Input-instance-key backdoors were more accurate on the backdoor task than the original task, harmed original-task accuracy, and were not completely dissipated by defences.Every defence improved both original-task accuracy and backdoor-task accuracy relative to the undefended setting, while parameter-dependent methods had unknown optimal settings.
  • Targeted attacks: Pattern-key backdoors achieved almost 100% backdoor-task success while usually preserving the original task, but defences reduced the backdoor at the cost of original-task performance.The resulting defence choice is a trade-off between attack mitigation and preserving the original model task.
  • Privacy attacks: Membership inference was barely effective, with attacker accuracy and AUC close to 0.5, whereas differential privacy stopped membership leakage but considerably reduced federated-task accuracy.Transfer learning might reduce the impact of differential privacy on accuracy in the tested setting.
  • Privacy attacks: Feature-inference attacks often extracted private information from gradients; Gaussian noise stopped several attacks, but gradient compression failed completely against Geiping et al. in some cases.Geiping et al.’s attack was independent of local training stage, while Zhu and Han’s and Wei et al.’s attacks became harder as the model was more trained.

6. Guidelines for the application of defences against adversarial attacks

The survey organizes defence categories by whether they completely, partially, or unsuccessfully address model and privacy attacks, while noting that coverage differs across attack types. DP-based defences partially protect against model attacks, whereas property inference lacks a complete defence.

  • The framework helps choose defence categories because most defences target particular attacks but may also affect other attack types.The survey presents the relationships as category-level guidance rather than claiming that every defence in a category has identical performance.
  • The guidelines map defence categories to model and privacy attack categories using complete, partial, no, and unknown coverage.Complete defence stops attacks; partial defence significantly reduces attack performance without stopping it.
  • DP-based defences designed for privacy attacks partially defend against model attacks, especially model attacks based on DP, but model defences do not reciprocate.
  • Defences against model attacks are generally more settled than privacy-attack defences, and no complete defence is identified for property inference attacks.
  • The survey concludes that no defence currently prevents all federated-learning attack types simultaneously.

7. Lessons learned

The survey finds that federated-learning threats and defences remain an expanding research area, with model attacks generally more effective than privacy attacks. Privacy attacks often require difficult-to-achieve settings, while defences face coverage, computational, and privacy–performance trade-offs.

  • Research on federated-learning vulnerabilities and defences is continuing to grow and is expected to remain active in coming years.
  • Model attacks are easier to defend against than privacy attacks but are more effective, with even simple attacks harming the federated model.
  • Privacy attacks generally require unusual settings and resources that are difficult to achieve in real federated-learning scenarios.The cited constraints include data, raw computing power, and time.
  • Inference-attack defences often reduce model-attack performance, whereas model-attack defences do not show the reverse effect; DP-inspired mechanisms are therefore highlighted for generic attacks.
  • DP and perturbation defences require fine-tuning and computational resources, while the field lacks consensus on measuring privacy–performance trade-offs.These constraints may make some defences unsuitable for low-power federated-learning devices.

8. Challenges of addressing federated learning threats

The survey identifies unresolved challenges in matching defences to the full attack surface, balancing protection with model performance, handling non-IID clients, and extending analysis beyond horizontal federated learning. It also treats privacy and security as cross-cutting requirements for emerging federated-learning variants.

  • Defences vs. attacks: A major challenge is identifying all exploitable federated-learning vulnerabilities and developing a defence effective against any attack.The survey describes this goal as still far from being achieved because attacks and defences expose different perspectives on the problem.
  • Trade-off in defences: Defences must balance attack prevention against preserving performance on the original task, especially when differential privacy requires substantial noise.The survey calls for more efficient DP methods and broader DP-based protection.
  • Non-IID assumptions: Non-IID client data makes it difficult to distinguish adversarial clients from clients whose valuable updates reflect highly skewed distributions.Existing approaches include anomaly detection for non-IID data and methods that do not rely on data distributions, but problems remain.
  • Generalised FL: Most identified attacks concern horizontal federated learning, leaving vulnerabilities and information leakage in vertical and federated transfer learning less fully explored.
  • Combination with other trends: Data security and privacy remain cross-cutting requirements when federated learning is combined with personalization, fairness, and other emerging approaches.

9. Conclusions

The paper surveys federated-learning attacks and defences through taxonomies, experiments, and defence-selection guidelines. It concludes that threat research remains ongoing, with further vulnerabilities and resilient protection mechanisms still requiring study.

  • Federated learning introduces adversarial-attack and defence challenges alongside its privacy-preserving and distributed-learning goals.
  • The paper develops attack and defence taxonomies, conducts an experimental study, and proposes guidelines, lessons learned, and challenges.
  • Federated-learning threat research remains ongoing because additional vulnerabilities and mechanisms for resilient, privacy-preserving learning still need investigation.
Loading 2201.08135v1…