Source-linked AI summary
FedSecurity: Benchmarking Attacks and Defenses in Federated Learning and Federated LLMs
Shanshan Han, Baturalp Buyukates, Zijian Hu, Han Jin, Weizhao Jin, Lichao Sun, Xiaoyang Wang, Wenxuan Wu, Chulin Xie, Yuhang Yao, Kai Zhang, Qifan Zhang, Yuhui Zhang, Carlee Joe-Wong, Salman Avestimehr, Chaoyang He
TL;DR
Federated learning faces security vulnerabilities, yet attacks and defenses lack a comprehensive benchmark, and their effectiveness against LLMs remains unclear. FedSecurity benchmarks attacks and defenses through FedAttacker and FedDefender; its evaluations report that random and zero-mode Byzantine attacks substantially degrade accuracy, while label flipping and Byzantine flipping have milder effects.
Problem
Federated learning is vulnerable to attacks on model integrity and data privacy, while comprehensive comparisons of defenses and evidence about attack and defense effectiveness against LLMs remain limited.
Source: §1 · p. 2Method
FedSecurity adds FedAttacker and FedDefender to FedML to simulate attacks and defenses during federated learning, with configurable benchmarking.
Source: §1 · p. 2, §2.3 · p. 3Results
Random and zero-mode Byzantine attacks substantially degrade accuracy, while label flipping and Byzantine flipping have milder effects; defenses may slightly improve accuracy against low-impact attacks.
Source: §5.1 · pp. 7–8Takeaways & Limitations
Experiments with BERT and Pythia showcase FedSecurity's scalability and applicability to federated LLM scenarios.
Source: §5.2 · p. 8FedSecurity does not yet support asynchronous or vertical federated learning.
Source: §6 · p. 10
Abstract
from arXiv · showhide
This paper introduces FedSecurity, an end-to-end benchmark that serves as a supplementary component of the FedML library for simulating adversarial attacks and corresponding defense mechanisms in Federated Learning (FL). FedSecurity eliminates the need for implementing the fundamental FL procedures, e.g., FL training and data loading, from scratch, thus enables users to focus on developing their own attack and defense strategies. It contains two key components, including FedAttacker that conducts a variety of attacks during FL training, and FedDefender that implements defensive mechanisms to counteract these attacks. FedSecurity has the following features: i) It offers extensive customization options to accommodate a broad range of machine learning models (e.g., Logistic Regression, ResNet, and GAN) and FL optimizers (e.g., FedAVG, FedOPT, and FedNOVA); ii) it enables exploring the effectiveness of attacks and defenses across different datasets and models; and iii) it supports flexible configuration and customization through a configuration file and some APIs. We further demonstrate FedSecurity's utility and adaptability through federated training of Large Language Models (LLMs) to showcase its potential on a wide range of complex applications.
1 INTRODUCTION
Federated learning supports collaborative training without centralizing client data, but its adversarial vulnerabilities and the limited evidence for defenses on LLMs motivate a unified benchmark. FedSecurity provides attack and defense components with flexible configuration and support for varied models, optimizers, and federated LLMs.
1 INTRODUCTION
- FL’s collaborative training can expose model integrity and training-data privacy to adversarial clients, despite keeping client data local.Attacks can submit spurious models, plant backdoors, or reconstruct training data from updates or gradients.Source: §1 · p. 2
- A comprehensive benchmark for comparing FL attacks and defenses under unified settings is lacking, and their effectiveness against LLMs remains poorly understood.The paper notes that attack and defense effectiveness may diminish with LLMs’ large parameter counts and complex datasets.Source: §1 · p. 2
- FedSecurity pairs FedAttacker, which simulates attacks, with FedDefender, which provides defenses, as a benchmark component for FedML.It implements attacks and defenses from the literature and supports configuration through YAML and APIs.Source: §1 · p. 2
- FedSecurity supports varied models and FL optimizers and can simulate attacks and defenses during federated LLM training.The paper also identifies a real-world FL application using edge devices as clients.Source: §1 · pp. 2–3
- The authors caution that defenses may reduce aggregation accuracy and call for practical strategies that activate only after detecting attacks and identify malicious clients without harming benign models.They report that existing defenses may compromise accuracy even when no attack occurs, while attacks are infrequent in practice.Source: §1 · p. 3
2 PRELIMINARIES AND OVERVIEW
This section situates FedSecurity among existing FL benchmarks, characterizes adversaries, and outlines how the framework inserts attacks and defenses into FedML training. Its overview spans client and server stages of the FL workflow.
2.1 Existing Benchmark Frameworks
- Blades implements attacks and corresponding defenses but omits data reconstruction attacks; FederatedScope [99] implements model- or gradient-based reconstruction attacks, omits prevalent Byzantine attacks, and provides no FL defenses.FedSecurity covers model poisoning, data poisoning, and data reconstruction attacks; it integrates defenses and offers flexible configurations and APIs for customizing attack and defense strategies.Source: §2.1 · p. 3
2.2 Adversarial Model
- The section distinguishes active adversaries, who manipulate data or models, from passive adversaries, who infer sensitive information from model updates or gradients.Examples include malicious clients, global sybils, external attackers, adversarial servers, and adversarial clients.Source: §2.2 · p. 3
2.3 Overview of FedSecurity
- FedSecurity is an external FedML component that injects attacks and defenses at different FL training stages without altering the existing FL procedures.The overview identifies server aggregation and client training as algorithmic injection points.Source: §2.3 · p. 3, Figure 2 · p. 4
- FedSecurity uses separate FedAttacker and FedDefender instances to simulate attacks and defenses within the FL system.The instances are initialized once and accessed by other objects using a singleton design pattern.Source: §2.3 · p. 3
2.3.1 Injection of attacks.
- The overview presents attack and defense injection across server aggregation and client training stages.Algorithm 1 is for server aggregation, and Algorithm 2 is for client training.Source: §2.3 · pp. 3–4, §2.3.1 · p. 4
1 Function client_training(dataset) begin
FedAttacker places attack mechanisms at FL stages suited to their targets, while FedDefender exposes server-side functions for intervening before, during, or after aggregation. The design also provides inputs and APIs for customizing these mechanisms.
1 Function client_training(dataset) begin
- Data poisoning alters client datasets, model poisoning tampers with submitted client models, and data reconstruction infers training information from models, updates, or gradients.The overview places data and model poisoning before aggregation at the server and reconstruction at the server, which can access local and global models across rounds.Source: §1 · p. 4
2.3.2 Injection of defenses.
- FedDefender manipulates local models or aggregation through before-, on-, and after-aggregation functions to counter attacks or protect and improve the global model.After-aggregation functions may add noise or clip the aggregated model.Source: §2.3.2 · pp. 4–5
- FedDefender’s flexible APIs provide local models and the global model to customized defenses, with local_models represented as per-round sample-count and client-model tuples.The paper identifies Table 3 as a summary of customization APIs for attack types.Source: §2.3.2 · pp. 4–5, Table 3 · p. 5
3 IMPLEMENTATION OF ATTACKS
FedAttacker supports model poisoning, data poisoning, and data reconstruction, with APIs for injecting these attacks into FL. The section describes representative mechanisms and the steps for integrating new attacks.
3 IMPLEMENTATION OF ATTACKS
- FedAttacker injects model poisoning, data poisoning, and data reconstruction attacks at different FL training stages and provides APIs for them.The section summarizes customization APIs and introduces each attack type.Source: §3 · p. 5
3.1 Model Poisoning Attacks
- Model poisoning modifies client models before aggregation; Byzantine examples set weights to zero, assign random values, or reverse the update direction.The flipping mode forms a local model as w_g + (w_g - w_l), where w_g is the global model and w_l the real local model.Source: §3.1 · p. 5
3.2 Data Poisoning Attacks
- Label flipping is a data poisoning attack in which a global sybil or malicious clients relabel samples from a source class to an incorrect target class.FedAttacker offers APIs to control each local dataset.Source: §3.2 · p. 5
3.3 Data Reconstruction Attacks
- Data reconstruction attacks use trained global or local models, or model updates, to infer training data under an assumption of no leakage during trusted local training.The paper gives DLG as an example that infers local training data from publicly shared gradients.Source: §3.3 · p. 5
3.4 Integration of a New Attack
- To add an attack, users select its type, implement the relevant API functions, and register its name in FedAttacker’s corresponding enabler functions.Examples include attack_model(*), poison_data(*), reconstruct_data(*), and is_model_poisoning_attack().Source: §3.4 · p. 5
4 IMPLEMENTATION OF DEFENSES
FedDefender injects defensive functions at different stages of server-side aggregation. A defense can use one or multiple stages, with APIs supporting each stage.
4 IMPLEMENTATION OF DEFENSES
- FedDefender accepts local_models as sample-count/model tuples and auxiliary_info as defense-specific information, with APIs for before-, on-, and after-aggregation functions.Table 4 summarizes the defense APIs.Source: §4 · p. 6, Table 4 · p. 6
4.1 Before-Aggregation Defenses
- Krum selects the single local model most likely benign as the global model; m-Krum instead selects the m client models with the lowest scores.Krum is presented as a before-aggregation defense operating on local models at the server before aggregation.Source: §4.1 · p. 6
4.2 On-Aggregation Defenses
- RFA replaces simple averaging with the geometric median of client models, computed in practice using the Smoothed Weiszfeld Algorithm.The geometric median is chosen to minimize the sum of distances to the client models in that iteration.Source: §4.2 · p. 6
4.3 After-Aggregation Defenses
- CRFL clips the aggregated global model to bound its norm, then adds Gaussian noise before the server distributes it to clients.Source: §4.3 · p. 6
4.4 Integration of a New Defense
- To add a defense, users choose its aggregation stage and implement the corresponding API functions; defenses spanning multiple stages require all relevant functions.The defense name must also be added to the enabler functions to activate it.Source: §4.4 · p. 6
5 EVALUATIONS
The evaluations benchmark attacks and defenses across standard FL settings, varied models and datasets, federated LLMs, and real edge devices. Results show that defense effectiveness depends on the attack and task, while m-Krum mitigates attacks in several tested settings.
5.1 Evaluations on FL
- Random- and zero-mode Byzantine attacks substantially degrade test accuracy, while label flipping and flipping-mode Byzantine attacks have milder effects; random mode is the strongest tested attack.The authors use random mode as the default attack in subsequent experiments.Source: Figure 3 · p. 7, §5.1 · p. 7
- Defenses can reduce accuracy even with benign clients, and their effects vary by attack: m-Krum counters random-mode Byzantine attacks, while RFA can reduce accuracy under label flipping.The authors attribute benign-setting losses to excluding, reweighting, or changing the aggregation of local models.Source: Figure 4 · p. 7, Figure 5 · p. 7, Figure 6 · p. 7
- Across i.i.d. data, 100-client training, and 1–3 malicious clients, m-Krum keeps test accuracy close to the no-attack case or relatively consistent.In the 100-client setting, m-Krum aggregates five selected local models.Source: Figure 7 · p. 7, Figure 8 · p. 7, Figure 12 · p. 7
- Defense performance varies across models: some mechanisms fail on particular tasks, including m-Krum on RNN and Foolsgold on CNN.The authors link this to model selection or reweighting that can remove local models important in early training, leaving test accuracy unchanged in later iterations.Source: Figure 10 · p. 7, Figure 11 · p. 7, §5.1 · p. 8
5.2 Evaluations on Federated LLMs
- For federated LLMs, m-Krum brings BERT accuracy closer to the attack-free case and mitigates the increased test loss from Byzantine attacks on Pythia-1B.The evaluations use 10 clients for BERT and 7 for Pythia-1B; the authors note that defenses storing intermediate results may face significant cache requirements with LLMs.Source: §5.2 · p. 8, Figure 13 · p. 9, Figure 14 · p. 9
5.3 Evaluation in Real-World Applications
- On 70 Theta edge devices, m-Krum mitigates random-mode Byzantine attacks in an MNIST logistic-regression task with 7 malicious clients.The setup uses 35 of 70 local models in aggregation per round and accounts for CPU-only devices, connectivity, latency, and storage constraints.Source: §5.3 · pp. 8–9, Figure 15 · p. 9
6 CONCLUSION
FedSecurity benchmarks attacks injected during FL training and defenses intended to mitigate their impact. It does not yet support asynchronous or vertical FL.
6 CONCLUSION
- FedAttacker simulates attacks during FL training, while FedDefender facilitates strategies to mitigate their impact.The benchmark is open-sourced, and the authors invite contributions of novel attack and defense strategies.Source: §6 · pp. 9–10
- FedSecurity does not yet support asynchronous FL or vertical FL.Source: §6 · p. 10
- The authors invite research-community contributions to enrich the open-source benchmark with novel attack and defense strategies.Source: §6 · p. 10