Source-linked AI summary
BackDFL: A Unified Benchmark For Backdoor Attacks and Defenses In Decentralized Federated Learning
Mouhamed Amine Bouchiha, Gregory Blanc, Yufei Han
TL;DR
DFL security lacks rigorous evidence under realistic adaptive backdoor threats because prior evaluations use fragmented protocols and simplified assumptions. BackDFL provides a reproducible benchmark spanning attacks, defenses, datasets, and configurations, and finds that DFL defenses fail even with modest malicious participation while topology strongly affects robustness. The paper concludes that adaptive local defenses can improve resilience in some settings, but robust peer-to-peer DFL remains highly susceptible to contamination.
Problem
Existing DFL security studies use simplified threat models, inconsistent configurations, and fragmented evaluations, leaving backdoor robustness insufficiently characterized.
Method
BackDFL is a unified reproducible benchmark integrating six targeted attacks, five untargeted attacks, fourteen defenses, and seven datasets.
Results
State-of-the-art Byzantine-robust DFL methods fail at 15% malicious participation, while defense robustness varies substantially across communication graph topologies.
Takeaways & Limitations
Robust peer-to-peer DFL remains highly susceptible to backdoor contamination, although an adaptive lightweight BALANCE variant can improve resilience with low computation.
Takeaways & Limitations
The threat model fixes the communication topology and assumes attackers lack prior knowledge of the graph and neighborhood structure.
Abstract
from arXiv · showhide
Decentralized Federated Learning (DFL) promises trust-free collaborative learning by replacing the centralized parameter server with peer-to-peer model exchange. However, this architectural shift fundamentally reshapes the threat landscape. Without globally coordinated aggregation, DFL becomes particularly susceptible to backdoor attacks, in which malicious participants implant persistent hidden behaviors while maintaining high clean-task performance. In this paper, we argue that the robustness of DFL has been significantly overestimated. Existing studies rely on simplified threat models, non-adaptive adversaries, fragmented evaluation protocols, inconsistent communication topologies, and ad hoc training configurations, leading to an incomplete understanding of DFL security. To address these limitations, we present BackDFL, a unified benchmark for systematically evaluating DFL under realistic and adaptive backdoor attacks. Through extensive experiments, BackDFL exposes critical failure modes of decentralized learning. Our results demonstrate that both state-of-the-art Byzantine-robust DFL methods and adapted FL backdoor defenses fail under modest malicious participation rates (as low as 15%), especially in heterogeneous settings, while their robustness varies substantially across communication graph topologies.
1 Introduction
DFL removes centralized coordination through peer-to-peer aggregation, but this architecture creates serious backdoor-security challenges. BackDFL addresses fragmented evaluations with a unified benchmark and comparative defense analysis.
- DFL replaces the central server with peer-to-peer aggregation, reducing centralized trust dependencies and single points of failure.
- Adaptive backdoor attacks can preserve benign accuracy while implanting targeted misclassification in decentralized models.
- Existing DFL defense evaluations use naive backdoor scenarios that do not capture realistic adaptive threats.
- Centralized FL defenses benefit from server-wide visibility and cannot be directly applied to fully decentralized systems.
- BackDFL provides a unified reproducible benchmark and evaluates thirteen defenses to expose their failure modes.
2 Related Work
Prior backdoor research is largely centralized and difficult to compare across studies. Differences in configurations and simplified threat models limit reproducible assessment of DFL security.
- Centralized FL backdoor research benefits from parameter-server coordination and large-scale benchmarking, but remains tied to centralized assumptions.
- Studies vary in graph topology, local training, optimizers, and hyperparameters, hindering meaningful comparison of robustness claims.
- Most prior evaluations use static triggers, fixed poisoned datasets, and non-adaptive adversaries.
3 Problem Definition
Fully decentralized FL replaces the parameter server with direct peer-to-peer exchange but leaves backdoor security largely unexplored. BackDFL therefore targets rigorous, reproducible, and comparable evaluation of DFL attacks and defenses.
- DFL uses direct peer-to-peer model exchange over a communication graph instead of a parameter server, while backdoor attacks remain largely unexplored.
- BackDFL unifies DFL configurations, training protocols, attack definitions, threat models, implementation practices, and evaluation metrics.
- RQ1 asks which centralized defenses remain effective after adaptation to DFL and why others fail.
- RQ2 examines established stealthy attacks including Neurotoxin, A3FL, and IBA in DFL settings.
- RQ3 and RQ4 study how data and model characteristics, and graph topology, affect robustness.
4 System & Threat Models
The paper models DFL as local training and neighbor aggregation over a fixed communication graph, then evaluates adaptive attackers that compromise a minority of clients. Attackers manipulate data or updates using locally observed information, while defenders lack attacker identities and strategies.
- System Model: The DFL system connects clients through an undirected graph, with each client holding private data and exchanging parameters with directly connected neighbors.
- System Model: Each round combines local optimization with decentralized aggregation of neighboring models.
- System Model: The aggregation parameter α controls the trade-off between self-reliance and consensus, while AGG can implement rules such as FedAvg or Median.
- Threat Model: The attacker compromises a default 15% fraction of clients and can inject poisoned samples or manipulate shared model updates.
- Threat Model: The topology is fixed, attackers lack graph knowledge, and defenders do not know which clients are malicious or the attack strategy.
5 Proposed BackDFL
BackDFL is a modular, configurable framework for reproducibly evaluating backdoor attacks and defenses across centralized and decentralized federated learning. It combines parallel FL/DFL experiment flows with shared algorithmic components and broad attack, defense, and dataset coverage.
- BackDFL provides a modular framework for evaluating backdoor attacks and defenses in centralized and decentralized federated learning.Its independent components support extension and integration of datasets, models, attacks, defenses, and experiment flows.
- BackDFL uses shared attack and defense modules across centralized and decentralized flows while retaining topology management for DFL experiments.This design allows the same framework to evaluate both FL settings with distinct experiment pipelines.
- YAML-based configuration files define experiments and feed parallel centralized and decentralized flows, including configurable graph topologies for DFL.The framework separates configuration, data and models, experiment flows, attack and defense modules, foundational FL code, and output metrics.
- The framework supports evaluation on seven datasets alongside established attack, defense, and model implementations.Implemented examples include DBA, Neurotoxin, A3FL, IBA, BALANCE, SCCLIP, M-Krum, Weak-DP, FLAME, and DeepSight.
- BackDFL integrates six targeted backdoor attacks, five untargeted attacks, and fourteen defenses spanning adapted FL and DFL-specific methods.The targeted attacks include fixed-trigger, optimized-trigger, and gradient-manipulation categories.
6 Evaluation & Results
BackDFL evaluates attacks and defenses across datasets, models, centralized and decentralized settings, and communication topologies. Results show that adaptive attacks, non-IID data, and graph structure expose substantial weaknesses in existing defenses.
- Evaluation setup: BackDFL evaluates six backdoor attacks and defenses across seven benchmark datasets under non-IID settings.The benchmark pairs datasets with adapted neural-network architectures and reports clean-task accuracy alongside attack success rate.
- Centralized versus decentralized FL: DFL defenses degrade because clients observe limited neighborhoods and contaminated local reference models can make malicious updates appear consistent.In decentralized settings, Weak-DP, DeepSight, and SPP reach approximately 1.0 ASR with maximal durability, while average accuracy remains acceptable.
- Attack effectiveness: 0.57/0.33/0.22 mean ASR under IID and 0.74/0.52/0.35 under non-IID rank IBA, A3FL, and Neurotoxin respectively.IBA reaches ASR ≥0.500 in 75% of defense–dataset–partition combinations, compared with 35% for A3FL and 21% for Neurotoxin.
- Dataset and partition effects: IBA exceeds 0.677 ASR against every defense on CIFAR-10 in both IID and non-IID partitions.On GTSRB IID, several defenses hold IBA below 0.10 ASR, whereas under non-IID GTSRB only ABALANCE and UBAR remain below the failure threshold.
- Defense effectiveness: ABALANCE and FLAME are the most consistent defenses overall, with mean ASR of 0.347 and 0.341.Similarity- and clustering-based defenses, including M-Krum, MMAD, and DFL-Dual, show the largest IID-to-non-IID ASR increases; SCCLIP has mean ASR 0.685.
- Accuracy–robustness trade-off: Aggressive filtering does not consistently improve robustness without accuracy costs.DeepSight has CIFAR-10 IID Min. ACC 0.810 and mean ASR 0.450, while UBAR has the worst Min. ACC under non-IID.
- Graph topology: 65.3% versus 97.8% peak ASR across communication topologies shows that graph structure strongly affects attack severity.Only ABALANCE and UBAR suppress the backdoor on every tested topology, while shortcut-rich graphs can accelerate both diffusion and mitigation.
7 Conclusion
BackDFL shows that decentralized federated learning remains highly vulnerable to adaptive backdoor contamination, even with modest malicious participation and realistic non-IID data. Defense robustness also fails to generalize across graph topologies, while adaptive BALANCE improves resilience with low computation.
- 15% malicious rates can defeat state-of-the-art Byzantine-robust DFL methods under realistic non-IID distributions.
- Adaptive BALANCE improves resilience by making its acceptance criterion responsive to neighbor updates without overly aggressive filtering.
- Defense effectiveness varies across graph topologies, motivating adaptive mechanisms that remain robust across diverse networks.
A Detailed Experimental Settings
The experiments adapt model architectures to each dataset’s characteristics, using convolutional networks for image and traffic-sign data and an MLP for sensor-derived features.
- MNIST uses a SimpleCNN with two convolutional layers and fully connected layers of sizes 120, 84, and 10.
- FashionMNIST uses FashionCNN with two convolutional blocks containing 32 and 64 filters, followed by fully connected layers.
B Sensitivity Analysis
Sensitivity analyses examine malicious participation, mixing ratio, data heterogeneity, and local training epochs. Results show that defense robustness and backdoor persistence depend strongly on these settings, with ABALANCE generally remaining among the most stable methods.
- Impact of malicious rate: 15–25% malicious clients leave robust defenses at Final.ASR 0.11–0.12 on MNIST.
- Impact of mixing ratio: At α = 0.3, permissive defenses reach Max.ASR values of 0.39–0.50 as reliance on neighbor aggregation strengthens.
- Impact of mixing ratio: FLAME and ABALANCE maintain Max.ASR ≤0.07 at α = 0.3, although stronger self-reliance reduces utility.
- Impact of local epochs: ABALANCE degrades more gracefully than FLAME as local training epochs increase, while Min.ACC remains largely stable.