Source-linked AI summary
BackdoorBench: A Comprehensive Benchmark of Backdoor Learning
Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, Chao Shen
TL;DR
Backdoor-learning evaluations are often insufficient because the field develops rapidly across diverse settings and prior methods are difficult to implement or reproduce. BackdoorBench addresses this gap with a modular codebase, a standardized protocol, and broad attack-defense evaluations, revealing patterns across poisoning ratios, datasets, and models.
Problem
Rapid development, diverse settings, and implementation or reproduction difficulties make backdoor-learning evaluations insufficient for verifying claims and comparing methods.
Method
BackdoorBench provides an extensible modular codebase, a standardized complete-learning protocol, and analysis tools for reproducible backdoor evaluations.
Results
8,000 evaluations cover 8 attacks against 9 defenses across 5 poisoning ratios, 5 models, and 4 datasets, enabling analysis of backdoor-learning factors.
Takeaways & Limitations
BackdoorBench provides a basis for comparing existing methods, clarifying current progress, and identifying research directions from comprehensive evaluations.
Takeaways & Limitations
BackdoorBench currently focuses mainly on computer vision and supervised learning, with planned expansion to NLP, speech, and reinforcement learning.
Abstract
from arXiv · showhide
Backdoor learning is an emerging and vital topic for studying deep neural networks' vulnerability (DNNs). Many pioneering backdoor attack and defense methods are being proposed, successively or concurrently, in the status of a rapid arms race. However, we find that the evaluations of new methods are often unthorough to verify their claims and accurate performance, mainly due to the rapid development, diverse settings, and the difficulties of implementation and reproducibility. Without thorough evaluations and comparisons, it is not easy to track the current progress and design the future development roadmap of the literature. To alleviate this dilemma, we build a comprehensive benchmark of backdoor learning called BackdoorBench. It consists of an extensible modular-based codebase (currently including implementations of 8 state-of-the-art (SOTA) attacks and 9 SOTA defense algorithms) and a standardized protocol of complete backdoor learning. We also provide comprehensive evaluations of every pair of 8 attacks against 9 defenses, with 5 poisoning ratios, based on 5 models and 4 datasets, thus 8,000 pairs of evaluations in total. We present abundant analysis from different perspectives about these 8,000 evaluations, studying the effects of different factors in backdoor learning. All codes and evaluations of BackdoorBench are publicly available at \url{https://backdoorbench.github.io}.
1 Introduction
Backdoor learning poses a serious security threat, but rapid development and inconsistent evaluations make new methods difficult to verify and compare. BackdoorBench addresses this gap with a modular benchmark, standardized protocol, comprehensive evaluations, and analysis tools.
- Motivation: Rapidly evolving attacks and defenses, diverse threat models, and reproduction difficulties leave evaluations insufficient across methods, models, and datasets.This limits verification of reported performance and the assumptions behind new methods.
- Impact: The benchmark aims to clarify current progress, facilitate comparisons, and reveal intrinsic properties that can guide future backdoor-learning research.The stated goal includes supporting both new attack and defense design and exploration of underlying reasons and properties.
- Benchmark: BackdoorBench provides an extensible modular codebase containing 8 SOTA attacks, 9 SOTA defenses, and 5 analysis tools.The modules cover attacks, defenses, evaluation, and analysis, with additional methods and tools continuously updated.
- Benchmark: The benchmark standardizes the complete backdoor-learning procedure to support fair and reproducible evaluations.Its protocol covers data preparation, attack, defense, saving, evaluation, and analysis of intermediate and final outputs.
- Evaluation: 8,000 attack-defense evaluations cover every pair of 8 attacks and 9 defenses across 5 poisoning ratios, 5 models, and 4 datasets.These evaluations support analysis of factors affecting backdoor learning.
2 Related work
Prior work spans backdoor attacks, defenses, and broader robustness benchmarks, but existing backdoor-learning libraries provide limited coverage and analysis relative to BackdoorBench.
- Backdoor attacks: Backdoor attacks are categorized by threat model into data poisoning and training-controllable attacks, with variation in trigger visibility, locality, and fusion.Examples include visible or invisible triggers and local or global trigger coverage.
- Backdoor defenses: Backdoor defenses are categorized by stage as pre-training, in-training, or post-training, targeting poisoned data, model features, or backdoor effects.Post-training defenses constitute most existing methods described in the passage.
- Related benchmarks: Adversarial-robustness libraries such as CleverHans, Foolbox, AdvBox, RobustBench, RobustART, ARES, and ART mainly address test-stage adversarial examples.The passage contrasts these resources with the smaller set of backdoor-learning benchmarks.
- Related benchmarks: TrojanZoo is the most similar benchmark, but BackdoorBench differs in programming style and emphasizes comprehensive analysis and findings.TrojanZoo uses object-oriented modules, whereas BackdoorBench uses a procedural, function-based design.
3 Our benchmark
BackdoorBench combines categorized attack and defense implementations with modular data, attack, defense, evaluation, and analysis components. A standardized protocol connects these modules for flexible, reproducible experiments.
- 3.1 Implemented algorithms: The benchmark selects 8 attacks using classic-baseline or recent state-of-the-art status and coverage of threat models and trigger characteristics.BadNets, Blended, and LC are classic methods; five others are recently published.
- 3.1 Implemented algorithms: Attack categories encode threat model, trigger visibility, coverage, fusion mode, and whether triggers are agnostic or sample-specific.The listed characteristics distinguish visible or invisible, local or global, additive or non-additive, and shared or specific triggers.
- 3.1 Implemented algorithms: The 9 defense methods are organized by input, output, defense stage, and strategy, including detection, identification, poison handling, mitigation, and inhibition.Their table also records each method’s motivation, assumption, or observation.
- 3.2 Codebase: The modular codebase contains input, attack, defense, and evaluation-and-analysis modules.The input module supplies clean data and model architectures, while attack and defense modules support different threat models and defense inputs.
- 3.2 Codebase: Evaluation uses clean accuracy, attack success rate, and robust accuracy, with five tools for visualizing features, pixel or frequency contributions, and neuron activity.The protocol supports pure attack, pure defense, and joint attack-defense modes across the full evaluation workflow.
4 Evaluations and analysis
BackdoorBench evaluates attack-defense pairs across datasets, models, and poisoning ratios, revealing that performance depends strongly on dataset, poisoning ratio, defense behavior, and model architecture.
- Experimental setup: The benchmark evaluates 8 attacks against 9 defenses, plus undefended models, across 5 poisoning ratios, 4 datasets, and 5 models, totaling 8,000 evaluations.Performance is measured with C-Acc, ASR, and R-Acc.
- Results overview: In Figure 2, most defenses mitigate backdoor effects without significantly harming clean accuracy under the tested PreAct-ResNet18 and 5% poisoning setting.The distribution compares C-Acc and ASR, with defenders favoring high C-Acc and low ASR.
- Results overview: On CIFAR-10 and GTSRB, most attack-defense pairs lie near ASR + R-Acc = 1, whereas pairs on CIFAR-100 and Tiny ImageNet often fall away from this anti-diagonal.The paper relates this difference to the greater difficulty of recovering correct predictions when datasets have more classes.
- Effect of poisoning ratio: ABL isolates all 500 samples as poisoned at 10% poisoning but none as poisoned at 5% in the LC-on-CIFAR-10 example, changing later unlearning behavior.The second-row t-SNE visualizations in Figure 4 support this explanation.
- Effect of poisoning ratio: Higher poisoning ratios do not consistently improve attacks; some defenses instead show sharp ASR drops as poisoning increases.The analysis attributes this pattern to higher poisoning ratios making poisoned and clean samples more distinguishable to adaptive defenses.
- Effect of model architectures: Model architecture substantially changes attack and defense performance: EfficientNet-B3 resists several attacks, while PreAct-ResNet18 is easier to backdoor and ANP often removes its backdoors less successfully.FT is particularly effective on MobileNetV3-Large, while NC and ANP remove backdoors in most DenseNet-161 cases; ANP is less effective against SIG on EfficientNet-B3.
5 Conclusions, limitations and societal impacts
The paper concludes that BackdoorBench provides a comprehensive benchmark for comparing backdoor attacks and defenses and analyzing their behavior. Its current scope is mainly computer vision and supervised learning, while the technology may support both beneficial and malicious uses.
- Conclusions: BackdoorBench combines a modular codebase implementing 8 attacks and 9 defenses with 8,000 attack-defense evaluations and thorough analysis.The benchmark is intended to clarify current progress and support comparisons and new research questions.
- Limitations: BackdoorBench currently focuses mainly on algorithms and evaluations in computer vision and supervised learning.The authors plan future expansion to NLP, speech, and reinforcement learning.
- Societal impacts: The benchmark may facilitate new backdoor-learning algorithms, whose implementations could be used for beneficial or malicious purposes.The paper identifies intrinsic-property research, regulations, and laws as possible approaches to adverse impacts.
Checklist
The checklist reports that the paper describes its contributions, limitations, societal impacts, reproducibility materials, and asset provenance. Supplementary material also documents implemented attack and defense algorithms.
- Checklist: The authors report describing the paper’s contributions, scope, limitations, societal impacts, and compliance with ethics guidelines.
- Reproducibility and assets: The paper provides code, data, instructions, training details, and hyper-parameter information needed to reproduce the main experiments.The implementations and modifications of existing algorithms are documented in the BackdoorBench repository.
- Implemented algorithms: The supplementary material introduces eight implemented attacks and nine implemented defenses, including BadNets, Blended, label consistent, fine-tuning, fine-pruning, NAD, and Neural Cleanse.The passages describe the operational ideas of several listed methods, such as patch insertion, alpha blending, clean-sample fine-tuning, neuron pruning, and attention distillation.
A.3 Implementation details and computational complexities
The benchmark uses documented software and hardware settings, fixed or consistently selected hyper-parameters, and broad evaluations across datasets, models, poisoning ratios, attacks, and defenses. Its analyses identify dataset variation, randomness, poisoning-ratio behavior, and hyper-parameter sensitivity as important factors.
- Implementation details: The experiments use GPU servers with RTX3090 GPUs, 320 GB RAM, and software versions documented in the BackdoorBench repository.
- Hyper-parameter settings: Hyper-parameter settings for all implemented attacks and defenses are documented, enabling reproduction of the reported 8,000 evaluation pairs.
- Hyper-parameter settings: The benchmark avoids separate searches for every method because suitable hyper-parameter ranges can vary across datasets, architectures, and opposing methods.The authors argue that exhaustive per-evaluation tuning could make comparisons unfair and lacks practical selection rules.
- Hyper-parameter settings: The evaluations adopt original-paper values when available, otherwise calibrate values against reported results and keep them fixed across other settings.Consistent values are intended to support fair comparisons, while the study focuses on shared high-level factors rather than fine-grained parameter effects.
- Results overview: 5% and 10% poisoning-ratio results show attack-defense performance distributions across PreAct-ResNet18 and VGG-19, with some defense patterns moving closer to the anti-diagonal as poisoning increases.
- Effect of dataset: Across datasets, Blended is relatively stable, BadNets fluctuates most, and AC and Spectral Signature are comparatively unaffected by dataset changes.The comparison uses PreAct-ResNet18 with a 5% poison ratio and reports ASR through bar heights across four datasets.
- Effect of poisoning ratio with randomness: Five-seed reruns show small standard deviations for the poisoning-ratio trends overall, but low-ratio poisoning identification under ABL and NC can be unstable.For NC, instability in the initial trigger-search detection step causes large ASR variation when detection succeeds in some runs but not others.
- Sensitivity to hyper-parameters: Hyper-parameter sensitivities differ across methods, datasets, and architectures, making practical parameter selection challenging.Examples include stronger or more detectable BadNets triggers, frequency-dependent SIG behavior, and diverse ANP threshold effects.
B.6 Analysis of quick learning of backdoor
The paper analyzes backdoor quick learning using losses, accuracies, GSNRs, gradient norms, and gradient similarities during training. Across five attacks, poisoned samples learn rapidly and exhibit stronger early gradient signals than clean samples.
- Analysis design: The analysis tracks losses, accuracies, GSNRs, average-gradient norms, and pairwise cosine similarities for clean and poisoned samples across training epochs.
- Observed learning dynamics: For BadNets, Blended, SSBA, LC, and LF, poisoned-test loss drops quickly and converges lower than clean-test loss during early training.The experiments use CIFAR-10, PreAct-ResNet18, and a 10% poisoning ratio.
- Gradient analysis: Early poisoned samples have significantly higher GSNRs and gradient norms than clean samples, consistent with higher ASR and lower poisoned-test loss.
B.7 Analysis of backdoor forgetting
The paper uses forgetting events to compare learning dynamics for clean and poisoned training samples. Clean-sample forgetting is similar across cases, while poisoned-sample forgetting depends on the poisoning ratio and supports analysis of individual-sample contributions.
- Definition and analysis: A forgetting event occurs when a correctly classified training sample becomes misclassified at the next epoch.The formal definition compares f_θt(x)=y with f_θt+1(x)≠y.
- Clean samples: Clean training-sample forgetting events follow an exponential distribution and remain similar across different cases.
- Poisoned samples: At 0.1% and 0.5% poisoning, poisoned samples often have more forgetting events than clean samples; at 5% and 10%, they often have fewer.
- Interpretation: The forgetting-event analysis is compatible with higher poisoning ratios producing a quickly learned, stable mapping from poisoned samples to the target class.It also provides a fine-grained way to analyze each training sample’s contribution to backdoor learning.
B.8 Analysis of trigger generalization of backdoor attacks
BackdoorBench identifies trigger generalization as the property that a model trained with one trigger can also respond to other triggers. Experiments with Blended and SSBA show that this property depends on trigger characteristics and can undermine trigger uniqueness.
- Trigger generalization means a backdoored model trained with one trigger can be activated by other triggers.
- For Blended, training with a 30% transparency trigger does not easily transfer to lower-transparency test triggers.
- SSBA models can be activated by many strings sharing the training trigger’s length, indicating trigger generalization across several attacks.
- Trigger generalization undermines the uniqueness assumption behind using backdoors for intellectual-property protection.
B.9 Evaluation on vision transformer
The benchmark extends backdoor evaluation to a pretrained ViT-b-16 on CIFAR-10 and finds that the architecture remains vulnerable. The tested defenses perform poorly, while visualization tools support analysis of model decisions in image and frequency domains.
- The evaluation tests pretrained ViT-b-16 with 16 × 16 input patches on CIFAR-10 at a 10% poison ratio.
- Without defense, ViT-b-16 achieves very high ASR across all evaluated attacks, revealing vulnerability to backdoor attacks.
- FT and NC reduce ASR significantly but also downgrade clean accuracy, whereas ABL leaves ASR largely unchanged except for BadNets.
- The results imply that defenses effective on CNNs may not be suitable for ViT architectures.
- Frequency saliency analysis estimates the connection between model prediction and image frequency components using gradient norms and the chain rule.
B.11.2 Visualization results
Visualization analyses examine which image regions or frequency components support predictions under varied attacks and defenses. The results show attack-specific frequency preferences and defense-related shifts in model attention.
- Shapley Value and Grad-CAM visualize regions contributing to predictions for poisoned samples across attacks and defenses.
- Most backdoor models attend to high-frequency regions, while LF models concentrate more on low-frequency regions.
- SSBA and WaNet show distinct frequency-domain patterns despite similar-looking spatial images.
- After FT on a BadNet model, low-frequency regions regain attention, consistent with FT removing backdoors embedded by BadNet and WaNet.
C BackdoorBench in Natural Language Processing
BackdoorBench extends beyond its original computer-vision benchmark into NLP by implementing two attacks and one defense. The attacks remain effective at low poisoning ratios, while ONION performs poorly against these invisible triggers.
- The NLP extension implements LWS and HiddenKiller attacks together with the ONION defense.
- Both evaluated NLP attacks achieve high attack success rates even at low poisoning ratios.
- ONION performs poorly because it targets obvious sentence outliers, whereas HiddenKiller and LWS use invisible triggers without special tokens.