Source-linked AI summary
Anti-Backdoor Learning: Training Clean Models on Poisoned Data
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, Xingjun Ma
TL;DR
The paper asks whether clean models can be trained directly on backdoor-poisoned data, beyond existing detection and erasing defenses. It proposes Anti-Backdoor Learning, a two-stage gradient-ascent scheme, and reports near-clean accuracy with attack success rates close to random guessing across 10 attacks.
Problem
Existing defenses detect or erase backdoors, but whether training can prevent poisoned data from producing a backdoored model remains unclear.
Method
ABL treats poisoned-data training as clean and backdoor dual-task learning, using early local gradient ascent to isolate low-loss backdoor examples and later global gradient ascent to unlearn their target-label correlation.
Results
Across 10 state-of-the-art attacks, ABL-trained models achieve almost the same clean accuracy as models trained on clean data, with attack success rates close to random guessing.
Takeaways & Limitations
ABL provides a way to train clean models on poisoned data without prior knowledge of the backdoored-data distribution.
Takeaways & Limitations
ABL is less effective against Blend and SIG attacks because their trigger patterns can resemble natural artifacts in clean data, making isolation and unlearning harder.
Abstract
from arXiv · showhide
Backdoor attack has emerged as a major security threat to deep neural networks (DNNs). While existing defense methods have demonstrated promising results on detecting or erasing backdoors, it is still not clear whether robust training methods can be devised to prevent the backdoor triggers being injected into the trained model in the first place. In this paper, we introduce the concept of \emph{anti-backdoor learning}, aiming to train \emph{clean} models given backdoor-poisoned data. We frame the overall learning process as a dual-task of learning the \emph{clean} and the \emph{backdoor} portions of data. From this view, we identify two inherent characteristics of backdoor attacks as their weaknesses: 1) the models learn backdoored data much faster than learning with clean data, and the stronger the attack the faster the model converges on backdoored data; 2) the backdoor task is tied to a specific class (the backdoor target class). Based on these two weaknesses, we propose a general learning scheme, Anti-Backdoor Learning (ABL), to automatically prevent backdoor attacks during training. ABL introduces a two-stage \emph{gradient ascent} mechanism for standard training to 1) help isolate backdoor examples at an early training stage, and 2) break the correlation between backdoor examples and the target class at a later training stage. Through extensive experiments on multiple benchmark datasets against 10 state-of-the-art attacks, we empirically show that ABL-trained models on backdoor-poisoned data achieve the same performance as they were trained on purely clean data. Code is available at \url{https://github.com/bboylyg/ABL}.
1 Introduction
Backdoor attacks can implant task-irrelevant trigger–class correlations through small amounts of poisoned data, while existing defenses leave open whether clean models can be trained directly on poisoned datasets. ABL addresses this question by exploiting faster backdoor learning and target-class dependence, achieving near-clean performance against diverse attacks.
- Backdoor attacks inject triggers into a small fraction of training data, causing normal clean-test behavior but target-class predictions when triggers appear.
- DNNs are vulnerable because backdoor triggers are easy to implant but difficult to detect or erase, creating significant security threats.
- Existing defenses primarily detect or erase backdoors, leaving unresolved whether a clean model can be trained directly on poisoned data.
- On CIFAR-10, 4 of 6 attacks achieve nearly 100% ASR at a poisoning rate of 0.5%.
- Backdoor examples are learned faster than clean examples, with stronger attacks producing faster loss reduction; backdoor learning also depends on a specific target class.
- ABL uses gradient ascent to isolate low-loss backdoor examples early and unlearn their target-class correlation later, training clean models without prior knowledge of poisoned-data distribution.
- ABL remains robust across 10 state-of-the-art attacks, with clean accuracy nearly matching clean-data training and ASR close to random guessing.
2 Related Work
Prior work focuses on detecting backdoors or erasing their effects from models. Anti-backdoor learning instead trains clean models directly on poisoned datasets, targeting settings where poisoned examples and their distribution are unknown.
- Backdoor Attack: Backdoor attacks optimize trigger stealthiness, low poisoning rate, and high attack success rate through patterns ranging from simple pixels to complex blended or invisible signals.
- Backdoor Defense: Backdoor defenses use detection to identify anomalous data or models, but detection does not itself remove the trigger’s impact.
- Anti-Backdoor Learning: Anti-backdoor learning trains clean models directly on poisoned datasets without altering models or input data, even when poisoned examples are unknown.
3 Anti-Backdoor Learning
ABL reformulates training on poisoned data as separate clean and backdoor tasks, then uses two-stage gradient ascent to isolate and unlearn backdoor examples. The method addresses rapid backdoor learning and target-class correlation during training.
- Problem formulation: The ABL formulation decomposes poisoned-data training into clean and backdoor tasks, with standard training learning both and producing a backdoored model.
- Distinctive learning behaviors: Backdoor examples can be identified through atypical learning behavior because their task loss drops abruptly during early training.The backdoor task is described as much easier than the clean task, and its training loss drops more rapidly as poisoning rate increases.
- Distinctive learning behaviors: Filtering low-loss examples alone is ineffective because some backdoor examples retain high loss early, while clean examples also become low-loss later.Powerful attacks can succeed with only 50 or 100 backdoor examples, and later filtering becomes inaccurate as clean losses decrease.
- Anti-Backdoor Learning method: ABL first applies local gradient ascent to isolate low-loss backdoor examples without prior knowledge of their distribution.The loss is boosted toward threshold γ when an example’s loss falls below γ; otherwise it remains unchanged.
- Anti-Backdoor Learning method: ABL then applies global gradient ascent to isolated backdoor examples while continuing to learn the remaining data as clean examples.This later stage exploits the association between backdoor triggers and a particular target class to unlearn the backdoor.
4 Experiments
Across benchmark attacks and datasets, ABL substantially reduces attack success while preserving clean accuracy, with performance depending on isolation rate, turning epoch, and gradient-ascent strength.
- Effectiveness against existing defenses: ABL achieves the best average ASR reduction against most attacks across CIFAR-10, GTSRB, and an ImageNet subset while maintaining high clean accuracy.Against NAD, average ASR is lower by 12.71%, 11.90%, and 7.35% on the three datasets, respectively.
- Per-attack behavior: ABL is not uniformly best for every attack: NAD performs best against Blend on CIFAR-10 and SIG on GTSRB, while MCR performs best against Blend on GTSRB and the ImageNet subset.Blend and SIG are harder to isolate because their triggers can resemble natural image artifacts.
- Effectiveness against existing defenses: ABL’s average clean accuracy is near that of models trained on completely clean data and exceeds NAD by 4.39%, 8.21%, and 5.07% across the three datasets.The reported ABL-versus-NAD CA pairs are 84.76% vs. 80.37%, 95.25% vs. 87.04%, and 86.87% vs. 81.80%.
- Isolation-rate sensitivity: An isolation rate below 5% works reasonably well against all six classic attacks, but higher isolation lowers ASR while harming clean accuracy.The experiment varies p from 0.01 to 0.2, revealing a trade-off between unlearning more backdoor examples and preserving clean examples.
- Training schedule: Epoch 20 produces the best overall results among turning epochs 10, 20, 30, and 40, while delayed switching slightly hinders defense performance.Epoch 20 corresponds to approximately 20%–30% of the full training process.
- Gradient-ascent mechanism: LGA separates backdoor and clean examples across tested γ values; larger γ widens loss and performance gaps but γ = 1.5 causes larger fluctuations.The separation is visualized using training losses on backdoor and clean subsets together with ASR and CA.
- Isolation analysis: Maintaining a loss gap between clean and backdoor examples is central to extracting a high-purity isolated backdoor set.Without this separation, isolating backdoor examples from clean examples is difficult.
- Stress testing: With only 1% isolation, ABL reduces ASR from 100% to 4.98%, 16.11%, 27.28%, and 25.74% for BadNets, Trojan, Blend, and Dynamic at 50% poisoning.ABL breaks at 70% poisoning, but remains effective up to 70% for BadNets and 50% for the other three attacks.
5 Conclusion
ABL exploits two weaknesses of backdoor attacks: backdoor examples are learned faster than clean examples, and backdoor learning correlates them with the target label. Its two stages use local and global gradient ascent to isolate backdoor data and unlearn the backdoor model.
- Backdoor examples are easier and faster to learn than clean examples, while backdoor learning strongly correlates them with the target label.
- ABL uses local gradient ascent early to maximize the loss gap and isolate backdoored data through their low loss.
- ABL uses global gradient ascent late to unlearn the backdoored model with the isolated backdoor data.
- ABL effectively defends against 10 state-of-the-art backdoor attacks across varied experimental settings.
Broader Impact
ABL addresses the difficulty of ensuring training-data purity by enabling secure learning with data that may not be fully trusted. The approach is also presented as potentially useful for preventing overfitting in clean settings.
- Training-data quality and purity are difficult to guarantee, including for high-quality datasets collected from the internet.
- ABL introduces a direction for secure and robust learning with not-fully-trusted data.
- Even in clean settings, ABL can prevent deep learning models from overfitting to certain patterns.
A.1 Datasets and Classifiers
The experiments use multiple datasets, classifiers, attack settings, defenses, and training configurations. ABL isolates 1% of potential backdoor examples before subsequent clean-accuracy recovery and final unlearning stages.
- Experimental setup: The experiments summarize datasets and DNN models in Table 4 and use specified attack settings with clean accuracy and attack success rate metrics.
- Training configuration: Models are trained for 100 epochs with SGD, scheduled learning-rate reductions, weight decay of 10^-4, and momentum of 0.9.
- ABL configuration: ABL trains for 20 epochs before isolation, then recovers clean accuracy for 60 epochs and applies LGGA for the final 20 epochs.
- Baselines: The explored unlearning methods include pixel noise, gradient noise, label shuffling, label uniform, label smoothing, self-learning, and finetuning variants.
- Baselines: Additional baselines retrain from scratch, finetune all layers, finetune the last layers, or finetune the last block of an ImageNet model.
- Experimental analysis: Backdoor examples used in the experiments are illustrated in Figure 5, while BadNets training loss is examined at poisoning rates of 1%, 5%, and 10%.
B.3 Comparison of training data detection
The study compares ABL with established detection methods and examines training-loss or isolation performance across attacks, poisoning rates, datasets, and isolation strategies. ABL achieves the best detection precision against all six evaluated CIFAR-10 attacks.
- Detection comparison: ABL is compared with Activation Cluster and Spectral Signature Analysis for detecting backdoor training examples.
- Detection comparison: Figure 7 reports detection precision, defined as TP/(TP + FP), for 1% isolated backdoor examples.
- Isolation comparison: Figures 8 and 9 evaluate isolation based on flooding loss and label smoothing against four backdoor attacks on CIFAR-10.
- Training-loss analysis: Figure 10 compares training loss on clean and backdoor examples for GTSRB and an ImageNet subset under a 10% poisoning rate.
- Detection comparison: ABL achieves the best detection precision against all 6 backdoor attacks evaluated on CIFAR-10.
B.4 Results of Tuning Epochs
ABL performs best when its training-stage transition occurs at epoch 20, corresponding to roughly 20%–30% of training.
- Epoch 20 produces the best defense results among the four tested turning epochs.The evaluated epochs are 10, 20, 30, and 40.
B.5 Results of Detection Rate under different γ
ABL’s local gradient ascent is critical for isolating backdoor examples: without it, precision is extremely low, whereas γ ≥0.5 yields perfect precision.
- 100% precision is achieved when γ ≥0.5 for the 1% isolated backdoor examples.The experiments use BadNets with 10% poisoning on CIFAR-10, GTSRB, and an ImageNet subset.
- Without local gradient ascent, the precision of isolated examples is extremely low.The comparison is made at the end of training epoch 20 using γ = 0.
B.6 Results of Alternative Backdoor Isolation Methods
Alternative isolation and unlearning methods are substantially less effective than ABL in the reported experiments, although flooding loss can isolate backdoor data comparably in some settings.
- Flooding loss achieves similar performance to ABL against four backdoor attacks on CIFAR-10.ABL’s local gradient ascent performs mildly better against the Blend and Dynamic attacks in attack success rate reduction.
- Label smoothing produces much poorer attack success rate performance against Dynamic, Trojan, and Blend attacks than the reported ABL defense.The comparison uses a smoothing value of 0.4.
- All tested unlearning methods except ABL fail to defend against any BadNets attack in the 1% isolation setting.Their attack success rate remains almost unchanged at 100%, which the passage attributes to 9% backdoored data remaining in the potential clean set.
B.8 Results of Computational Complexity for ABL
ABL adds limited computational overhead for isolation and can operate under low poisoning rates or limited clean data, while its scope remains tied to the stated traditional single-dataset setting.
- ABL’s isolation operation adds less than 10% of standard training time on CIFAR-10 and 3% on the ImageNet subset.The reference training times are approximately 40 minutes and 80 minutes for 100 epochs, respectively.
- Against a 1% poisoning rate, ABL with 1% isolated data achieves greater attack success rate reduction with similar clean accuracy than under 10% poisoning.This comparison is reported on CIFAR-10 with WRN-16-1.
- The stated threat model assumes poisoning of training data without manipulating the training procedure, with the defender controlling training but lacking poison proportion and distribution knowledge.The focus is traditional machine learning with a single dataset and model; federated-learning attacks require a different defense setting.
- ABL can unlearn backdoors when only 1% clean training data is available by using Neural Cleanse to reverse-engineer trigger patterns.The reverse-engineered patterns are then used in ABL’s maximization term for model unlearning.