Source-linked AI summary
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, Xingjun Ma
TL;DR
Backdoor attacks can preserve clean accuracy while causing triggered misclassification, motivating defenses that remove embedded triggers. NAD uses a clean-data-finetuned teacher to guide attention-based finetuning of the backdoored student, and empirically performs strongly across six attacks while remaining resilient to experimental settings.
Problem
Backdoor attacks can control predictions through trigger patterns while leaving clean-example performance unaffected, and existing defenses may fail to eradicate their effects.
Method
NAD finetunes a backdoored student on a small clean subset while aligning its intermediate-layer attention with a teacher obtained by finetuning on the same subset.
Results
NAD achieves superior performance against 6 state-of-the-art backdoor attacks compared with 3 other defenses and is resilient to changes in experimental settings.
Takeaways & Limitations
NAD provides a strong baseline for mitigating backdoor threats in model deployment and can be employed without exhaustive hyperparameter tuning.
Takeaways & Limitations
The defense setting assumes the defender has a small subset of clean training data for finetuning the backdoored model.
Abstract
from arXiv · showhide
Deep neural networks (DNNs) are known vulnerable to backdoor attacks, a training time attack that injects a trigger pattern into a small proportion of training data so as to control the model's prediction at the test time. Backdoor attacks are notably dangerous since they do not affect the model's performance on clean examples, yet can fool the model to make incorrect prediction whenever the trigger pattern appears during testing. In this paper, we propose a novel defense framework Neural Attention Distillation (NAD) to erase backdoor triggers from backdoored DNNs. NAD utilizes a teacher network to guide the finetuning of the backdoored student network on a small clean subset of data such that the intermediate-layer attention of the student network aligns with that of the teacher network. The teacher network can be obtained by an independent finetuning process on the same clean subset. We empirically show, against 6 state-of-the-art backdoor attacks, NAD can effectively erase the backdoor triggers using only 5\% clean training data without causing obvious performance degradation on clean examples. Code is available in https://github.com/bboylyg/NAD.
1 INTRODUCTION
Backdoor attacks exploit poisoned training data to make DNNs associate triggers with target labels, while preserving clean-example performance. NAD addresses this threat through distillation-guided finetuning and attention-map analysis.
- Backdoor attacks poison a small proportion of training data to associate a trigger pattern with a target label.
- Complex triggers and untrusted model sources make backdoors difficult to detect and remove with standard finetuning or neural pruning.
- NAD is a distillation-guided finetuning approach that uses a teacher network and clean data to align student intermediate-layer attention.
- NAD is presented as a defense approach for a wide range of backdoor attacks.
- Attention maps are proposed as an intuitive way to evaluate defenses because they can highlight backdoored regions in network topology.
2 RELATED WORK
Related work distinguishes backdoor attacks and defenses, including detection and trigger erasing, while positioning NAD as a combination of knowledge distillation and neural attention transfer. Figure 1 illustrates NAD’s two-step erasing pipeline.
- Backdoor Attack: Backdoor attacks inject trigger patterns, with or without target labels, into subsets of training data.
- Backdoor Defense: Backdoor defenses primarily use detection or trigger erasing strategies.
- Backdoor Defense: Detection methods identify poisoned models or filter suspicious inputs, but backdoor effects remain in the model.
- NAD Pipeline: NAD first finetunes a backdoored network into a teacher, then distills attention representations between teacher and student networks.
- Knowledge Distillation: NAD combines knowledge distillation with neural attention transfer to supervise intermediate representations during defense.
3 PROPOSED APPROACH
NAD uses a clean-data defense setting in which a finetuned teacher guides the original backdoored student. Its loss aligns layer-wise attention representations while preserving classification performance.
- Defense Setting: The defense assumes access to a backdoored model and a small clean training subset, aiming to erase triggers while retaining clean-sample performance.
- Overview: NAD uses a finetuned network as teacher and the original backdoored network as student in an attention distillation process.
- Attention Representation: The attention operator maps a C×H×W activation map to a flattened H×W attention representation.
- Attention Representation: Asum includes all activation regions, while larger p in Ap sum emphasizes regions with the highest neuron activations and Amean averages activation regions.
- Attention Distillation Loss: For ResNets, attention representations are computed after each residual group, with the teacher fixed during distillation.
- Overall Training Loss: The total loss combines student cross-entropy with NAD losses across residual groups, weighted by β.
- Teacher Network: The teacher is obtained by finetuning the backdoored student on the same clean subset, and NAD is applied once for only a few epochs.
4 EXPERIMENTS
Across six backdoor attacks, NAD substantially lowers attack success while preserving clean accuracy better than the compared defenses. Additional analyses examine data availability, attention alignment, teacher–student choices, teacher architectures, and parameter tuning.
- Overall effectiveness: 7.22% average ASR: NAD reduces the attack success rate from nearly 100% across six attacks, outperforming finetuning, Fine-pruning, and MCR.The competing methods reduce average ASR to 37.36%, 67.00%, and 25.59%, respectively.
- Overall effectiveness: 2.66% ACC drop: NAD has the least prominent clean-accuracy degradation among the evaluated erasing methods.The paper reports that all erasing methods negatively affect ACC, but NAD’s drop is merely 2.66%.
- Effectiveness under different clean-data budgets: 99.04% to 35.93% ASR: with only 1% clean training data, NAD still reduces the average attack success rate while sacrificing 4.69% ACC.With 20% clean data, NAD and MCR defend against all six attacks almost 100% of the time; NAD also converges faster than MCR.
- Attention-map analysis: Attention-map evidence shows NAD distracts CL-backdoored models from triggered regions and activates benign areas more strongly than MCR.For BadNets, standard finetuning, MCR, and NAD pay almost no attention to the injected bottom-right trigger; for CL, only MCR and NAD do so.
- Parameter sensitivity: β balances ASR reduction against ACC preservation; increasing it improves robustness but can severely reduce clean accuracy.At β = 50,000, the purified network lost more than 50% ACC; the suggested strategy is to increase β until ACC falls below an acceptable threshold.
- Teacher–student analysis: B-F teacher and B student is the standard NAD combination, while a clean-from-scratch teacher can reduce ASR but may significantly degrade ACC.Here B denotes the backdoored network, B-F the finetuned backdoored network, and C the model trained from scratch on 5% clean data.
- Teacher architectures: All five tested teacher architectures effectively purify a WRN-16-1 student against BadNets using 5% clean data.The tested teachers are WRN-10-2, WRN-16-1, WRN-16-2, WRN-40-1, and WRN-40-2.
- Training behavior: NAD students can overfit partially purified teachers, but early stopping can address this behavior.The overfitting experiment ran NAD for up to 20 epochs on CIFAR-10 against BadNets and CL attacks.
5 CONCLUSION
NAD is a knowledge-distillation-based backdoor defense that achieves superior performance against six state-of-the-art attacks compared with three other defenses. Attention maps provide an intuitive visual evaluation, while results remain resilient to experimental-setting changes.
- NAD is a knowledge distillation based framework for defending DNNs against backdoor attacks.
- NAD achieves superior performance against 6 state-of-the-art backdoor attacks compared with 3 other backdoor defense methods.
- Attention maps offer an intuitive visual way to evaluate backdoor defenses by displaying backdoored regions in a network’s topology.
- NAD remains fairly resilient to changes in experimental settings and can be employed without exhaustive hyperparameter tuning.
- NAD provides a strong baseline for mitigating backdoor threats during model deployment.
A MORE IMPLEMENTATION DETAILS
The implementation evaluates six backdoor attacks using CIFAR-10, GTSRB, and WideResNet models, with attack-specific triggers and configurations. Defense baselines use standardized finetuning, pruning, and mode-connectivity procedures.
- BadNets uses a 3 × 3 checkerboard trigger at the bottom right, with 10% injection and 100% attack success rate.
- Trojan uses a reverse-engineered 3 × 3 square trigger, with 5% poisoned training data and 100% attack success rate.
- Blend uses random patterns with 10% injection, α = 0.2, and 99.97% attack success rate.
- Clean-label uses PGD perturbations bounded by L∞ maximum perturbation ϵ = 0.15 and a 3 × 3 grid trigger, achieving 99.21% attack success rate.
- SIG and Refool achieve 99.91% and 95.16% attack success rates, respectively, under their specified trigger configurations.SIG uses ∆ = 20 and f = 6; Refool follows open-source code.
- Defense baselines use converged standard finetuning, last-convolutional-layer pruning matched to NAD’s ACC, and open-source mode connectivity repair.
B COMPARISON WITH DATA AUGMENTATION TECHNIQUES
The study evaluates Cutout and Mixup independently as data-augmentation approaches for erasing backdoor triggers, using fixed patch and mixing configurations.
- Cutout masks one random 3 × 3 patch from each training image to test its independent effectiveness in erasing backdoor triggers.
- Mixup is evaluated independently with its default α value of 1 for backdoor-trigger erasure.
- Table 4 compares Mixup and Cutout on their ability to erase backdoor triggers.
C MORE RESULTS OF MODE CONNECTIVITY REPAIR (MCR)
The MCR comparison uses 5% clean CIFAR-10 finetuning data and evaluates endpoint-model choices against NAD. NAD converges faster and achieves lower ASR than MCR in the BadNets comparison.
- MCR and NAD are compared on CIFAR-10 using 5% clean finetuning data against BadNets.
- MCR is evaluated with identical backdoored endpoints and with backdoored and finetuned-backdoored endpoints.
- Different MCR endpoint models do not further improve its performance.
- MCR takes longer to converge than NAD, yet its ASR remains higher than NAD’s against BadNets.
D EXPERIMENTAL RESULTS OF TRIGGER RECOVERING TECHNIQUE
The experiments compare NAD with trigger-recovery and retraining-based defenses using 5% clean data, including MESA-recovered triggers. NAD reduces CL attack success more than either retraining approach, while BadNets ASR reaches similar levels across methods.
- The MESA comparison reports the best averaging results after changing only the available training-data proportion to 5%.
- NAD reduces CL ASR by 16 percentage points versus retraining with the original trigger and by 22 percentage points versus the MESA-generated trigger.
E EXPERIMENTAL RESULTS OF HYPER-PARAMETER
The hyper-parameter analysis examines NAD under different β values. The authors report that their β settings are only rough estimates and that more granular tuning could improve performance.
- β values were selected using only a rough estimate across all backdoor attacks.
- More granular β tuning could provide better results than the reported settings.
F EXPERIMENTAL RESULTS OF DIFFERENT ATTENTION FUNCTIONS
NAD is evaluated with four attention functions against BadNets on CIFAR-10 using ASR and clean ACC, with results summarized in a comparative table. Additional figures cover all-target attack erasure.
- Four attention functions—Amean, A2mean, Asum, and A2sum—are compared for NAD against BadNets.
- The attention-function comparison uses attack success rate and clean accuracy as evaluation metrics.
- Figure 10 presents NAD results for erasing an all-target BadNets attack.
G EXPERIMENTAL RESULTS OF ITERATIVE NAD
The experiments examine iterative distillation, attention behavior, overfitting, and adaptive triggers. One NAD iteration is sufficient in the reported setting, while early stopping addresses student overfitting and advanced adaptive attacks remain future work.
- Iterative NAD: One distillation iteration is sufficient: further iterations did not reduce attack success and sometimes increased it by 2%.
- All-target attacks: NAD reduces all-target BadNets ASR from 79% to 9.7% using 5% clean training data.
- Attention behavior: Attention maps are used to align intermediate student and teacher attention rather than directly aligning feature maps.
- Overfitting in NAD: Early stopping around epoch 5 addresses NAD student overfitting and allows clean accuracy to recover and stabilize.
- Adaptive attacks: The adaptive attack places a weaker trigger at the image center, and the authors leave more advanced adaptive attacks for future work.