PaperlayerView exact revision on arXiv

Source-linked AI summary

Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks

Yige Li, Xixiang Lyu, Nodens Koren, Lingjuan Lyu, Bo Li, Xingjun Ma

arXiv:2101.05930v2cs.LGcs.AI

TL;DR

Backdoor attacks can preserve clean accuracy while causing triggered misclassification, motivating defenses that remove embedded triggers. NAD uses a clean-data-finetuned teacher to guide attention-based finetuning of the backdoored student, and empirically performs strongly across six attacks while remaining resilient to experimental settings.

  • Problem

    Backdoor attacks can control predictions through trigger patterns while leaving clean-example performance unaffected, and existing defenses may fail to eradicate their effects.

    Source: §1 · p. 1

  • Method

    NAD finetunes a backdoored student on a small clean subset while aligning its intermediate-layer attention with a teacher obtained by finetuning on the same subset.

    Source: §1 · p. 1, §3 · p. 3

  • Results

    NAD achieves superior performance against 6 state-of-the-art backdoor attacks compared with 3 other defenses and is resilient to changes in experimental settings.

    Source: §5 · p. 10

  • Takeaways & Limitations

    NAD provides a strong baseline for mitigating backdoor threats in model deployment and can be employed without exhaustive hyperparameter tuning.

    Source: §5 · p. 10

    The defense setting assumes the defender has a small subset of clean training data for finetuning the backdoored model.

    Source: §3 · p. 3

Abstract

from arXiv · show

Deep neural networks (DNNs) are known vulnerable to backdoor attacks, a training time attack that injects a trigger pattern into a small proportion of training data so as to control the model's prediction at the test time. Backdoor attacks are notably dangerous since they do not affect the model's performance on clean examples, yet can fool the model to make incorrect prediction whenever the trigger pattern appears during testing. In this paper, we propose a novel defense framework Neural Attention Distillation (NAD) to erase backdoor triggers from backdoored DNNs. NAD utilizes a teacher network to guide the finetuning of the backdoored student network on a small clean subset of data such that the intermediate-layer attention of the student network aligns with that of the teacher network. The teacher network can be obtained by an independent finetuning process on the same clean subset. We empirically show, against 6 state-of-the-art backdoor attacks, NAD can effectively erase the backdoor triggers using only 5\% clean training data without causing obvious performance degradation on clean examples. Code is available in https://github.com/bboylyg/NAD.

1 INTRODUCTION

Backdoor attacks exploit poisoned training data to make DNNs associate triggers with target labels, while preserving clean-example performance. NAD addresses this threat through distillation-guided finetuning and attention-map analysis.

  • Backdoor attacks poison a small proportion of training data to associate a trigger pattern with a target label.Source: §1 · p. 1
  • Complex triggers and untrusted model sources make backdoors difficult to detect and remove with standard finetuning or neural pruning.Source: §1 · p. 1
  • NAD is a distillation-guided finetuning approach that uses a teacher network and clean data to align student intermediate-layer attention.Source: §1 · pp. 1–2
  • NAD is presented as a defense approach for a wide range of backdoor attacks.Source: §1 · p. 2
  • Attention maps are proposed as an intuitive way to evaluate defenses because they can highlight backdoored regions in network topology.Source: §1 · p. 2

2 RELATED WORK

Related work distinguishes backdoor attacks and defenses, including detection and trigger erasing, while positioning NAD as a combination of knowledge distillation and neural attention transfer. Figure 1 illustrates NAD’s two-step erasing pipeline.

Backdoor Attack

  • Backdoor attacks inject trigger patterns, with or without target labels, into subsets of training data.Source: §2 · p. 2

Backdoor Defense

  • Backdoor defenses primarily use detection or trigger erasing strategies.Source: §2 · p. 2
  • Detection methods identify poisoned models or filter suspicious inputs, but backdoor effects remain in the model.Source: §2 · p. 2

NAD Pipeline

  • NAD first finetunes a backdoored network into a teacher, then distills attention representations between teacher and student networks.Source: Figure 1 · p. 3

Knowledge Distillation

  • NAD combines knowledge distillation with neural attention transfer to supervise intermediate representations during defense.Source: §2 · p. 3

3 PROPOSED APPROACH

NAD uses a clean-data defense setting in which a finetuned teacher guides the original backdoored student. Its loss aligns layer-wise attention representations while preserving classification performance.

Defense Setting

  • The defense assumes access to a backdoored model and a small clean training subset, aiming to erase triggers while retaining clean-sample performance.Source: §3 · p. 3

Overview

  • NAD uses a finetuned network as teacher and the original backdoored network as student in an attention distillation process.Source: §3.1 · p. 3

Attention Representation

  • The attention operator maps a C×H×W activation map to a flattened H×W attention representation.Source: §3.1 · p. 4
  • Asum includes all activation regions, while larger p in Ap sum emphasizes regions with the highest neuron activations and Amean averages activation regions.Source: §3.1 · p. 4

Attention Distillation Loss

  • For ResNets, attention representations are computed after each residual group, with the teacher fixed during distillation.Source: §3.1 · p. 4

Overall Training Loss

  • The total loss combines student cross-entropy with NAD losses across residual groups, weighted by β.Source: §3.1 · p. 4

Teacher Network

  • The teacher is obtained by finetuning the backdoored student on the same clean subset, and NAD is applied once for only a few epochs.Source: §3.1 · p. 4

4 EXPERIMENTS

Across six backdoor attacks, NAD substantially lowers attack success while preserving clean accuracy better than the compared defenses. Additional analyses examine data availability, attention alignment, teacher–student choices, teacher architectures, and parameter tuning.

Overall effectiveness

  • 7.22% average ASR: NAD reduces the attack success rate from nearly 100% across six attacks, outperforming finetuning, Fine-pruning, and MCR.The competing methods reduce average ASR to 37.36%, 67.00%, and 25.59%, respectively.Source: §4.2 · p. 5
  • 2.66% ACC drop: NAD has the least prominent clean-accuracy degradation among the evaluated erasing methods.The paper reports that all erasing methods negatively affect ACC, but NAD’s drop is merely 2.66%.Source: §4.2 · p. 6

Effectiveness under different clean-data budgets

  • 99.04% to 35.93% ASR: with only 1% clean training data, NAD still reduces the average attack success rate while sacrificing 4.69% ACC.With 20% clean data, NAD and MCR defend against all six attacks almost 100% of the time; NAD also converges faster than MCR.Source: §4.2 · p. 6

Attention-map analysis

  • Attention-map evidence shows NAD distracts CL-backdoored models from triggered regions and activates benign areas more strongly than MCR.For BadNets, standard finetuning, MCR, and NAD pay almost no attention to the injected bottom-right trigger; for CL, only MCR and NAD do so.Source: §4.3 · p. 7

Parameter sensitivity

  • β balances ASR reduction against ACC preservation; increasing it improves robustness but can severely reduce clean accuracy.At β = 50,000, the purified network lost more than 50% ACC; the suggested strategy is to increase β until ACC falls below an acceptable threshold.Source: §4.3 · p. 8

Teacher–student analysis

  • B-F teacher and B student is the standard NAD combination, while a clean-from-scratch teacher can reduce ASR but may significantly degrade ACC.Here B denotes the backdoored network, B-F the finetuned backdoored network, and C the model trained from scratch on 5% clean data.Source: §4.4 · pp. 8–9

Teacher architectures

  • All five tested teacher architectures effectively purify a WRN-16-1 student against BadNets using 5% clean data.The tested teachers are WRN-10-2, WRN-16-1, WRN-16-2, WRN-40-1, and WRN-40-2.Source: §4.4 · p. 9

Training behavior

  • NAD students can overfit partially purified teachers, but early stopping can address this behavior.The overfitting experiment ran NAD for up to 20 epochs on CIFAR-10 against BadNets and CL attacks.Source: §4.4 · p. 9

5 CONCLUSION

NAD is a knowledge-distillation-based backdoor defense that achieves superior performance against six state-of-the-art attacks compared with three other defenses. Attention maps provide an intuitive visual evaluation, while results remain resilient to experimental-setting changes.

  • NAD is a knowledge distillation based framework for defending DNNs against backdoor attacks.Source: §5 · p. 9
  • NAD achieves superior performance against 6 state-of-the-art backdoor attacks compared with 3 other backdoor defense methods.Source: §5 · p. 10
  • Attention maps offer an intuitive visual way to evaluate backdoor defenses by displaying backdoored regions in a network’s topology.Source: §5 · p. 10
  • NAD remains fairly resilient to changes in experimental settings and can be employed without exhaustive hyperparameter tuning.Source: §5 · p. 10
  • NAD provides a strong baseline for mitigating backdoor threats during model deployment.Source: §5 · p. 10

A MORE IMPLEMENTATION DETAILS

The implementation evaluates six backdoor attacks using CIFAR-10, GTSRB, and WideResNet models, with attack-specific triggers and configurations. Defense baselines use standardized finetuning, pruning, and mode-connectivity procedures.

B COMPARISON WITH DATA AUGMENTATION TECHNIQUES

The study evaluates Cutout and Mixup independently as data-augmentation approaches for erasing backdoor triggers, using fixed patch and mixing configurations.

C MORE RESULTS OF MODE CONNECTIVITY REPAIR (MCR)

The MCR comparison uses 5% clean CIFAR-10 finetuning data and evaluates endpoint-model choices against NAD. NAD converges faster and achieves lower ASR than MCR in the BadNets comparison.

D EXPERIMENTAL RESULTS OF TRIGGER RECOVERING TECHNIQUE

The experiments compare NAD with trigger-recovery and retraining-based defenses using 5% clean data, including MESA-recovered triggers. NAD reduces CL attack success more than either retraining approach, while BadNets ASR reaches similar levels across methods.

  • The MESA comparison reports the best averaging results after changing only the available training-data proportion to 5%.Source: D EXPERIMENTAL RESULTS OF TRIGGER… · p. 15
  • NAD reduces CL ASR by 16 percentage points versus retraining with the original trigger and by 22 percentage points versus the MESA-generated trigger.Source: Table 6 · p. 15

E EXPERIMENTAL RESULTS OF HYPER-PARAMETER

The hyper-parameter analysis examines NAD under different β values. The authors report that their β settings are only rough estimates and that more granular tuning could improve performance.

F EXPERIMENTAL RESULTS OF DIFFERENT ATTENTION FUNCTIONS

NAD is evaluated with four attention functions against BadNets on CIFAR-10 using ASR and clean ACC, with results summarized in a comparative table. Additional figures cover all-target attack erasure.

G EXPERIMENTAL RESULTS OF ITERATIVE NAD

The experiments examine iterative distillation, attention behavior, overfitting, and adaptive triggers. One NAD iteration is sufficient in the reported setting, while early stopping addresses student overfitting and advanced adaptive attacks remain future work.

Iterative NAD

All-target attacks

Attention behavior

Overfitting in NAD

Adaptive attacks

Related papers

Loading 2101.05930v2…