PaperlayerView exact revision on arXiv

Source-linked AI summary

Evolutionary Trigger Detection and Lightweight Model Repair Based Backdoor Defense

Qi Zhou, Zipeng Ye, Yubo Tang, Wenjian Luo, Yuhui Shi, Yan Jia

arXiv:2407.05396v2cs.CRcs.AI

TL;DR

Backdoor defenses must detect triggers that can cause target-label predictions, including when trigger sizes and numbers vary. The paper proposes CETF for evolutionary trigger detection and lightweight unlearning for model repair, reporting robust detection and effective mitigation while identifying BN layers as a backdoor-related mechanism.

  • Problem

    Backdoors can make triggered inputs predict a target label, and existing defenses are challenged by triggers with variable sizes and numbers.

    Source: §I · p. 1

  • Method

    CETF uses GradCAM-guided evolutionary search and clean-image pasting to detect triggers, which are then used in lightweight model unlearning.

    Source: §I · p. 2, E. Pasting and Filtering · p. 6

  • Results

    The authors report that CETF is robust to triggers varying in size and number, and that the defense mitigates backdoors effectively and efficiently without influencing inference performance.

    Source: §I · p. 2

  • Takeaways & Limitations

    The authors propose BN-focused repair methods and interpret their findings as evidence that backdoors are related to BN-layer statistical information.

    Source: §I · p. 2, §V · p. 12

Abstract

from arXiv · show

Deep Neural Networks (DNNs) have been widely used in many areas such as autonomous driving and face recognition. However, DNN model is fragile to backdoor attack. A backdoor in the DNN model can be activated by a poisoned input with trigger and leads to wrong prediction, which causes serious security issues in applications. It is challenging for current defenses to eliminate the backdoor effectively with limited computing resources, especially when the sizes and numbers of the triggers are variable as in the physical world. We propose an efficient backdoor defense based on evolutionary trigger detection and lightweight model repair. In the first phase of our method, CAM-focus Evolutionary Trigger Filter (CETF) is proposed for trigger detection. CETF is an effective sample-preprocessing based method with the evolutionary algorithm, and our experimental results show that CETF not only distinguishes the images with triggers accurately from the clean images, but also can be widely used in practice for its simplicity and stability in different backdoor attack situations. In the second phase of our method, we leverage several lightweight unlearning methods with the trigger detected by CETF for model repair, which also constructively demonstrate the underlying correlation of the backdoor with Batch Normalization layers. Source code will be published after accepted.

I. INTRODUCTION

Backdoor attacks can leave clean-input predictions normal while directing triggered inputs to a target label, making detection difficult when defenders lack prior knowledge of triggers. The paper proposes a two-stage defense that detects triggers with CETF and uses them for lightweight model repair, alongside a proposed connection between backdoors and Batch Normalization (BN) layers.

I. INTRODUCTION

  • Existing defenses face difficulty handling triggers with variable sizes and numbers while mitigating backdoors and preserving model performance.Backdoors may be hard to detect because abnormal behavior appears only when inputs contain triggers, which defenders may not know in advance.Source: §I · p. 1
  • CETF uses GradCAM and evolutionary search to locate a small influential region, then pastes it onto clean images to test for a trigger.The authors report robustness to triggers varying in size and number.Source: §I · p. 2
  • The authors propose that backdoors are associated with statistical information in BN layers and introduce BN-focused repair methods.They report modifying only BN parameters in their repair methods and frame the finding as relevant to backdoor interpretability.Source: §I · p. 2
  • The proposed defense uses CETF-extracted triggers for unlearning, and the authors report effective and efficient mitigation without affecting inference performance.The paper describes the extracted triggers as approximating those used in training, supporting lightweight repair.Source: §I · p. 2

II. RELATED WORK

The related work surveys visible and stealthy backdoor attacks, including triggers that vary across inputs, and describes physical-world vulnerabilities of stealthy attacks. It compares model-repair and sample-preprocessing defenses, highlighting concerns about clean-input accuracy, thresholds, and trigger search.

A. Backdook Attacks

  • Patch-based BadNets and Trojan Attack use visible triggers, while other attacks use invisible, blended, or input-varying triggers.The section notes that stealthy attacks can be vulnerable in physical settings, where camera distance, angle, and illumination may affect triggers.Source: A. Backdook Attacks · pp. 2–3
  • The authors propose that backdoors are hidden in BN layers, while prior work discussed skip connections without directly demonstrating a strong backdoor correlation.The section says suppressing skip connections can negatively affect inference performance.Source: A. Backdook Attacks · p. 3

B. Backdoor Defense

  • Backdoor defenses are grouped into model repair and sample preprocessing; Fine-pruning and Neural Cleanse repair models through pruning or unlearning.The section notes that pruning can affect clean-input performance and fine-tuning can require high computation cost.Source: B. Backdoor Defense · p. 3
  • STRIP detects poisoned inputs through prediction diversity after blending, but broken triggers in the blend can weaken separation.CETF instead substitutes trigger pixels onto clean images to keep triggers intact.Source: B. Backdoor Defense · p. 3
  • Februus removes and repaints influential regions on every input, while NEO uses random trigger search; the section identifies clean-accuracy, threshold, and search-efficiency concerns.Februus depends on a salient-map threshold despite unknown trigger size, and NEO's random search can be ineffective and inefficient.Source: B. Backdoor Defense · p. 3

III. METHOD

The method combines a CAM-focus evolutionary trigger filter with model repair. GradCAM narrows the search to an influential image region, and evolutionary optimization searches within that region for a trigger.

A. Threat Model

  • A backdoored model can classify clean inputs correctly but map trigger-poisoned inputs to the target label in the threat model.The described setting includes a user outsourcing DNN training to a third party.Source: A. Threat Model · pp. 3–4

B. Overview

  • CETF narrows the trigger-search range with GradCAM, validates an evolutionary search result, and passes extracted triggers to unlearning for model repair.The pipeline is presented as two phases: trigger detection followed by model repair.Source: B. Overview · p. 4, Figure 2 · p. 4

C. CAM-Focus

  • GradCAM highlights important regions, but a fixed threshold may miss part of an influential object; here it provides only a rough prior for the later search.The method uses weights above 0.7 and dilates the minimum bounding rectangle to form that prior region.Source: C. CAM-Focus · p. 4

D. Evolutionary Trigger Detecting

  • Customized Differential Evolution searches for the minimum influential part of the image within the CAM-focus prior through initialization, mutation, crossover, and selection.The method uses the evolutionary search to locate the trigger more precisely within the prior region.Source: D. Evolutionary Trigger Detecting · p. 4

1) Initialization:

The trigger-search population consists of rectangular image regions that are iteratively mutated, crossed over with their parents, and selected by fitness.

1) Initialization:

  • The search initializes 40 individuals, each encoding a rectangular region by its upper-left position, height, and width.Each individual is a four-entry vector.Source: 1) Initialization · p. 4

2) Mutation:

  • Mutation generates an individual by adding a scaled difference between two randomly chosen population members to the current individual.The scaling factor is α = 0.3.Source: 2) Mutation · pp. 4–5

3) Crossover:

  • Crossover forms each offspring from parent or mutated entries, with p = 0.5 giving the two individuals equal expected contribution.The choice for each entry uses a uniform random draw.Source: 3) Crossover · p. 5

4) Selection:

  • Selection keeps whichever has the larger fitness value from each parent–offspring pair.Source: 4) Selection · p. 5

5) Objective function:

The objective rewards candidate regions that affect predictions and penalizes their area. The search stops at convergence or after its generation limit, and a clean input can bypass later processing when no influential region is found.

5) Objective function:

  • The fitness function rewards a region that changes the input’s prediction and causes auxiliary images to predict the input’s class, while penalizing region area.The area term is normalized by total image area, and the coefficient 1000 balances the terms’ magnitudes.Source: 5) Objective function · p. 5
  • The evolutionary search runs for at most 100 generations and stops early when the best fitness fails to improve for 10 consecutive generations.The passage reports that 100 generations are sufficient for convergence.Source: 5) Objective function · p. 5
  • For a clean input, a search with fitness below 0 is treated as finding no influential region, so the input is classified as clean and skips the next step.This early decision saves computation and time.Source: 5) Objective function · p. 5

E. Pasting and Filtering

CETF tests whether a searched region acts as a trigger by pasting it onto clean images. The extracted region is then used in fine-tuning to unlearn the backdoor.

E. Pasting and Filtering

  • A searched region is treated as a trigger when pasting it onto clean images at its original position makes a majority of predictions change to the original input’s prediction.Otherwise, the input is treated as clean.Source: E. Pasting and Filtering · p. 6

1) Na¨ıve Unlearning:

  • For repair, CETF’s extracted trigger is pasted onto clean images with true labels for fine-tuning, instead of using Neural Cleanse’s reversed trigger.The authors expect the extracted region to resemble the original trigger more closely and report that this makes repair possible with less computation and data than Neural Cleanse.Source: 1) Na¨ıve Unlearning · p. 6

2) BN-Unlearning and BN-Cleaning:

The paper proposes BN-only repair methods based on evidence that backdoors are closely related to BN layers and their statistical information. BN-Unlearning fine-tunes BN parameters, while BN-Cleaning updates running statistics without labels or backpropagation.

2) BN-Unlearning and BN-Cleaning:

  • The authors propose BN-Unlearning and BN-Cleaning, two repair methods that modify only Batch Normalization layer parameters or statistics.Their experiments are presented as evidence that backdoors are closely related to BN layers and their statistical information.Source: 2) BN-Unlearning and BN-Cleaning · pp. 6–7
  • BN running statistics accumulate batch statistics during training and are used at inference, while the BN scale and offset parameters are typically updated by gradient descent.This distinction underlies the proposed cleaning method's focus on running mean and variance rather than scale and offset.Source: 2) BN-Unlearning and BN-Cleaning · pp. 6–7
  • BN-Unlearning tunes only BN-layer parameters, based on the proposed account that poisoned training shapes specific BN activations that can activate the target class.The authors describe clean inputs as producing activations below zero for those specific features, while poisoned inputs produce activations above zero.Source: 2) BN-Unlearning and BN-Cleaning · pp. 6–7
  • BN-Cleaning modifies only BN running mean and variance, which the authors identify as controls on relative activation magnitudes.The method updates these statistics using trigger-pasted images during forward propagation; the paper says it does not require true labels or backpropagation and costs far less than naïve unlearning.Source: 2) BN-Unlearning and BN-Cleaning · p. 7
  • BN-Cleaning can use unlabeled images because its running-statistic updates are independent of the loss function.The authors say its computation cost is far lower than naïve unlearning because other layers are frozen and backpropagation is unnecessary.Source: 2) BN-Unlearning and BN-Cleaning · p. 7

IV. EXPERIMENTS

The experiments evaluate clean accuracy and attack success rate across face-recognition and traffic-sign-classification settings. They describe datasets, models, patch-trigger attacks, and the attacked models’ pre-defense metrics, but the supplied passages do not report defense outcomes.

1) Metrics:

  • Clean accuracy and attack success rate are the two metrics used to assess retained classification performance and backdoor mitigation.The paper expects defenses to keep accuracy high while reducing attack success rate.Source: 1) Metrics · p. 7

2) Models and Datasets:

  • The evaluation uses FaceScrub for face recognition and GTSRB for traffic-sign classification, with BadNets and Trojan Attack settings across ResNet20, VGG16, and MobileNet.The experiments poison 10% of training data; VGG16 is used without BN layers for Trojan Attack to match the original attack setting.Source: 2) Models and Datasets · p. 7

3) Triggers and Defenses:

  • Before defense, ResNet20, VGG16, and MobileNet have clean accuracies of 99.97%, 95.52%, and 93.30%, and attack success rates of 99.18%, 100%, and 99.39%, respectively.The evaluated triggers include a fixed 2 × 2 yellow block, randomly positioned circular British-flag logos, and model-dependent Trojan Attack triggers; four comparison defenses are selected.Source: 3) Triggers and Defenses · p. 7

4) Settings for Robustness:

CETF-based defenses reduce attack success while maintaining clean-input accuracy, and the authors report strong results with limited data and computation. The experiments examine parameter choices, repair procedures, and several backdoor settings.

4) Settings for Robustness:

  • The robustness setup tests MobileNet on FaceScrub with 2- or 3-trigger inputs and random-size triggers measuring 10–50 pixels in width and height.The random-size-trigger evaluation uses 1,000 FaceScrub test samples and counts the proportion of initially successful poisoned inputs that still attack after defense as ASR.Source: 4) Settings for Robustness · pp. 7–8

5) Implementation Details for Repair:

  • Repair data are formed by pasting extracted triggers onto two randomly selected validation images per class; naïve unlearning updates all layers, while BN methods update only BN parameters.BN-cleaning updates the BN running mean and variance during forward propagation; the BN methods are not applied to VGG16 because it has no BN layer.Source: 5) Implementation Details for Repair · p. 8

6) Additional Experiments for Verification:

  • Additional verification uses ResNet20 models trained on GTSRB with Input-Aware, Blended Injection, WaNet, and SIG attacks, varying fine-tuning data from one to four images per class.These are non-patch-based attacks, including image-specific triggers, pixel blending, backdoor signals, and warping-based triggers.Source: 6) Additional Experiments for… · p. 8

1) Performance of CETF:

  • CETF+naïve unlearning has the strongest ASR reduction, while CETF-only maintains clean-input accuracy; Neural Cleanse ASRs are 7.63% for VGG16 and 6.49% for MobileNet.The authors report that CETF-based defenses need 2 images per class and 3 unlearning epochs, compared with Neural Cleanse’s larger data and computation requirements.Source: 1) Performance of CETF · pp. 8–9, Table III · p. 9
  • 40 DE individuals are selected because trigger-search performance tends to stabilize there, while scaling factor α=0.3 is reported as the best choice and nearby values also yield low ASRs.The authors note that 50 individuals are slightly better for MobileNet, potentially because of DE randomness, but require more computation.Source: 1) Performance of CETF · p. 10, Table IV · p. 10, Table V · p. 10
  • Clean and poisoned inputs show a clear divergence in prediction transitions, supporting the 50% threshold used to classify inputs as poisoned.In the illustrated DE process, optimization finishes after 11 generations as the searched region becomes more accurate at covering the trigger.Source: 1) Performance of CETF · pp. 9–10, Figure 5 · p. 9

2) Robustness Analysis:

CETF is tested against multiple and variable-size triggers, with results indicating robustness in both settings. The reported comparison also describes limitations of other defenses in these scenarios.

Multi-trigger and random-size attacks

  • Against multi-trigger attacks using randomly pasted 2 or 3 triggers on FaceScrub validation data, CETF reduced ASR to 0; for random-size-trigger attacks, CETF had the best performance.Neural Cleanse’s performance was attributed to instability across trigger variants, and NEO performed better than the other three methods but remained far inferior to CETF.Source: 2) Robustness Analysis · pp. 10–11

Multi-trigger attack

  • For multiple triggers, the authors report that Neural Cleanse may not simulate all trigger variants, while Februus may leave triggers that GradCAM does not highlight simultaneously.They also state that NEO struggles when distributed triggers cannot be covered by a fixed-size block.Source: 2) Robustness Analysis · p. 10

Random-size-trigger attack

  • For random-size triggers, the authors report that Februus may fail when a trigger is too large for GradCAM to contain, whereas NEO performs better than in the multi-trigger setting but remains inferior to CETF.STRIP nearly maintains its performance from the single fixed-size-trigger setting.Source: 2) Robustness Analysis · pp. 10–11

3) Performance of Repair:

The paper evaluates naïve unlearning, BN-unlearning, and BN-cleaning using triggers extracted by CETF. BN-focused repair is reported to mitigate backdoors with little data and short repair times, while BN-cleaning has exceptions.

Performance of Repair

  • BN-unlearning reduces ASRs to 0 while maintaining classification accuracy with only 1 image per class; BN-cleaning yields ASRs of 0.26% and 0.02% in the reported one-image-per-class setting.All three repair methods are reported to produce stable results with 2 images per class.Source: 3) Performance of Repair · p. 11, Figure 9 · p. 12
  • For Input-Aware, Blended Injection, WaNet, and SIG, ASRs fall to 0 after naïve unlearning and BN-unlearning; BN-cleaning does not reach 0 for Blended Injection and WaNet.BN-cleaning mainly updates running mean and variance, while the authors say γ and β may also need updating for perfect effect.Source: Table VIII · p. 11, 3) Performance of Repair · pp. 11–12
  • BN-unlearning and BN-cleaning take less time than naïve unlearning, with ResNet20 repair taking less than 1 second for 3 epochs.For MobileNet trained on 526 face classes, all three methods take less than 3 seconds per epoch.Source: Table IX · p. 11, 3) Performance of Repair · p. 12
  • The authors interpret BN-unlearning and BN-cleaning results as evidence for a backdoor relationship with Batch Normalization layers and as a way to repair models by updating BN parameters alone.The experiments also evaluate the repair methods across several attack types.Source: 3) Performance of Repair · pp. 11–12

V. CONCLUSION

The paper concludes that CETF detects triggers through evolutionary search and verification on clean images, while CETF-extracted triggers support successful model repair by unlearning. It also identifies BN-layer dependence and models without BN layers as directions for further study.

V. CONCLUSION

  • CETF searches for triggers using redesigned Differential Evolution optimization and verifies them on clean images; the extracted triggers are reported to be accurate, with a huge decrease in attack success rate.Robustness testing is also reported, but the supplied passage does not specify its findings.Source: §V · p. 12
  • Unlearning methods using CETF-extracted triggers successfully repair backdoored models, and BN-unlearning and BN-cleaning are proposed as guides to understanding backdoor attacks.These repair methods modify the distribution of statistical information in BN layers.Source: §V · p. 12
  • Future work includes examining other data that may influence BN statistical distributions and studying backdoors in models without BN layers.The authors also propose theoretically analyzing how triggers influence BN layers during training.Source: §V · p. 12

Related papers

Loading 2407.05396v2…