PaperlayerView exact revision on arXiv

Source-linked AI summary

DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation

Han Qiu, Yi Zeng, Shangwei Guo, Tianwei Zhang, Meikang Qiu, Bhavani Thuraisingham

arXiv:2012.07006v2cs.CRcs.LG

TL;DR

Untrusted third-party resources can introduce difficult-to-detect DNN backdoors, while existing defenses lack comprehensive coverage. DeepSweep evaluates data-augmentation policies that fine-tune infected models and preprocess inference inputs, identifying a solution that mitigates eight mainstream attacks and outperforms five existing defenses.

  • Problem

    Existing defenses are not comprehensive against diverse DNN backdoor techniques and arbitrary trigger designs introduced through untrusted third-party resources.

  • Method

    DeepSweep systematically searches augmentation functions and combinations, using one policy for infected-model fine-tuning and another for inference-input preprocessing.

  • Results

    The identified end-to-end solution mitigates 8 mainstream backdoor attacks and beats 5 state-of-the-art existing solutions.

  • Takeaways & Limitations

    DeepSweep provides an extensible framework for evaluating and identifying defenses against considered categories of DNN backdoor attacks.

  • Takeaways & Limitations

    DeepSweep does not guarantee protection against brand-new attack types fundamentally different from those represented in its Attack Database.

Abstract

from arXiv · show

Public resources and services (e.g., datasets, training platforms, pre-trained models) have been widely adopted to ease the development of Deep Learning-based applications. However, if the third-party providers are untrusted, they can inject poisoned samples into the datasets or embed backdoors in those models. Such an integrity breach can cause severe consequences, especially in safety- and security-critical applications. Various backdoor attack techniques have been proposed for higher effectiveness and stealthiness. Unfortunately, existing defense solutions are not practical to thwart those attacks in a comprehensive way. In this paper, we investigate the effectiveness of data augmentation techniques in mitigating backdoor attacks and enhancing DL models' robustness. An evaluation framework is introduced to achieve this goal. Specifically, we consider a unified defense solution, which (1) adopts a data augmentation policy to fine-tune the infected model and eliminate the effects of the embedded backdoor; (2) uses another augmentation policy to preprocess input samples and invalidate the triggers during inference. We propose a systematic approach to discover the optimal policies for defending against different backdoor attacks by comprehensively evaluating 71 state-of-the-art data augmentation functions. Extensive experiments show that our identified policy can effectively mitigate eight different kinds of backdoor attacks and outperform five existing defense methods. We envision this framework can be a good benchmark tool to advance future DNN backdoor studies.

1 INTRODUCTION

DeepSweep addresses difficult-to-detect DNN backdoors introduced through untrusted third-party resources, where existing defenses lack comprehensive coverage. It combines data augmentation for model fine-tuning and inference preprocessing, then searches augmentation policies across attacks and functions.

  • Motivation: Untrusted third parties can poison datasets or modify models, producing normal benign predictions but attacker-controlled errors on trigger-bearing samples.Small malicious-sample ratios can suffice, making these backdoors difficult to detect.
  • Motivation: Existing defenses do not comprehensively cover diverse attack techniques and trigger patterns, including all-to-all attacks and complex global triggers.Backdoor triggers can have arbitrary size, location, and content, complicating universal defense design.
  • Approach: DeepSweep uses two augmentation policies: one fine-tunes the infected model on transformed clean samples, while the other preprocesses inputs to perturb triggers during inference.The two stages jointly target the model backdoor and the sample trigger.
  • Approach: DeepSweep searches a library of 71 augmentation functions and their combinations against representative attacks stored in a backdoor database.Policies may contain multiple functions, and the search identifies functions and combinations effective within considered categories.
  • Results: Six shortlisted functions form two policies that significantly reduce success rates for eight common backdoor attacks across multiple triggers, attack modes, datasets, and models.The solution also outperforms five state-of-the-art defenses, including Neural Cleanse and Fine-pruning.
  • Significance: The framework is extensible and is intended to support systematic evaluation of new attacks and augmentation functions for future backdoor-defense research.The paper presents DeepSweep as a public framework for researchers and practitioners.

2 PRELIMINARIES OF BACKDOOR ATTACKS

A backdoor attack compromises a DNN while preserving benign behavior and causes trigger-bearing inputs to receive attacker-selected or otherwise incorrect labels. Attackers can implant backdoors through model modification or poisoned training data, using varied trigger designs.

  • Backdoor definition: A backdoor attack tampers with a victim model so benign samples remain accurate while trigger-bearing samples receive attacker-specified wrong labels.The attacker may pre-determine the target label or produce an arbitrary unmatched label.
  • Backdoor definition: The attack can be represented by model-parameter modification Δ𝜃 and an attacker-specified trigger 𝛿.The compromised model is written as f𝜃+Δ𝜃.
  • Injection mechanisms: Adversaries can implant backdoors by directly modifying neurons or poisoning training datasets with trigger-patched samples assigned incorrect labels.Poisoned samples are incorporated into the training set before model training.
  • Trigger patterns: Trigger designs vary widely and are used to categorize attacks, including small local pixel patterns such as colored or white squares.The supplied passage introduces local patterns as one trigger category.

3 DEFENSE ANALYSIS

Backdoor defenses are evaluated against robustness, comprehensiveness, functionality preservation, and lightweight operation. Existing detection and trigger-invalidation methods rely on restrictive assumptions or struggle with complex triggers, motivating DeepSweep’s combined invalidation strategy.

  • Threat model: The threat model assumes a defender receives a compromised model or trains one from poisoned data, while an adversary submits trigger-bearing samples during inference.The defender aims to invalidate the backdoor from the compromised model.
  • Defense requirements: A suitable defense should be robust, comprehensive across triggers and target labels, functionality-preserving on clean samples, and lightweight.These requirements include low attack success rates and resistance to evasion when the defense is known.
  • Existing defenses: Detection approaches may assume a single target label and simple small triggers, causing them to fail on all-to-all attacks or complex global patterns.These assumptions prevent such methods from meeting the comprehensiveness requirement.
  • Existing defenses: Model-classification defenses can cover more attacks only by mimicking possible backdoors, which is costly and makes the solution non-lightweight.The passage characterizes the number of possible attack implementations as too large for practical coverage.
  • Existing defenses: Online trigger-detection methods can be non-robust when benign-image overlays overlap triggers, while visualization methods require exact knowledge of trigger patterns.These limitations restrict robustness or comprehensiveness.
  • DeepSweep motivation: Trigger-invalidation preprocessing can handle simple triggers but may fail against complex global patterns because backdoor models and triggers can be highly robust.DeepSweep therefore combines backdoor invalidation with trigger invalidation rather than relying on trigger preprocessing alone.

4 FRAMEWORK DESIGN

DeepSweep is an extensible framework that combines an attack database, augmentation library, two-stage defense pipeline, and evaluation engine. It systematically selects augmentation functions for model fine-tuning and inference preprocessing to mitigate diverse backdoor attacks.

  • DeepSweep integrates an Attack Database, Augmentation Library, two-stage pipeline, and Evaluation & Validation Engine for systematic backdoor-defense analysis.
  • The Attack Database collects eight backdoor attack kinds across three datasets, including trojan, BadNet, and invisible attacks with varied triggers and modes.
  • The pipeline fine-tunes an infected model on transformed clean samples, then preprocesses clean and trigger-patched inference samples with another transformation policy.
  • DeepSweep evaluates augmentation functions by clean-sample accuracy and trigger-patched Attack Success Rate, then ranks and combines shortlisted functions into policies.
  • Algorithm 1 takes an augmentation library and attack search set, applies ACC and ASR thresholds, and outputs fine-tuning and inference policies.

5 A DEFENSE SOLUTION DISCOVERED BY DEEPSWEEP

DeepSweep constructs fine-tuning and inference transformation policies from augmentation functions selected under accuracy and attack-success-rate criteria. The resulting policies combine six functions for fine-tuning with a lighter three-operation policy for inference preprocessing.

  • Candidate selection: DeepSweep selects defense candidates with ACC of at least 70% and ranks the top six by lowest average ASR.Algorithm 1 scans the augmentation library to identify qualified functions; Table 2 reports the six highest-ranked candidates.
  • Inference policy: The inference transformation policy uses a subset of the fine-tuning policy to reduce online inference cost while achieving better defense results than using the entire fine-tuning policy.The selected subset is motivated by lightweight online inference.
  • Fine-tuning policy: The fine-tuning policy includes all six selected augmentation functions, T1–T6, applied to clean samples.The policy can substantially distort samples, so experiments fine-tune with 10,000 transformed clean samples for five epochs.
  • Fine-tuning policy: Fine-tuning the infected model with transformed clean samples alters its classification boundaries against trigger-patched malicious samples.The passage also reports improved generalization to transformed inference samples.
  • Inference policy: The inference policy uses GCSM, DDSM, and SAT to smooth pixels, scale down, and remap pixels away from their original coordinates.This three-operation policy is described as more lightweight than the fine-tuning policy for online efficiency.

6 EVALUATION

DeepSweep is evaluated against searched and validation attacks, prior defenses, and model explanations. The identified two-policy solution reduces attack success while preserving acceptable accuracy, generalizes beyond searched attacks, and remains lightweight.

  • DeepSweep evaluates its defense on searched attacks, validation attacks, prior defenses, and model explanations.The experiments assess effectiveness, generalization, comparative performance, and interpretability.
  • 6.1 Effectiveness against Searched Attacks: Compared with no defense, the two-policy solution mitigates all three searched backdoor attacks while keeping ASR very small and accuracy loss acceptable.The solution combines Pf for fine-tuning with Pi for inference transformation.
  • 6.1 Effectiveness against Searched Attacks: Inference-only preprocessing leaves L2 invisible and watermark Trojan attacks with high ASR and significantly reduces model accuracy, highlighting the need for fine-tuning transformation.The inference-only strategy satisfactorily reduces ASR only for the L0 invisible attack.
  • 6.1 Effectiveness against Searched Attacks: Fine-tuning alone performs worse than the optimal solution for ACC and ASR, and its ASR reaches 100% on the Trojan SQ attack despite being zero on Trojan WM.This result indicates that inference transformation is necessary for better generalizability.
  • 6.1 Effectiveness against Searched Attacks: Using the fine-tuning policy during both stages performs slightly worse than pairing it with the lightweight inference policy.The results support using distinct policies and reduce inference overhead.
  • 6.2 Effectiveness against Validation Attacks: The solution generalizes to unseen validation attacks, achieves the lowest ASR in most attacks, and averages 0.053 ASR with 0.831 ACC across eight attacks.Offline fine-tuning requires only 5 epochs, while online inference uses simple transformations.
  • 6.3 Comparisons with Existing Works: DeepSweep reports better robustness and model usability than Neural Cleanse, Fine-pruning, FLIP, and ShrinkPad-4 across the evaluated comparisons.The paper attributes competing-method failures to invisible or all-to-all attacks, fixed pruning criteria, and complex triggers.

7 DISCUSSION

DeepSweep’s policies cover the evaluated attack categories but do not guarantee protection against fundamentally new or semantic backdoor designs. The framework also leaves policy optimization and broader domain coverage as open issues.

  • 7.1 Optimization of the Policies: The identified policies may not be optimal because shortlisted functions are stacked without automatic optimization or simplification.Repeated operations such as image resizing could be merged, and other operations might outperform the evaluated functions.
  • 7.2 Comprehensiveness of our Solution: DeepSweep cannot guarantee defense against all backdoor attacks because trigger patterns, sizes, and formats can be arbitrary.The authors describe universal defense as challenging without restrictions on attack designs.
  • 7.2 Comprehensiveness of our Solution: Semantic backdoors remain especially difficult to detect because their triggers can resemble normal image content.Examples include special glasses and cars with special colors; few defenses consider such attacks.
  • 7.2 Comprehensiveness of our Solution: DeepSweep is intended to identify defenses for specified attack categories, not guarantee coverage of fundamentally different new attacks.Adding more attacks to the Attack Database is proposed to improve comprehensiveness.
  • 7.4 Adaptive Attackers: A more sophisticated adversary may attempt robust backdoors and triggers that resist both transformation policies.Random transformations make their effects harder to determine deterministically, and multiple inference policies are suggested as a possible enhancement.
  • 7.3 Other Domains: The study focuses on image classification, while other domains such as natural language processing require domain-specific augmentation techniques.The authors suggest text deletion, insertion, or shuffling as possible future adaptations.

8 CONCLUSION

DeepSweep systematically evaluates augmentation-based defenses for DNN backdoors and produces an end-to-end solution. The framework mitigates eight mainstream attacks and outperforms five existing solutions, while being open-sourced for continued benchmarking.

  • 8 CONCLUSION: DeepSweep systematically evaluates and identifies defense solutions against DNN backdoor attacks.The framework uses data augmentation to transform infected models and inference samples.
  • 8 CONCLUSION: DeepSweep’s end-to-end solution mitigates 8 mainstream backdoor attacks and beats 5 state-of-the-art existing solutions.The comparison covers comprehensiveness, model usability, and robustness.
  • 8 CONCLUSION: The framework is open-sourced to support backdoor-defense research and benchmarking.The authors plan to maintain it with new attacks and augmentation functions.
  • B ALGORITHMS AND PARAMETERS: The augmentation library groups transformation functions into affine, compression/quantization, noise/channel-distortion, and advanced-transformation classes.The classification is presented alongside the library’s function details.

B ALGORITHMS AND PARAMETERS

This section documents the augmentation candidates and preprocessing hyperparameters used by DeepSweep. The hyperparameter settings are listed in Table 8.

  • B ALGORITHMS AND PARAMETERS: DeepSweep’s augmentation candidates are detailed together with their adopted hyperparameters.The hyperparameter settings are provided in Table 8.
  • B ALGORITHMS AND PARAMETERS: Table 8 specifies the hyperparameter settings used during preprocessing.The table is identified as the source of preprocessing settings in this paper.

B.1 Optical Distortion

DeepSweep upgrades Optical Distortion by sampling distortion values randomly from a uniform distribution, varying the distortion applied across samples and times.

  • B.1 Optical Distortion: Optical Distortion samples a random distortion value from a uniform distribution bounded by the distortion limit.This differs from the referenced prior implementation.
  • B.1 Optical Distortion: The random process gives different samples different distortion scales across different times, helping the infected model adapt to remapping distortions.The same passage also introduces the paper’s improved Random Pincushion Distortion in Algorithm 2.

B.2 Gamma Compression and Extension

Gamma Compression and Gamma Extension are fine-tuned as median-filter components, using distinct gamma values and look-up tables to reshape pixel values. The resulting mappings help median filters smooth low-value pixels more effectively.

  • Gamma Compression and Extension: Gamma Compression and Gamma Extension are fine-tuned and used in the median filters, forming GCSM and GESM.The paper sets Gamma Compression to 0.6 and Gamma Extension to 2.6.
  • Gamma Compression: 0.6 maps larger original pixel values toward 255, helping larger values bend in so median filtering more efficiently smooths low-value pixels.The mapping is illustrated by the middle LUT in Figure 8.
  • Gamma Extension: 2.6 is used for Gamma Extension to provide the opposite LUT-based remapping of pixel values.The supplied passage introduces this extension but truncates its further explanation.

B.4 Stochastic Affine Transformation

DeepSweep adopts Stochastic Affine Transformation with fixed translation, scaling, and rotation parameters. The transformation updates valid pixel coordinates and handles scaling through cropping or padding before restoring image dimensions.

  • Stochastic Affine Transformation: DeepSweep adopts Stochastic Affine Transformation with T=0.16, S=0.16, and R=4, matching the parameters in prior work.These are the SAT parameters used in Algorithm 5.
  • Coordinate Update: SAT assigns each transformed pixel from a displaced coordinate when the translated position remains within image width and height bounds.The update is applied only when x+Δx lies in (0,w) and y+Δy lies in (0,h).
  • Scaling: For scaling greater than one, SAT crops the transformed image to (h,w), whereas scaling below one triggers padding to that size.The algorithm distinguishes the δ_s>1 and δ_s<1 cases.
  • Augmentation Library: The augmentation library contains four main classes and 71 transformation functions in total.The table summarizes the library used throughout the paper.
Loading 2012.07006v2…