PaperlayerView exact revision on arXiv

Source-linked AI summary

Rethinking the Backdoor Attacks' Triggers: A Frequency Perspective

Yi Zeng, Won Park, Z. Morley Mao, Ruoxi Jia

arXiv:2104.03413v4cs.LGcs.CR

TL;DR

Backdoor attacks have been extensively studied in the image domain, but their frequency-domain behavior has been underexplored. This paper analyzes trigger spectra, detects existing triggers, and develops smooth alternatives, finding widespread high-frequency artifacts and effective frequency-based detection.

  • Problem

    Frequency-domain analysis of image backdoor triggers has been missing despite the security risks posed by triggered model behavior.

  • Method

    The paper analyzes existing triggers in the frequency domain, builds a supervised frequency-based detector, and designs smooth triggers through bilevel optimization and a heuristic algorithm.

  • Results

    98.50% average detection is achieved under attack-agnostic settings, while proposed smooth triggers reach 97.25% ASR and 84.54% clean accuracy at poison ratio 0.13.

  • Takeaways & Limitations

    Frequency analysis is important for designing backdoor attacks and defenses, and detectors trained on strong smooth triggers can generalize to unseen weak smooth triggers.

  • Takeaways & Limitations

    The attacker model assumes full knowledge of the training set, inference set, and potential target model.

Abstract

from arXiv · show

Backdoor attacks have been considered a severe security threat to deep learning. Such attacks can make models perform abnormally on inputs with predefined triggers and still retain state-of-the-art performance on clean data. While backdoor attacks have been thoroughly investigated in the image domain from both attackers' and defenders' sides, an analysis in the frequency domain has been missing thus far. This paper first revisits existing backdoor triggers from a frequency perspective and performs a comprehensive analysis. Our results show that many current backdoor attacks exhibit severe high-frequency artifacts, which persist across different datasets and resolutions. We further demonstrate these high-frequency artifacts enable a simple way to detect existing backdoor triggers at a detection rate of 98.50% without prior knowledge of the attack details and the target model. Acknowledging previous attacks' weaknesses, we propose a practical way to create smooth backdoor triggers without high-frequency artifacts and study their detectability. We show that existing defense works can benefit by incorporating these smooth triggers into their design consideration. Moreover, we show that the detector tuned over stronger smooth triggers can generalize well to unseen weak smooth triggers. In short, our work emphasizes the importance of considering frequency analysis when designing both backdoor attacks and defenses in deep learning.

1. Introduction

The paper examines backdoor triggers through frequency analysis, finding persistent high-frequency artifacts and proposing smooth triggers without such artifacts.

  • Motivation: Backdoor attacks manipulate training data or model parameters so triggered inputs produce attacker-chosen outputs while clean performance can remain strong.Examples include stop-sign relabeling and face misidentification.
  • Prior trigger designs: Existing triggers seek visual inconspicuousness through commonplace objects, norm-constrained perturbations, or GAN-generated patterns.
  • Frequency analysis: Frequency analysis finds severe high-frequency artifacts across existing trigger designs, datasets, and resolutions, arising from trigger patterns or insertion methods.
  • Detection: 98.5% detection rate is achieved for existing triggers without prior knowledge of attack types, with high detection maintained across differing input distributions and datasets.
  • Smooth triggers: The paper designs smooth triggers through bilevel optimization and a practical heuristic after direct low-pass filtering proves insufficient for attack success.The proposed triggers outperform simple low-pass-filtered triggers and support studying detectability and defense design.
  • Implications: The work emphasizes frequency analysis for designing both backdoor attacks and defenses.

2. Related Work

Related work spans trigger generation, backdoor-data detection, poisoned-model detection, and attack invalidation; this paper adds frequency-domain analysis of backdoor data.

  • Backdoor Trigger Generation: Backdoor trigger research evolved from BadNets and blending attacks toward more effective and stealthy triggers, including model-parameter insertion and GAN synthesis.
  • Backdoor Data Detection: Prior backdoor-data detection identifies input-space outliers or analyzes model responses, features, influence functions, and saliency maps.
  • Backdoor Data Detection: This work examines backdoor data in the frequency domain to provide a simple detection method distinct from model-space and response-based approaches.
  • Poisoned Model Detection: Poisoned-model detectors use meta-classifiers or trigger reconstruction, but reconstruction methods can be ineffective for smooth triggers when they assume local image patching.
  • Attack Invalidation: Attack invalidation includes ensembles, differential privacy, input preprocessing, and data augmentation to disrupt backdoors in models or triggers.

3. Frequency Artifacts

The paper uses DCT to compare clean and triggered images, showing that existing triggers generate high-frequency artifacts through localized, global, or GAN-based mechanisms.

  • Frequency analysis: The paper studies existing triggers in the frequency domain because visually inconspicuous triggers may still reveal spectral artifacts.
  • DCT preliminaries: Type-II 2D-DCT converts images into sums of cosine functions with varying magnitudes and frequencies for frequency-domain analysis.
  • Examining images with triggers using DCT: Figure 1 compares clean and triggered mean spectra using CIFAR-10 small-input data and PubFig large-input data, with trigger-specific heatmaps.The figure uses 10000 CIFAR-10 samples and 1000 PubFig samples; PubFig values are clipped and exponentiated for visualization.
  • Examining images with triggers using DCT: Clean natural-image spectra are dominated by low frequencies, whose contribution decreases toward higher frequencies as pixel values generally change gradually.
  • Examining images with triggers using DCT: Images patched with different triggers contain strong high-frequency components, and these artifacts persist across CIFAR-10, PubFig, GTSRB, and TSRD.
  • Analyzing causes of high-frequency artifacts: The artifact analysis considers additive patching and GAN-based generation, with patching divided into local and large-size or global categories.
  • Analyzing causes of high-frequency artifacts: Localized triggers can carry high frequencies intrinsically, and DCT linearity adds their spectrum to the original image spectrum.
  • Analyzing causes of high-frequency artifacts: Large-size or global triggers produce artifacts through reduced neighboring-pixel correlation or high-frequency content intrinsic to the trigger.

4. Frequency-Based Backdoor Data Detection

The paper uses frequency-domain representations to detect backdoored data across attack scenarios and datasets, including settings where detector and test-data distributions differ. The detector achieves high detection rates, while fine-tuning and larger training sets improve transferability across datasets.

  • Detection framework: The detector targets both training-set poisoning and direct model-weight modification without prior knowledge of the trigger pattern or target model.It can reject potential poisoned data during training or reject triggers during inference.
  • Detection framework: 98.5% average BDR is achieved by a supervised frequency-domain detector distinguishing clean and backdoored data.ACC and BDR evaluate clean-data retention and backdoor rejection, respectively.
  • Detection framework: Frequency-based detection rejects backdoored data more accurately than image-domain detection without sacrificing much clean data.Image-domain detection performs poorly overall, while higher BDR on PubFig is accompanied by lower average ACC.
  • Transferability: Detectors trained on one dataset can transfer to another, but dataset-distribution differences can substantially reduce ACC.Comparisons include CIFAR-10-to-GTSRB and GTSRB/CIFAR-10-to-TSRD transfer settings.
  • Transferability: Fine-tuning with 200 samples, half clean and half randomly perturbed GTSRB samples, raises ACC without sacrificing too much BDR.The fine-tuning samples are not used in the test set.
  • Transferability: After fine-tuning with 100 clean TSRD samples in a 200-sample dataset, both transferred detectors achieve satisfying detection with acceptable ACC.The TSRD dataset is used only for testing because of its limited size.
  • Transferability: A combined CIFAR-10 and GTSRB training set improves average detection efficiency on TSRD, supporting transfer learning from universal high-frequency artifacts.The paper links transferability to training-set size and notes that public clean datasets can support detection without the original training set.

5. Creating Smooth Triggers

The paper generates smooth backdoor triggers through bilevel optimization that balances attack loss with trigger roughness. These triggers avoid high-frequency artifacts while retaining strong attack performance and informing frequency-aware defenses.

  • Methodology: The method formulates smooth-trigger generation as bilevel optimization, minimizing poisoned-data loss and trigger roughness under image-range constraints.The roughness term uses a preset low-pass filter, while λ controls the trade-off between smoothness and perturbation scale.
  • Methodology: Low-pass filtering alone is ineffective, whereas the proposed bilevel approach produces more effective smooth triggers.Directly filtered random patches cannot achieve satisfying functionality, motivating the approximate bilevel solution.
  • Methodology: The heuristic algorithm iteratively searches for a smooth pattern that leads clean samples toward a target label.It uses gradients from a pretrained classifier and updates the trigger with the low-pass-filtered perturbation.
  • Attack Results and Evaluations: Neither the smooth trigger nor its patched images contain high-frequency components, exhibiting frequency spectra similar to natural images.Figure 3 visualizes the trigger and averaged patched-image heatmap in image and frequency domains.
  • Attack Results and Evaluations: 97.25% ASR and 84.54% clean accuracy were achieved on CIFAR-10 with a 0.13 poison ratio, close to the 85.50% baseline accuracy.The attack reached around 95% ASR within one training epoch, while training accuracy remained below 30%.
  • Impacts over Defenses: Upgrading MNA for smooth-trigger generation increased its AUC from 0.0776 to 0.694 and achieved 42.85% detection accuracy.The case study indicates that frequency-aware defense design can improve robustness to smooth triggers.
  • Impacts over Defenses: 89.37% detection averaged over unseen low-pass-filtered triggers reduced the strongest smooth trigger’s ASR to 19.72%, or 18.03% after poisoned-sample removal.Fine-tuning on smooth-trigger samples detected the smooth trigger at 82.49% accuracy and generalized better than fine-tuning on filtered triggers.

6. Conclusion

The paper analyzes backdoor attacks and defenses through the frequency domain, identifying high-frequency artifacts and proposing frequency-invisible triggers. It also reports attack-agnostic detection and shows that frequency-invisible attacks matter for backdoor defenses.

  • 98.50% average detection rate is achieved under attack-agnostic settings by exploiting high-frequency artifacts.
  • Existing attack triggers exhibit severe high-frequency artifacts, motivating frequency-domain analysis of backdoor data.
  • The paper proposes an effective way to generate triggers that are invisible in the high-frequency domain.
  • The results suggest that existing backdoor defenses can benefit from considering frequency-invisible attacks.
  • The type-II 2D-DCT maps image data to a frequency representation for frequency-domain analysis.

B. Visual Examples of Different Triggers

The visual comparison presents paired image-domain and frequency-domain effects of samples patched with different triggers across several datasets. These comparisons reveal severe high-frequency artifacts beyond the CIFAR-10 examples.

  • Figures 7 and 8 compare samples patched with different triggers in the image and frequency domains on GTSRB and TSRD.
  • Severe high-frequency artifacts are visible across the GTSRB and TSRD comparisons, consistent with results on CIFAR-10.
  • Extended pair-to-pair image- and frequency-domain comparisons are also provided for CIFAR-10 and PubFig.

C. Visual Examples of the Random Puturbation used in Developing the Detector

Figure 5 shows clean and randomly perturbed images used to develop the detector. The perturbations simulate high-frequency artifacts from major trigger-patching methods rather than reproducing the tested triggers.

  • Figure 5 contrasts one clean image with five randomly perturbed results produced by different approaches.
  • The perturbations include white-rectangle patching, random-valued rectangle patching, Gaussian noise, random shadows, and random blending.
  • The rectangle perturbations simulate localized triggers analyzed among the detector’s major patching methods.
  • These random perturbations differ in shape and values from the tested triggers and are used only to simulate their high-frequency artifacts.

D. Linear Separability & Input Space

The appendix examines how input-space size relates to the efficiency of linear models for detecting triggered samples. Experiments use PubFig datasets with ten input-width values from 32 to 224.

  • Figure 6 depicts the relationship between input width and linear-model detection efficiency.
  • Ten PubFig input-width values ranging from 32 to 224 are tested for linear-model detection efficiency.
  • Detection efficiency is evaluated using F1-score and overall accuracy on triggered samples.

E. DNN Model Architechures and Ablation Study

The section selects simple detector architectures through ablation studies tailored to large- and small-input-space frequency data. It evaluates frequency comparisons across multiple datasets and reports the resulting detector design rationale.

  • Model Ablation Study: The ablation study seeks the simplest DNN architecture that maintains detection performance on small-input-space frequency data.The analysis gradually adds hidden layers, convolutional layers, and larger kernels.
  • Frequency Comparisons: Frequency comparisons are conducted on GTSRB, TSRD, CIFAR-10, and PubFig using test-set averages ranging from 4,170 to 10,000 samples.PubFig visualizations are clipped to the range (1.5,4.5).
  • Large-Input-Space Detector: A linear model achieves outstanding detection efficiency on the large-input-space PubFig dataset, so no further ablation is performed there.The adopted linear detector is described in Table 6.
  • Small-Input-Space Detector: Small-input-space DCT data depict high-frequency artifacts less effectively because they contain fewer high-frequency coefficients.A comparable fully connected linear model therefore does not achieve satisfying detection efficiency under the same framework.
  • Small-Input-Space Detector: A simple 6-layer CNN is used for small-input-space detection experiments after the linear model fails to meet the required detection efficiency.The CNN architecture is reported in Table 7 and is trained for 150 epochs on CIFAR-10-derived data.

F. Target Model for Evaluating the Smooth Trigger

The smooth-trigger evaluation uses a target model trained on CIFAR-10 and GTSRB. The section reports the model architecture and its clean-sample baseline for CIFAR-10.

  • Target Model: The smooth-trigger experiments use a target model whose layer sizes are reported for CIFAR-10 and GTSRB.The architecture is documented in Table 8.
  • Training Setup: The CIFAR-10 target model reaches a baseline ACC of 85.50% on clean samples.The model is trained with Adam at learning rate 0.05 for 150 epochs.

G. Smooth Trigger on the GTSRB Dataset

The proposed smooth trigger is evaluated on GTSRB in both image and frequency domains using the established attack pipeline. Incorporating the strongest smooth trigger into detector development substantially reduces the attack success rate.

  • Attack Setup: The GTSRB smooth-trigger evaluation uses the established pipeline with poison rate 0.1 and dominant target label 1.Figure 11 presents the generated trigger in the image and frequency domains.
  • Frequency Characteristics: Both the smooth trigger and patched images exhibit frequency spectra similar to natural images.The trigger is multiplied by 5 for visualization, and the heatmap averages 10,000 patched samples.
  • Detector Fine-Tuning: 85.53% BDR and 0.8628 F1 are recovered after incorporating the strongest smooth trigger into the detector through fine-tuning.The fine-tuning has limited effect on detection efficiency for other attack triggers.
  • Attack Mitigation: 97.86% to 13.27%: inference-time detector rejection constrains the poisoned model’s ASR on GTSRB.Applying the detector during training further reduces ASR to 13.03%.
  • Cross-Dataset Finding: The GTSRB results closely match CIFAR-10 findings and further support frequency-domain analysis for backdoor attack and defense development.The paper presents these results as reinforcing its broader remarks.
Loading 2104.03413v4…