PaperlayerView exact revision on arXiv

Source-linked AI summary

Invisible Backdoor Attack with Sample-Specific Triggers

Yuezun Li, Yiming Li, Baoyuan Wu, Longkang Li, Ran He, Siwei Lyu

arXiv:2012.03816v3cs.CR

TL;DR

Existing backdoor defenses often exploit sample-agnostic triggers, motivating a sample-specific alternative. SSBA encodes attacker-specified strings into benign images as invisible additive triggers and learns their target-label mapping through standard training. Experiments report effective attacks, including high ASR with low poisoning rates and performance on out-of-dataset images, while the method assumes limited attacker control.

  • Problem

    Existing backdoor attacks commonly use sample-agnostic triggers, which current defenses can reconstruct or detect across poisoned samples.

  • Method

    SSBA modifies selected training samples with invisible, sample-specific additive noises generated by encoding attacker-specified strings into benign images, without altering other training components.

  • Results

    The method is effective across target labels and settings, reaches ASR > 95% on both datasets by poisoning only 2% of training samples, and also works with out-of-dataset images.

  • Takeaways & Limitations

    Sample-specific triggers break the sample-agnostic assumption underlying current defenses, enabling the proposed attack to bypass them.

  • Takeaways & Limitations

    The attack requires choosing the poisoning rate because increasing it decreases stealthiness.

Abstract

from arXiv · show

Recently, backdoor attacks pose a new security threat to the training process of deep neural networks (DNNs). Attackers intend to inject hidden backdoors into DNNs, such that the attacked model performs well on benign samples, whereas its prediction will be maliciously changed if hidden backdoors are activated by the attacker-defined trigger. Existing backdoor attacks usually adopt the setting that triggers are sample-agnostic, $i.e.,$ different poisoned samples contain the same trigger, resulting in that the attacks could be easily mitigated by current backdoor defenses. In this work, we explore a novel attack paradigm, where backdoor triggers are sample-specific. In our attack, we only need to modify certain training samples with invisible perturbation, while not need to manipulate other training components ($e.g.$, training loss, and model structure) as required in many existing attacks. Specifically, inspired by the recent advance in DNN-based image steganography, we generate sample-specific invisible additive noises as backdoor triggers by encoding an attacker-specified string into benign images through an encoder-decoder network. The mapping from the string to the target label will be generated when DNNs are trained on the poisoned dataset. Extensive experiments on benchmark datasets verify the effectiveness of our method in attacking models with or without defenses.

1. Introduction

Backdoor attacks exploit opaque DNN training by poisoning samples so models behave normally on benign inputs but switch to an attacker-chosen label when triggered. This paper argues that sample-agnostic triggers enable defenses and proposes invisible, sample-specific triggers instead.

  • Backdoor attacks poison training samples with attacker-specified triggers and target labels, causing malicious predictions when the trigger is present.The attacked model otherwise behaves normally on benign samples, and invisible triggers can make attacks stealthy.
  • Existing defenses often succeed because different poisoned samples share the same sample-agnostic trigger, enabling trigger reconstruction or detection.
  • The proposed attack uses invisible, sample-specific triggers while modifying only selected training samples, not the training loss or model structure.
  • An encoder-decoder network embeds an attacker-specified string into benign images as additive noise, allowing training to learn its mapping to a target label.
  • The paper reports that the sample-specific, invisible attack breaks the fundamental assumption of current defenses and is verified by extensive experiments.

2. Related Work

Related work frames backdoor attacks as a growing DNN-training security threat and distinguishes visible attacks from invisible attacks. Representative methods poison images with fixed visual patterns, while defenses include pruning, saliency analysis, and STRIP.

  • Backdoor attacks threaten DNN training and can be categorized by whether their triggers are visible or invisible to humans.
  • Visible Backdoor Attack: BadNets poisons images by stamping an attacker-chosen trigger onto samples and assigning them a target label before DNN training.
  • Backdoor Defense: Pruning-based defenses remove neurons associated with dormant backdoor behavior or high activation values during benign inference.
  • Backdoor Defense: Saliency-map defenses locate likely trigger regions using critical-image regions and anomaly detection, while STRIP tests prediction randomness after image superimposition.

3. A Closer Look of Existing Defenses

The paper argues that mainstream defenses implicitly assume triggers are sample-agnostic. Their effectiveness is therefore expected to suffer when trigger behavior varies across samples.

  • Mainstream backdoor defenses largely rely on the assumption that triggers are sample-agnostic.
  • STRIP identifies suspicious samples through prediction consistency after superimposing image patterns, but its success also relies on input-agnostic triggers.

4. Sample-specific Backdoor Attack (SSBA)

SSBA assumes attackers can poison training data but cannot alter other training components or inference. It generates sample-specific invisible triggers by encoding strings into images, then relies on standard training to associate those strings with target labels.

  • 4.1. Threat Model: The threat model allows poisoning some training data but excludes changing the loss, training schedule, model structure, or inference process.
  • 4.2. The Proposed Attack: The poisoned dataset combines modified samples with remaining benign samples, with poisoning rate γ determined by the modified subset.
  • 4.2. The Proposed Attack: The poisoning rate γ controls stealthiness: smaller γ produces a more stealthy attack.
  • 4.2. The Proposed Attack: A sample-specific attack requires triggers generated from different images to differ whenever the source images differ.
  • 4.2. The Proposed Attack: An encoder-decoder network generates invisible additive-noise triggers by embedding a representative target-label string into benign images.
  • 4.2. The Proposed Attack: Users train DNNs with the poisoned dataset and standard training, so the model learns the string-to-target-label mapping for later trigger activation.

5. Experiment

Experiments evaluate the sample-specific invisible attack on ImageNet and MS-Celeb-1M against multiple baselines and defenses. The attack maintains high effectiveness while showing resistance to pruning, trigger synthesis, STRIP, Spectral Signatures, and related detection methods.

  • Experimental Settings: Experiments use ImageNet object classification and MS-Celeb-1M face recognition with ResNet-18, comparing BadNets, Blended Attack, and Standard Training.The ImageNet subset contains 200 classes, 100,000 training images, and 10,000 test images; MS-Celeb-1M is used for face recognition.
  • Experimental Settings: The attack poisons 10% of training samples toward target label y_t = 0, while baseline triggers use a bottom-right white square with a cross-line.Blended Attack uses 10% trigger transparency; the proposed triggers are generated by an encoder trained on benign training data.
  • Main Results: ASR exceeds 99% on both datasets, while benign-test accuracy decreases by less than 1% relative to Standard Training.The proposed method is on par with BadNets and higher than Blended Attack in ASR, with smaller benign-accuracy reductions than both baselines.
  • Main Results: After pruning 20% of neurons, the attack retains ASR above 95%, whereas BadNets and Blended Attack suffer dramatic ASR declines.The proposed attack’s ASR decreases by less than 5% as the fraction of pruned neurons increases; Blended Attack falls below 10% on both datasets.
  • Main Results: Neural Cleanse and Grad-CAM fail to identify the proposed trigger regions as effectively as they identify the localized triggers of baseline attacks.Neural Cleanse uses anomaly indices and synthesized triggers, while Grad-CAM distinguishes BadNets and Blended Attack regions but fails for the proposed attack.
  • Discussion: Spectral Signatures is disturbed because clean samples unexpectedly receive large outlier scores, and out-of-dataset inference samples achieve nearly 100% ASR.The out-of-dataset inference test uses Microsoft COCO images and synthetic random images.
  • Discussion: The attack reaches ASR above 95% with only 2% poisoning, while inconsistent triggers sharply reduce ASR on ImageNet but not on MS-Celeb-1M.Increasing poisoning raises ASR while benign accuracy remains almost unchanged, but reduces stealthiness; facial-feature similarity may improve trigger generalization.

6. Conclusion

The paper identifies sample-agnostic triggers as the reason existing backdoor attacks are readily mitigated and introduces sample-specific backdoor attack (SSBA) using invisible triggers. Experiments verify effectiveness against models with and without defenses.

  • Existing defenses readily alleviate attacks because different poisoned samples contain the same, sample-agnostic trigger.
  • SSBA generates sample-specific invisible additive noises by encoding an attacker-specified string into benign images.
  • The mapping from the encoded string to the target label is learned during training on the poisoned dataset.
  • Extensive experiments verify the method's effectiveness against models with or without defenses.

Appendix

Table 6 reports benign accuracy and attack success rate for methods using VGG-16, with boldface marking the best result and underlining marking the second-best result.

  • Table 6 compares methods using benign accuracy (BA) and attack success rate (ASR) with VGG-16.Boldface denotes the best result among attacks, while underlining denotes the second-best result.

1. More Results of Methods with VGG-16

With VGG-16, the proposed attack maintains high attack success and benign accuracy across both datasets and remains resistant to pruning, Neural Cleanse, STRIP, and SentiNet. These results support its effectiveness beyond the main ResNet-18 experiments.

  • More Results of Methods with VGG-16: The VGG-16 appendix evaluates attack success rate and benign accuracy on ImageNet and MS-Celeb-1M using the same settings as the main manuscript.
  • Attack Effectiveness: ASR > 98.5% on both datasets; the proposed attack is on par with BadNets and higher than Blended Attack.These results indicate that sample-specific invisible additive noises can serve as effective triggers despite their greater complexity than BadNets' white-square trigger.
  • Resistance to Fine-Pruning: On ImageNet, ASR remains > 95% after pruning 60% of neurons, while Blended Attack falls below 10%.All attacks lose ASR as pruning increases, but the proposed attack's decline is slower.
  • Resistance to Neural Cleanse: The proposed attack is more resistant to Neural Cleanse on ImageNet, and its synthesized triggers on MS-Celeb-1M are meaningless despite having the highest anomaly index.Neural Cleanse's smaller anomaly index corresponds to greater resistance, while synthesized-trigger visualizations provide an additional comparison.
  • Resistance to STRIP: The proposed attack has significantly higher STRIP entropy than baselines on both datasets, making it more resistant to STRIP.Higher entropy indicates greater difficulty for STRIP to defend.
  • Resistance to SentiNet: Grad-CAM fails to detect trigger regions, and poisoned samples have significantly different Grad-CAMs, enabling the attack to bypass SentiNet.SentiNet identifies trigger regions from similarities among Grad-CAMs of different samples.

2. Detailed Settings of DF-TND and Spectral Signature

The appendix details hyperparameter handling for DF-TND and Spectral Signature, whose default settings are unsuitable for the evaluated datasets. DF-TND is tuned for fair comparison, while Spectral Signature is evaluated through outlier-score distributions.

  • DF-TND: DF-TND's vanilla setting targets CIFAR rather than the evaluated datasets, and its performance is sensitive to hyperparameter values.
  • DF-TND: DF-TND is tuned over nine combinations of batch size b and sparsity parameter γ, selecting b = 10, γ = 0.0001 for ImageNet and b = 20, γ = 0.00001 for MS-Celeb-1M.
  • DF-TND: For DF-TND, a target label appearing earlier in descending logit-increase order indicates better defensive performance.
  • Spectral Signature: Spectral Signature is implemented based on Trojan-Zoo because its original code is unavailable.
  • Spectral Signature: Because Spectral Signature's CIFAR default outlier threshold is unsuitable, the appendix reports each test sample's outlier-score distribution instead.The defense fails when clean samples have larger outlier scores.

3. More Comparisons with Adapted Methods

The paper compares its method with adapted versions of prior methods on ResNet-18, using ImageNet performance and Grad-CAM visualizations as comparison outputs.

  • Scope and adaptation: Prior works [31] [34] [50] are excluded from the main scope because their tasks or threat models differ.The authors nevertheless adapt [34] [50] for comparison in their scenario.
  • Scope and adaptation: The adaptations replace the original AlexNet and CNN+LSTM backbones with ResNet-18 and remove the clean-label setting for fair comparison.The original validations used AlexNet for [34] and CNN+LSTM for [50].
  • Quantitative comparison: Table 7 reports BA/ASR percentages for ResNet-18 on ImageNet.The supplied passage identifies the table’s metrics and evaluation setting but provides no values.
  • Visualization comparison: Figure 18 presents Grad-CAM visualizations of poisoned samples generated by different methods.The figure caption identifies the visualization but does not state an outcome comparison.
Loading 2012.03816v3…