PaperlayerView exact revision on arXiv

Source-linked AI summary

Hidden Trigger Backdoor Attacks

Aniruddha Saha, Akshayvarun Subramanya, Hamed Pirsiavash

arXiv:1910.00033v2cs.CV

TL;DR

The paper addresses backdoor attacks whose poisoned data or triggers can be detected during training. It proposes correctly labeled, visually natural poisons optimized across source images and trigger locations, and reports successful attacks on unseen images with a secret random-location trigger while clean performance remains high.

  • Problem

    Existing backdoor attacks can expose poisoned data through incorrect labels or visible triggers, while defenses are difficult when no explicit trigger appears during training.

  • Method

    The attack optimizes poisons to be pixel-close to target images and feature-close to source images patched with a secret trigger, using multiple random sources and locations.

  • Results

    The attack fools models on unseen images with a random-location trigger, reducing validation accuracy from 98% to 40% while the trigger occupies less than 2% of the image area.

  • Takeaways & Limitations

    The results identify a practical vulnerability in deep learning models deployed with adversarially sourced training data and motivate stronger backdoor defenses.

  • Takeaways & Limitations

    Defending against attacks without explicit triggers in the training data remains challenging, and a single poison with a fixed source and location may not generalize.

Abstract

from arXiv · show

With the success of deep learning algorithms in various domains, studying adversarial attacks to secure deep models in real world applications has become an important research topic. Backdoor attacks are a form of adversarial attacks on deep networks where the attacker provides poisoned data to the victim to train the model with, and then activates the attack by showing a specific small trigger pattern at the test time. Most state-of-the-art backdoor attacks either provide mislabeled poisoning data that is possible to identify by visual inspection, reveal the trigger in the poisoned data, or use noise to hide the trigger. We propose a novel form of backdoor attack where poisoned data look natural with correct labels and also more importantly, the attacker hides the trigger in the poisoned data and keeps the trigger secret until the test time. We perform an extensive study on various image classification settings and show that our attack can fool the model by pasting the trigger at random locations on unseen images although the model performs well on clean data. We also show that our proposed attack cannot be easily defended using a state-of-the-art defense algorithm for backdoor attacks.

1. Introduction

The paper introduces hidden trigger backdoor attacks whose poisoned images look natural, carry correct labels, and conceal the trigger until testing. Experiments show the attack can fool models on unseen images while preserving clean-data performance.

  • Backdoor attacks poison training data so a hidden trigger causes source images to be classified as a target category.The model remains correct on regular clean data, masking the compromise.
  • Backdoor attacks are especially relevant to transfer learning because victims may download poisoned data from unreliable web sources.
  • The proposed attack uses correctly labeled poisoned images without visible triggers, making visual identification difficult.
  • The trigger remains secret until test time, potentially leaving defenders little time to respond.
  • 98% to 40%: a secret trigger at a random location reduced validation accuracy on unseen images while occupying less than 2% of image area.

2. Related work

Prior backdoor methods commonly expose poisoned data through incorrect labels or visible triggers, while later approaches hide triggers in different ways. This work extends clean-label poisoning toward a trigger-based attack that generalizes across unseen images and locations, challenging existing defenses.

  • Earlier backdoor attacks mislabeled patched source images, allowing victims to identify poisoned examples through visual inspection.
  • The proposed method keeps the trigger undisclosed during training while using it at test time on unseen images and arbitrary locations.
  • Some newer methods hide triggers with additive perturbations or clean-label poisoning, but they differ in triggering requirements or attack form.
  • Adapting defenses to attacks without explicit training-time triggers is challenging.
  • A statistical defense based on separating clean and poisoned feature clusters failed to find most of the proposed poisoned data.

3. Method

The method constructs poisoned images that are pixel-close to target images but feature-close to source images patched with a secret trigger. It optimizes multiple poisons over random source images and trigger locations so the trained model can generalize the backdoor beyond the poisoning examples.

  • The attacker supplies poisoned data for fine-tuning a pretrained classifier, aiming to change predictions only when a secret trigger is shown.
  • The trigger is pasted onto source images using a patch and binary mask, with different locations obtained by varying the mask.
  • Poisoned images are optimized to remain visually close to target images while matching patched source images in the model’s feature space.
  • Generalization across source images and trigger locations: A single poison generated for one source image and fixed trigger location may fail to generalize across other patched source images.
  • Generalization across source images and trigger locations: Randomly sampling source images and trigger locations during optimization targets generalization to novel images and locations.
  • Generalization across source images and trigger locations: Multiple poisoned images are jointly optimized by assigning sampled patched sources to nearby poison images in feature space and reducing pairwise distances.
  • The attack is evaluated by adding correctly labeled poisons to the target category and checking for high clean-image accuracy but low patched-source accuracy.

4. Experiments

The experiments evaluate hidden-trigger poisoning across ImageNet settings, using correctly labeled poisoned images that resemble targets visually but align with triggered sources in feature space. The attack fools models at test time while keeping patched sources and triggers absent from training data.

  • Experimental setup: Triggers are random 4 × 4 color matrices resized to the desired patch size, with one trigger sampled for each source-target experiment.The reference ImageNet trigger size is 30x30 on 224x224 images, and the trigger is pasted at random locations during testing.
  • Experimental setup: The setup generates poisoned images from source-target pairs, labels them as the target category, and adds them to the target training set.Poisoned images are optimized using AlexNet fc7 features; the reference ImageNet setup adds 100 poisons to 800 clean target images.
  • Hidden-trigger construction: Poisoned targets are visually similar to clean targets but close to patched sources in feature space, while the victim never sees patched source images during finetuning.Figure 2 illustrates the poisoned classifier shifting some patched sources across the decision boundary after poisoned target data are added.
  • Evaluation: The evaluation averages accuracy over 10 random patch locations and 10 random source-target pairs across ImageNet and CIFAR10 settings.The experiments include random, hand-picked, and dog-only ImageNet pairs, plus random CIFAR10 pairs.
  • Comparison: The hidden-trigger threat model achieves results comparable to BadNets despite hiding triggers during training and using clean labels.Unlike BadNets, the poisoned training data contain neither visible triggers nor incorrectly labeled patched sources.

4.2. Ablation study on ImageNet random pairs

The ablation study varies perturbation size and poison count independently to assess their effects on attack efficiency. Poison count has a stronger observed effect than perturbation size.

  • Perturbation ϵ values of 8, 16, and 32 have little influence on attack efficiency.The study generates poisons separately for each perturbation setting.
  • 400 poisoned images added to 800 clean target images produce the best-performing attack, with 33% of the data poisoned.More poisoned data empirically exerts greater influence on the decision boundary during finetuning.

4.3. Finetuning more layers

The attack is evaluated when progressively more fully connected layers are finetuned after optimizing poisons in the conv5 embedding space. Finetuning more layers weakens the attack but leaves low patched-data accuracy alongside high clean accuracy.

  • Finetuning more layers weakens the attack because the fc7 feature space used by the attack evolves during finetuning.The study compares finetuning only the final layer with finetuning fc6, fc7, and fc8.
  • Patched-data accuracy remains below 65% while clean accuracy exceeds 98% when all fully connected layers are finetuned.Poisons are optimized using conv5 as the embedding space, and finetuned layers are initialized from scratch.

4.4. CIFAR10 random pairs

The CIFAR10 evaluation uses 10 randomly selected category pairs and a simplified AlexNet classifier. The accompanying comparisons examine attack success under hidden-trigger training and varying poison injection rates.

  • The CIFAR10 evaluation covers 10 randomly selected pairs of categories using a simplified AlexNet with four convolutional and two fully connected layers.The convolutional layers use 64, 192, 384, and 256 kernels, while the fully connected layers use 512 and 10 neurons.
  • A successful attack is defined by low accuracy on patched data while maintaining high accuracy on clean data.Table 2 states this criterion for interpreting the ablation results.
  • The hidden-trigger threat model achieves attack success rates similar to BadNets despite hiding the trigger during training.The comparison is reported in Table 3.
  • For the multi-class single-source attack on a 1000-class ImageNet classifier, attack success rate increases with the number of injected poisons.The experiment uses 10 random ImageNet source-target pairs.

4.5. ImageNet hand-picked pairs

The study hand-picks 20 ImageNet classes using PASCAL VOC classes as a semantic-distance reference, then forms 10 pairs for the poisoning attack.

  • 20 ImageNet classes were hand-picked using PASCAL VOC classes as a reference for controlling semantic distance.
  • 10 pairs were created from the 20 selected classes and evaluated with the poisoning attack using reference ImageNet parameters.

4.6. ImageNet “dog” pairs

The paper evaluates hidden trigger poisoning on visually similar dog categories and in broader multi-class settings. These experiments examine attack behavior across binary and multi-class classifiers, including the effects of fine-tuning more layers.

  • 10 random pairs of visually similar dog categories were sampled from ImageNet for binary-classifier poisoning experiments.
  • Allowing more network layers to adjust decreases attack efficiency but preserves an approximately 30% gap between clean and patched validation accuracy.
  • The multi-class experiments combined 10 random pairs into 20 ImageNet categories, using 400 poisoned images added to the target category.
  • 84.3 ± 9.2% clean validation accuracy and 69.3 ± 14.8% attack success were obtained for single-source multi-class attacks.
  • 88.5 ± 0.3% clean validation accuracy and 30.7±6.3% attack success were obtained for multi-source attacks, compared with 5% random chance.

4.8. Spectral signatures for backdoor attack detection

The paper tests whether spectral signatures can detect hidden-trigger poisoned images. Under a favorable assumption that the defense knows the poisoned category, the defense usually detects few or none of the poisoned images.

  • The evaluation applies Tran et al.'s spectral-signatures defense to search for poisoned data in the target class.
  • Using the default 85% threshold, the defense should identify 135 of 800 images despite only 100 being poisoned.
  • The defense finds no poisoned images in most category pairs and nearly half in only one pair, even when given the poisoned category.
  • The authors attribute the weak detection to limited empirical separation between target and poisoned data.

5. Conclusion

The paper concludes that hidden trigger attacks use natural-looking, correctly labeled poisoned data while keeping the trigger secret until test time. Experiments across datasets and settings show attack effectiveness and weak defense performance.

  • The attack activates when a small patch is added at a random location on an unseen image during testing.
  • Poisoned data look natural, carry clean labels, and do not reveal the trigger before the actual attack.
  • Spectral signatures does not find many of the poisoned images, with the table reporting zero detections for many category pairs.
  • The attack works across two datasets and various settings.
  • The authors identify hidden trigger attacks as a vulnerability requiring better defenses before deployment in critical real-world applications.
Loading 1910.00033v2…