PaperlayerView exact revision on arXiv

Source-linked AI summary

Label-Consistent Backdoor Attacks

Alexander Turner, Dimitris Tsipras, Aleksander Madry

arXiv:1912.02771v2stat.MLcs.CRcs.LG

TL;DR

Backdoor attacks are effective but standard poisoned samples are often visibly mislabeled, creating a detection problem for human inspection. The paper uses adversarial perturbations and generative-model latent embeddings to create plausible, label-consistent samples that remain hard to classify. It shows that effective label-consistent backdoor attacks are possible and argues that human inspection may be insufficient against them.

  • Problem

    Standard backdoor attacks rely on often-mislabeled poisoned inputs, motivating attacks whose inputs remain consistent with their labels and evade human inspection.

  • Method

    The paper perturbs original inputs to make them harder to classify while preserving labels, using adversarial perturbations and generative-model latent embeddings.

  • Results

    The proposed methods produce effective backdoor attacks while maintaining label consistency.

  • Takeaways & Limitations

    Backdoor attacks can be made significantly harder for humans to detect, indicating a need for principled defenses beyond human inspection.

  • Takeaways & Limitations

    Label consistency is maintained under restricted perturbations, specifically τ = 0.2 and ε = 300 in ℓ2-norm, based on inspected images.

Abstract

from arXiv · show

Deep neural networks have been demonstrated to be vulnerable to backdoor attacks. Specifically, by injecting a small number of maliciously constructed inputs into the training set, an adversary is able to plant a backdoor into the trained model. This backdoor can then be activated during inference by a backdoor trigger to fully control the model's behavior. While such attacks are very effective, they crucially rely on the adversary injecting arbitrary inputs that are---often blatantly---mislabeled. Such samples would raise suspicion upon human inspection, potentially revealing the attack. Thus, for backdoor attacks to remain undetected, it is crucial that they maintain label-consistency---the condition that injected inputs are consistent with their labels. In this work, we leverage adversarial perturbations and generative models to execute efficient, yet label-consistent, backdoor attacks. Our approach is based on injecting inputs that appear plausible, yet are hard to classify, hence causing the model to rely on the (easier-to-learn) backdoor trigger.

1 Introduction

Backdoor attacks can control model predictions through trigger-activated behavior, but standard poisoned samples are often visibly mislabeled and vulnerable to human detection. This work creates label-consistent poisoned inputs by making samples harder to classify while preserving plausible labels.

  • Motivation: Backdoor attacks plant trigger-activated behavior in models trained on poisoned datasets, while appearing normal without the trigger.The trigger can force a chosen target label on previously arbitrary inputs during inference.
  • Motivation: Standard attacks randomly trigger natural samples and relabel them, producing suspiciously incorrect input-label pairs.Human inspection may reveal the attack because the poisoned images are clearly mislabeled.
  • Research question: The paper asks whether backdoor attacks can succeed when poisoned inputs and labels appear consistent to humans.Such attacks would be difficult to detect even through human inspection.
  • Key insight: The authors argue that poisoned inputs must be hard to classify from salient natural features, otherwise models may ignore the trigger.Using correctly labeled inputs alone is reported to be ineffective.
  • Approach: The proposed methods perturb original inputs while keeping changes sufficiently minor for label consistency, using GAN latent-space interpolation or adversarial perturbations.Both methods produce successful backdoor attacks while maintaining label consistency.
  • Additional contributions: The work also develops less conspicuous, augmentation-robust triggers and studies how models memorize backdoor triggers.The trigger modifications are illustrated alongside label-consistent poisoned inputs.

2 Background

The paper frames backdoor attacks within standard supervised classification and a threat model where an adversary injects limited training samples. It defines attack success and label consistency while situating the approach among poisoning and backdoor literature.

  • 2.1 Setting: In standard classification, model parameters minimize a loss over labeled training examples sampled from a data distribution.The model maps inputs to labels through parameters learned from the training set.
  • 2.1 Setting: Training emphasizes difficult examples because stochastic-gradient updates give greater weight to inputs on which the model performs poorly.This motivates using hard-to-classify poisoned inputs to influence the trained model.
  • 2.2 Threat model: The adversary chooses a target label and trigger function, creates limited arbitrary samples, and injects them into the training set.The general threat model assumes access to the model, training procedure, and underlying data distribution, with a weaker setting also considered.
  • 2.2 Threat model: Attack success rate is the fraction of non-target test samples classified as the target label after the trigger is applied.The metric evaluates triggered behavior on previously unseen test samples.
  • 2.2 Threat model: A label-consistent attack requires every injected poisoned sample to appear correctly labeled to a human.This definition focuses on avoiding suspicious samples in the training set.
  • Related work: Prior poisoning work commonly targets test-accuracy degradation, while prior label-consistent attacks addressed other settings such as targeted poisoning.Earlier backdoor research also studied less visible triggers and detection based on trigger or latent structure.

3 Limitations of standard backdoor attacks

Standard backdoor attacks are effective but expose themselves through poisoned samples that are often mislabeled and therefore detectable by sanitization or human inspection.

  • Standard backdoor attack: Standard attacks select natural samples, apply a trigger, relabel them to the adversary’s target, and inject them into training.The resulting model is intended to associate the trigger with the chosen target label.
  • Detectability: The injected samples are likely mislabeled, making them vulnerable to outlier detection followed by human inspection.A simple sanitization scheme trains a classifier on a small clean dataset before inspecting identified outliers.

4 Towards label-consistent backdoor attacks

Label-consistent backdoor attacks address the detectability of standard attacks by making poisoned inputs appear correctly labeled while remaining difficult to classify. The paper proposes latent-space interpolation and adversarial perturbations, alongside less conspicuous and augmentation-resistant triggers.

  • 4.1 An (ineffective) baseline: Restricting standard backdoor poisoning to correctly labeled target-class inputs is ineffective because salient features let the model ignore the trigger.Even at 25% poisoning, only one class exceeds 50% attack success; correctly labeled samples provide little incentive to associate the trigger with the target label.
  • 4 Towards label-consistent backdoor attacks: The proposed strategy makes poisoned inputs harder to classify while preserving label consistency through minor transformations of natural inputs.The underlying rationale is that difficult-to-learn inputs increase reliance on the easier-to-learn backdoor trigger.
  • 4.2 Method 1: Latent space interpolation: GAN-based interpolation moves a target-class input toward an incorrect class in latent space, producing a smooth transition whose transformed sample retains the target label.The method embeds both inputs, interpolates their latent representations using τ, and applies the trigger to the resulting sample.
  • 4.3 Method 2: Adversarial perturbations: Adversarial perturbations maximize an independently trained model’s loss within a small ℓp-norm constraint, making poisoned inputs harder to learn without changing their ground-truth labels.The perturbation is computed before trigger application, using PGD and an independent pre-trained model unavailable to the adversary during training.
  • 4.4 Improving backdoor trigger design: Trigger design is improved by reducing pixel perturbation amplitude and replicating the pattern across four corners so it remains less visible and survives common augmentations.Reduced-amplitude triggers can remain learnable, while four-corner placement makes the trigger invariant under flips and visible under crops.

5 Evaluating our attacks

The evaluation shows that label-consistent attacks achieve high attack success with few poisoned inputs, while improved triggers preserve effectiveness under reduced visibility and data augmentation.

  • 5.1 Basic attack: Larger perturbations produce higher attack success rates, although label plausibility decreases as perturbation strength grows.The experiments vary GAN interpolation coefficient τ and ℓp-norm bound ε; the authors restrict perturbations to τ = 0.2 and ε = 300 in ℓ2-norm to preserve label-consistency.
  • 5.1 Basic attack: More than 50% attack success rate is achieved on half the classes using only 75 label-consistent poisoned inputs.The perturbation-based attack contrasts with a label-consistent baseline that is ineffective below 300 injected inputs.
  • 5.1 Basic attack: 92% test accuracy with data augmentation and 87% without indicates that attacks generally do not affect standard accuracy.The exception is 100% poisoning, where the model predicts incorrectly on the entire target class.
  • 5.2 Improved trigger attacks: With reduced trigger visibility, attacks remain successful when poisoning more than 6% of the target class, or 300 images.At 1.5% poisoning, less visible triggers are ineffective; for 6% poisoning, increasing amplitude from 16 to 64 improves performance approximately log-linearly.
  • 5.2 Improved trigger attacks: Data augmentation makes the one-corner trigger essentially ineffective but improves attack success for the modified four-corner trigger.Without augmentation, the two trigger designs perform similarly.
  • 5.2 Improved trigger attacks: At least 300 injected inputs are required for successful attacks with the less visible pattern and data augmentation, while a fully visible inference trigger can raise success rates to essentially 100%.The injected inputs appear benign and similar to natural samples.

6 Exploring the underlying attack mechanism

The comparison favors adversarial perturbations over GAN interpolation, and supporting experiments indicate that attack success depends on creating hard-to-classify inputs while preserving useful input features.

  • 6.1 On the relative performance of latent interpolations and adversarial examples: ℓp-bounded adversarial perturbations are more effective than GAN-based interpolation, especially when the allowed perturbation is large.Both methods construct inputs that are hard to classify, but their attack performance differs.
  • 6.1 On the relative performance of latent interpolations and adversarial examples: GAN-interpolated inputs may be blurry and lack salient features, limiting attack success on natural inference inputs.The proposed explanation is that weak trigger associations learned from such inputs do not transfer well to natural inputs.
  • 6.1 On the relative performance of latent interpolations and adversarial examples: Adversarial perturbations introduce features of an incorrect class that can remain visible while making inputs harder to classify.This provides a proposed explanation for their stronger relative performance.
  • 6.2 On the mechanism behind the attack: Small Gaussian noise can improve attack effectiveness, whereas increasing noise variance has an adverse effect.The result supports the hypothesis that modest difficulty increases preserve original features better than excessive noise.
  • 6.2 On the mechanism behind the attack: Poisoned inputs have similar or smaller training loss than clean inputs, while poisoned inputs without the trigger retain high loss throughout training.This pattern indicates that the model predicts poisoned samples correctly with the trigger but not without it.

7 Conclusion

The paper establishes label-consistent backdoor attacks by synthesizing inputs that are difficult to classify from natural features, making the attacks harder for humans to detect.

  • 7 Conclusion: Label-consistency is identified as a key desired property because clearly mislabeled poisoned samples make attacks likely to be detected.Previous backdoor attacks generally lack this property.
  • 7 Conclusion: Effective label-consistent backdoor attacks are constructed using adversarial perturbations and latent embeddings from generative models.The central design is to make poisoned inputs difficult to classify based on their natural salient features.
  • 7 Conclusion: Hard-to-learn samples provide a framework for developing more insidious attacks and motivate principled defenses against attacks that are harder for humans to detect.The authors state that further work may produce attacks that are more effective and bypass sophisticated filtering schemes.

A Experimental setup

The experiments use a residual network trained with momentum optimization, alongside WGAN-based interpolation and PGD-based adversarial perturbation implementations.

  • A Experimental setup: The model is a standard ResNet with three residual-layer groups and five residual units per group.The groups use filter sizes 16, 16, 32, and 64.
  • A Experimental setup: Training uses momentum 0.9, weight decay 0.0002, batch size 50, batch normalization, and a stepped learning-rate schedule.The step size starts at 0.1, decreases to 0.01 at 40,000 steps, and to 0.001 at 60,000 steps.
  • A Experimental setup: GAN-based interpolation uses a WGAN and latent-space optimization over 1,000 gradient-descent steps with step size 0.1.The GAN is trained on images from the two classes involved in interpolation.
  • A Experimental setup: Adversarial examples are generated with a 100-step PGD attack using step size 1.5 ε/100.The attack is run on adversarially trained models using a publicly available implementation.

B Reproducing standard backdoor attacks

The reproduced Gu et al. attack remains highly effective on CIFAR-10 with roughly 75 poisoned samples, but its labels are visibly incorrect.

  • The reproduction uses a standard ResNet architecture on CIFAR-10.The original setting did not thoroughly study the number of poisoned examples required.
  • Approximately 75 poisoned samples are sufficient for a highly successful attack.Figure 11 reports 75 samples, corresponding to 0.15% of the training data.
  • The attack remains very effective across different target labels.
  • Most poisoned image labels are clearly incorrect, creating a human-inspection liability.Such labels could reveal the attack if the poisoned inputs are inspected.

C Weaker adversaries

The framework is evaluated against weaker adversaries using pixel interpolation and adversarial perturbations. These attacks outperform naive baselines, although weaker adversarial perturbations trail the white-box attack.

  • The evaluation considers attackers with less knowledge of the model architecture and training data.
  • Pixel-space interpolation: Pixel-space interpolation generates poisoned samples by linearly mixing an original image with a target image.The interpolation coefficient τ controls the mixture.
  • Pixel-space interpolation: Pixel-space interpolation outperforms the naive baseline but remains weaker than GAN-based interpolation.
  • Weaker adversarial perturbations: The weaker adversarial setting targets an adversarially trained VGG-style model trained on randomly selected CINIC-10 images.
  • Weaker adversarial perturbations: Weaker adversarial perturbations remain effective and significantly outperform the other baselines.They are less powerful than the white-box attack.

D Omitted Figures

The omitted figures document label-consistent attack behavior, trigger-visibility effects, hard-to-classify poisoned inputs, and visual plausibility across interpolation and perturbation methods.

  • Both label-consistent attacks achieve significantly higher success rates than the baseline across target classes.The attacks use GAN-based interpolation with τ = 0.2 and adversarial perturbations bounded by ε = 300 in ℓ2 norm.
  • The per-class comparison includes the restricted Gu et al. baseline, pixel-wise interpolation, GAN-based interpolation, and adversarial examples.
  • Using a full-visibility trigger during inference greatly increases attack success relative to reduced visibility.The reduced-visibility experiment uses trigger amplitude 16.
  • With data augmentation, experiments compare amplitude-32 training triggers against full-amplitude inference triggers.The figures cover GAN-based interpolation and adversarial perturbation attacks using a four-corner pattern.
  • Poisoned inputs without the trigger retain high training loss, indicating reliance on the backdoor trigger for classification.
  • Manual inspection finds the proposed poisoned inputs appear label-consistent rather than clearly mislabeled.The inspection compares the original attack with proposed attacks and reports no clearly mislabeled examples in the displayed samples.
Loading 1912.02771v2…