PaperlayerView exact revision on arXiv

Source-linked AI summary

Dynamic Backdoor Attacks Against Machine Learning Models

Ahmed Salem, Rui Wen, Michael Backes, Shiqing Ma, Yang Zhang

arXiv:2003.03675v2cs.CRcs.LGstat.ML

TL;DR

Existing backdoor attacks rely on fixed trigger patterns and locations, motivating techniques that are harder for current defenses to detect. The paper proposes Random Backdoor, BaN, and c-BaN, which generate dynamic triggers for DNNs. Across three benchmark datasets, these techniques achieve near-perfect backdoor success while preserving utility and bypassing evaluated defenses.

  • Problem

    Existing DNN backdoor attacks use static trigger patterns and locations, which current backdoor defenses leverage for detection.

  • Method

    The paper proposes Random Backdoor, BaN, and c-BaN, including generative and target-label-conditioned trigger generation.

  • Results

    Across three benchmark datasets, all techniques achieve almost perfect backdoor success while preserving model utility and bypassing state-of-the-art defenses.

  • Takeaways & Limitations

    Dynamic triggers broaden backdoor attacks beyond fixed patterns and locations while retaining strong attack performance against evaluated defenses.

  • Takeaways & Limitations

    Random Backdoor and BaN require different target labels to use disjoint trigger-location sets; c-BaN is introduced to overcome this limitation.

Abstract

from arXiv · show

Machine learning (ML) has made tremendous progress during the past decade and is being adopted in various critical real-world applications. However, recent research has shown that ML models are vulnerable to multiple security and privacy attacks. In particular, backdoor attacks against ML models have recently raised a lot of awareness. A successful backdoor attack can cause severe consequences, such as allowing an adversary to bypass critical authentication systems. Current backdooring techniques rely on adding static triggers (with fixed patterns and locations) on ML model inputs which are prone to detection by the current backdoor detection mechanisms. In this paper, we propose the first class of dynamic backdooring techniques against deep neural networks (DNN), namely Random Backdoor, Backdoor Generating Network (BaN), and conditional Backdoor Generating Network (c-BaN). Triggers generated by our techniques can have random patterns and locations, which reduce the efficacy of the current backdoor detection mechanisms. In particular, BaN and c-BaN based on a novel generative network are the first two schemes that algorithmically generate triggers. Moreover, c-BaN is the first conditional backdooring technique that given a target label, it can generate a target-specific trigger. Both BaN and c-BaN are essentially a general framework which renders the adversary the flexibility for further customizing backdoor attacks. We extensively evaluate our techniques on three benchmark datasets: MNIST, CelebA, and CIFAR-10. Our techniques achieve almost perfect attack performance on backdoored data with a negligible utility loss. We further show that our techniques can bypass current state-of-the-art defense mechanisms against backdoor attacks, including ABS, Februus, MNTD, Neural Cleanse, and STRIP.

1 Introduction

The paper introduces dynamic backdoor attacks for DNN image classifiers, replacing fixed trigger patterns and locations with flexible, varying triggers. Its three techniques aim to retain attack performance while reducing the efficacy of existing defenses.

  • Motivation: Current backdoor attacks use fixed trigger patterns and locations, a static property leveraged by most existing defenses.Badnets, for example, uses a white square consistently placed in the input’s top-left corner.
  • Contributions: The paper proposes three dynamic techniques: Random Backdoor, BaN, and c-BaN.BaN and c-BaN algorithmically generate triggers, while c-BaN additionally conditions generation on the target label.
  • Random Backdoor: Random Backdoor samples a trigger from a uniform distribution and places it at a random location for each input during training.The generated triggers are mixed with clean data to train the backdoored model.
  • Backdoor Generating Network (BaN): BaN jointly trains a generative model with the backdoor model to produce triggers from latent codes and place them at random input locations.Its framework allows the adversary to adapt the generator’s loss function to specific requirements, including defense evasion.
  • Conditional Backdoor Generating Network (c-BaN): c-BaN inputs the target label into BaN to generate label-specific triggers that can occupy overlapping locations across target labels.This removes the requirement that different target labels use disjoint trigger-location sets.
  • Evaluation: Across three benchmark datasets, the techniques achieve approximately 100% backdoor accuracy with negligible utility loss and bypass evaluated state-of-the-art defenses.BaN matches clean-model accuracy on CelebA and MNIST, while all three techniques achieve nearly clean-model accuracy on CIFAR-10.

2 Preliminaries

The paper formalizes backdoor attacks as hidden model behavior activated by trigger-bearing inputs and motivates dynamic triggers as harder to detect than static ones. It also states the training-time threat model and the backdoor adding operation.

  • Backdoor setting: A backdoored model misclassifies inputs containing secret triggers to attacker-selected target labels while preserving clean-data functionality.Training uses both clean data and backdoored data formed by adding triggers to a subset of clean inputs.
  • Static versus dynamic triggers: Static attacks use fixed trigger patterns and locations, whereas dynamic backdoors vary both properties.The paper argues that varying patterns and locations makes detection harder because existing defenses assume static behavior.
  • Backdoor construction: The backdoor adding function A inserts trigger t_i at location κ using a binary mask, producing a modified input x_bd.The operation is defined as A(x,t_i,κ)=t_i·κ+x·(1−κ).
  • Threat model: The adversary is assumed to control target-model training and access training data, with a relaxed setting allowing only training-data poisoning.After publication, the trigger may be added digitally or physically before querying the model.

3 Dynamic Backdoors

The paper introduces three dynamic backdoor techniques that vary trigger patterns and locations, including generative methods and a conditional generator for target-specific triggers. The methods support single- and multiple-target-label settings with increasing flexibility.

  • Overview: The paper proposes Random Backdoor, Backdoor Generating Network (BaN), and conditional Backdoor Generating Network (c-BaN).BaN and c-BaN algorithmically generate triggers, while c-BaN additionally conditions generation on the target label.
  • Random Backdoor: Random Backdoor samples trigger values from a distribution and places each trigger at a random location during training.The basic implementation samples t from U(0,1) and κ from a predefined location set K; other distributions can alter trigger appearance.
  • Random Backdoor: For multiple labels, Random Backdoor partitions the possible locations into disjoint subsets so each target label has its own location set.The paper allows arbitrary partitions subject to the restriction that no location is shared by multiple target labels.
  • BaN: BaN jointly trains a trigger-generating network with the backdoored model, using a latent code to generate triggers that are placed randomly.Unlike an image-generating GAN, BaN generates backdoor triggers and learns patterns jointly with the target model.
  • c-BaN: c-BaN adds the target label as an input to BaN, generating label-specific triggers that allow different target labels to use any location in K.This removes the earlier requirement that multiple labels have disjoint trigger-location sets.

4 Evaluation

The evaluation uses MNIST, CelebA, and CIFAR-10 with dataset-specific models and measures backdoor success rate and model utility. Utility is assessed by comparing backdoored-model accuracy with clean-model accuracy on clean data.

  • Experimental setup: The experiments evaluate Random Backdoor, BaN, and c-BaN on MNIST, CelebA, and CIFAR-10 benchmark datasets.The target models include custom CNNs for MNIST and CelebA and VGG-19 for CIFAR-10.
  • Datasets: MNIST contains 70,000 grayscale 28 × 28 images across 10 balanced digit classes, while CIFAR-10 contains 60,000 colored 32×32 images across 10 classes.CelebA contributes more than 200K celebrity face images with 40 binary attributes; the experiments select three attributes and form eight classes.
  • Metrics: Backdoor success rate measures accuracy on backdoored data, while model utility compares backdoored-model and clean-model accuracy on clean data.Closer clean-data accuracies indicate better model utility.

4.3 Random Backdoor

Random Backdoor constructs dynamic triggers with varied patterns and locations while preserving clean-model utility and achieving a fully functioning backdoor across MNIST, CelebA, and CIFAR-10.

  • 100% backdoor success rate was achieved for both single and multiple target labels across all three datasets.
  • Single Target Label: In the single-target setting, MNIST and CelebA matched clean-model accuracy at 99% and 70%, while CIFAR-10 dropped by less than 2%.
  • Random Backdoor triggers differed in appearance and location across randomly sampled MNIST inputs.
  • Multiple Target Labels: Multiple-target evaluation retained approximately 100% backdoor success across all target labels while focusing utility assessment on clean testing data.
  • For CIFAR-10, Random Backdoor reached 92% accuracy versus 92.4% for the clean model.
  • Across target labels, the technique used visibly different trigger patterns and locations, including vertically varying positions.

4.4 Backdoor Generating Network (BaN)

BaN algorithmically generates dynamic triggers and maintains clean-model performance while supporting nearly perfect backdoor success for single and multiple target labels.

  • Single Target Label: BaN achieved a perfect backdoor success rate with negligible utility loss in the single-target setting.
  • 99%, 92.4%, and 70% clean-test accuracy on MNIST, CIFAR-10, and CelebA matched the corresponding clean models.
  • Adding one dropout layer increased trigger diversity while retaining 99% clean accuracy and 100% backdoor success on MNIST.
  • Multiple Target Labels: BaN achieved approximately 100% backdoor success for all target labels in the multiple-target setting.
  • For CIFAR-10, BaN reached 92.1% accuracy on clean data versus 92.4% for the clean model.
  • BaN triggers varied in pattern and location across different target labels and across inputs sharing the same target label.

4.5 conditional Backdoor Generating Network (c-BaN)

c-BaN generates target-specific dynamic triggers and, alongside the other proposed techniques, preserves clean accuracy while achieving near-perfect attack performance and evading several defenses.

  • All three dynamic techniques achieved almost perfect backdoor success while maintaining clean-test performance similar to clean models.
  • On CIFAR-10, c-BaN, BaN, and Random Backdoor achieved 92%, 92.1%, and 92% accuracy versus 92.4% for the clean model.
  • Grad-CAM maps showed backdoored models focusing mainly on triggers for backdoored inputs and main objects for clean inputs.
  • Model-based Defense: ABS detected no backdoor in the evaluated dynamic models, while Neural Cleanse classified all models as clean.
  • The authors attribute defense failures to violated static-trigger assumptions and backdoors implemented for all possible labels.
  • Model-based Defense: MNTD scores fell from 67.08(± 20.49) for static backdoors to 3.05(± 0.82), 0.54(± 0.83), and 1.47(± 0.87) for Random Backdoor, BaN, and cBaN.
  • Data-based Defense: For some target labels, dynamic backdoored entropy distributions overlapped clean distributions, making threshold-based detection difficult without increased false positives.
  • Data-based Defense: Februus reduced attack success from 100% to approximately 80.5%, 81.7%, and 72% for Random Backdoor, BaN, and cBaN, versus 0.25% for static backdoors.

4.7 Evaluating Different Hyperparameters

Hyperparameter experiments show that dynamic backdoors tolerate more locations, require sufficiently large triggers, and can remain effective when triggers are made substantially more transparent.

  • Number of Locations: Using 8 or 16 possible locations, c-BaN achieved 92% clean accuracy and 100% backdoor accuracy on CIFAR-10.
  • Trigger Size: On MNIST, a trigger size of 5 achieved 99% clean-test accuracy and 100% backdoored-test accuracy.
  • Trigger Size: Smaller triggers made backdoor implementation harder and reduced model utility.
  • Transparency of the Triggers: With transparency scale 0.5 or higher, c-BaN retained 99% clean accuracy and 100% backdoored accuracy on MNIST.
  • Transparency of the Triggers: With image-sized triggers at scale 0.1, CIFAR-10 attacks retained 100% success with a 0.3% utility drop.

4.8 Relaxing the Threat Model (Transferability of the Triggers)

The threat model can be relaxed so the adversary poisons only the training dataset, while dynamic triggers remain effective and transferable from pre-trained generators.

  • Random Backdoor requires only poisoning training images and their target labels, without changing the target model’s training procedure.
  • Pre-trained BaN and c-BaN models can generate triggers without being trained jointly with the target model.
  • CIFAR-10-pre-trained BaN and c-BaN models were used to generate poisoned MNIST datasets, demonstrating transfer across data distributions.
  • 100% attack success rate was achieved while preserving the utility of jointly trained backdoored models.
  • Poisoning can be lowered to about 10% when the generator and target models are not jointly trained.

4.9 Possible Defenses

The proposed autoencoder defense can remove dynamic triggers effectively on simple data, but reconstruction difficulty on complex data causes substantial utility and attack-performance losses. Data augmentation reduces attack success but does not prevent the attacks and can also reduce utility.

  • Autoencoder defense: The defense denoises every input with an autoencoder trained on clean data before forwarding it to the target model.
  • Autoencoder defense: On MNIST, autoencoder denoising successfully defends against c-BaN with less than 1% utility loss.
  • Autoencoder defense: On CIFAR-10, clean-data accuracy drops by 4.8% and backdoored-data accuracy drops by 25% because complex details hinder reconstruction.
  • Data augmentation: Resizing reduces ASR to approximately 57.4% and 66.5% for Random Backdoor and c-BaN, while utility falls by 15.4% and 15.9%, respectively.
  • Data augmentation: Data augmentation can reduce attack performance but cannot prevent dynamic backdoor attacks and may substantially reduce utility.

5 Related Work

The paper positions dynamic backdoors against static attacks and existing defenses, while distinguishing its trigger structure, threat model, and broader applicability from related methods.

  • Backdoor attacks: BadNets and Trojan attacks use static trigger patterns and locations, whereas this work introduces dynamic patterns and locations.
  • Backdoor attacks: The related input-aware dynamic backdoor generates a unique trigger per input, while this work uses square-like triggers that can be applied to physical objects or images.
  • Backdoor attacks: This work can relax its threat model to dataset poisoning alone and can generate multiple triggers for the same image.
  • Scope and related attacks: The paper focuses on image classification, while related work extends backdoor attacks to federated learning, video recognition, transfer learning, and NLP.
  • Defenses against backdoor attacks: Backdoor defenses include model-based methods such as Neural Cleanse and data-based methods such as STRIP and Februus.
  • Other attacks against machine learning: Other related machine-learning attacks include poisoning, adversarial examples, membership inference, model stealing, model inversion, property inference, and dataset reconstruction.

6 Conclusion

The paper introduces dynamic backdoor attacks that vary trigger patterns and locations, evaluates three techniques across benchmark datasets, and reports near-perfect attack success with preserved utility and defense bypass.

  • The paper addresses the vulnerability of critical ML applications to backdoor attacks that intentionally misclassify inputs containing triggers.
  • The proposed dynamic backdoors allow multiple trigger patterns and locations, unlike current static attacks.
  • Random Backdoor samples triggers uniformly at random locations, BaN constructs triggers generatively, and c-BaN generates label-specific triggers.
  • Across three benchmark datasets, all techniques achieve almost perfect backdoor success rates while preserving model utility and bypassing state-of-the-art defenses.

A Additional Experimental Results

Figure 14 examines cBaN performance when data augmentation techniques are used during target-model training, reporting utility and attack success rate (ASR).

  • Figure 14 reports cBaN's utility and ASR when data augmentation techniques are applied during target-model training.
Loading 2003.03675v2…