PaperlayerView exact revision on arXiv

Source-linked AI summary

Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation

Cong Liao, Haoti Zhong, Anna Squicciarini, Sencun Zhu, David Miller

arXiv:1808.10307v1cs.CRcs.LGstat.ML

TL;DR

The paper studies how adversaries can implant targeted backdoors in CNN classifiers without materially disrupting normal performance. It develops two stealthy perturbation-mask strategies for pre-training and model-update settings, reporting attack success above 90% with around 1% injection and below 1% accuracy loss, including under minimal attacker knowledge.

  • Problem

    Security-critical deep learning deployments are vulnerable to stealthy poisoning attacks that implant attacker-chosen target-label behavior while preserving normal model efficacy.

  • Method

    The paper designs patterned static and targeted adaptive perturbation masks and evaluates their injection before training and during model updating.

  • Results

    Above 90% attack success rate, around 1% injection rate, and below 1% classification-accuracy loss are reported, including under the weakest adversary assumption.

  • Takeaways & Limitations

    Perturbation-mask backdoors can be both visually stealthy and effective across varied model-learning settings and adversary assumptions.

  • Takeaways & Limitations

    Potential defenses based on labeling test samples are human-laborious, while feature-space clustering is described as sophisticated and speculative future work.

Abstract

from arXiv · show

Deep learning models have consistently outperformed traditional machine learning models in various classification tasks, including image classification. As such, they have become increasingly prevalent in many real world applications including those where security is of great concern. Such popularity, however, may attract attackers to exploit the vulnerabilities of the deployed deep learning models and launch attacks against security-sensitive applications. In this paper, we focus on a specific type of data poisoning attack, which we refer to as a {\em backdoor injection attack}. The main goal of the adversary performing such attack is to generate and inject a backdoor into a deep learning model that can be triggered to recognize certain embedded patterns with a target label of the attacker's choice. Additionally, a backdoor injection attack should occur in a stealthy manner, without undermining the efficacy of the victim model. Specifically, we propose two approaches for generating a backdoor that is hardly perceptible yet effective in poisoning the model. We consider two attack settings, with backdoor injection carried out either before model training or during model updating. We carry out extensive experimental evaluations under various assumptions on the adversary model, and demonstrate that such attacks can be effective and achieve a high attack success rate (above $90\%$) at a small cost of model accuracy loss (below $1\%$) with a small injection rate (around $1\%$), even under the weakest assumption wherein the adversary has no knowledge either of the original training data or the classifier model.

1 INTRODUCTION

The paper addresses stealthy backdoor injection attacks against security-critical deep learning systems. It proposes barely perceptible perturbation-based backdoors that preserve normal accuracy while enabling targeted misclassification.

  • Motivation: Deep learning’s use in security-critical applications creates attack opportunities for adversaries to exploit deployed models.Examples include traffic-sign recognition, spam filtering, and face-recognition systems.
  • Problem: A backdoor injection attack poisons training data with perturbed samples carrying attacker-chosen target labels.The small poisoning fraction is intended to avoid undermining the model’s normal functioning.
  • Approach: The proposed attack considers injection before model training and during incremental model updating.Both settings use a small number of crafted backdoor samples inserted into training data.
  • Results: Above 90% attack success rate, below 1% classification-accuracy loss, and around 1% injection rate are reported across evaluated scenarios.The attack remains successful even when the adversary lacks knowledge of the original training data and classifier model.
  • Contributions: The paper proposes patterned static and targeted adaptive perturbation masks that are visually difficult to notice and minimally impair normal model behavior.The stated contribution also covers effectiveness across different learning settings and adversary assumptions.
  • Motivation: Earlier backdoor images could contain visually identifiable anomalies, undermining stealthiness.This motivates the paper’s focus on less perceptible perturbations.

2 BACKDOOR INJECTION ATTACK

The paper defines backdoor injection as targeted poisoning that teaches a CNN an additional, attacker-selected mapping while preserving normal classification performance. It formalizes goals, threat models, and adversary knowledge levels.

  • Attack Definition: A poisoned dataset can associate crafted instances with a target label, causing the learned model to recognize both normal and abnormal mappings.The abnormal input-label pattern constitutes the backdoor.
  • Attack Goals: The attack aims to trigger targeted misclassification of backdoor instances without significantly degrading overall model performance.The target class is specified by the adversary for samples drawn from a particular source class.
  • Attack Goals: The attacker’s required goals are high attack success, high backdoor stealthiness, and low performance impact.For image classification, stealthiness includes visual imperceptibility and resistance to machine detection.
  • Adversary Model: The paper distinguishes full, partial, and minimal knowledge according to the adversary’s access to model details and training data.Minimal Knowledge assumes neither model specifics nor original training data are known.
  • Adversary Capabilities: Under minimal knowledge, the adversary may rely on a general understanding of the model’s functionality and data type or collect similarly distributed data.Greater knowledge permits leveraging model characteristics or training instances when constructing backdoor samples.

3 ATTACK OVERVIEW

The attack pipeline generates a perturbation mask, injects masked samples with a target label, and trains or updates a CNN using poisoned data. It supports both pre-training and model-update injection settings.

  • Attack Pipeline: The attack has three phases: generating a perturbation mask, injecting backdoor samples, and training with poisoned data.These phases form the paper’s overall attack procedure.
  • Backdoor Generation: A perturbation mask changes relative pixel intensities rather than adding a concrete visual pattern, making the backdoor less discernible.The resulting image remains bounded by [0, 255], and samples sharing a mask receive the same target label.
  • Backdoor Generation: The paper proposes static patterned and targeted adaptive perturbation masks for generating CNN backdoors.The adaptive strategy is based on a targeted class of samples.
  • Injection Settings: Backdoor Injection Before model training inserts a small number of crafted samples into the original dataset before training a new model.Backdoor Injection During model updating instead inserts samples into data used to update an existing pretrained model.
  • Injection Settings: During updating, backdoor samples are inserted into sequential data batches and used to update all CNN parameters.The attacker can adjust injection timing, amount, and rate, while continued pristine-data updates can gradually remove the backdoor.
  • Poisoned Training: The poisoned training procedure minimizes cross-entropy loss over ordinary training samples and injected backdoor samples.The model is expected to assign the target label the highest posterior probability for a tested backdoor instance.

4 BACKDOOR GENERATING STRATEGIES

The paper develops two stealthy backdoor-generation strategies: a repeated patterned static mask and a data- and model-aware targeted adaptive mask. Both create poisoned samples through small image perturbations, with the adaptive method iteratively moving samples toward a target-class decision boundary.

  • The attack requires perturbations that are both learnable by the model and minimally visible to human observers.The perturbation must balance attack effectiveness with visual stealthiness.
  • 4.1 Patterned Static Perturbation Mask: The static strategy treats the repeated perturbation as an image pattern that CNN filters can learn through spatially local correlations.Its main limitation is that the same repeated pattern is used regardless of image content or classifier.
  • 4.1 Patterned Static Perturbation Mask: The patterned static mask repeats one fixed intensity change at corresponding positions across equal-sized image sub-regions.The mask is added to the original image to create a backdoor sample; examples use intensity changes of 6 and 10.
  • 4.2 Targeted Adaptive Perturbation Mask: The adaptive strategy uses training data and an existing classifier to generate a backdoor specific to the attack scenario.It iteratively computes additional perturbations for samples that are not yet classified as the target class.
  • 4.2 Targeted Adaptive Perturbation Mask: For each source-class sample, targeted DeepFool computes a minimum perturbation toward the target-class decision boundary and accumulates it into a universal mask.The method constrains the perturbation with an l∞ bound and can stop after a maximum iteration threshold.

5 EXPERIMENTAL EVALUATION

Experiments evaluate backdoor injection across datasets, perturbation types, attack settings, adversary knowledge assumptions, injection rates, and stealthiness metrics. Adaptive perturbations generally achieve attack success rates above 90% while keeping test accuracy loss near or below 1%, including under limited knowledge.

  • 5.5 Evaluation of Backdoor Injection Attack Under Various Scenarios: Adaptive perturbation generally exceeds 90% attack success rate while keeping test accuracy loss below or near 1%; static perturbation remains below 90% and can lose around 3.1% accuracy.These comparisons hold across corresponding BIB and BID scenarios, with static attacks sometimes failing.
  • 5.5.1 Attack Performance: BID attacks generally affect test accuracy more than BIB attacks, while adaptive perturbations limit this impact and usually retain similar success rates across settings.The paper attributes the greater BID impact to catastrophic forgetting during incremental learning.
  • 5.5.1 Attack Performance: 6,000 adaptive backdoor samples achieved above 90% average attack success in BIB-MK at a 2.8% injection ratio and max intensity change 10.BID-MK achieved comparably decent performance at a 1.4% injection rate, while the best static results remained 88.22% and 87.73%.
  • 5.5.3 Effect of Source of Injection Data and Knowledge of Pretrained Model: Adaptive perturbation is less dependent on injection-data source in BIB, with attack success increasing from 91.6% to 97.64% across the reported knowledge conditions.Static perturbation increased from 50.85% to 88.22% in the corresponding comparison.
  • 5.5.4 Effect of Max Intensity Change: Increasing injected samples generally raises attack success for both perturbation types, while increasing max intensity improves adaptive perturbation more evidently than static perturbation.Adaptive perturbations can remain effective at lower injection rates, whereas static perturbation benefits more strongly from additional samples.
  • 5.6 Evaluation of Perturbation Stealthiness: 99.5% and 99.1% pHash similarity scores show that static and adaptive perturbed images largely preserve original image content.Static perturbation had slightly higher similarity because it changes only one quarter of the pixels, whereas adaptive perturbation changes more pixels.
  • 5.7.2 Evaluation on CIFAR-10 dataset: Both perturbation masks exceeded 98% success with 0.5% accuracy loss after roughly 500 batches on CIFAR-10.The experiment used 10 injected samples per batch of size 128 and max intensity change 10.

6 POSSIBLE DEFENSES

The paper discusses defenses that attempt to destroy perturbation patterns, detect anomalous label frequencies or test-time errors, and identify backdoor-specific feature clusters. Each approach has practical constraints, including accuracy loss, required knowledge, labor, rarity of exploitation, or speculative complexity.

  • Statistical detection: Class-frequency analysis requires knowledge of true class priors and may have limited success on small training sets.Controlling the injection ratio can also avoid injecting too many samples with the same target label.
  • Test-time detection: Test-set label analysis is laborious because it requires ground-truth labels, and infrequent backdoor use can make malicious samples rare among normal data.Consequently, the target label need not dominate the labels of misclassified data.
  • Feature-space detection: High-dimensional clustering could detect separate backdoor and normal-pattern clusters within the target class, but the strategy remains sophisticated and speculative.The paper identifies accurate cluster detection as a subject for future work.

7 RELATED WORK

Related work spans evasion attacks, poisoning attacks, neural-network trojans, and image steganography. This paper differs from conventional poisoning by embedding a test-time backdoor while preserving regular-data accuracy and functioning under limited attacker knowledge.

  • Evasion attacks: Evasion attacks craft adversarial samples that fool machine-learning models at test time, including imperceptible image perturbations that cause neural networks to misclassify.The literature subsequently refined methods for generating adversarial examples.
  • Poisoning attacks: Conventional poisoning attacks inject malicious training samples to degrade the efficacy of the resulting model.This literature includes attacks against both traditional and deep-learning models.
  • Backdoor injection: The proposed attack instead embeds a backdoor exploitable at test time without degrading accuracy on regular data.This distinguishes it from poisoning methods whose objective is general model degradation.
  • Neural-network trojans: Neural-network trojan studies include direct parameter manipulation, reverse-engineered poisoning of public models, and countermeasures against trojan triggers.The cited work also examines backdoor poisoning in outsourced training.
  • Image steganography: Image steganography shares the goal of adding secret information or code to images, but its techniques and application area differ from this attack.The paper suggests future exploration of steganographic techniques for hidden information learned by deep models.

8 CONCLUSION

The paper proposes stealthy backdoor injection attacks using crafted perturbation masks across multiple threat models and injection timings. Experiments show successful, stealthy attacks, while perturbation intensity and injection rate affect efficacy to a limited extent; extensions include multiple masks and other domains.

  • 8 CONCLUSION: The paper proposes backdoor injection attacks using two stealthy perturbation masks that achieve high attack success with little effect on model performance.The contribution also considers realistic threat models and injection scenarios.
  • 8 CONCLUSION: Experiments demonstrate that the attack strategies are both stealthy and successful.The experiments evaluate the effects of perturbation maximum intensity change and injection rate.
  • 8 CONCLUSION: Perturbation maximum intensity change and injection rate affect attack efficacy only to a limited extent.The conclusion frames both factors as having bounded influence in the reported experiments.
  • 8 CONCLUSION: A proposed refinement is to inject multiple perturbation masks simultaneously, potentially making attacks harder to detect and enabling multiple backdoor targets.The paper also proposes applying attack variants beyond image classification.
  • 8 CONCLUSION: The adaptive-mask setup varies DT across cases defined by the adversary’s knowledge and capability.The listed cases use surrogate models, existing pretrained models, and different data subsets.

B MODEL STRUCTURES AND TRAINING SETUP

The model-structure summary notes that the first three models use dropout before the output layer. The specified keep probability is 0.5.

  • B MODEL STRUCTURES AND TRAINING SETUP: The first three models use a dropout layer with keep probability 0.5 before the output layer.This is part of the reported model structures and training setup.

B.1 ConvNet for GTSRB

The GTSRB experiments use a ConvNet and a LeNet-5-based surrogate model, with the ConvNet incorporating a concatenation before its final fully connected layer.

  • B.1 ConvNet for GTSRB: The ConvNet has 3 convolutional layers and 1 fully connected layer.Its architecture is identified as the ConvNet for GTSRB.
  • B.1 ConvNet for GTSRB: After the third convolutional layer, the ConvNet concatenates its output with the output after the second max-pooling layer.The concatenated representation serves as input to the last fully connected layer.
  • B.1 ConvNet for GTSRB: The surrogate model adapts LeNet-5 to the GTSRB dataset and contains 3 convolutional layers and 2 fully connected layers.Its architecture is listed in Table 8.

B.3 LeNet-5 for MNIST

The MNIST model is based on LeNet-5, with its main architecture documented separately from the GTSRB architectures.

  • B.3 LeNet-5 for MNIST: The MNIST model is based on LeNet-5.Its architecture is presented in Table 9.
  • B.3 LeNet-5 for MNIST: Table 9 is labeled as the main architecture of LeNet-5 for MNIST.The supplied passage identifies the table but does not provide layer-level details.

B.4 VGG-CIFAR10

The CIFAR-10 experiments use a VGG-16 adaptation organized into convolutional and fully connected layer groups, optimized with stochastic gradient descent.

  • B.4 VGG-CIFAR10: The VGG-CIFAR10 model adapts VGG-16 to CIFAR-10 and contains 5 convolution-layer groups and 1 fully connected-layer group.The model has 13 convolutional layers and 2 fully connected layers in total.
  • B.4 VGG-CIFAR10: The model uses the dropout configuration from and stochastic gradient descent with an initial learning rate of 0.001.The supplied passage also states that updating stops once validation accuracy and attack success rate meet the specified condition, but the sentence is truncated.

C GENERATING ADAPTIVE PERTURBATION MASK FOR DIRECT TARGETED MISCLASSIFICATION

Direct targeted misclassification requires balancing attack success against perturbation magnitude because larger adaptive masks undermine stealthiness. The experiment varies magnitude constraints across target pairs and finds that exceeding 90% success requires a maximum intensity of 40.

  • C GENERATING ADAPTIVE PERTURBATION MASK FOR DIRECT TARGETED MISCLASSIFICATION: The targeted-misclassification problem constrains perturbation magnitude to preserve stealthiness.The paper distinguishes this setting from work that does not impose the same small-perturbation requirement.
  • C GENERATING ADAPTIVE PERTURBATION MASK FOR DIRECT TARGETED MISCLASSIFICATION: The experiment selects 5 target pairs and varies the adaptive-mask magnitude constraint from 10 to 40.Testing-set instances from each source class are evaluated for misclassification into the corresponding target class.
  • C GENERATING ADAPTIVE PERTURBATION MASK FOR DIRECT TARGETED MISCLASSIFICATION: Above 90% attack success requires an adaptive perturbation mask with maximum intensity as large as 40.A maximum intensity of 10 rarely succeeds at targeted misclassification.
  • C GENERATING ADAPTIVE PERTURBATION MASK FOR DIRECT TARGETED MISCLASSIFICATION: A maximum intensity of 40 inevitably fails to achieve the stealthiness required by the attack design.The result exposes a trade-off between direct targeted-misclassification effectiveness and perceptual stealthiness.
Loading 1808.10307v1…