A new Backdoor Attack in CNNs by training set corruption without label poisoning

Mauro Barni; Kassem Kallas; Benedetta Tondi

Source-linked AI summary

A new Backdoor Attack in CNNs by training set corruption without label poisoning

Mauro Barni, Kassem Kallas, Benedetta Tondi

arXiv:1902.11237v1Feb 12, 2019cs.CRcs.CVcs.LG

TL;DR

The paper addresses the reduced stealthiness of backdoor attacks that poison training labels. It proposes corrupting only target-class samples without label poisoning and shows this approach works on MNIST and traffic-sign classification, while requiring more corrupted samples than label-poisoning attacks.

Problem
Prior backdoor attacks poison labels, making corrupted samples detectable when their content conflicts with their labels.
Method
The attack adds a weak backdoor signal to training samples from the target class, then adds that signal at test time to induce classification as the target.
Results
The attack works on MNIST and traffic-sign classification, including traffic-sign attack success rates of 80% and 56% for two targets at test-time signal strength ∆ts = 40.
Takeaways & Limitations
Avoiding label poisoning improves training-set stealthiness and lets the attacker choose the source class at test time, but requires corrupting a larger training-sample fraction.

Abstract

from arXiv · show

Backdoor attacks against CNNs represent a new threat against deep learning systems, due to the possibility of corrupting the training set so to induce an incorrect behaviour at test time. To avoid that the trainer recognises the presence of the corrupted samples, the corruption of the training set must be as stealthy as possible. Previous works have focused on the stealthiness of the perturbation injected into the training samples, however they all assume that the labels of the corrupted samples are also poisoned. This greatly reduces the stealthiness of the attack, since samples whose content does not agree with the label can be identified by visual inspection of the training set or by running a pre-classification step. In this paper we present a new backdoor attack without label poisoning Since the attack works by corrupting only samples of the target class, it has the additional advantage that it does not need to identify beforehand the class of the samples to be attacked at test time. Results obtained on the MNIST digits recognition task and the traffic signs classification task show that backdoor attacks without label poisoning are indeed possible, thus raising a new alarm regarding the use of deep learning in security-critical applications.

I. INTRODUCTION

The paper addresses backdoor attacks against CNNs by proposing training-set corruption that preserves labels while inducing target-class misclassification at test time. It argues this approach improves stealth and flexibility, though it requires corrupting a larger fraction of training samples.

Threat and prior attacks: Backdoor attacks can induce targeted test-time misclassification by manipulating network parameters or poisoning the training set.The paper focuses on attacks that make backdoor instances receive an attacker-specified target label.
Problem with prior attacks: Label poisoning makes corrupted samples easier to detect because their content can mismatch their assigned labels.Such samples may be identified through dataset inspection or pre-classification.
Proposed approach: The proposed attack corrupts only training samples from the target class by adding a weak signal while leaving their labels unchanged.At test time, adding the signal to a sample from another class should induce classification as the target class while pristine samples remain correctly classified.
Contribution and evaluation: Creating a label-preserving backdoor is difficult because the network must rely on the signal without abandoning genuine target-class features.The paper reports that poisoned labels make this easier, whereas the proposed approach requires corrupting a larger training fraction.
Contribution and evaluation: The paper evaluates the attack on MNIST digit recognition and traffic-sign classification.The introduction identifies both tasks as experimental settings for demonstrating feasibility.

II. BACKDOOR ATTACK FORMULATION

The formalized attack injects an imperceptible signal into a fraction of target-class training samples without changing labels, aiming to trigger target classification across source classes while preserving clean accuracy. Its success depends on balancing the attacked fraction and signal strength.

II-A. Attcker’s model: The attacker seeks a signal that causes triggered inputs to be recognized as a target class without reducing performance on uncorrupted samples.The signal should also be imperceptible or difficult to perceive during dataset inspection.
II-A. Attcker’s model: The attacker is assumed to know nothing about the CNN model.This defines the model-knowledge constraint in the attacker model.
II-A. Attcker’s model: The attacker can access only a fraction α of target-class training samples and cannot change their labels.The attack requires trading off the attacked fraction against signal strength to preserve stealth.
II-B. Attack formalization: The attack corrupts a fraction α of samples in target class t within a pristine training set partitioned into class-specific subsets.The corrupted samples form the backdoor subset used to construct the poisoned training data.
II-B. Attack formalization: Each selected target-class image is transformed by additive perturbation x_bi = x_i + v, where v is the backdoor signal.The perturbation is applied in the image domain.
II-B. Attack formalization: The poisoned dataset replaces each selected pair (x_i, t) with (x_bi, t), preserving the original target label during training.The model is trained on this modified dataset rather than on the pristine set.
II-B. Attack formalization: The attack succeeds when a triggered sample from any non-target class is classified as t, formally f(x + v) = t.The signal should be sufficient for target classification but not necessary, so clean target samples retain their discriminative features.
II-B. Attack formalization: The attacked fraction α must be large enough for the network to learn the signal but not so large that clean target classification is impaired.The formulation also extends to multiple target classes, each with its own signal and target output.

III. METHODOLOGY

The experiments implement task-specific, weak backdoor signals for MNIST and GTSRB, using ramp, triangle, and sinusoidal patterns chosen for detectability by the network and stealthiness to human inspection. Testing can use stronger signals to increase attack effectiveness.

Datasets and tasks: The study evaluates MNIST digit recognition and traffic-sign classification as two recognition tasks.These tasks provide the experimental settings for the proposed attack.
Datasets and tasks: MNIST uses 28 × 28 grayscale images from 10 digit classes, while GTSRB experiments use 16 selected traffic-sign classes.The MNIST split contains about 6000 training and 1000 testing images per class.
Network setup: The experiments use a CNN tailored to MNIST and standard LeNet-5 for traffic signs.The MNIST model uses convolutional and fully connected layers, while the traffic-sign network reaches about 0.98 accuracy without attacks.
Training setup: Training uses Adam with learning rate 10^-3 and momentum 0.99, together with standard Keras image augmentation.The models and attack procedure are implemented in Python through Keras.
Backdoor-signal design: Signal design balances easy network detection against weak visual strength and limited disruption of pristine-image features.The required signal form depends on the classification task and target class.
Backdoor-signal design: Figure 1 illustrates clean and altered MNIST and GTSRB images using ∆ = 40 for MNIST and ∆ = 20, f = 6 for GTSRB.The examples show the task-specific signals superimposed on representative images.
Backdoor-signal design: For MNIST, the study uses ramp and triangle signals; for textured traffic-sign images, it uses a horizontal sinusoid.The MNIST ramp adds a slowly varying background, while the traffic-sign sinusoid is defined by amplitude ∆ and frequency f.
Testing procedure: Testing may use the same or a larger signal strength, improving attack effectiveness without compromising training-time stealthiness.The paper distinguishes training-time stealth from test-time signal strength.

IV. EXPERIMENTAL RESULTS

Experiments on MNIST and traffic signs show that label-preserving backdoor attacks can induce target-class predictions while retaining accurate classification of pristine samples. Attack effectiveness increases with test-time signal strength, supports multiple targets, and requires corrupting more training samples than label-poisoning attacks.

MNIST classification: MNIST experiments used a ramp signal to corrupt α = 0.3 of target-digit samples during training, then applied the signal to other digits at test time.The representative setup used ∆tr = 30, with the target class being digit ’3’.
MNIST classification: Nearly perfect pristine-sample accuracy was retained, while the backdoor usually caused attacked MNIST samples to be classified as the target digit.Using stronger test-time signals, ∆ts = 40 and 60, improved attack effectiveness without changing the training-time corruption.
Traffic Signs classification: Traffic-sign experiments targeted the speed limit 50 sign and caused several signs from different classes to be classified as that target when the sinusoidal signal was added.The setup used α = 0.2, ∆tr = 20, f = 6, and ∆ts = 30; speed-limit signs were generally easier to attack.
Traffic Signs classification: For two traffic-sign targets, attack success rates reached 80% and 56% at ∆ts = 40, and 90% and 67% at ∆ts = 60.The rates correspond to targets t1 = 1 and t2 = 7, respectively, averaged over the 7 best results.

V. CONCLUDING REMARKS

The paper introduces a backdoor attack without label poisoning and evaluates it on digit recognition and traffic sign classification. The approach improves stealthiness and flexibility but requires corrupting an order of magnitude more samples than attacks with label corruption.

The proposed attack does not poison the labels of corrupted training samples, improving stealthiness by avoiding sample–label mismatch detection.
The attack improves flexibility because only target-class samples need corruption during training, while the source class can be selected at test time.
Corrupting an order of magnitude more samples is the cost of avoiding label corruption.
The attack was implemented for digit recognition and traffic sign classification.
The attack successfully compromised digit-recognition networks with an invisible signal and was effective to some extent for traffic signs with a nearly invisible signal.