Defending Neural Backdoors via Generative Distribution Modeling

Ximing Qiao; Yukun Yang; Hai Li

Source-linked AI summary

Defending Neural Backdoors via Generative Distribution Modeling

Ximing Qiao, Yukun Yang, Hai Li

arXiv:1910.04749v2Oct 10, 2019cs.LGstat.ML

TL;DR

Neural backdoor defenses struggle with complex triggers because valid triggers form an unknown distribution rather than a single recoverable point. The paper introduces MESA, a sampling-free generative method that models this distribution and supports retraining-based defense. Distribution-based defense reduces original-trigger ASR from 92.3% ∼ 99.8% to 1.2% ∼ 5.9%.

Problem
Complex backdoor triggers make single reversed-trigger defenses unreliable because valid triggers form an unknown distribution that cannot be directly sampled.
Method
MESA ensembles entropy-maximized sub-models to recover the valid trigger distribution without direct sampling, then uses it to retrain the backdoored model.
Results
92.3% ∼ 99.8% ASR before defense decreases to 1.2% ∼ 5.9% after distribution-based defense, while the single-trigger baseline reaches 2.4% ∼ 51.4%.
Takeaways & Limitations
Modeling the full valid trigger distribution is important for reliable backdoor removal across the evaluated Cifar10/100 trigger settings.
Takeaways & Limitations
The implementation handles only non-structured triggers with fixed known shape and size, and assumes the defender knows the trigger size.

Abstract

from arXiv · show

Neural backdoor attack is emerging as a severe security threat to deep learning, while the capability of existing defense methods is limited, especially for complex backdoor triggers. In the work, we explore the space formed by the pixel values of all possible backdoor triggers. An original trigger used by an attacker to build the backdoored model represents only a point in the space. It then will be generalized into a distribution of valid triggers, all of which can influence the backdoored model. Thus, previous methods that model only one point of the trigger distribution is not sufficient. Getting the entire trigger distribution, e.g., via generative modeling, is a key to effective defense. However, existing generative modeling techniques for image generation are not applicable to the backdoor scenario as the trigger distribution is completely unknown. In this work, we propose max-entropy staircase approximator (MESA), an algorithm for high-dimensional sampling-free generative modeling and use it to recover the trigger distribution. We also develop a defense technique to remove the triggers from the backdoored model. Our experiments on Cifar10/100 dataset demonstrate the effectiveness of MESA in modeling the trigger distribution and the robustness of the proposed defense method.

1 Introduction

The paper identifies valid trigger distributions as the central challenge for robust backdoor defense and proposes MESA to model them without direct sampling. Experiments on Cifar10/100 show that distribution-based defense improves robustness across trigger types.

1 Introduction: Backdoor attacks inject a predefined trigger during training, then cause the model to output a trigger-associated target class at inference.The attack uses one trigger across inputs, unlike perturbation-based adversarial attacks that recalculate perturbations per input.
1 Introduction: Existing defenses are limited for complex triggers: reversed triggers vary across runs, making backdoor removal unpredictable.Testing-stage defenses are effective when the reversed trigger resembles the original trigger.
1 Introduction: Reverse-engineered triggers form a continuous valid trigger distribution, and some achieve higher ASR than the original trigger.When this distribution is sufficiently wide, a single reversed trigger cannot reliably approach the original trigger.
1 Introduction: Because the trigger distribution is unknown and validity requires testing through the backdoored model, GANs and VAEs cannot directly sample it; high-dimensional brute force and MCMC are impractical.The paper frames trigger modeling as a high-dimensional sampling-free generative modeling problem.
1 Introduction: MESA ensembles sub-models trained by entropy maximization to approximate the valid trigger distribution, then uses that distribution to retrain and fix the backdoored model.The defense detects the target class, constructs the distribution, and retrains the model.
1 Introduction: 92.3% ∼ 99.8% ASR before defense falls to 1.2% ∼ 5.9% after distribution-based defense, versus 2.4% ∼ 51.4% for the single-trigger baseline.Experiments cover 51 black-white 3 × 3 triggers and random color triggers on Cifar10/100.

2 Background

The background defines neural backdoors, reviews training- and testing-stage defenses, and explains why conventional generative models do not fit trigger modeling. It motivates entropy maximization as a sampling-free alternative.

2.1 Neural backdoors: Neural backdoors combine a trigger x, target class c, application rule Apply(·, x), and poison ratio r to alter model behavior during training.Attackers optimize losses on triggered inputs and clean inputs with probabilities r and 1 − r, respectively.
2.1 Neural backdoors: The Apply function overwrites an image with a trigger at a random or fixed location, while studied triggers include physical, trojaning, single-pixel, clean-label, and invisible perturbation forms.These examples illustrate the range of trigger constructions considered in prior work.
2.2 Backdoor defenses: Training-stage defenses detect abnormal activations in poisoned training data but cannot discover backdoors already embedded in pretrained models.Their access to training data defines their main scope boundary.
2.2 Backdoor defenses: Testing-stage defenses reverse-engineer triggers in pixel space and use retraining or pruning to remove the backdoor.Retraining minimizes the loss on triggered images with clean labels, while pruning removes neurons sensitive to the reversed trigger.
2.3 Entropy maximization: GANs, VAEs, autoregressive models, and normalizing flows require samples from a true data distribution, which backdoor modeling cannot provide.Trigger validity is instead assessed through the backdoored model.
2.3 Entropy maximization: For deterministic generators, output entropy equals mutual information between noise and output, enabling MINE-based entropy estimation and maximization.The paper adopts this estimator to train its sampling-free generative model.

3 Method

MESA formulates backdoor defense as sampling-free generative modeling of an unknown valid-trigger distribution, using an ensemble of entropy-trained sub-models and a retraining-based removal procedure.

Problem formulation: MESA models an unknown trigger distribution f over trigger space X using a generator ˜G driven by Gaussian noise, without directly sampling from f.The testing function F serves as a surrogate, returning attack success information for candidate triggers.
Staircase approximation: MESA partitions the surrogate function F with thresholds β_i, assigning each sub-model G_i to learn one portion of the trigger distribution.The sets defined by thresholding F identify the regions modeled by individual sub-models.
Model ensemble: Each sub-model is trained by entropy maximization, while the ensemble ˜G randomly selects sub-models according to categorical weights γ_i.The thresholds are chosen to cover [0, 1], and the mixture can use entropy- and derivative-based weights.
Neural implementation: For high-dimensional triggers, MESA parameterizes each sub-model as a neural network and replaces entropy with a mutual-information estimator plus a relaxed constraint loss.Independent noise variables support mutual-information estimation, and sub-models failing their thresholded average ASR are skipped.
Backdoor defense: The defense detects attacked target classes with MESA, retrains models using generated triggers and clean labels, then selects the best threshold or uniformly mixes sub-models.Trigger-augmented examples are used with probability r, typically no greater than 1%, to preserve clean-data performance.

4 Experiments

Experiments evaluate MESA on fixed 3×3 triggers using Cifar10/100 backdoored ResNet-18 models, testing approximation behavior, target detection, defense robustness, and visualization. MESA reliably suppresses trigger success across 51 black-white patterns and shows similar results for random-color triggers.

Experimental setup: Experiments use Cifar10/100, pre-trained ResNet-18 models, 3×3 triggers, a 1% poison rate, and 10 training epochs.Triggers overwrite images at random locations; attacks preserve clean-data performance.
Experimental setup: MESA models and retrains on 8K Cifar10 testing images, evaluating defense on a separate 2K-image split without clean-data degradation.The implementation uses three-layer fully connected networks and retains the trigger application rule.
MESA approximation: Sub-model behavior depends on α: very small values concentrate outputs, whereas very large values over-expand the generated trigger set.The study samples 2K triggers and projects them into two dimensions with PCA while fixing βi = 0.8.
MESA approximation: Decreasing βi expands each sub-model’s output range, with higher-index ranges mostly covered by lower-index ranges as expected from staircase approximation.This test fixes α = 0.1 and uses β3 = 0.9, β2 = 0.8, and β1 = 0.5.
Backdoor defense: Target detection separates the true class from other classes: average reversed-distribution ASR exceeds 94.3% for the target and remains below 5.8% otherwise.The experiment covers all ten Cifar10 classes for the 51 triggers.
Backdoor defense: Across 51 black-white 3×3 triggers, MESA reduces original-trigger ASR from above 92% to below 9.1% for every tested βi.With βi = 0.9, average after-defense ASR is 3.4%, near the 2.4% ideal-defense result.
Backdoor defense: The baseline has high variance on difficult triggers, while random-color triggers produce results similar to black-white triggers.Figure 5 further shows trigger-specific distributions, sometimes with multiple modes and little similarity to baseline reversed triggers.

5 Conclusion and future works

The paper identifies valid trigger distributions as central to robust backdoor defense and proposes MESA to model them without sampling. Experiments show reliable backdoor removal, while the current implementation remains limited to fixed-size, non-structured triggers whose size is known.

Conclusion: The paper identifies valid trigger distributions as a main challenge and uses MESA-based distribution modeling for robust backdoor removal.The conclusion reports reliable removal on Cifar10 and emphasizes the importance of modeling trigger distributions.
Conclusion: The baseline single-reversed-trigger defense is unstable and performs 8× worse than the proposed method in the extreme case.This comparison motivates modeling the full valid-trigger distribution rather than one reversed trigger.
Future works: The current implementation handles only non-structured triggers with fixed shape and size, assuming the defender knows the trigger size.Future extensions suggested by the authors include convolutional generators and learned transparency channels.

A Derivation of MESA algorithm and proofs

MESA approximates an unknown trigger distribution by ensembling entropy-maximized sub-models arranged through staircase approximation. The derivation establishes convergence of the resulting density approximation as the ensemble size grows.

MESA construction: Each MESA sub-model learns a portion of the trigger distribution, while entropy maximization avoids direct sampling.The sub-model density is obtained from an entropy-maximization problem over a constrained region.
Staircase approximation: MESA assigns weights γi = exp(h(Gi(Z))) to sub-model densities, producing a weighted ensemble approximation.The construction uses nested regions and combines their densities through weighted summation.
Staircase approximation: As N →∞, the staircase ensemble approximates the cumulative distribution F in probability.For points between successive nested regions, the approximation error is bounded by a term that vanishes as N grows.
Density approximation: The same approximation strategy extends from F to its density f by recalculating weights from the function g.Only ratios among the γi values matter because the absolute scale is normalized.
Density approximation: Under F = g ◦ f with strictly increasing g and minimal slope ω, the estimated density converges to f in probability as N →∞.The proof uses g^-1 and normalization to show that the ensemble density error tends to zero.

The impact of trigger color, dataset, and target class

The experiments examine whether defense performance changes with trigger color, dataset, and target-class choices. Random-color triggers produce results similar to black-white triggers, while black-white patterns provide broader coverage of special cases.

Trigger color: Black-white triggers are out of the Cifar10/100 image color distribution, but the paper argues this is not problematic for backdoor defense.Backdoor triggers are described as deliberately out-of-distribution and relatively unrelated to the dataset.
Trigger color: Black-white triggers better cover corner cases than naive RGB randomization, which almost never produces a 3×3 black square.A 3×3 black square requires 27 exact zero-valued color components.
Defense results: The defense achieves approximately 2% average after-defense ASR for random-color triggers, compared with approximately 4% for previous black-white results.Random-color and black-white experiments show similar defensive outcomes across the evaluated settings.
Dataset and target class: Experiments cover Cifar10 with a fixed target class and Cifar100 with a random target class to assess dataset and target-class effects.Figure 1 uses target class 0 for Cifar10 and random targets for Cifar100.

Model ensembling and hyper-parameter selection

Model ensembling is used both to characterize the trigger-distribution shape and to provide defense without parameter tuning. The paper reports that β must reflect attack strength, whereas α is less sensitive within a tested range.

Model ensembling: The optimal defense retrains the backdoored model with the original trigger, while distribution reversal is an alternative intended to cover it.Experiments on black-white and random-color triggers across Cifar10 and Cifar100 support this distinction.
Model ensembling: Ensembling supports visualization of trigger-distribution cross-sections and robust defense without parameter tuning.The ensemble is not claimed to outperform every individual sub-model.
Hyper-parameters: Parameter α balances hinge loss and regularization, and values between 0.1 and 1 are empirically acceptable.The paper reports limited sensitivity to α over this interval.
Hyper-parameters: Parameter β reflects attack strength, and inappropriate values can make the reversed-trigger distribution too sparse or narrow for effective defense.The paper identifies β as more consequential for defense effectiveness than α.

Main experiments

The main experiments evaluate MESA-based defense across Cifar10 and Cifar100 using varied trigger colors and target classes. They search trigger distributions with a generator, retrain defended models, and compare against optimization and original-trigger baselines.

Experimental setup: Experiments use Cifar10’s 50,000 training images and 10,000 testing images, with 80% of images for trigger-distribution search and 20% for defense testing.Reported accuracy results are evaluated on the testing set.
Search distribution: The trigger-search generator is a three-layer perceptron with 64-dimensional input, 512 hidden units, and 27 outputs for a 3×3 RGB trigger.A statistic network estimates entropy, and the input noise follows a uniform distribution.
Defense procedure: Defense retrains the backdoored model with the attack parameters and ensembles three models using β values 0.9, 0.8, and 0.5.Across 10 defense rounds, the models generate 10 different triggers using sampling counts of 4, 3, and 3.
Baselines: The baseline reverse-engineers 10 trigger patterns per backdoored model through pixel-space optimization, while the ideal case uses the original trigger.Baseline defense performance is summarized by the mean and standard deviation across the searched patterns.
Additional evaluation: Supplementary experiments extend evaluation to a larger dataset, different trigger colors, and different target classes while keeping the environment similar.Visualization uses 2,000 generator samples for t-SNE and 128 samples for PCA.
Experimental setup: The study evaluates black-white and independent uniform RGB triggers, attacks random Cifar100 classes, and uses a larger hidden layer for Cifar100.The Cifar100 model is ResNet-18 with reported original accuracy of 76.23%.