Source-linked AI summary
Bypassing Backdoor Detection Algorithms in Deep Learning
Te Juin Lester Tan, Reza Shokri
TL;DR
Backdoor detectors often rely on latent differences between clean and poisoned inputs, creating a gap for adaptive attackers. This paper trains an adversarially embedded backdoor to reduce those differences and reports successful evasion of several state-of-the-art defenses.
Problem
Existing backdoor defenses commonly assume attackers are unaware of the detection algorithm, although detectors rely on latent-representation differences between clean and adversarial inputs.
Method
The attack combines data poisoning with adversarial regularization that penalizes latent-distribution differences while preserving classification performance.
Results
97.3% attack success rate remains against spectral-signature detection and 96.2% against activation clustering on VGG/CIFAR-10, demonstrating evasion of several state-of-the-art defenses.
Takeaways & Limitations
Sophisticated attackers can hide backdoor signals in latent representations, rendering defenses based on those signals ineffective in the evaluated settings.
Abstract
from arXiv · showhide
Deep learning models are vulnerable to various adversarial manipulations of their training data, parameters, and input sample. In particular, an adversary can modify the training data and model parameters to embed backdoors into the model, so the model behaves according to the adversary's objective if the input contains the backdoor features, referred to as the backdoor trigger (e.g., a stamp on an image). The poisoned model's behavior on clean data, however, remains unchanged. Many detection algorithms are designed to detect backdoors on input samples or model parameters, through the statistical difference between the latent representations of adversarial and clean input samples in the poisoned model. In this paper, we design an adversarial backdoor embedding algorithm that can bypass the existing detection algorithms including the state-of-the-art techniques. We design an adaptive adversarial training algorithm that optimizes the original loss function of the model, and also maximizes the indistinguishability of the hidden representations of poisoned data and clean data. This work calls for designing adversary-aware defense mechanisms for backdoor detection.
1. Introduction
Backdoor attacks embed an adversarial rule that activates only when inputs contain a trigger, while detection methods seek latent-representation differences between poisoned and clean inputs. The paper proposes adversarial embedding to hide those differences and evade detection.
- Backdoor attacks: Backdoor attacks manipulate training data or model parameters so the model behaves adversarially on inputs containing an embedded trigger.On clean inputs, the model behaves normally; triggers may be instance-specific, pixel-based, or semantic.
- Backdoor detection: Existing detectors identify backdoor inputs or responsible model activations by distinguishing hidden representations of triggered and clean inputs.Figure 1 illustrates inputs with and without a trigger, while detection methods target latent signatures.
- Threat model: Prior defenses commonly assume that the attacker does not know which detection algorithm or latent statistics the defender uses.The paper frames this assumption as a central weakness of existing backdoor defenses.
- Adversarial embedding: The proposed attack uses adversarial regularization to maximize latent indistinguishability between adversarial and benign inputs, targeting specific or generic detectors.A discriminator identifies hidden-layer differences, while the classifier is trained to maximize the discriminator’s loss.
- Results: 97.3% attack success rate remains against spectral-signature detection and 96.2% against activation clustering on VGG/CIFAR-10, versus 1.5% and 1.9% for static attacks.Against feature pruning, removing the backdoor from the adversarially embedded model reduces accuracy to 20%, with 10% representing random guessing.
2. Prior Backdoor Detection Algorithms
Prior defenses exploit differences between clean and backdoor latent representations to prune neurons or filter poisoned training samples. They include trigger reverse engineering, activation-based pruning, spectral analysis, and clustering.
- Backdoor representation: A backdoor instance combines features from its true class with a trigger and is misclassified as the attacker’s target label.Clean and backdoor instances can share a target label while differing in their high-level features and latent representations.
- Defense categories: Latent-representation defenses either prune neurons responsible for the backdoor rule or filter poisoned samples before retraining the model.These form the two main categories of defenses analyzed in the paper.
- Trigger reverse engineering: Reverse engineering searches for the smallest perturbation that forces each target class, then flags unusually small perturbations as candidate triggers.The method assumes that a backdoor trigger is a small input perturbation.
- Feature pruning: Feature pruning ranks hidden neurons by the absolute difference between their mean activations on clean and triggered inputs, then prunes until backdoor behavior disappears.This assumes the backdoor rule produces large activation changes in neurons representing backdoor features.
- Dataset filtering: Spectral-signature defenses apply singular value decomposition to latent-representation covariance, whereas activation clustering reduces representations and separates them into two clusters.Clustering uses independent component analysis followed by k-means and exclusionary reclassification to identify poisoned samples.
3. Adversarial Backdoor Embedding Attack
The attack embeds backdoors while reducing the latent-representation differences that detection defenses use to separate clean and poisoned inputs. It combines targeted representation manipulation with adversarial network regularization so the classifier remains accurate while a discriminator cannot distinguish the two input types.
- The attack objective is to create poisoned models whose backdoor inputs remain indistinguishable from clean inputs in latent representations.
- The representation penalty adds a loss term that penalizes large activation-distribution differences between clean and backdoor inputs.The penalty can target an anticipated defense or generalize across multiple defenses.
- 3.1. Targeted Adversarial Embedding: For targeted embedding, the attacker minimizes clean–backdoor activation differences in selected backdoor neurons so pruning defenses prioritize other neurons.The method scales activations in the backdoor-neuron set by k, while preserving the signal pathways; mean square error is used for Lrep.
- 3.2. Adversarial Embedding: The general attack models the network as H followed by classifier C, then trains discriminator D on latent representations to distinguish clean from backdoor inputs.The classifier is trained for accurate predictions while the representation extractor makes D unable to classify the latent representations reliably.
4. Evaluation
The evaluation tests targeted and adversarial embedding against pruning and dataset-filtering defenses across CIFAR-10 and GTSRB with VGG and DenseNet models. Adversarial embedding reduces the latent-representation differences used by these defenses, allowing backdoors to persist despite pruning or filtering.
- Evaluation setup: The evaluation spans two image datasets and two convolutional architectures, using poisoned models as baselines for defense comparisons.The datasets are CIFAR-10 and GTSRB; the architectures are DenseNet-BC and VGG.
- Targeted embedding: Pruning fails on some baseline settings, with model accuracy falling below 40% before the backdoor is removed.The unsuccessful settings are DenseNet-CIFAR-10 and VGG-GTSRB; DenseNet-GTSRB removes the backdoor with about an 8% accuracy reduction.
- Targeted embedding: Targeted adversarial embedding causes pruning to select clean rather than backdoor neurons, so accuracy falls while attack success rate increases.Removing clean neurons increases the relative magnitude of backdoor signals, strengthening the attack during pruning.
- Adversarial embedding: Adversarial embedding preserves significant attack success as pruning reduces accuracy, preventing complete backdoor removal without substantial retraining.The corresponding losses converge during adversarial training, while the defense cannot fully eliminate the backdoor at the original accuracy.
- Dataset filtering: Spectral-signature filtering leaves more than half of the poisoned samples and yields above 90% attack success after retraining, whereas the baseline attack success is 0% to 1.5%.Adversarial embedding makes poisoned and clean samples less distinguishable by the filtering statistic.
- Dataset filtering: Activation clustering is also weakened: adversarial embedding lowers the adjusted Rand index and leaves retrained models with attack success above 75%.In the baseline, clustering achieves an adjusted Rand index above 0.95 and retraining reduces attack success to 0% to 1.9%.
5. Related Work
Prior work studies backdoor injection through poisoned data or model updates and develops defenses based on latent representations, pruning, and input anomaly removal. Baseline poisoned inputs often form separable latent patterns, but adversarial embedding can make poisoned and clean representations overlap, weakening these defenses.
- Backdoor attacks: Backdoors can be injected through poisoned training samples or malicious model updates, including in federated learning.Prior work reports that 50 poisoned samples can yield attack success above 90%, and that backdoors may persist after model repurposing.
- Backdoor defenses: Several defenses detect or remove backdoors by pruning neurons, filtering latent representations, or removing anomalous input features.Pruning-based mitigation may require fine-tuning, while other methods operate directly on the input space.
- Latent-space detection: Spectral-signature filtering identifies poisoned samples when their latent representations have higher top-eigenvector correlation than clean samples.Adversarial embedding instead gives poisoned and clean samples similar correlation distributions, so filtering removes comparable portions of both.
- Adaptive evasion: Other adversarial-learning studies similarly evade detectors by preserving smooth image structure while avoiding detector-specific input artifacts.This line of work motivates evaluating defenses against attackers aware of their detection signals.
- Latent-space detection: Activation clustering separates baseline poisoned representations into a distinct cluster, whereas adversarial embedding distributes poisoned samples across both clusters.The resulting clusters contain substantial numbers of poisoned samples rather than isolating them cleanly.
6. Conclusions
The paper introduces a backdoor embedding attack that bypasses several prior detection algorithms. Its central result is that attackers can hide backdoor-image signals in learned latent representations, undermining defenses that rely on those representations.
- Conclusions: The proposed backdoor embedding attack successfully bypasses several prior backdoor detection algorithms.The attack hides backdoor-image signals in learned latent representations, rendering those defense algorithms ineffective.