Source-linked AI summary
Universal Litmus Patterns: Revealing Backdoor Attacks in CNNs
Soheil Kolouri, Aniruddha Saha, Hamed Pirsiavash, Heiko Hoffmann
TL;DR
Backdoor detection is difficult when users have only an untrusted pretrained CNN and lack its training data or reliable clean-data tests. The paper introduces Universal Litmus Patterns, optimized universal images whose outputs are classified to reveal backdoors. Across four datasets and thousands of networks, ULPs achieved strong detection performance, while fully adaptive attackers could bypass the detector.
Problem
Detect backdoors in pretrained CNNs without knowing the target class or trigger and without access to poisoned training data.
Method
Learn universal litmus patterns and classify the pooled logits produced when they pass through a model as clean or poisoned.
Results
AUC was close to 1 on CIFAR10 and MNIST, 0.96 on GTSRB with ResNet18, and 0.94 on Tiny-ImageNet across thousands of networks.
Takeaways & Limitations
A small set of ULPs can detect malicious networks efficiently and generalize to disjoint triggers and new architectures.
Takeaways & Limitations
Fully adaptive poisoned models with access to the ULPs and binary classifier successfully bypassed the detector, although their responses remained discriminative across classes.
Abstract
from arXiv · showhide
The unprecedented success of deep neural networks in many applications has made these networks a prime target for adversarial exploitation. In this paper, we introduce a benchmark technique for detecting backdoor attacks (aka Trojan attacks) on deep convolutional neural networks (CNNs). We introduce the concept of Universal Litmus Patterns (ULPs), which enable one to reveal backdoor attacks by feeding these universal patterns to the network and analyzing the output (i.e., classifying the network as `clean' or `corrupted'). This detection is fast because it requires only a few forward passes through a CNN. We demonstrate the effectiveness of ULPs for detecting backdoor attacks on thousands of networks with different architectures trained on four benchmark datasets, namely the German Traffic Sign Recognition Benchmark (GTSRB), MNIST, CIFAR10, and Tiny-ImageNet. The codes and train/test models for this paper can be found here https://umbcvision.github.io/Universal-Litmus-Patterns/.
1. Introduction
Deep networks’ widespread deployment increases the importance of detecting stealthy backdoor attacks, especially when models come from unknown parties and training data are unavailable. The paper proposes Universal Litmus Patterns to identify corrupted CNNs without clean-data testing.
- Motivation: Backdoor attacks insert triggers that cause targeted misclassification while potentially preserving normal behavior on clean inputs.The attacker introduces a trigger into training data so source-class images are classified as a target class.
- Motivation: Evaluating a model only on clean test data is insufficient for detecting stealthy backdoors, and original training data are often unavailable.These constraints motivate a detector that probes the model directly.
- Approach: Universal Litmus Patterns are optimized universal test images whose network outputs indicate whether a CNN is clean or backdoored.The detector uses the outputs produced when ULPs are fed through the network.
- Approach: ULP detection requires no access to training data or clean test data and uses only a small set of universal patterns.This makes the approach suitable for examining downloaded or third-party models.
- Results: AUC was close to 1 on CIFAR10 and MNIST, 0.96 on GTSRB with ResNet18, and 0.94 on Tiny-ImageNet.The experiments covered thousands of trained networks across four benchmark datasets.
2. Related Work
Prior backdoor defenses often depend on poisoned training data, clean data, or computationally demanding searches over class pairs and trigger patterns. ULPs instead learn transferable probing patterns whose model outputs are classified to detect backdoors.
- Existing detection methods: Earlier detection methods commonly analyze poisoned training data, its outliers, or neural activations generated from that data.These approaches can identify anomalous samples or activation patterns associated with triggered inputs.
- Assumptions and gap: Many existing defenses assume access to the infected training dataset, an assumption that fails for outsourced training and downloaded pretrained models.The related-work discussion identifies model-only settings as requiring different detection strategies.
- Neural Cleanse: Neural Cleanse detects attacks by optimizing minimal triggers that force source-class images toward target classes.Its search may need to consider potentially all source-target class pairs.
- Neural Cleanse: Neural Cleanse is computationally demanding when the attacked source class is unknown and requires a strong prior to distinguish benign perturbations from real triggers.These requirements limit its practicality for broad model screening.
- Universal Litmus Patterns: ULPs learn universal, transferable patterns and classify the resulting logits to identify networks containing backdoor attacks without infected training data.The paper presents this as a different angle from minimal-trigger optimization.
3. Methods
The method targets third-party CNNs that may behave normally on clean data but misclassify inputs containing unknown triggers. It learns universal input patterns and classifies model outputs to distinguish clean from backdoored networks without poisoned training data.
- 3.1. Threat Model: The threat model assumes a third party inserts a targeted trigger into a CNN, causing source-class images with that trigger to receive a target label.The attacker poisons selected training images by adding a patch and assigning target labels.
- 3.2. Defense Goals: The defense assumes no prior knowledge of the attack’s target class or trigger and no access to the poisoned training dataset.The goal is to identify untrusted third parties that provide infected pretrained CNNs.
- 3.3. Formulation: The formulation learns Universal Litmus Patterns and a binary classifier that separates infected from clean models using their outputs on those patterns.The model set may contain different architectures, depths, neuron counts, and activation functions, making model classification nonstandard.
- 3.3. Formulation: The pooled logits from ULPs are passed to a classifier that estimates whether each network contains a backdoor.The experiments use concatenation for pooling and a softmax classifier; max-pooling and averaging were also tested.
- 3.3. Formulation: Total variation regularization is used to constrain the learned patterns, with λ controlling the regularization strength.The regularizer is R(z_m) = ∥∇z_m∥_1.
- Baselines: Random patterns provide an ablation baseline, while Neural-Cleanse searches source-target class pairs for universal patches that successfully induce targeted misclassification.The random-pattern baseline shares the pooling and classifier with ULPs, isolating the effect of jointly optimizing the input patterns.
4. Experiments
Across four datasets and thousands of CNNs, ULPs detected poisoned models, including unseen triggers and randomized architectures, while requiring only a few forward passes. They generally outperformed baselines and remained effective under subtle poisoning, although fully adaptive attackers could bypass the detector.
- Experimental setup: About 2,000 CNNs per dataset were trained, with clean and poisoned models matched on clean-data performance while poisoned models exceeded 90% attack success.Experiments covered MNIST, CIFAR10, GTSRB, and Tiny-ImageNet, using mutually exclusive train/test triggers where specified.
- Overall detection results: ULPs consistently outperformed the two baselines by a large margin across the benchmark datasets.Table 1 reports AUC scores for the detection methods on MNIST, CIFAR10, GTSRB, and Tiny-ImageNet.
- MNIST experiments: Nearly 1 AUC was achieved on MNIST with M = 10 ULPs, while detection accuracy stayed above 95% even when poisoned samples comprised only 1% of training data.With noise inputs, accuracy fell to almost 60% at the 1% poisoning ratio.
- GTSRB experiments: On GTSRB VGG-like models, ULPs reached AUC = 0.9 with M = 10 versus AUC = 0.74 for Neural-Cleanse.The models included Spatial Transformer Networks, and Neural-Cleanse was reported as 90,000 times slower.
- Tiny-ImageNet experiments: On Tiny-ImageNet ResNet-like models, ULPs achieved AUC = 0.94 and 95.8% detection accuracy with M = 5 patterns.The models averaged 45.1% top-1 accuracy on clean test data and used randomly located 7x7 triggers.
- Efficiency and generalizability: ULP detection required O(M) forward passes and ran in approximately 10–20 msec, compared with 3–30 minutes for Neural-Cleanse on the reported datasets.Neural-Cleanse was not run on Tiny-ImageNet because of its computational burden.
- Efficiency and generalizability: ULPs trained on fixed architectures generalized to randomized architectures and across VGG-like and ResNet-like model types on GTSRB.The generalizability experiments used random VGG-like architectures and compared ULPs trained on fixed VGG or ResNet architectures.
- Adaptive attacker: Fully adaptive poisoned models could bypass the ULP detector, although their responses remained discriminative among clean, poisoned, and adaptively poisoned classes.The authors suggest hardening the defense with a more complex binary classifier or advanced model augmentations.
5. Discussion
Universal Litmus Patterns provide an efficient method for detecting backdoor attacks in neural networks. The discussion highlights architecture-agnostic operation, strong detection with few patterns, generalization beyond optimization conditions, and an intuition based on learned trigger features.
- 5. Discussion: Universal Litmus Patterns detect backdoor attacks by probing neural networks with optimized input images.The method uses network outputs to indicate whether a model is clean or contains a backdoor attack.
- 5. Discussion: The approach requires only model input-output behavior and does not require access to the training data.This makes the method agnostic to the network architecture.
- 5. Discussion: A small set of at most 10 ULPs detected malicious networks with relatively high accuracy and outperformed a Neural Cleanse baseline.Each ULP requires only one forward pass through a CNN, whereas Neural Cleanse tests potentially many class-label pairs.
- 5. Discussion: ULPs generalized to triggers disjoint from those used for optimization and to models different from the source models, including random architectures.Figure 9 compares pooled outputs from clean, poisoned, and adaptively poisoned models using M = 10 input ULPs in the discriminant subspace.
- 5. Discussion: The proposed intuition is that poisoned CNNs learn triggers as salient features, while each ULP combines a wide variety of triggers.Presenting such a pattern should therefore elicit a positive response from a poisoned network with high probability.