Source-linked AI summary
Transferable Clean-Label Poisoning Attacks on Deep Neural Nets
Chen Zhu, W. Ronny Huang, Ali Shafahi, Hengduo Li, Gavin Taylor, Christoph Studer, Tom Goldstein
TL;DR
Deep neural networks trained on data from untrusted sources are vulnerable to poisoning attacks, motivating transferable clean-label attacks without access to the victim model. The paper constructs feature-space polytopes around targets and improves transferability with Dropout and multi-layer enforcement.
Problem
Data collected from untrusted sources can let attackers inject malicious training examples that control model behavior, motivating effective transferable clean-label poisoning attacks.
Method
The attack optimizes correctly labeled poisons to form a convex polytope around the target in substitute-model feature space, using Dropout and multiple layers to improve transferability.
Results
The polytope objective achieves better black-box success than feature collision, while multi-layer enforcement produces high success rates in end-to-end learning contexts.
Takeaways & Limitations
Transferability can be enhanced by sampling varied substitute networks with Dropout and enforcing the polytope objective across multiple layers.
Takeaways & Limitations
The attacker must know the victim’s training distribution well enough to collect a similar training set for substitute models.
Abstract
from arXiv · showhide
Clean-label poisoning attacks inject innocuous looking (and "correctly" labeled) poison images into training data, causing a model to misclassify a targeted image after being trained on this data. We consider transferable poisoning attacks that succeed without access to the victim network's outputs, architecture, or (in some cases) training data. To achieve this, we propose a new "polytope attack" in which poison images are designed to surround the targeted image in feature space. We also demonstrate that using Dropout during poison creation helps to enhance transferability of this attack. We achieve transferable attack success rates of over 50% while poisoning only 1% of the training set.
1. Introduction
The paper examines clean-label poisoning as a security risk in web-sourced training data and targets transferable attacks against unknown black-box image classifiers. It proposes a polytope-based objective, strengthened with Dropout and multi-layer enforcement, to improve attack transferability.
- Web-scraped datasets can expose deep networks to poisoning attacks that manipulate training data to control inference behavior.
- Clean-label attacks preserve correct labels, allowing malicious images to be placed online for collection by scraping systems or unsuspecting victims.
- Prior clean-label poisoning demonstrations used white-box access, leaving transfer to unknown black-box deep image classifiers unexplored.
- Transferable poisoning is difficult because poisoning changes the victim’s decision boundary unpredictably and may provide no direct access to the victim model.
- The proposed attack trains substitute models and optimizes poisons into a feature-space polytope enclosing the target, improving black-box transfer over feature collision.
- Dropout during poison crafting and enforcement across multiple network layers further improve transferability in transfer-learning and end-to-end settings.
2. The Threat Model
The threat model assumes clean-label poisons are injected into image-classification training data while the target remains unmodified at inference. The attacker lacks access to the victim model but can collect a similar training distribution and craft attacks for transfer learning or end-to-end training.
- The attacker adds perturbed samples with labels true to their class, aiming to make the trained victim classify an unmodified test image as a specified alternative class.
- The victim model is inaccessible, so the attacker must know the victim’s training distribution well enough to collect a similar dataset for substitute models.
- Transfer learning freezes a pretrained feature extractor and fine-tunes an application-specific linear classifier on features from another dataset.
- In the toy SVM example, feature collision can leave the target correctly classified even when poisons are closest, whereas the polytope attack minimizes the poisons’ segment distance to the target.
- End-to-end training jointly updates the feature extractor and classifier, imposing stricter requirements because poisons can alter the extractor’s parameters.
3. Transferable Targeted Poisoning Attacks
Targeted poisoning is harder than evasion because training on poisoned data changes the victim’s decision boundary. The paper replaces strict feature collision with a convex-polytope constraint to improve transferability and reduce visible target patterns.
- Threat and challenge: Targeted poisoning must make a trained victim classify an unmodified target as a specified alternative class.Unlike targeted evasion, the attacker modifies the training distribution rather than only perturbing the test input.
- Feature Collision Attack: Feature Collision crafts correctly labeled poisons from a targeted class by bringing their feature representations close to the target’s.The input-space term preserves the base image’s human-perceived label, while the feature-space term encourages collision.
- Transferability challenge: Different feature extractors create different feature spaces, so collisions optimized on substitute models may not transfer to an unknown victim model.This mismatch makes black-box Feature Collision attacks difficult even when the poison is close to the target for substitute models.
- Imperceptibility: Feature Collision can produce obvious target patterns, whereas Convex Polytope poisons show almost no target patterns and have enhanced imperceptibility.The stricter collision objective imposes many feature constraints, encouraging visible target structure in the poisons.
- Convex Polytope Attack: The Convex Polytope Attack places poison features so the target lies inside their convex polytope rather than forcing every poison to collide with it.This relaxed condition allows poisons to lie farther from the target while altering labels across a larger region of feature space.
- Convex Polytope Attack: The convex-polytope objective improves transferability because the unknown victim need not align the target with one specific poison feature.For Feature Collision, success requires the target to lie within a ρ-ball around a particular poison; the polytope permits a broader condition.
- Multi-layer and randomized attacks: When the victim trains its feature extractor end-to-end, applying the polytope objective at only one feature space is insufficient for successful transfer.The paper addresses this by applying the attack across multiple network layers, while Dropout randomization supplies many substitute networks without proportional memory costs.
4. Experiments
The experiments evaluate Convex Polytope (CP) poisoning across architectures, training-set overlap, and training regimes. CP generally transfers better than Feature Collision (FC), but end-to-end transfer requires enforcing the objective in multiple layers.
- Experimental setup: CIFAR10 experiments use 48,000 images for pre-training and a 500-image clean fine-tuning set, with five ship poisons targeting frog images excluded from training.The next 50 frog images are evaluated independently as targets.
- Validation accuracy: The attack preserves victim accuracy compared with fine-tuning on the corresponding clean dataset.This supports the clean-label requirement that poisoning should not cause an obvious accuracy drop.
- Experimental setup: Victims span six substitute architectures and two black-box architectures, with different random seeds distinguishing gray-box from black-box evaluation.Gray-box victims share an architecture with substitute models; black-box victims use architectures absent from the substitute set.
- Transfer learning: CP achieves success rates higher or close to 0.5 in most transfer-learning cases, whereas FC never exceeds 0.5.The comparison uses substitute architectures from set S1.
- Importance of Training Set: With 0% training-set overlap, CP still transfers to structurally different black-box models, although higher-capacity models transfer better than lower-capacity models.The authors attribute this pattern to greater overfitting by higher-capacity models and conclude that transfer can remain powerful when the victim generalizes well.
- End-to-End Training: In end-to-end training, last-layer CP produces almost zero successful attacks, motivating enforcement of the CP objective across multiple substitute-model layers.Multi-layer enforcement yields success rates above 0.6 in the gray-box setting, but transfer to GoogLeNet remains difficult.
5. Conclusion
The paper introduces a convex-polytope objective for transferable clean-label targeted poisoning. Dropout ensembles and multilayer enforcement improve transferability, while substitute-model data distributions also affect performance.
- Conclusion: The main contribution constructs a convex polytope around the target in feature space, enabling a linear classifier overfitting the poisoned data to assign the target the poisons’ class.The objective is designed to enhance transferability of clean-label targeted poisoning attacks.
- Conclusion: Dropout during poison crafting samples across networks with different structures, providing one practical way to improve transferability.The paper describes this as an ensemble-like use of Dropout.
- Conclusion: Enforcing the convex-polytope objective in multiple layers enables attack success in end-to-end learning contexts.This extends the approach beyond attacks restricted to the final feature layer.
- Conclusion: Transferability can depend on the data distribution used to train the substitute model.The conclusion identifies substitute-model training data as an additional factor affecting transfer.
A. Proof of Proposition 1
The proposition’s proof shows that linear classification constraints define a convex region containing the target feature representation when the target is a convex combination of poison features.
- Convex region: Each class constraint defines a convex half-space, so their intersection is also convex.The proof uses the convexity of the region satisfying all linear inequalities.
- Convex region: If the target feature is a convex combination of points in that intersection, it also lies in the intersection and satisfies the classification constraints.This establishes why surrounding the target with poison features supports the desired label under the proposition’s conditions.
- Proof construction: The proof considers the closest point in the poison-feature set to the target and uses a smooth classifier function to derive the needed derivative condition.The zero-distance case directly satisfies the convex-combination condition; otherwise the closest-point construction supplies the proof step.
B. Comparison of Validation Accuracies
The validation experiment reports that the accuracy drop caused by poisoning is not obvious compared with fine-tuning on the corresponding clean dataset.
- Validation accuracy: The poisoned dataset causes no obvious drop in fine-tuned model accuracy relative to the corresponding clean dataset.Maintaining accuracy is treated as part of making the poisoning attack undetectable.
C. Details of the qualitative example
The qualitative example uses a fish target and five hook images from WebVision, with all five poison examples shown in Figure 9. Figure 8 reports accuracies on the whole CIFAR10 test for models trained or fine-tuned on different datasets.
- Qualitative example: Both the target fish image and five hook images used to craft the poisons come from WebVision.WebVision shares the ImageNet taxonomy.
- Qualitative example: Figure 8 compares whole-CIFAR10 test accuracies for models trained or fine-tuned on different datasets.Fine-tuned models are initialized from networks trained on the first 4800 images of each class.
- Qualitative example: Figure 9 presents all five poison examples.