Source-linked AI summary
Adversarial Examples Make Strong Poisons
Liam Fowl, Micah Goldblum, Ping-yeh Chiang, Jonas Geiping, Wojtek Czaja, Tom Goldstein
TL;DR
The paper asks whether adversarial examples can be used to poison training data more effectively than methods designed for poisoning. It introduces adversarial poisoning and finds that these examples sharply degrade clean-data performance because they encode discriminative features for incorrect classes, while requiring a clean model for poison generation.
Problem
Automated training-data curation increases exposure to availability attacks that degrade models’ clean-data performance.
Method
Adversarial poisoning generates training-data perturbations using a fixed model trained on clean data, including a class-targeted variant.
Results
Adversarial attacks against a fixed network are more potent availability poisons than existing methods, often reducing accuracy below random-guessing levels.
Takeaways & Limitations
Adversarial examples contain discriminatory features for the wrong class, making them effective poisons and useful for studying data obfuscation.
Takeaways & Limitations
Poison generation requires a clean trained model, making the attack less general purpose.
Abstract
from arXiv · showhide
The adversarial machine learning literature is largely partitioned into evasion attacks on testing data and poisoning attacks on training data. In this work, we show that adversarial examples, originally intended for attacking pre-trained models, are even more effective for data poisoning than recent methods designed specifically for poisoning. Our findings indicate that adversarial examples, when assigned the original label of their natural base image, cannot be used to train a classifier for natural images. Furthermore, when adversarial examples are assigned their adversarial class label, they are useful for training. This suggests that adversarial examples contain useful semantic content, just with the ``wrong'' labels (according to a network, but not a human). Our method, adversarial poisoning, is substantially more effective than existing poisoning methods for secure dataset release, and we release a poisoned version of ImageNet, ImageNet-P, to encourage research into the strength of this form of data obfuscation.
1 Introduction
The paper connects test-time adversarial examples to training-time availability poisoning, finding that adversarially crafted data can sharply degrade clean-data performance. It attributes this strength to adversarial features that carry information associated with incorrect classes.
- Adversarial examples form stronger availability attacks than existing poisoning methods, despite being designed for test-time model manipulation.
- Models trained on adversarial examples can perform below random-guessing accuracy on clean test data.
- Adversarially perturbed training data can prevent models from classifying the original clean training samples.
- Random additive noise does not produce the same failure, suggesting adversarial perturbations inject features associated with incorrect labels rather than merely leaving the natural image manifold.
- Relabeling adversarial examples with their crafting network’s incorrect predictions substantially improves training performance, approaching that of clean-image training.
2 Related Work
Related work distinguishes data poisoning, adversarial examples, and adversarial training by their optimization objectives and roles in attacking or training models.
- Classical data-poisoning attacks use bilevel optimization, but exact solutions become intractable for neural networks.
- Adversarial attacks create small, constrained input perturbations that maximize loss and can cause catastrophic misclassification.
- Some studies characterize adversarial examples as off-manifold, while others develop methods for producing on-manifold attacks.
- Adversarial training crafts adversarial versions of mini-batches and trains neural networks on those versions.
3 Adversarial Examples as Poisons
The paper treats adversarial examples as availability poisons: simple, transferable perturbations that degrade models trained on poisoned data, including at ImageNet scale and under partial-data settings. Its class-targeted variant is especially potent, while defenses generally fail to restore clean-data accuracy.
- The poisoner perturbs the victim’s training data without knowing its model, training routine, or architecture, and success is measured on clean data.
- Adversarial poisoning replaces the intractable bilevel poisoning objective with adversarial examples crafted on a fixed pretrained network.The method uses loss-maximizing perturbations under an ℓ∞ constraint; PGD is the strongest tested attack.
- Even under tight ℓ∞ constraints, adversarial poisons more than halve test accuracy across VGG19, ResNet-18, GoogLeNet, DenseNet-121, and MobileNetV2 on CIFAR-10.
- Compared with the previous best loss-minimization method, adversarial poisoning degrades a CIFAR-10 victim’s validation accuracy by a factor of more than three.
- The method transfers across black-box victim architectures and training settings, while class-targeted facial-recognition poisons reduce protected-class accuracy to 8%.On WebFace, this is half the classification success of the compared unlearnable-examples method.
- Adversarial training improves defense performance but is computationally expensive and reduces natural accuracy; other tested defenses can perform worse than using no defense.
4 Analysis
The experiments support the view that adversarial poisons contain useful features associated with the adversarial class, rather than merely encoding decision-boundary information. Relabeling poisons with their adversarial labels substantially improves validation accuracy, while models trained on original labels fail on clean data.
- Victim networks can almost perfectly fit adversarial examples but perform poorly on the original unperturbed training images.This separates successful optimization on the poison data from generalization to clean data.
- Class-targeted adversarial poisons make victim networks confuse the same classes as the crafting network.The victim-network classification patterns match the attack patterns used to generate the poisons.
- Relabeling each poison with its adversarial class label raises validation accuracy from 6.25% to 75.69% on a victim ResNet-18.For example, a dog image perturbed toward the cat class is trained with the cat label; this correction requires access to the crafting model and original clean dataset.
- Figure 2 compares predictions after attacking the crafting network with clean-test predictions from a new network trained on adversarial poisons.
- The relabeling result supports the conclusion that adversarial examples contain useful semantic features, but those features are assigned the wrong labels.A comparison with labeled SVHN data shows that unrelated out-of-distribution data does not achieve comparable CIFAR-10 validation accuracy.
5 Limitations
The method requires a clean trained model to generate adversarial examples and remains vulnerable to adversarial training as a defense.
- Adversarial poisoning requires a clean trained model whose parameters remain fixed during poison generation.This assumption may fit practitioners such as social media companies but makes the attack less general purpose.
- Adversarial training is a second limitation because it has been successful as a defense against availability poisoning attacks.
6 Conclusions
The paper finds that adversarial attacks against a fixed network are potent availability poisons and that their effectiveness reflects discriminatory features assigned to the wrong class.
- Adversarial attacks against a fixed network often degrade victim accuracy below random-guessing levels, outperforming existing availability-poisoning methods.
- Across multiple settings, the experiments indicate that adversarial examples contain discriminatory features for the wrong class.
- The class-targeted attack uses this wrong-class feature structure to poison models across multiple datasets.
A.1 Training/Crafting details
The experiments use dataset-specific training and crafting procedures for CIFAR-10, ImageNet, and facial recognition, with fixed training schedules and label permutations in the image experiments.
- For CIFAR-10, the crafting network is trained for 40 epochs, while victim models train for 100 epochs with three SGD learning-rate drops.
- For ImageNet, poisons are crafted with a pretrained ResNet18 in batches of 25,000 before training a randomly initialized victim model for 100 epochs.Standard ImageNet resizing, center crops, and normalization are used during victim training.
- Class-targeted attacks use a random label permutation; in the reported CIFAR-10 and ImageNet settings, label i maps to i + 3.
- The facial-recognition experiments modify only selected users’ face images after splitting WebFace into 80% training and 20% testing data.Fifty identities are randomly selected as the users whose identities should be hidden.
A.1.1 Hardware and time considerations
Crafting and training costs vary substantially with dataset and hyperparameter choices. A typical CIFAR-10 experiment requires roughly six hours on four RTX 2080 Ti GPUs, but pretrained models and one restart can reduce this time.
- A typical CIFAR-10 crafting experiment takes roughly 6 hours using 4 RTX 2080 Ti GPUs.Crafting and training time vary with dataset and hyperparameter choices.
- Using pretrained models and one restart reduces the typical CIFAR-10 experiment time by roughly a factor of 8.
A.2 Visualization
The visualizations show adversarial and Webface poison perturbations at specified ε levels. Increasing perturbation strength trades visual similarity for potency, while poisoned examples can make models perform below random accuracy.
- Increasing ε creates a trade-off between visual similarity and perturbation potency.
- Figure 3 compares an unaltered CIFAR-10 frog image with perturbations at ε = 4/255 and ε = 8/255.Networks trained on perturbations including the ε = 8/255 example achieve below random accuracy.
- Figure 4 visualizes perturbed identities from the Webface dataset using ε = 8/255 poisoned images.
A.3 Adversary comparison
Across adversary comparisons and ablations, PGD-based adversarial poisoning is effective, transfers across architectures, and generally favors class-targeted attacks over untargeted attacks. Its effectiveness depends on crafting choices, with robust crafting models producing ineffective poisons and untargeted attacks showing instability.
- A.3 Adversary comparison: PGD-based attacks supersede other common adversarial attacks in poison efficiency.The comparison uses black-box victim models with random initialization on CIFAR-10.
- A.4 Crafting Ablations: More PGD optimization steps make adversarial poisons more effective, while fewer steps still degrade validation accuracy.
- A.4 Crafting Ablations: Adversarial poisons transfer across different victim network architectures.
- A.4 Crafting Ablations: Adversarially trained crafting models produce ineffective poisons because they use less perturbation-brittle discriminatory features.
- A.6 Adversarial Labels: Training with adversarial labels can enable classification of cats without showing the network cat images.
- A.8 ImageNet Comparison: The class-targeted ImageNet attack far outperforms prior gradient-alignment poisoning when comparing 100-epoch training with prior 40-epoch training.
- A.9 Instability of Untargeted Attacks: Class-targeted attacks are more stable and potent than untargeted attacks, whose effectiveness can depend strongly on poison initialization.Untargeted attacks can significantly degrade validation accuracy but may produce much worse outcomes across poison datasets.