Source-linked AI summary
Adversarial Example Defenses: Ensembles of Weak Defenses are not Strong
Warren He, James Wei, Xinyun Chen, Nicholas Carlini, Dawn Song
TL;DR
The paper asks whether combining weak defenses can create a substantially stronger defense against adversarial examples. It evaluates three ensemble strategies with adaptive, defense-specific attacks and finds that all evaluated components and combined defenses can be defeated with low distortion. The results imply that ensembles of weak defenses are insufficient for strong protection against adaptive attackers.
Problem
The paper examines whether combining multiple individually weak defenses can produce a defense significantly stronger than each component.
Method
The authors study three ensemble defenses and evaluate their components and composites using adaptive attacks tailored to the defenses.
Results
All evaluated defenses and components can be defeated by adaptive adversaries generating adversarial examples with low distortion.
Takeaways & Limitations
Ensembled defenses do not provide significantly more resilience than their individual components, so ensembles of weak defenses are insufficient against adaptive attackers.
Takeaways & Limitations
The paper assumes a full-knowledge adaptive adversary, while noting that its optimization-based attack is not guaranteed to succeed in every general case.
Abstract
from arXiv · showhide
Ongoing research has proposed several methods to defend neural networks against adversarial examples, many of which researchers have shown to be ineffective. We ask whether a strong defense can be created by combining multiple (possibly weak) defenses. To answer this question, we study three defenses that follow this approach. Two of these are recently proposed defenses that intentionally combine components designed to work well together. A third defense combines three independent defenses. For all the components of these defenses and the combined defenses themselves, we show that an adaptive adversary can create adversarial examples successfully with low distortion. Thus, our work implies that ensemble of weak defenses is not sufficient to provide strong defense against adversarial examples.
1 Introduction
The paper asks whether combining multiple weak defenses can produce a defense significantly stronger than its components. Across three ensemble strategies, adaptive attacks defeat both components and composites with low distortion, indicating that ensembling is insufficient for strong defense.
- Adversarial examples threaten neural networks because small input changes can cause incorrect predictions, including in security-critical autonomous systems.
- The paper systematically investigates whether ensembles of individually weak defenses become significantly stronger than their components.The authors study three ensemble instances, including defenses intentionally designed to combine compatible components and an ensemble of independent mechanisms.
- Adaptive attacks defeat feature squeezing, the ensemble-of-specialists defense, and an ensemble of recently proposed detectors.The attacks target both individual components and composite defenses.
- Ensembled defenses provide no significantly greater resilience than their individual components, so combining weak defenses is insufficient against adaptive attackers.The evaluation also finds that adaptive adversarial examples transfer across several defense or detection proposals.
2 Overview
The overview defines adversarial examples and threat models, then describes the paper’s adaptive attack evaluation. Experiments use optimization-based attacks and measure distortion primarily with the L2 norm on MNIST and CIFAR-10.
- 2.1 Background: Adversarial Examples: An adversarial example is close to a correctly classified input but causes the classifier to produce an incorrect output.The paper considers both untargeted and targeted adversarial examples.
- 2.2 Threat Models: The assumed adversary knows the model architecture, parameters, and defense strategies, making it strictly more powerful than a static adversary.
- 2.3 Problem Statement: The paper evaluates whether three defense ensembles outperform their component defenses against adaptive attacks.Two ensembles intentionally combine components designed to work together, while the third combines independent, mutually compatible detection mechanisms.
- 2.4 Experimental Setup: The evaluation uses 100 randomly sampled, correctly classified test images from MNIST and CIFAR-10 with the model architectures specified by the defense papers.
- 2.4 Experimental Setup: The attacks use optimization to minimize a loss balancing input distortion and misclassification.The loss includes a classifier-logit penalty and a constant controlling the relative weighting between distortion and misclassification.
- 2.4 Experimental Setup: Distortion is measured as the L2 norm between the adversarial example and the original input unless otherwise specified.Input-image dimensions are scaled to the interval [0,1].
3 Adaptive attacks on feature squeezing
The paper attacks feature squeezing adaptively, targeting both its individual transformations and the composite detector. Across MNIST and CIFAR-10, adversarial examples remain effective after squeezing and evade detection with low distortion.
- Background: feature squeezing defense: Feature squeezing reduces image fidelity through color-depth reduction or median-filter spatial smoothing before classification.The composite detector compares softmax outputs for the original, reduced-color-depth, and smoothed images using an L1 score.
- 3 Adaptive attacks on feature squeezing: Adaptive attacks explicitly search for adversarial examples that remain adversarial after each squeezing transformation.The attack first targets robustness to individual components, then constructs examples whose predictions remain similar across transformed inputs.
- 3.1.1 Evading color-depth-reduction defense: 3.86 average distortion sufficed for MNIST examples that remained misclassified after 1-bit color-depth reduction.All original images yielded successful examples; the unreduced 8-bit baseline required 1.38 average distortion.
- 3.1.1 Evading color-depth-reduction defense: 0.945 average distortion sufficed for CIFAR-10 examples after 3-bit color-depth reduction, compared with 0.214 without reduction.The authors report that the required distortion remained small and imperceptible despite increasing relative to the baseline.
- 3.1.2 Evading spatial smoothing: 1.29 average distortion defeated 3×3 median smoothing on MNIST without increasing distortion over the 1.38 unsmoothed baseline.Adversarial examples were generated for all original images, and larger median filters did not require greater distortion.
- Background: Composite feature squeezing defense: 4.76 average distortion bypassed combined 1-bit color reduction and 2×2 smoothing on MNIST, while 0.601 bypassed the analogous CIFAR-10 combination.Both attacks succeeded on all original images, produced misclassified examples, and evaded detection; the CIFAR-10 distortion was lower than for color reduction alone.
4 Evading ensemble of specialists
The specialists+1 defense combines a generalist with specialists trained on related class subsets, but adaptive attacks evade the ensemble with small distortions and high confidence.
- Defense background: The specialists+1 ensemble uses a generalist classifier alongside specialists trained to classify selected class subsets and their complements.For each class, subsets are derived from frequent non-targeted FGSM confusions, with an additional generalist covering all classes.
- Defense background: The system accepts a class when the generalist and all applicable specialists agree; otherwise, it uses the ensemble majority vote.
- Adaptive attack: The original defense reduced confidence against static FGSM, DeepFool, and related attacks, motivating evaluation against an adaptive attacker.
- Adaptive attack: The adaptive attack targets random classes and optimizes for simultaneous agreement by the generalist and all applicable specialists.
- Results: 2.50 average distortion defeated the ensemble on MNIST for all original images, despite distortion being 81.2% higher than for a non-ensemble classifier.
- Results: The specialists+1 ensemble therefore fails to ensure low confidence, allowing unanimously target-classified adversarial examples to evade detection.
5 Evading ensemble of detectors
The paper evaluates an ensemble of three independent detectors and constructs adaptive attacks that bypass their combined detection with imperceptible distortion.
- Defense construction: The ensemble combines color-depth, hidden-layer, and final-layer density-based detectors that were not designed to work together.
- Defense construction: The combined system reports an input as adversarial if any detector’s output probability exceeds one half.
- Adaptive attack: The attack constructs a classifier G that incorporates detector outputs, enabling direct optimization against the ensemble.
- Adaptive attack: If the optimized example avoids class N + 1, it is classified by the original network without being detected by any detector.
- Results: 0.18 L2 distortion defeated the three-detector network on CIFAR-10, compared with 0.11 for an unsecured network, a 60% increase that remained imperceptible.
- Results: Adversarial examples transfer across individual detectors, helping explain why combining them does not significantly improve robustness.
6 Conclusion
The study finds that adaptive adversaries can defeat the evaluated ensemble defenses and their components with low distortion. It concludes that weak-defense ensembles do not significantly improve robustness and stresses strong, adaptive evaluation.
- The paper examines three ensembles: two intentionally designed to combine compatible components and one combining unrelated detectors.
- Adaptive adversaries generate low-distortion adversarial examples that fool all evaluated defenses and components.
- Feature squeezing is at best marginally stronger than color-depth reduction alone, while specialists+1 raises required distortion only slightly.
- Adversarial examples transfer across individual detectors, and combining weak defenses does not significantly improve system robustness.
- Defense evaluations should use strong attacks and adaptive adversaries that know the mechanisms being tested.
A Gumbel-Softmax reparameterization
The Gumbel-Softmax approach reparameterizes discrete image optimization so gradients can update continuous distribution parameters. It generates adversarial images, though not at the lowest distance to unquantized originals.
- Quantized images are sampled from one Gumbel-Softmax distribution per channel and pixel, with continuous parameters optimized during backpropagation.
- The attack connects a discrete-image sampling network to a classifier and optimizes the sampling distributions rather than the image directly.
- On MNIST, 1-bit quantization produced adversarial examples for all originals with average L2 distortion 7.48, versus 2.56 for simple rounding.
- On CIFAR-10, 3-bit quantization produced adversarial examples for all originals with average L2 distortion 2.32, versus 2.27 for simple rounding.
- The reparameterized optimization succeeds but does not achieve the lowest distance to unquantized originals because generated images are necessarily quantized.
- The resulting distance is not much larger than the minimum distance from each original image to its normally quantized version.