PaperlayerView exact revision on arXiv

Source-linked AI summary

Certified Adversarial Robustness with Additive Noise

Bai Li, Changyou Chen, Wenlin Wang, Lawrence Carin

arXiv:1809.03113v6cs.LGcs.CRstat.ML

TL;DR

Adversarial defenses often lack adaptive-attack resistance, theoretical guarantees, or scalability to large models and datasets. The paper connects certified adversarial robustness with additive random noise and introduces stability training, obtaining competitive provable and empirical robustness, especially for strong attacks.

  • Problem

    Existing defenses are often heuristic and vulnerable to adaptive attacks, while certified methods frequently struggle to provide non-trivial robustness for large-scale models and datasets.

  • Method

    The paper derives noise-based certified bounds using Rényi divergence and uses stability training to improve robustness to additive random noise.

  • Results

    The defense provides competitive provable and empirical robustness against state-of-the-art models, with especially strong robustness under strong attacks.

  • Takeaways & Limitations

    Additive noise enables a scalable framework that combines certified and empirical robustness, while stability training improves the certified bounds.

  • Takeaways & Limitations

    The method has a gap between theoretical lower bounds and empirical accuracy, and stability training alone is not claimed to provide adversarial robustness.

Abstract

from arXiv · show

The existence of adversarial data examples has drawn significant attention in the deep-learning community; such data are seemingly minimally perturbed relative to the original data, but lead to very different outputs from a deep-learning algorithm. Although a significant body of work on developing defensive models has been considered, most such models are heuristic and are often vulnerable to adaptive attacks. Defensive methods that provide theoretical robustness guarantees have been studied intensively, yet most fail to obtain non-trivial robustness when a large-scale model and data are present. To address these limitations, we introduce a framework that is scalable and provides certified bounds on the norm of the input manipulation for constructing adversarial examples. We establish a connection between robustness against adversarial perturbation and additive random noise, and propose a training strategy that can significantly improve the certified bounds. Our evaluation on MNIST, CIFAR-10 and ImageNet suggests that the proposed method is scalable to complicated models and large data sets, while providing competitive robustness to state-of-the-art provable defense methods.

1 Introduction

Adversarial examples expose a gap between deep-network performance and reliable robustness guarantees. The paper develops a noise-based certified framework and training strategy intended to scale while improving certified bounds.

  • Adversarial examples are minimally perturbed inputs that can produce different classifier outputs despite little or no visible image difference.
  • Many existing defenses are heuristic, vulnerable to adaptive attacks, or unable to obtain non-trivial certified robustness at scale.
  • The framework analyzes model robustness using Rényi divergence between outputs on natural and adversarial examples with added random noise.
  • The analysis yields a higher upper bound on tolerable perturbation size than prior work [18].
  • Stability training accounts for the connection between adversarial robustness and additive random noise, improving certified bounds.
  • Experiments evaluate theoretical and empirical performance across the proposed framework, with results competitive with state-of-the-art methods.

2 Background and Related Work

Prior certified-robustness analyses often depend on restrictive activations or architectures and are difficult to scale. Related work also connects adversarial robustness with robustness to additive noise, but uses different analytical perspectives.

  • Distributionally robust optimization certifies robustness to changes in data-generating distributions through divergence or Wasserstein neighborhoods.
  • Other methods certify minimum perturbation distortion, but some analyses are restricted to shallow networks, specific structures, or specialized relaxations.
  • Certified-bound analyses generally rely on particular activation functions or model structures, limiting scalability to flexible architectures and large datasets.
  • Earlier work links adversarial robustness to additive noise through decision-boundary curvature, concentration of measure, or near-zero error under large noise.

3 Preliminaries

The paper studies image classification with stochastic classifiers whose outputs are class-probability distributions. It situates adversarial defense alongside empirical adversarial training and certified approaches, while targeting both forms of robustness.

  • 3.1 Notation: Images are represented in [0,1]^(h×w×c), and a k-class DNN classifier maps inputs to class labels or multinomial output distributions.
  • 3.2 Rényi Divergence: Rényi divergence is introduced as the theoretical quantity underlying the paper’s certified-robustness analysis.
  • 3.3 Adversarial Examples: An adversarial example is a norm-bounded input x′ satisfying D(x,x′)<ϵ while changing the classifier’s output; the paper focuses mainly on ℓ2 and also evaluates ℓ∞.
  • 3.4 Adversarial Defense: Adversarial defense models are designed to remain robust to adversarial examples.
  • 3.4 Adversarial Defense: Adversarial training improves empirical robustness by augmenting training data with adversarial examples, while TRADES adds a regularizer for them.
  • 3.4 Adversarial Defense: The proposed approach aims to provide both empirical and provable robustness, which are often treated as separate research directions.

4 Certified Robust Classifier

The certified classifier adds random noise before prediction and uses output probabilities to bound tolerable perturbations. Rényi-divergence analysis supports certification for ℓ2 perturbations with Gaussian noise and extends to ℓ1 with Laplacian noise.

  • 4 Certified Robust Classifier: Random noise is added to input pixels before classification to reduce the effects of adversarial perturbations and produce an upper bound on tolerable attack size.
  • 4 Certified Robust Classifier: Algorithm 1 samples noisy predictions, estimates class probabilities, and returns the majority class together with the certified attack size.
  • 4 Certified Robust Classifier: The proof targets label preservation: if x is classified as class c, every x′ with ∥x−x′∥2≤L should also receive class c.
  • 4 Certified Robust Classifier: Rényi-divergence bounds ensure that sufficiently small divergence preserves the index of the largest class probability.
  • 4 Certified Robust Classifier: For ℓ2 perturbations, Theorem 2 compares noisy class distributions generated at x and x′ under Gaussian noise to certify unchanged prediction.
  • 4 Certified Robust Classifier: For ℓ1 perturbations, the framework replaces Gaussian with Laplacian noise; the resulting bound is tight in the binary case.
  • 4 Certified Robust Classifier: Certification applies to any classifier with i.i.d. Gaussian or Laplacian test noise, but probability estimates require confidence-interval adjustment and larger samples increase computational burden.
  • 4 Certified Robust Classifier: The noise level σ creates a trade-off: larger noise can delay bound degeneration but can also reduce classification accuracy and worsen the bound.

5 Improved Certified Robustness

The framework connects adversarial robustness with robustness to additive Gaussian noise, then uses stability training to improve certified robustness. Its testing-time noise is theoretically supported, while broader noise-robustness methods can be adapted within the framework.

  • Larger separation between p(1) and p(2) under additive Gaussian noise yields a larger certified robustness bound.The paper associates this separation with more confident classification under noise.
  • Robustness to additive Gaussian noise is easier to achieve than robustness to carefully crafted adversarial examples, motivating the framework's training strategy.
  • The method requires added Gaussian noise during testing and is supported theoretically.
  • The framework can adapt methods for robustness to random noise to improve classification accuracy under Gaussian noise and thereby adversarial robustness.The paper specifically discusses stability training, while noting a broader literature on noise-robust methods.
  • Stability training encourages perturbed inputs to produce similar classifier outputs and was previously used to improve robustness on noisy images.It adds a regularization term based on the distance between predictions for original and perturbed inputs.
  • Within this framework, stability training improves Algorithm 1's adversarial robustness and yields Stability Training with Noise (STN).The paper distinguishes this use from stability training alone, which targets classification accuracy under Gaussian perturbation.

6 Experiments

Experiments evaluate STN's certified and empirical robustness on MNIST, CIFAR-10, and ImageNet, including comparisons with PixelDP, TRADES, and prior certified methods. Stability training improves certified bounds, while noise creates a trade-off between small- and large-attack performance.

  • Theoretical Bound: Larger σ delays degeneration of the certified lower bound but worsens bounds for small L∗ because added noise reduces classification accuracy.The experiments expose a trade-off between robustness to larger perturbations and clean classification performance.
  • Theoretical Bound: Stability training significantly improves the lower bound, and STN bounds dominate PixelDP's accuracy lower bounds in Figure 1.PixelDP is represented using its optimal parameter pair found by grid search.
  • Theoretical Bound: STN's certified bound is close to [17] on MNIST and better on CIFAR-10 while retaining high natural accuracy under stronger attacks.The comparison reports a trade-off between certified bounds, robustness accuracy, and natural-example accuracy.
  • Theoretical Bound: STN produces a higher certified bound than PixelDP on ImageNet, a dataset containing over 1 million images and 1,000 classes.The framework adds almost no extra computational burden during training, enabling the ImageNet evaluation.
  • Empirical Results: Figure 3 compares TRADES and STN across MNIST and CIFAR-10, ℓ2 and ℓ∞ attacks, and white-box and black-box settings.White-box evaluations use PGD for ℓ∞ attacks and Carlini & Wagner attacks for ℓ2 attacks; black-box results transfer attacks generated against Madry's model.
  • Empirical Results: STN performs slightly worse than TRADES for small attacks but becomes better as attack size increases, especially for ℓ2-bounded distortions.The authors attribute this pattern to random noise reducing small-attack accuracy while helping against stronger attacks.
  • Empirical Results: Figure 4 shows that additive noise hurts under small attacks but helps against large attacks, while σ = 0 is vulnerable.The authors note that an adaptive noise amount could improve results but is impractical because attack size is unknown beforehand.

7 Comparison to [40]

The paper compares its framework with [40], which provides a tighter ℓ2 bound, and identifies stability training, real-attack evaluation, and norm flexibility as distinguishing contributions. It reports a tight certified ℓ1 bound using Laplacian noise.

  • Comparison to [40]: [40] provides a tighter ℓ2 bound, but STN uniquely combines stability training with certified robustness and empirical attack evaluation.The paper states that stability training generally outperforms Gaussian augmentation, as shown in Figure 1, while [40] does not evaluate real attacks.
  • Comparison to [40]: STN extends beyond [40]'s isotropy-dependent analysis by yielding a tight certified ℓ1 bound through Laplacian noise.The paper presents norm flexibility as a distinction between the two analyses.

8 Conclusions

The framework connects certified adversarial robustness with robustness to additive random perturbations and uses stability training to improve defense models. Experiments report competitive provable and empirical robustness, especially against strong attacks, while leaving a gap between theoretical and empirical performance.

  • The analysis connects robustness to adversarial attacks with robustness to additive random perturbations.
  • Stability training is proposed to improve the robustness of defense models.
  • Experimental results show competitive provable and empirical robustness compared to state-of-the-art models.
  • The defense model yields strong robustness when strong attacks are considered.
  • A noticeable gap remains between theoretical lower bounds and empirical accuracy, suggesting the upper bound may not be tight or stronger attacks may reduce empirical results.

A Comparison between Bounds

The paper compares its certified bound with PixelDP through simulation over valid top-two output probabilities and derives the comparison using Rényi-divergence optimization. Its bound is strictly higher and can exceed PixelDP's by more than twofold when the probabilities are far apart.

  • The simulation compares the proposed bound with PixelDP over valid first and second largest output probabilities.The comparison requires p(1) ≥ p(2) ≥ 0, p(1) + p(2) ≤ 1, and p(1) + p(2) ≥ 0.2.
  • For fixed σ, a grid search selects ε and δ to maximize PixelDP's bound for a fair comparison.
  • The proposed bound is strictly higher than PixelDP's across the simulated p(1) and p(2) settings.
  • When p(1) and p(2) are far apart, the proposed bound is more than twice as high as PixelDP's.
  • The comparison analyzes the bound through Rényi-divergence minimization over distributions whose largest-probability index changes.The proof assumes an ordered distribution and derives the constrained solution using KKT conditions.

C Proof of Theorem 2

The proof uses the Rényi divergence between Gaussian-noised inputs to bound output-distribution divergence under an input perturbation. It then applies the probability condition to show that the noisy classifier preserves its prediction.

  • Rényi divergence between Gaussian distributions with common covariance depends on the squared ℓ2 distance between their means.
  • Theorem 2 considers natural and adversarial inputs within ℓ2 distance L and compares their classifier output distributions under Gaussian noise.
  • If the stated condition on the first and second largest output probabilities holds, the noisy classifier gives the same prediction on both inputs.
  • The proof applies data processing to relate divergence between noised inputs to divergence between the classifier's output distributions.

D.1 Gradient-Free methods

The evaluation includes Boundary Attack, a gradient-free method, alongside other attacks on MNIST and CIFAR-10. The figures compare STN robustness across attack types for ℓ2 and ℓ∞ perturbations, while the reported Boundary Attack results are ineffective against the models.

  • Boundary Attack constructs adversarial examples along the decision boundary without model-gradient information.It uses rejection sampling and complements gradient-based attacks.
  • The experiments test Boundary Attack on MNIST and CIFAR-10 alongside other attacks.
  • Figure 6 compares MNIST adversarial robustness across attack types for ℓ2 and ℓ∞ perturbations.
  • Boundary Attack is not effective against the models, consistent with observations about gradient-free attacks against randomized models.The results are included as a sanity check.
  • Figure 7 compares CIFAR-10 adversarial robustness across attack types for ℓ2 and ℓ∞ perturbations.
Loading 1809.03113v6…