PaperlayerView exact revision on arXiv

Source-linked AI summary

Stochastic Activation Pruning for Robust Adversarial Defense

Guneet S. Dhillon, Kamyar Azizzadenesheli, Zachary C. Lipton, Jeremy Bernstein, Jean Kossaifi, Aran Khanna, Anima Anandkumar

arXiv:1803.01442v1cs.LGstat.ML

TL;DR

Adversarial perturbations can fool neural networks while remaining imperceptible, motivating defenses that improve reliability. The paper casts defense as a stochastic minimax strategy and proposes SAP, which prunes activations by magnitude and rescales survivors. SAP can be applied to pretrained models without additional training, and experiments report improved accuracy and calibration, including additive benefits with adversarial training.

  • Problem

    Imperceptible adversarial perturbations can cause neural-network misclassification, threatening the reliability of systems deployed in the wild.

  • Method

    SAP uses a stochastic defense policy that preferentially retains larger-magnitude activations, prunes others, and scales survivors during inference.

  • Results

    SAP improves accuracy and calibration against adversarial examples, with additive benefits when combined with adversarial training.

  • Takeaways & Limitations

    SAP guards pretrained networks against adversarial examples without requiring additional training and can also be effective in reinforcement learning.

  • Takeaways & Limitations

    Why SAP works while other stochastic methods do not remains an open question.

Abstract

from arXiv · show

Neural networks are known to be vulnerable to adversarial examples. Carefully chosen perturbations to real images, while imperceptible to humans, induce misclassification and threaten the reliability of deep learning systems in the wild. To guard against adversarial examples, we take inspiration from game theory and cast the problem as a minimax zero-sum game between the adversary and the model. In general, for such games, the optimal strategy for both players requires a stochastic policy, also known as a mixed strategy. In this light, we propose Stochastic Activation Pruning (SAP), a mixed strategy for adversarial defense. SAP prunes a random subset of activations (preferentially pruning those with smaller magnitude) and scales up the survivors to compensate. We can apply SAP to pretrained networks, including adversarially trained models, without fine-tuning, providing robustness against adversarial examples. Experiments demonstrate that SAP confers robustness against attacks, increasing accuracy and preserving calibration.

1 INTRODUCTION

Neural networks can be fooled by imperceptible adversarial perturbations, creating reliability concerns for deployment. The paper proposes Stochastic Activation Pruning (SAP), which protects pretrained networks without additional fine-tuning.

  • Imperceptible, carefully chosen perturbations can induce high-probability misclassification in powerful convolutional neural networks.
  • Adversarial vulnerability threatens the suitability of machine-learning systems for real-world and security applications.
  • Existing defenses include adversarial training with the same model or ensembles, and learning flat representations sensitive to fewer input dimensions.
  • SAP stochastically prunes activations during the forward pass, preferentially retaining larger-magnitude activations and scaling survivors.
  • SAP can be applied post-hoc to pretrained networks without additional fine-tuning.

2 PRELIMINARIES

The paper formalizes neural networks, supervised learning, and bounded adversarial perturbations, then introduces FGSM as a first-order attack based on the loss gradient.

  • An n-layer neural network composes layer functions, each applying a linear transformation followed by a nonlinearity.
  • Learning minimizes a loss such as cross entropy over parameters learned from labeled data.
  • An adversary seeks a bounded perturbation ∆x satisfying ∥∆x∥∞≤λ that changes the model’s classification.
  • Neural-network nonlinearities make the adversarial optimization generally non-convex, motivating a first-order loss approximation.
  • FGSM sets ∆x = λ · sign(J (θ, x, y)) and requires model access to compute the gradient.

3 STOCHASTIC ACTIVATION PRUNING

SAP frames defense and attack as a minimax game and implements the defender’s stochastic policy by sampling activations in proportion to magnitude, pruning others, and reweighting survivors. Its expected activations are preserved, enabling post-hoc use on pretrained models while balancing accuracy against robustness.

  • 3 STOCHASTIC ACTIVATION PRUNING: The defense is formulated as a minimax zero-sum game whose optimal defender and adversary strategies are generally stochastic policies.
  • 3 STOCHASTIC ACTIVATION PRUNING: The method retains high-magnitude nodes and rescales survivors to preserve activation dynamic range, while remaining applicable post-hoc to trained models.
  • 3 STOCHASTIC ACTIVATION PRUNING: SAP converts each activation layer into a multinomial distribution whose probabilities are proportional to absolute activation values.
  • 3 STOCHASTIC ACTIVATION PRUNING: The algorithm samples activations with replacement, sets unsampled activations to zero, and scales sampled activations by inverse sampling propensity.
  • 3 STOCHASTIC ACTIVATION PRUNING: SAP preserves each activation in expectation, so sufficiently many examples can yield performance similar to the unpruned model and support application without fine-tuning.
  • 3.1 ADVANTAGE AGAINST ADVERSARIAL ATTACK: SAP’s benefit requires balancing samples per layer so accuracy is negligibly affected while robustness is retained.
  • 3.1 ADVANTAGE AGAINST ADVERSARIAL ATTACK: Unlike dropout’s uniform sampling, SAP favors large-magnitude activations, which supports post-hoc application without significantly reducing accuracy.
  • 3 STOCHASTIC ACTIVATION PRUNING: Against SAP, an adversary can estimate the expected loss with Monte Carlo sampling before applying FGSM.

4 EXPERIMENTS

Experiments evaluate SAP, dropout, adversarial training, reinforcement learning, and other stochastic defenses under adversarial perturbations. SAP improves robustness while generally preserving calibration, whereas dropout and several alternative stochastic strategies provide little or no robustness.

  • Experimental setup: SAP is evaluated on pretrained neural networks for image classification and reinforcement learning, using stochastic forward passes and Monte Carlo gradient estimates for adversarial examples.The image-classification evaluation measures accuracy and calibration; the reinforcement-learning evaluation measures average game rewards across perturbation strengths.
  • Image classification: 89.8% is the dense model’s clean CIFAR-10 accuracy, but accuracy falls to 66.3% at λ = 1 and 56.4% at λ = 2 under small perturbations.These perturbations are described as hardly perceptible in the input images.
  • Image classification: 12.2%, 16.3%, and 12.8% are SAP-100’s absolute accuracy increases at λ = 1, 2, and 4, respectively, while clean accuracy decreases by 6.5%.At λ = 16, SAP-100 instead shows a 5.2% absolute accuracy decrease.
  • Adversarial training and calibration: ADV+SAP-100 performs best at larger λ values and is closest to linear calibration there, while the other evaluated models are well calibrated for λ ≤ 4.For smaller λ values, SAP-100 achieves high accuracy; the dense model is not linear-calibrated for any λ ≠ 0.
  • Reinforcement learning: SAP-100 increases rewards for nonzero perturbations in nearly all Atari cases, including a 3425.9% relative increase for λ = 1 in Bowling, but decreases performance without perturbation.The exception is λ = 1 in BattleZone.
  • Additional baselines: Alternative Gaussian-noise strategies generally perform worse than the dense baseline, while dropout does not provide additional robustness against adversarial examples.Increasing stochastic weight noise can also reduce nonzero-λ accuracy, and higher sparsity can rapidly reduce clean accuracy.

5 RELATED WORK

Prior work has focused on generating adversarial examples and training models on them, including ensemble-based approaches.

  • FGSM generates adversarial examples by taking one input update in the sign direction of the loss gradient.
  • Iterative FGSM methods use smaller steps to better approximate the gradient when constructing attacks.
  • Adversarial training injects generated examples into training, using the target model alone or ensembles of pretrained models.

6 CONCLUSION

The paper concludes that SAP protects networks against adversarial examples without additional training, while improving accuracy and calibration in image classification. Combining SAP with adversarial training provides additive benefits, and SAP also shows effectiveness in reinforcement learning.

  • SAP guards networks against adversarial examples without requiring additional training.
  • Applying SAP to image classifiers improves both accuracy and calibration in adversarial settings.
  • Combining SAP with adversarial training yields additive benefits.
  • Additional experiments show that SAP can also be effective against adversarial examples in reinforcement learning.

A REINFORCEMENT LEARNING MODEL ARCHITECTURE

The reinforcement-learning model uses a convolutional architecture that processes four grayscale 84 × 84 observations and produces action Q-values. Training uses RMSProp, replay minibatches, periodic target updates, and 100M steps per game.

  • Training uses RMSProp with minibatches of 32, learning rate 0.00025, momentum 0.95, and discount factor γ = 0.99.
  • Agents are trained for 100M steps per game using replay from the 1M most recent transitions and target updates every 10000 steps.
  • The input is a 4 × 84 × 84 tensor containing rescaled grayscale versions of the last four observations.
  • Three convolutional layers use 32, 64, and 64 filters, followed by fully connected layers of size 512 and an action-value output.

B OTHER METHODS

The paper evaluates multiple stochasticity methods added to pretrained dense models on CIFAR-10 against adversarial examples.

  • The experiments test a variety of methods added to pretrained models against adversarial examples.
  • The CIFAR-10 experiments continue the evaluation using the dense model.

B.1 RANDOM NOISY WEIGHTS (RNW)

Random noisy weights add Gaussian noise to every weight, but the resulting models remain similar to the dense model and do not improve adversarial robustness.

  • Gaussian noise with mean 0 and constant standard deviation s is added independently to each weight tensor entry.
  • The noisy-weight models behave very similarly to the dense model.
  • No significant robustness improvement against adversarial examples is observed across the tested noise levels.
  • As s increases, accuracy decreases for non-zero λ.

B.2 RANDOMLY SCALED WEIGHTS (RSW)

Randomly scaled weights multiply each weight by Gaussian noise centered at one, but this approach remains similar to the dense model and provides no adversarial robustness.

  • Each weight is multiplied by a Gaussian scale factor with mean 1 and standard deviation s.
  • Randomly scaled-weight models perform similarly to the dense model.
  • The method provides no robustness against adversarial examples.
  • The models follow a trend similar to the random noisy-weight models.

B.3 DETERMINISTIC WEIGHT PRUNING (DWP)

Deterministic pruning retains only the largest-magnitude weights, while subsequent stochastic pruning and activation perturbation variants are evaluated as alternative defenses. Across these tests, the alternatives largely remain similar to dense models or lose accuracy without added robustness.

  • B.3 DETERMINISTIC WEIGHT PRUNING (DWP): Deterministic weight pruning keeps the top k% entries by absolute value and sets the remaining weights to zero.
  • B.3 DETERMINISTIC WEIGHT PRUNING (DWP): At low pruning levels, deterministic pruning behaves similarly to the dense model even against adversarial examples.
  • B.3 DETERMINISTIC WEIGHT PRUNING (DWP): At higher sparsity, accuracy in the no-perturbation case drops quickly.
  • B.4 STOCHASTIC WEIGHT PRUNING (SWP): Stochastic weight pruning samples weight-tensor entries with replacement, using k% of the entries as the number of samples drawn.
  • B.4 STOCHASTIC WEIGHT PRUNING (SWP): Stochastic weight-pruning models show no evident adversarial robustness and remain similar to the dense model across the tested sample percentages.
  • B.5 RANDOM NOISY ACTIVATIONS (RNA) / B.6 RANDOMLY SCALED ACTIVATIONS (RSA): Random Gaussian noise or scaling applied directly to activations also fails to provide robustness against adversarial examples.
Loading 1803.01442v1…