PaperlayerView exact revision on arXiv

Source-linked AI summary

Training Robust Deep Neural Networks via Adversarial Noise Propagation

Aishan Liu, Xianglong Liu, Chongzhi Zhang, Hang Yu, Qiang Liu, Dacheng Tao

arXiv:1909.09034v2cs.LGcs.CV

TL;DR

Deep networks can be misled by adversarial examples and corruption, while many defenses remain vulnerable to generalized noise. ANP injects adversarial noise into hidden layers during training. Experiments report strong robustness across adversarial and corrupted inputs, and the framework can be combined with other adversarial training methods.

  • Problem

    Many adversarial defenses remain vulnerable to generalized noise, while noise in hidden representations can affect predictions.

    Source: §I · p. 1

  • Method

    ANP injects adversarial noise into hidden layers in a layer-wise manner using backward-forward training.

    Source: §I · p. 2, §VI · p. 11

  • Results

    Across visual-classification experiments, ANP provides strong robustness against varied noise and compares favorably with common defenses, including on adversarial and corruption benchmarks.

    Source: §I · p. 2, §VI · p. 11

  • Takeaways & Limitations

    Experiments indicate that ANP can further improve model robustness when combined with other adversarial training methods.

    Source: §VI · p. 11

    The reasons that shallow-layer noise improves robustness while deep-layer perturbation has the opposite effect remain unclear.

    Source: §VI · p. 12

Abstract

from arXiv · show

In practice, deep neural networks have been found to be vulnerable to various types of noise, such as adversarial examples and corruption. Various adversarial defense methods have accordingly been developed to improve adversarial robustness for deep models. However, simply training on data mixed with adversarial examples, most of these models still fail to defend against the generalized types of noise. Motivated by the fact that hidden layers play a highly important role in maintaining a robust model, this paper proposes a simple yet powerful training algorithm, named \emph{Adversarial Noise Propagation} (ANP), which injects noise into the hidden layers in a layer-wise manner. ANP can be implemented efficiently by exploiting the nature of the backward-forward training style. Through thorough investigations, we determine that different hidden layers make different contributions to model robustness and clean accuracy, while shallow layers are comparatively more critical than deep layers. Moreover, our framework can be easily combined with other adversarial training methods to further improve model robustness by exploiting the potential of hidden layers. Extensive experiments on MNIST, CIFAR-10, CIFAR-10-C, CIFAR-10-P, and ImageNet demonstrate that ANP enables the strong robustness for deep models against both adversarial and corrupted ones, and also significantly outperforms various adversarial defense methods.

I. INTRODUCTION

The introduction identifies a gap in defenses against generalized noise and proposes ANP, which injects adversarial noise into hidden layers. It presents ANP as a flexible framework that can complement existing adversarial training.

I. INTRODUCTION

  • Prior adversarial training augments data with adversarial examples, but adversarially trained models can remain vulnerable to iterative attacks.Other defenses seek non-computable gradients, while a perturbation-based regularizer was also proposed to improve adversarial robustness.Source: §I · p. 1
  • Research on corruption robustness includes stability training limited to JPEG compression and a benchmark covering 75 corruption types.The introduction also notes that corruption such as snow and blur occurs in real-world settings.Source: §I · p. 1
  • Most existing adversarial defenses remain vulnerable to generalized noise, in part because they add adversarial noise only to input data.The introduction motivates hidden representations by noting that invasive noise can cause sharp variations in intermediate feature maps.Source: §I · p. 1
  • ANP injects adversarial noise into hidden layers during training to target robustness against both adversarial examples and corruption.Unlike traditional methods that perturb inputs, it uses a modified backward-forward process without significant computational overhead.Source: §I · p. 2
  • Experiments across MNIST, CIFAR-10, ImageNet, CIFAR-10-C, and CIFAR-10-P report strong adversarial and corruption robustness, with ANP comparing favorably against common defenses.The authors also report that noise need only be injected into shallow layers and that combining ANP with other adversarial training methods can further improve robustness.Source: §I · p. 2

II. RELATED WORKS

  • ANP generalizes input-only adversarial training by considering hidden-layer contributions and more flexible forms of adversarial noise.The paper distinguishes its same-mini-batch noise computation from layer-wise adversarial training, which uses gradients from a previous mini-batch.Source: §II · p. 2

III. PRELIMINARIES

The preliminaries specify the visual-recognition setting and distinguish adversarial examples from image corruption. The two forms of robustness are measured under different conditions.

A. Terminology and Notation

  • The paper frames its setting as learning a classification function from feature vectors and labels for visual recognition.The model maps feature vectors in X to labels in Y.Source: A. Terminology and Notation · p. 2

B. Adversarial Example

  • An adversarial example is a nearby input that causes the model to predict a label other than the ground-truth label.The distance metric quantifies how small the semantic distance between the original and altered inputs is.Source: B. Adversarial Example · p. 2

C. Corruption

  • Corruption robustness measures average-case performance over corruption functions, whereas adversarial robustness measures worst-case performance under small, classifier-tailored perturbations.Examples of corruption include Gaussian noise, defocus blur, and brightness changes.Source: C. Corruption · pp. 2–3

IV. PROPOSED APPROACH

ANP seeks stable hidden representations by injecting adversarial noise into hidden states rather than perturbing only inputs. Its backward-forward procedure uses gradients from standard backpropagation to update and propagate layer-wise noise.

A. Adversarial Formulation

  • ANP introduces adversarial noise at hidden layers to improve layer-wise resistance to small noise and stabilize predictions.The formulation aims for consistent predictions on clean inputs and noisy surrogates in their neighborhoods.Source: A. Adversarial Formulation · p. 3
  • The formulation treats the network as composed of layers and adds a noise term to each hidden state before computing the next layer.The final output is denoted as the network output with injected noise at all layers, and parameters are learned by minimizing an adversarial loss subject to an ℓp constraint.Source: A. Adversarial Formulation · p. 3

B. Noise Propagation

  • ANP approximates the inner noise optimization with k gradient-descent steps per training mini-batch, using the standard backward-forward process.The adversarial-noise gradient equals the hidden-state gradient already computed in standard backpropagation, avoiding additional computational cost.Source: B. Noise Propagation · p. 3
  • The noise update combines the previous layer noise with its gradient, with η and ε controlling their respective contributions.ε is normalized by k so that the total update magnitude across k steps equals ε.Source: B. Noise Propagation · p. 3

1) Learning with a Noise Register:

The noise-register procedure stores adversarial noise for each hidden layer during the backward pass and uses it during the forward pass. At inference, the register can be discarded.

1) Learning with a Noise Register:

  • For each mini-batch, ANP stores the adversarial noise for each hidden layer during backward propagation, then retrieves and adds it during forward propagation.Noise is added after the affine transformation and before the activation function.Source: 1) Learning with a Noise Register · pp. 3–4, Figure 1 · p. 4
  • The noise register adds no substantial computation or memory consumption beyond storing layer-wise noise, and it can be discarded at inference.Discarding the register means it does not affect model complexity at inference.Source: 1) Learning with a Noise Register · p. 3
  • ANP trains on clean examples, computing and propagating adversarial noise for each layer instead of feeding adversarial examples as training inputs.The hidden-layer noise is added to the affine-transformed values before activation, then propagated to the next hidden layer.Source: 1) Learning with a Noise Register · p. 4

V. EXPERIMENTS AND EVALUATION

The experiments compare ANP with adversarial defenses on image-classification benchmarks, evaluating both adversarial attacks and corruption. The setup spans multiple datasets, architectures, and attack types.

V. EXPERIMENTS AND EVALUATION

  • The evaluation follows prior guidelines and uses FoolBox with attack and defense hyperparameters drawn from the relevant papers.Further experimental details are in the Supplementary Material.Source: §V · p. 4, A. Experimental Setup · p. 4

A. Experimental Setup

  • Adversarial robustness is assessed on MNIST, CIFAR-10, and a 200-class subset of ImageNet using LeNet, four CIFAR-10 architectures, and two ImageNet architectures.The ImageNet subset uses 100K training and 10K test images.Source: A. Experimental Setup · p. 4
  • The attack suite includes six ℓ∞-norm methods—FGSM, BIM, Step-LL, MI-FGSM, PGD, and BPDA—and the ℓ2-norm C&W attack.Source: A. Experimental Setup · p. 4
  • Corruption robustness is tested on CIFAR-10-C and CIFAR-10-P, benchmarks covering 75 corruption techniques and varying severity.Source: A. Experimental Setup · p. 4
  • The evaluation compares ANP with NAT, PAT, EAT, LAT, and Rand, including two methods ranked first and second in the NeurIPS 2017 defense competition.EAT and Rand were ranked No.1 and No.2, respectively.Source: A. Experimental Setup · p. 4

B. Evaluation Criteria

The paper uses accuracy measures for adversarial attacks and corruption-specific metrics for corrupted inputs and noisy sequences. The corruption measures are lower-is-better.

B. Evaluation Criteria

  • Black-box attack defense is measured by worst-case top-1 accuracy across attacks generated from hold-out models, while white-box defense uses top-1 accuracy against direct attacks.Higher values indicate better performance for both accuracy measures.Source: 1) Adversarial robustness evaluation · p. 4

2) Corruption robustness evaluation:

  • Corruption robustness is evaluated with mCE, Relative mCE, and mFR, with lower values indicating better performance.mCE compares corruption error with a baseline; Relative mCE measures the gap between mCE and clean-data error; mFR measures classification changes between adjacent frames.Source: 2) Corruption robustness evaluation · p. 5
  • mCE averages 15 corruption errors, each formed by comparing a classifier's error on a corruption type and severity with baseline-model errors.The clean dataset top-1 error is also computed as part of the described evaluation procedure.Source: 2) Corruption robustness evaluation · p. 5
  • mFR is the average flip rate over noise sequences, where a flip records a classification change between adjacent frames.Source: 2) Corruption robustness evaluation · p. 5

3) Model Structure Robustness:

The paper introduces two structural robustness measures and investigates how perturbing different hidden layers affects robustness. Across layer-group, pairwise, and single-layer experiments, shallow layers are more consequential, though their perturbation can also affect clean accuracy.

3) Model Structure Robustness:

  • Empirical Boundary Distance estimates minimum movement to a prediction-changing decision boundary across sampled orthogonal directions, with larger values indicating stronger robustness.Source: 3) Model Structure Robustness · p. 5
  • ε-Empirical Noise Insensitivity measures loss differences between clean and polluted examples under a noise constraint, with lower values indicating stronger robustness.Polluted examples may include adversarial attacks, Gaussian noise, and blur.Source: 3) Model Structure Robustness · p. 5

C. Is It Necessary to Inject Noise into All Layers?

  • Injecting noise into top-m layers improves adversarial accuracy and negative mCE for m ≤ 4, but adding noise to more layers does not continually increase robustness.The figure caption reports the best performance when noise is injected only into the top-4 layers.Source: Figure 2 · p. 5, C. Is It Necessary to Inject Noise… · p. 6
  • Perturbing distant layer pairs can outperform perturbing neighboring pairs; the strongest reported pairs are layers 1 and 4, or layers 7 and 13.The experiments vary pair intervals and use layers 1 and 7 as base layers in the two figure panels.Source: C. Is It Necessary to Inject Noise… · p. 6, Figure 3 · p. 6
  • The influence of individual Batch Normalization statistics reassignment on robustness diminishes with layer depth, as the performance gap becomes smaller.The reassignment uses statistics from a corresponding vanilla model at inference.Source: C. Is It Necessary to Inject Noise… · p. 6, Figure 4 · p. 6
  • When noise is injected into only one hidden layer, robustness decreases as the perturbed layer gets deeper.The reported robustness measures are adversarial-example accuracy and negative mCE for corruptions.Source: C. Is It Necessary to Inject Noise… · p. 6, Figure 4 · p. 6
  • Shallow layers are more critical to robustness than deeper layers, but their contribution by depth is not strictly linear and perturbing them has a stronger negative influence on clean accuracy.The theoretical discussion attributes stronger clean-output changes to shallow-layer perturbations.Source: C. Is It Necessary to Inject Noise… · p. 6

3) Theoretical analysis:

The analysis represents ANP as a feed-forward network with layerwise perturbations and compares its output with that of the unperturbed model. It concludes that shallow-layer perturbations can influence clean accuracy more strongly, while shallow layers are more critical to robustness.

3) Theoretical analysis:

  • ANP's layerwise perturbations enter the network at the input and successive hidden-layer activations, defining the perturbed model output for comparison with the vanilla network.Source: 3) Theoretical analysis · pp. 6–7
  • Shallow-layer perturbations can influence clean-example output scores more strongly than deeper-layer perturbations, while each layer's contribution is not strictly linear with depth.Source: 3) Theoretical analysis · p. 7
  • Injecting noise only into shallow layers is sufficient to achieve better model robustness, and the paper uses this finding to guide later experiments.Source: 3) Theoretical analysis · p. 7

D. Adversarial Robustness Evaluation

In black-box evaluation, ANP generally leads the compared defenses on MNIST, CIFAR-10, and ImageNet. It also maintains clean-example accuracy close to vanilla models on MNIST and CIFAR-10 while defending against black-box attacks.

1) Black-box setting:

  • Black-box attacks use adversarial examples generated on hold-out models different from the target models.The evaluation generates 10,000 adversarial images from these hold-out models.Source: 1) Black-box setting · p. 7
  • ANP achieves the best defense performance in almost all black-box cases and provides robustness against single-step and iterative attacks.The evaluation covers MNIST, CIFAR-10, and ImageNet.Source: 1) Black-box setting · p. 7
  • On CIFAR-10 and MNIST, ANP keeps clean-example classification accuracy close to that of vanilla models despite noise introduction.The passage contrasts this with typical accuracy degradation of 5–10% when more noise is introduced.Source: 1) Black-box setting · p. 7

2) White-box setting:

ANP shows a significant advantage over other methods in the reported white-box evaluations, with a caveat on ImageNet PGD attacks. On CIFAR-10-C and CIFAR-10-P, it achieves the lowest reported corruption metrics; the section also evaluates structural robustness using two proposed metrics.

2) White-box setting:

  • ANP significantly outperforms other methods in the reported white-box evaluations, although it is slightly weaker than PAT against PGD on ImageNet.The authors note that clean accuracy drops for all methods on ImageNet.Source: 2) White-box setting · p. 7

F. Model Structure Robustness Evaluation

  • The section evaluates structural robustness with Empirical Boundary Distance and ε-Empirical Noise Insensitivity.The supplied passage names these metrics but does not report structural-robustness results here.Source: F. Model Structure Robustness Evaluation · p. 8

E. Corruption Robustness Evaluation

  • On corruption robustness, ANP achieves the lowest mCE (average corruption error) and mFR (average flip rate) among all methods, surpassing compared strategies by large margins of almost 6 and 30, respectively.CIFAR-10-C uses 10K images with 15 different corruption levels and 5 severity levels; CIFAR-10-P uses noise sequences with more than 30 frames per image.Source: E. Corruption Robustness Evaluation · p. 7

1) Empirical Boundary Distance:

ANP combined with TRADES or PAT outperforms the corresponding standalone methods, while ANP models also show strong margins against label changes and low noise insensitivity. The results support combining hidden-layer perturbations with other adversarial-training objectives.

1) Empirical Boundary Distance:

G. Combination with Other Adversarial Defense Strategies

  • ANP+TRADES and ANP+PAT outperform TRADES and PAT, respectively, and ANP+TRADES achieves the most robust performance among the compared methods.The framework is presented as complementary to these adversarial-training methods.Source: G. Combination with Other Adversarial… · p. 10

H. What Did Hidden Layers Do during Noise Propagation?

The section examines ANP through hidden-representation insensitivity and gradient alignment, then concludes that shallow-layer perturbation can train robust models while hidden layers differ in their effects on clean accuracy.

H. What Did Hidden Layers Do during Noise Propagation?

VI. CONCLUSION

  • ANP injects diversified noise layer by layer and uses the standard backward-forward process without additional computations.Source: §VI · p. 11
  • Experiments on visual classification demonstrate that ANP enables strong robustness for deep networks against various types of noise.Source: §VI · p. 11
  • Empirical studies find that perturbing shallow layers is sufficient for robust models, while theoretical proof finds stronger negative effects from shallow than deep layers on clean accuracy.Source: §VI · p. 11
  • ANP is orthogonal to most adversarial training methods, and experiments report stronger models when the methods are combined.Source: §VI · p. 11
  • Current ANP uses equal noise amounts across layers and manual choices, while the reasons shallow-layer noise helps and deep-layer perturbation has the opposite effect remain unclear.Source: §VI · pp. 11–12
Loading 1909.09034v2…