PaperlayerView exact revision on arXiv

Source-linked AI summary

Batch Normalization is a Cause of Adversarial Vulnerability

Angus Galloway, Anna Golubeva, Thomas Tanay, Medhat Moussa, Graham W. Taylor

arXiv:1905.02161v2cs.LGstat.ML

TL;DR

Batch norm is widely used to ease training, but robustness to plausible unseen inputs is a concern and the factors that reduce it warrant understanding. The paper tests batch norm’s effect and finds robustness trade-offs; removing it while adding ℓ2 regularization eliminates an inherent relationship between vulnerability and input dimension.

  • Problem

    The paper asks whether batch norm itself reduces robustness, since adversarial examples expose unstable predictions and robustness to plausible unseen inputs matters for deployment.

    Source: §1 · p. 1

  • Method

    The authors test batch norm’s robustness effects empirically and examine input-dimension vulnerability after removing batch norm and adding ℓ2 regularization.

    Source: §1 · pp. 1–2

  • Results

    Batch norm’s accelerated training and occasional clean-accuracy gains come at the cost of robustness to additive noise and adversarial perturbations.

    Source: §6 · p. 8

  • Takeaways & Limitations

    The results highlight the importance of identifying the distinct mechanisms of regularization, particularly when robustness matters.

    Source: §6 · p. 8

    The noise and corruption methods demonstrate a robustness disparity but are not intended as a formal security evaluation.

    Source: §3 · p. 2

Abstract

from arXiv · show

Batch normalization (batch norm) is often used in an attempt to stabilize and accelerate training in deep neural networks. In many cases it indeed decreases the number of parameter updates required to achieve low training error. However, it also reduces robustness to small adversarial input perturbations and noise by double-digit percentages, as we show on five standard datasets. Furthermore, substituting weight decay for batch norm is sufficient to nullify the relationship between adversarial vulnerability and the input dimension. Our work is consistent with a mean-field analysis that found that batch norm causes exploding gradients.

1 Introduction

Batch norm is widely used and can ease training, yet robustness to plausible unseen inputs remains a concern. The paper tests whether batch norm itself reduces robustness and reports that removing it while adding ℓ2 regularization eliminates an inherent relationship between vulnerability and input dimension.

1 Introduction

  • Batch norm is a standard network component that often reduces sensitivity to hyperparameter choice, while deployment raises concerns about robustness to plausible unseen inputs.The section frames robustness as a concern distinct from ease of training.Source: §1 · p. 1
  • Adversarial examples expose unstable predictions across state-of-the-art models, motivating research into factors that reduce robustness.The authors frame understanding those factors as a prerequisite to developing robustness-improvement methods.Source: §1 · p. 1
  • Robustness-improvement approaches often retain batch norm and patch models against specific attacks, but the tested assumption that batch norm itself does not reduce robustness was invalid.The authors note that robustness responds differently to disparate regularization mechanisms.Source: §1 · p. 1
  • Prior work observed that adversarial vulnerability can scale with input dimension, making it important to identify regularizers beyond their effect on test accuracy.The introduction identifies batch norm as a confounding factor in that prior analysis.Source: §1 · p. 1
  • Removing batch norm and adding ℓ2 regularization shows no inherent relationship between adversarial vulnerability and input dimension.This extends initialization-time analysis results to after training.Source: §1 · p. 2

2 Batch Normalization

Batch norm centers and scales each unit’s pre-activation within a mini-batch, then applies a learned scale and shift before the nonlinearity. The section argues that this suppresses useful information and can impair robustness by disrupting representations and relative distances.

2 Batch Normalization

  • Batch norm subtracts each unit’s batch mean, divides by its standard deviation plus a small constant, and applies learned scale and shift before the nonlinearity.The notation identifies α as the neuron index, l as the layer, and i as a mini-batch of B samples.Source: §2 · p. 2
  • At initialization, batch norm fixes all neurons’ first and second moments equally, independent of network width or depth.The procedure suppresses information in those moments, which the learned scale and shift cannot recover.Source: §2 · p. 2
  • Examples differing by one item in a mini-batch receive different representations, and stacking batch-norm layers amplifies that difference.The section also argues that normalized representations impair distinctions between high- and low-quality examples.Source: §2 · p. 2
  • The authors argue that information loss and disrupted relative input distances reduce adversarial and general robustness.Their batch-normalized linear-network visualization shows degraded class-relevant input distances; they contrast this with unnormalized networks that preserve class membership at arbitrary depth.Source: §2 · p. 2

3 Empirical Result

Across the evaluated datasets and robustness tests, batch normalization often improved clean accuracy but reduced accuracy under adversarial perturbations or noise. The experiments also examine corruption types, depth and batch size, and representation compression.

CIFAR-10

  • On CIFAR-10, batch norm reduced VGG test accuracy by 6 ± 1% under noise and by 17.3 ± 0.7% and 5.9 ± 0.4% under PGD-ℓ∞ and PGD-ℓ2.It reduced the clean generalization gap by 1.1 ± 0.2%.Source: Table 2 · p. 3, §3 · p. 3

CIFAR-10.1

  • On CIFAR-10.1, batch norm reduced test accuracy by 6 ± 1% under additive noise and by 15 ± 1% and 3.4 ± 0.6% under PGD-ℓ∞ and PGD-ℓ2.It slightly improved clean test accuracy by 2.0 ± 0.3%.Source: Table 2 · p. 3, §3 · p. 3

CIFAR-10-C

  • Across all 19 CIFAR-10-C corruptions, batch norm increased mCE by 1.9 ± 0.9% for VGG8, 2.0 ± 0.3% for VGG13, and 1.6 ± 0.4% for WRN; noise-category error was higher for every tested model at every intensity.The reported noise-category gaps were especially large for several VGG13 corruptions, including 20.9 ± 1.4% for Gaussian noise and 13.6 ± 0.6% for Impulse noise.Source: Table 4 · p. 4, §3 · p. 4

ImageNet

  • On ImageNet, batch norm consistently reduced top-5 accuracy under PGD by 8.54% to 11.00% absolute, though it improved top-5 accuracy on noise in some models.The top-1 PGD degradation ranged from 2.38% to 4.17%.Source: Table 5 · p. 4, §3 · p. 5

Depth and batch size

  • For depths with reasonable test accuracy, robustness decreased as batch size increased with batch norm, while its maximum trainable depth rose with batch size and then plateaued; unnormalized networks showed no such clean-accuracy–robustness tension.Unnormalized-network perturbation robustness increased with depth.Source: Figure 2 · p. 5, §3 · pp. 5–6

CIFAR-10-C

  • For WRN, batch norm improved accuracy on Contrast, Snow, and Spatter corruptions, while the authors suggest its texture reliance may contribute to poorer performance under spatially global noise.The respective accuracy improvements were 4.9 ± 1.1%, 2.8 ± 0.4%, and 2.3 ± 0.8%.Source: §3 · p. 5

Representation information

  • In batch-normalized models, lower estimated I(X; T) coincided with higher clean accuracy, but estimates near 10 bits occurred around the roughly 40-layer trainable-depth limit and low I(X; T) also appeared where BIM robustness was poor.The authors relate the latter observation to possible overfitting through excessive compression, while noting that this intuition needs further analysis.Source: Figure 3 · p. 6, §3 · p. 6

4 Vulnerability and Input Dimension

The experiments find that ℓ2 weight decay can counter input-dimension scaling of adversarial vulnerability, while batch norm reduces robustness as image size grows. Weight decay also improves robustness to noisy and adversarial inputs, though its effects on clean accuracy depend on the setting.

4 Vulnerability and Input Dimension

  • ℓ2 weight decay can approximately recover independence between adversarial vulnerability and input dimension with little to no clean-accuracy trade-off, unlike PGD adversarial training’s modest trade-off.The regularization constant λ corrects the loss scaling as the input norm increases with d.Source: §4 · p. 7
  • With ℓ2 weight decay, adversarial test-accuracy ratios were 0.98 ± 0.01, 0.96 ± 0.04, and 1.00 ± 0.03 at √d values of 56, 84, and 112.The corresponding clean-accuracy ratios were 0.999 ± 0.002, 0.996 ± 0.003, and 0.987 ± 0.004, relative to the original d = 28 dataset.Source: §4 · p. 7
  • Batch norm reduced robustness as image size increased, leaving roughly 40% less BIM robustness and 13–16% less noise robustness at all tested sizes.For the original resolution, it reduced test accuracy by 16.4 ± 0.4% for noise and 43.8 ± 0.5% for BIM-ℓ∞.Source: §4 · p. 7
  • For d = 28, using batch normalization yields approximately 66% accuracy for BIM and induces a substantial MNIST test-error increase of 1.69 ± 0.01%.With the same regularization constant but without batch normalization, clean test accuracy increases by 1.39 ± 0.04%, and BIM-ℓ∞ accuracy by 21.7 ± 0.4%.Source: §4 · p. 7
  • Across the reported conditions, batch norm reduced test accuracy on noisy and adversarially perturbed inputs, whereas weight decay increased it.The paper also reports that reducing weight decay to λ = 0 with batch norm degraded accuracy for an ε∞ = 0.1 perturbation by 79.3 ± 0.3%.Source: §4 · p. 7

5 Related Work

Related work discusses batch norm’s training and optimization effects, including gradient explosion in deep networks. The authors connect those ideas to their findings on robustness and activation patterns, and describe weight decay’s complementary role.

5 Related Work

  • [32] prove that stacking batch-normalized layers causes gradient explosion exponential in depth, while shallow networks can show the “helpful smoothing” described in.The paper reports that a single batch-normalized layer suffices to induce severe adversarial vulnerability.Source: §5 · p. 8
  • With two batch-norm layers, activations start more spread out, which the authors associate with easier clustering and faster training, while clusters remain more stationary.In the unnormalized network, points overlap at initialization, then spread, cluster, and tighten over training; the activation visualizations cover 500 epochs.Source: §5 · p. 8, Figure 4 · p. 8
  • Weight decay’s loss-scaling mechanism complements its reported increase in effective learning rate [31] [33] and reduces the generalization gap even in batch-normalized networks.When batch norm is not applied to every layer, the loss-scaling mechanism persists, though to a lesser degree.Source: §5 · p. 8

6 Conclusion

The conclusion emphasizes a trade-off: batch norm’s training speed and occasional clean-accuracy gains come with reduced robustness. It also rejects an inherent link between input dimension and vulnerability and highlights the need to distinguish regularization mechanisms.

6 Conclusion

  • Batch norm’s accelerated training and occasional higher clean test accuracy come at a robustness cost for both additive noise and adversarial perturbations.Source: §6 · p. 8
  • The authors conclude that adversarial vulnerability has no inherent relationship with input dimension.Source: §6 · p. 8
  • The results highlight the importance of identifying the distinct mechanisms of regularization techniques when robustness is a concern.Source: §6 · p. 8

A Supplement to Empirical Results

This supplement adds explanations and results for the empirical section, including the rationale for using VGG and a comparison of PGD with BIM.

A Supplement to Empirical Results

A.1 Why the VGG Architecture?

  • VGG was chosen for SVHN and CIFAR-10 because it is a contemporary architecture developed independently of batch norm, supporting an intervention focused on batch norm itself.The authors note that state-of-the-art architectures more closely tied to batch norm may be less suitable, and describe the architecture choice as otherwise arbitrary.Source: §A.1 · p. 11

A.2 Comparison of PGD to BIM

  • For CIFAR-10 VGG8, the batch-norm accuracy gap was 17 ± 1% under both BIM and PGD, despite lower test accuracy under PGD for each model.The comparison used 20 iterations; increasing VGG16 PGD iterations from 20 to 40 changed accuracy from 28.9 ± 0.2% to 28.5 ± 0.3%.Source: §A.2 · p. 11

A.3 Additional SVHN and CIFAR-10 Results for Deeper Models

On CIFAR-10 variants, reducing batch-normalization layers with Fixup improved accuracy under noise and adversarial attacks. Deeper SVHN VGG models were only trainable with batch normalization, but showed limited PGD-ℓ2 robustness gains.

A.3 Additional SVHN and CIFAR-10 Results for Deeper Models

  • Deeper SVHN VGG models were reportable only with batch norm after attempts beyond 8 layers failed; they gained little PGD-ℓ2 robustness but did better on PGD-ℓ∞.The deeper batch-normalized models did not obtain much better PGD-ℓ2 robustness.Source: §A.3 · p. 11, Table 8 · p. 11
  • Fixup reduces, but does not eliminate, batch normalization: its WideResNet retains one batch norm layer before classification.The comparison uses a 28-layer WideResNet with width factor 10 against the default batch-normalized architecture.Source: §A.3 · pp. 11–12
  • Fixup improves accuracy by 11.6 ± 1.9% on noisy CIFAR-10, 9.5 ± 1.5% on noisy CIFAR-10.1, 5.4 ± 0.6% on PGD-ℓ∞, and 1.1 ± 0.4% on PGD-ℓ2.Table 9 reports accuracies for WideResNet–28–10 on CIFAR-10 and CIFAR-10.1.Source: Table 9 · p. 12, §A.3 · p. 12
  • VGG8 reaches 78.9±0.6 under noise, versus 69.1±1.1 for WideResNet variants on noisy CIFAR-10; the WideResNets show little generalization gap between noisy CIFAR-10 and CIFAR-10.1.The reported noisy CIFAR-10.1 accuracies are 69.1±1.1 and 57.6±1.5 for the two WideResNet variants, compared with 67.8±0.9 and 58.3±1.2 on noisy CIFAR-10.Source: §A.3 · p. 12, Table 9 · p. 12

A.4 ImageNet Black-box Transferability Analysis

The ImageNet transfer analysis compares VGG variants with and without batch normalization under BIM-ℓ∞ attacks. Accuracy reductions were larger when unnormalized models generated attacks against unnormalized targets, and pretrained model records lacked training details.

A.4 ImageNet Black-box Transferability Analysis

  • For unnormalized targets, unnormalized sources cut top-1 accuracy by 16.5%–20.4%, versus 2.1%–4.9% for batch-normalized sources.Table 10 compares ImageNet validation accuracy for BIM-ℓ∞ examples transferred between VGG variants of different depths and normalization settings.Source: Table 10 · p. 12, §A.4 · p. 12
  • When the target is unnormalized, top-1 accuracy is 10.5%–16.4% higher and top-5 accuracy is 5.3%–7.5% higher if the source is normalized rather than unnormalized.The authors state this is consistent with their white-box results and not evidence of gradient masking.Source: §A.4 · p. 12
  • The authors suggest unnormalized networks use more stable features than batch-normalized networks.This interpretation follows the reported transfer-accuracy comparisons.Source: §A.4 · p. 12
  • The pretrained ImageNet models lacked hyperparameter settings and other training details, limiting what can be established about their training configurations.The authors state they believe the results are not sensitive to hyperparameters.Source: §A.4 · p. 13

A.5 Batch Norm Limits Maximum Trainable Depth and Robustness

Batch normalization limits the maximum trainable depth and, at depths that retain test accuracy, robustness declines as batch size increases. Unnormalized networks can attain good clean accuracy and robustness together after longer training.

A.5 Batch Norm Limits Maximum Trainable Depth and Robustness

  • Batch norm limits maximum trainable depth, and robustness decreases with batch size at depths that maintain test accuracy, around 25 or fewer layers.In unnormalized networks, clean accuracy and robustness show little to no relationship with depth or batch size.Source: §A.5 · p. 13
  • Some unnormalized-network outliers at large depths and batch sizes may reflect fewer parameter updates when batch size rises but epochs remain fixed.The authors present this as a possible explanation for the outliers.Source: §A.5 · p. 13
  • Unnormalized networks can achieve good robustness alongside good clean test accuracy after training for 40 epochs, while these regimes mostly do not overlap with batch norm.They take longer to converge, and more configurations attain higher clean test accuracy than batch-normalized networks after the longer training.Source: §A.5 · p. 13

B Weight Decay and Input Dimension

The section examines how input dimension affects adversarial vulnerability and how ℓ2 weight decay can correct loss scaling. Its reported results show dimension-invariant vulnerability with weight decay, while the proposed scaling account connects the loss to distance from the classification boundary.

B Weight Decay and Input Dimension

  • The model's score is a scaled signed distance to its classification boundary, and adding ℓ2 regularization controls loss scaling.The section defines the scaled loss as ζ(∥w∥2 × z) and describes regularization as controlling the scaling.Source: B Weight Decay and Input Dimension · p. 14
  • Figures 6(a)–6(c) illustrate quantities in the binary-classification loss, while Figures 6(d)–6(f) depict how λ changes loss scaling.For large regularization, all data points are penalized almost linearly.Source: B Weight Decay and Input Dimension · p. 14, Figure 6 · p. 14
  • In the single-linear-layer MNIST experiment, increasing input dimension increased adversarial vulnerability in accuracy and loss; predicted adversarial damage fell between the empirical ϵ = 0.05 and ϵ = 0.1 results except at width 112.At width 112, the observed damage was slightly greater than anticipated.Source: B Weight Decay and Input Dimension · pp. 14–15
  • At d = 112 and ϵ = 0.1, the accuracy ratio is 1.00 ± 0.03 with weight decay, compared to 0.10 ± 0.09 without.The authors choose λ dependent on input dimension to correct the loss scaling; Table 11 describes ratios for d > 28 relative to the d = 28 baseline.Source: B Weight Decay and Input Dimension · pp. 14–15, Table 11 · p. 15
  • Compared with PGD training, weight decay avoids an arbitrary ϵ hyperparameter, inner-loop training overhead, and attack-specificity; on 100 MNIST invariance examples, it achieves 86% correct versus 74% for batch norm.The authors caution that robustness to ϵ-perturbations may increase vulnerability to invariance-based examples, so traditional ℓp defenses are not the end goal.Source: B Weight Decay and Input Dimension · p. 15
  • Weight decay reduces the generalization gap even in batch-normalized networks; its loss-scaling effect persists, though to a lesser degree, when the last layer lacks batch norm.The passage also describes this scaling mechanism as complementary to other reported effects of weight decay.Source: B Weight Decay and Input Dimension · p. 15

C Adversarial Spheres

On the 2D Adversarial Spheres task, batch normalization was associated with reduced robustness and greater learning-rate sensitivity. Its representations also showed sample overlap and class mixing across network layers.

C Adversarial Spheres

  • 61 ± 1% degradation in absolute accuracy with batch norm compared to the baseline at d = 84.Source: C Adversarial Spheres · p. 16
  • In the batch-normalized representation, some samples overlapped by Layer 2 and classes were mixed by Layer 14.Source: Figure 7 · p. 16
  • At η = 0.01, the batch-normalized network was untrainable, while the unnormalized network trained at learning rates up to 0.1.Source: C Adversarial Spheres · p. 16
  • With input noise, the batch-normalized model classified 94.83% of test points correctly, versus 96.06% for the unnormalized model.Only models trained to 100% training accuracy at η = 0.001 were evaluated.Source: C Adversarial Spheres · p. 16
Loading 1905.02161v2…