PaperlayerView exact revision on arXiv

Source-linked AI summary

Batch Normalization Increases Adversarial Vulnerability and Decreases Adversarial Transferability: A Non-Robust Feature Perspective

Philipp Benz, Chaoning Zhang, In So Kweon

arXiv:2010.03316v2cs.LGcs.CVstat.ML

TL;DR

The paper asks whether BN’s accuracy gains favor robust or non-robust features, a question left open despite its effects on adversarial robustness. It proposes a framework to separate feature usefulness from robustness and finds evidence that normalization shifts models toward NRFs, with implications for transferable attacks.

  • Problem

    Although BN improves accuracy while reducing adversarial robustness, it remains unclear whether it favors learning robust features or non-robust features.

    Source: §1 · p. 1

  • Method

    The paper disentangles feature usefulness from feature robustness and uses Local Input Gradient Similarity as an indicator of feature robustness.

    Source: §1 · p. 2

  • Results

    BN and other normalization variants increase adversarial vulnerability, with evidence that BN shifts models toward NRFs and that DNNs learn RFs before NRFs.

    Source: §8 · p. 8

  • Takeaways & Limitations

    A substitute model without BN and early stopping its training can boost transferable attacks.

    Source: §1 · p. 2

    The authors do not claim that reducing ICS is definitively the reason BN favors NRFs and welcome further evidence.

    Source: §6 · pp. 6–7

Abstract

from arXiv · show

Batch normalization (BN) has been widely used in modern deep neural networks (DNNs) due to improved convergence. BN is observed to increase the model accuracy while at the cost of adversarial robustness. There is an increasing interest in the ML community to understand the impact of BN on DNNs, especially related to the model robustness. This work attempts to understand the impact of BN on DNNs from a non-robust feature perspective. Straightforwardly, the improved accuracy can be attributed to the better utilization of useful features. It remains unclear whether BN mainly favors learning robust features (RFs) or non-robust features (NRFs). Our work presents empirical evidence that supports that BN shifts a model towards being more dependent on NRFs. To facilitate the analysis of such a feature robustness shift, we propose a framework for disentangling robust usefulness into robustness and usefulness. Extensive analysis under the proposed framework yields valuable insight on the DNN behavior regarding robustness, e.g. DNNs first mainly learn RFs and then NRFs. The insight that RFs transfer better than NRFs, further inspires simple techniques to strengthen transfer-based black-box attacks.

1. Introduction

The paper examines BN’s effects on adversarial robustness, asking whether its improved optimization and accuracy favor robust or non-robust features. It proposes a way to separate feature usefulness from robustness and presents evidence that BN shifts models toward NRFs.

1. Introduction

  • BN improves clean accuracy but lowers robust accuracy, and the authors’ evidence supports a shift toward greater reliance on NRFs.The shift is also supported by analyses of corruption robustness and feature transferability.Source: §1 · pp. 1–2
  • The proposed framework separates feature usefulness from feature robustness, using Local Input Gradient Similarity (LIGS) as an indicator of feature robustness.Clean accuracy and robust accuracy measure usefulness and robust usefulness, respectively; the framework treats feature robustness as orthogonal to usefulness.Source: §1 · p. 2
  • Analysis finds that models learn RFs before NRFs, and may partly forget RFs as NRF-aligned accuracy rises past RF-aligned accuracy.The introduction also reports that BN is crucial for learning NRFs, while most other investigated network structures and optimization factors do not significantly influence feature robustness as indicated by LIGS.Source: §1 · p. 2
  • A substitute model without BN outperforms its BN counterpart for transferable attacks, and early stopping the substitute’s training can also boost transferability.The paper presents these as practical applications of its finding that RFs transfer better than NRFs.Source: §1 · p. 2

2. Related Work

Prior work has studied adversarial vulnerability, feature robustness, and transferability, while this paper adds a feature-robustness perspective to that literature. It also situates BN among alternative normalization methods and prior work on their robustness effects.

Adversarial Vulnerability and Transferability

  • Prior work attributes adversarial vulnerability to several factors, including highly predictive but brittle NRFs, and measures feature usefulness and robust usefulness with clean and robust accuracy.The paper adds feature robustness as a dimension orthogonal to feature usefulness and relates its findings to adversarial-example transferability.Source: §2 · p. 2

Batch normalization and beyond

  • Table 2 compares accuracy and robustness across normalization techniques.The supplied caption identifies accuracy on the left and robustness on the right but provides no result values.Source: Table 2 · p. 3
  • LN and IN use channel-dimension normalization, and GN includes them as special cases; the paper reports that LN, IN, and GN mirror BN’s trend of increased adversarial vulnerability.The cited literature also examines batch dependence, generalization, adversarial training, and robustness to common corruptions.Source: §2 · pp. 2–3

3. RFs vs. NRFs: Which Side does BN Favor?

The section frames BN’s effect through the distinction between robust and non-robust features, then tests whether normalization variants show related robustness patterns. Its reported results connect normalization with greater NRF reliance, while describing corruption-specific differences and a possible role for style information.

3.1. Background and Motivation

  • A feature is ρ-useful when its expected label-feature correlation is at least ρ; RFs remain useful under specified valid perturbations, whereas NRFs lack that robustness.The authors conjecture that BN shifts models toward greater reliance on NRFs.Source: §3.1 · p. 3

3.2. Empirical evidence

  • IN, LN, and GN also show increased adversarial vulnerability; IN generally has the lowest robust accuracy, except for ResNet50 on CIFAR10.The authors suggest IN’s removal of style information may contribute, since style information is likely to be robust features.Source: §3.2 · pp. 3–4
  • Adversarial training improves robustness to noise corruptions but reduces robustness to contrast and fog, a pattern confirmed in Figure 2.The authors’ feature-based explanation associates noise corruptions with NRFs and contrast and fog with RFs.Source: §3.2 · p. 4, Figure 2 · p. 4
  • Models without normalization are more robust to noise corruptions, while models with normalization are more robust to fog and contrast; the authors interpret this contrast as evidence of greater NRF reliance with normalization.Figure 2 also shows IN’s especially high robustness to contrast, which the authors relate to IN’s generally low robustness.Source: §3.2 · p. 4, Figure 2 · p. 4

4. Framework for disentangling usefulness and robustness

The framework separates a feature set’s usefulness from its robustness, using clean accuracy and robust accuracy to represent these dimensions. It measures feature robustness through local linearity, quantified by Local Input Gradient Similarity (LIGS).

  • Feature robustness is defined by local linearity: similarity between input-loss gradients at an original and a locally perturbed input.The paper adopts cosine similarity for this measure and calls it Local Input Gradient Similarity (LIGS).Source: §4 · p. 4
  • LIGS is one possible robustness metric; the authors note that other metrics may also be appropriate.Source: §4 · p. 4
  • The framework treats feature usefulness and feature robustness as distinct dimensions, with robust usefulness depending on both.The paper informally relates usefulness to the amount of useful features and robustness to the ratio of robust to non-robust features.Source: §4 · p. 5
  • Clean accuracy, robust accuracy, and LIGS measure feature usefulness, robust usefulness, and feature robustness, respectively.Source: §4 · p. 5

5. Disentangling usefulness and robustness of model features

The analyses use LIGS to separate feature robustness from usefulness and find that BN is associated with lower feature robustness under standard training. Experiments on disentangled and conflicting features further indicate that BN enables learning NRFs, while feature learning tends to progress from RFs to NRFs.

5. Disentangling usefulness and robustness of model features

  • With standard training, BN lowers LIGS throughout training and correspondingly lowers robust accuracy, while adversarial training keeps LIGS near 1.For standard training, LIGS decreases as training evolves; BN has limited influence on LIGS in adversarial training.Source: Figure 3 · p. 5, §5 · p. 5
  • Regularizing LIGS improves robust accuracy by a large margin while having only a small influence on clean accuracy.Source: Figure 4 · p. 5, §5 · p. 5

5.1. Training on a dataset of disentangled RFs and NRFs

  • Experiments hold other setup choices constant while varying the factor of interest, such as the presence of BN.Source: §5.1 · p. 5
  • On NRF-only bDrand, a model without BN fails to converge, whereas BN enables learning after about 25 epochs.Before convergence, LIGS rises; after convergence begins, it quickly falls, while LIGS on RF-rich bDR remains above 0.9.Source: Figure 5 · p. 6, §5.1 · p. 6
  • On the conflicting-feature dataset, BN-equipped models first learn RF-cued labels, then NRF-cued labels, whose accuracy eventually surpasses RF-aligned accuracy.RF-aligned accuracy peaks around 80% before sharply decreasing; without BN, the model learns RFs while ignoring NRFs.Source: Figure 6 · p. 6, §5.1 · p. 6

5.2. Exploration beyond (batch) normalization

  • Most tested architecture and optimization factors have little effect on LIGS, though weight decay and initial learning rate affect it differently across training stages.Source: §5.2 · p. 6

6. On the link between why BN favors NRFs and how BN helps optimization.

The paper links BN’s shift toward NRFs with the unresolved question of how BN improves optimization. Its exploratory evidence favors reduced internal covariate shift over optimization-landscape smoothing, but the authors do not present this as a settled explanation.

  • Why BN shifts models toward NRFs and how it improves optimization are described as closely related questions, while the optimization mechanism remains unresolved.Source: §6 · pp. 6–7
  • The authors report that greater training stability does not necessarily correspond to lower feature robustness, and suggest their evidence supports the reduced-ICS view.They contrast this with the smoothing-landscape view and report that normalization variants also reduce ICS and shift models toward NRFs.Source: §6 · p. 7
  • The authors explicitly leave the proposed optimization explanation open to future supporting or contradicting evidence.Source: §6 · p. 7

7. Implications of our findings for improving adversarial transferability

The paper uses the finding that RFs transfer better than NRFs to motivate two transfer-based attack techniques. BN-free substitute models and early stopping are reported to improve transferability, and the techniques are expected to complement existing methods.

  • The motivation is that RFs transfer better than NRFs, and adversarially trained models were preliminarily observed to transfer better to normal models.The paper notes that typical adversarial training requires more computation resources.Source: §7 · pp. 7–8
  • Substitute models without BN transfer significantly better than their BN-equipped counterparts across a range of architectures on ImageNet and CIFAR10.The paper reports results for multiple attack baselines, with full CIFAR10 results for DI-FGSM and TI-FGSM in the supplementary material.Source: §7 · p. 8, Table 3 · p. 8, Table 4 · p. 8
  • Early stopping improves transferability: performance rises sharply in early epochs and gradually declines in later epochs.This technique is motivated by the finding that models mainly learn RFs before NRFs.Source: §7 · p. 8, Figure 9 · p. 8
  • The transfer techniques are expected to complement existing transfer-based black-box attack methods.Source: §7 · p. 8

8. Conclusion

The paper argues that normalization increases adversarial vulnerability by shifting models toward non-robust features, and presents a framework and empirical analyses of that shift. It also relates feature robustness to frequency behavior and transferability.

8. Conclusion

  • Normalization increases adversarial vulnerability, and the authors attribute this to greater reliance on non-robust features; their framework finds models learn robust features before non-robust features.They report that batch normalization is crucial for learning non-robust features and that robust features are essential early in training.Source: §8 · p. 8
  • The authors report that robust features transfer better than non-robust features and use this insight to motivate techniques for strengthening transfer-based black-box attacks.They describe the work as the first to demonstrate that models learn robust features before non-robust features.Source: §8 · p. 8, A. Relation to frequency perspective · p. 11

A. Relation to frequency perspective

  • The authors connect their feature account to frequency analyses: noise corruptions mainly affect non-robust features and high-frequency components, while contrast and fog mainly affect robust features and low-frequency components.Their reported learning order from robust to non-robust features aligns with prior findings that models learn low-frequency before high-frequency components.Source: A. Relation to frequency perspective · p. 11

B.1. Setup for training models in Sec. 3.2

The supplementary sections document training and evaluation settings, feature-transfer analyses, and LIGS trends across normalization methods and ImageNet. Feature-transfer results favor features from models without normalization, while normalization consistently lowers LIGS in the reported settings.

B.1. Setup for training models in Sec. 3.2

  • CIFAR10 and SVHN models use the listed SGD training parameters, while the ResNet50 models train for 350 epochs with learning-rate drops at epochs 150 and 250.The ImageNet VGG models come from torchvision, and ImageNet ResNets use the training parameters in [17].Source: §B.1 · p. 11, Table 5 · p. 11

B.2. PGD attack for evaluating the adversarial robustness

  • Robust accuracy evaluations use l2-PGD for 10 iterations on 1,000 samples; the main Table 1 evaluations additionally describe l2 and l∞ PGD and CW attacks.For those Table 1 attacks, the stated perturbation magnitudes are 0.25 for l2 and 1/255 for l∞ on images in [0, 1].Source: §B.2 · p. 11, Table 6 · p. 11

B.3. LIGS metric

  • LIGS is measured by default using Gaussian noise with mean 0 and standard deviation 0.01; the perturbation comparison also uses uniform noise, FGSM, and l∞-PGD.The perturbation comparison reports a general decrease in LIGS during training, with values without BN consistently higher than with BN.Source: §B.3 · p. 11, Figure 10 · p. 11

C. Feature Transferability

  • The feature-transfer setup extracts features from standard and adversarially trained models as datasets for cross-evaluation on other models.The cited tables report separate evaluations for standard and adversarially trained source models.Source: C. Feature Transferability · pp. 11–12, Table 7 · p. 11, Table 8 · p. 12
  • Features extracted from robust models transfer better than those from standard models, while standard-model features without normalization transfer better than features from normalized models.The authors interpret this pattern as evidence that normalized models rely more on non-robust features; Table 7 and Table 8 provide the cross-evaluations.Source: Table 7 · p. 11, C. Feature Transferability · p. 12, Table 8 · p. 12

C.1. Extracting features as a dataset from a model

  • Feature extraction treats a model's penultimate-layer activations as its utilized features and optimizes to extract robust features for each sample.The procedure uses an l2-PGD optimization with 1,000 iterations and no perturbation-magnitude restriction.Source: §C.1 · p. 12

D. Influence of other normalization techniques on LIGS

E. Results on ImageNet with LIGS trend

  • For models trained on ImageNet, instance normalization (IN) yields the lowest accuracy and batch normalization (BN) the highest; BN, IN, LN, and GN consistently lead to lower LIGS.Source: E. Results on ImageNet with LIGS trend · p. 12

F. Description of bDR / bDNR / bDrand / bDdet

The section describes feature datasets used to compare learning from robust and non-robust features, then reports their LIGS trends. The bDdet experiment resembles bDConflict, which the authors use to avoid influence from non-robust features in D.

F. Description of bDR / bDNR / bDrand / bDdet

  • bDR contains mainly robust features relevant to a robust model, while bDNR contains features from a standard model and includes both robust and non-robust features.bDrand and bDdet contain useful non-robust features represented through adversarial examples; bDrand uses randomly chosen target classes, while bDdet uses an offset target class.Source: F. Description of bDR / bDNR / bDrand… · p. 12
  • The bDR-trained model has relatively high LIGS, the bDrand-trained model relatively low LIGS, and bDNR falls between them.The authors note that the middle position is expected because bDNR contains robust and non-robust features; Figure 13 presents the comparison.Source: F. Description of bDR / bDNR / bDrand… · p. 13, Figure 13 · p. 13
  • The bDdet results resemble bDConflict, which the authors use to avoid influence from non-robust features in D.Figure 14 presents the bDdet result.Source: F. Description of bDR / bDNR / bDrand… · p. 13, Figure 14 · p. 13

G. Evaluating adversarial robustness with FGSM attack

The FGSM comparison reports higher adversarial robustness for models with BN than without BN, contrary to the broader vulnerability pattern under stronger attacks. The authors relate this result to FGSM's dependence on local linearity.

G. Evaluating adversarial robustness with FGSM attack

  • Under FGSM, models with BN have higher adversarial robustness than models without BN.The authors report lower LIGS with BN and conjecture that FGSM is less effective on BN models because FGSM's success depends strongly on linearity; Table 10 compares robust accuracy.Source: G. Evaluating adversarial robustness… · p. 13, Table 10 · p. 13

H. Additional Transferability Results

The additional results compare transferability across adversarially trained and normalized models, and show that gradient behavior varies with normalization and model setting.

H. Additional Transferability Results

I. Visualization of the optimization landscape

  • On ResNet18, only BN yields a more predictive and stable gradient; on ResNet50, all four normalization methods stabilize gradients, with IN, LN, and GN effects smaller than BN's.Figures 16–17 plot loss variation, l2 gradient change, and β-smoothness for ResNet18 and VGG16, respectively.Source: §I · p. 14, Figure 16 · p. 14, Figure 17 · p. 15

H. Additional Transferability Results

I. Visualization of the optimization landscape

  • The shortcut has a trivial influence on gradient stability, while FixupIni yields a less stable gradient than BN.Figure 18 shows the ResNet18 shortcut comparison; Figure 19 compares BN and FixupIni on ResNet20 and ResNet56.Source: §I · p. 14, Figure 18 · p. 15, Figure 19 · p. 15

Related papers

Loading 2010.03316v2…