PaperlayerView exact revision on arXiv

Source-linked AI summary

Excessive Invariance Causes Adversarial Vulnerability

Jörn-Henrik Jacobsen, Jens Behrmann, Richard Zemel, Matthias Bethge

arXiv:1811.00401v4cs.LGcs.AIcs.CVstat.ML

TL;DR

Deep networks can fail under distribution shift because they are excessively invariant to task-relevant variations, not merely sensitive to small perturbations. The paper analyzes this failure with invertible networks and an information-theoretic view of cross-entropy, then proposes an objective that encourages representations to capture task-dependent variation. Across tasks and architectures, it finds excessive invariance and reports that the extended objective can reduce unwanted invariances.

  • Problem

    Deep networks show adversarial and out-of-distribution failures, but research has focused mainly on small perturbations while unrestricted adversarial examples remain difficult to formalize.

  • Method

    The paper uses fully invertible networks to expose invariant adversarial subspaces and extends cross-entropy with information-theoretic components separating semantic from nuisance features.

  • Results

    Excessive invariance appears across tasks and architectures, while the extended objective empirically removes unwanted invariances in targeted distribution-shift experiments.

  • Takeaways & Limitations

    Adversarial vulnerability can reflect insufficient representation of semantically meaningful variation, so controlling invariance provides a tractable route for studying and reducing this vulnerability.

  • Takeaways & Limitations

    The distribution-shift analysis assumes nuisance variables do not become more informative about labels and excludes synergistic interactions between semantic and nuisance variables.

Abstract

from arXiv · show

Despite their impressive performance, deep neural networks exhibit striking failures on out-of-distribution inputs. One core idea of adversarial example research is to reveal neural network errors under such distribution shifts. We decompose these errors into two complementary sources: sensitivity and invariance. We show deep networks are not only too sensitive to task-irrelevant changes of their input, as is well-known from epsilon-adversarial examples, but are also too invariant to a wide range of task-relevant changes, thus making vast regions in input space vulnerable to adversarial attacks. We show such excessive invariance occurs across various tasks and architecture types. On MNIST and ImageNet one can manipulate the class-specific content of almost any image without changing the hidden activations. We identify an insufficiency of the standard cross-entropy loss as a reason for these failures. Further, we extend this objective based on an information-theoretic analysis so it encourages the model to consider all task-dependent features in its decision. This provides the first approach tailored explicitly to overcome excessive invariance and resulting vulnerabilities.

1 INTRODUCTION

The paper reframes adversarial vulnerability as excessive invariance to task-relevant changes, complementing the usual focus on sensitivity to small perturbations. It argues that narrow learning and cross-entropy's information-theoretic limitations leave networks insensitive to much class-specific content, motivating an alternative objective.

  • Alternative viewpoint: The paper treats excessive invariance as a complementary source of adversarial vulnerability alongside sensitivity to small perturbations.It introduces invariance-based adversarial examples, in which class-specific content can change without changing network activations.
  • Alternative viewpoint: Almost any input can have its class-specific content changed arbitrarily without changing network activations.The paper illustrates this phenomenon on ImageNet and uses it to analyze unrestricted adversarial vulnerability.
  • Motivation: The paper links vulnerability to narrow learning, in which classifiers rely on only a few highly predictive features rather than all task-dependent information.This perspective is connected to networks' reliance on spectral or stationary statistics instead of more abstract features such as shape and appearance.
  • Motivation: Cross-entropy maximizes a bound on mutual information between labels and representations but gives no incentive to explain all class-dependent input aspects.The paper identifies this limitation as a major reason for excessive invariance.
  • Contributions: The paper reports invariance-based adversarial examples across various tasks and deep-network architectures.This is presented as evidence that excessive invariance is not specific to one task or architecture family.
  • Contributions: It proposes an invertible architecture and an alternative objective that explicitly addresses excessive invariance.The architecture enables class-specific image manipulations while leaving classifier-visible representation dimensions invariant, and the objective is designed to reduce this behavior.

2 TWO COMPLEMENTARY APPROACHES TO ADVERSARIAL EXAMPLES

The paper frames adversarial vulnerability as arising from two complementary failure modes: excessive sensitivity to task-irrelevant perturbations and excessive invariance to task-relevant changes. It analyzes the latter using invertible networks and metameric sampling, showing that image content can change substantially while classifier representations remain fixed.

  • Complementary failure modes: Adversarial examples arise either by crossing a classifier’s decision boundary through perturbations or by moving within its pre-image to a misclassified region.These viewpoints differ in their reference points but describe complementary routes to classifier failure.
  • Pre-images and invariance: Non-trivial pre-images arise when network layers are non-injective, such as through subsampling or ReLU, accumulating invariance that can become problematic.A subnetwork is invariant to perturbations that remain within the same pre-image.
  • Complementary failure modes: ϵ-bounded attacks expose excessive sensitivity to task-irrelevant changes, whereas invariance-based attacks expose excessive insensitivity to task-relevant changes.Invariance-based examples preserve the network output while changing the oracle-assigned class.
  • Invertible analysis: Fully invertible RevNets separate logits zs from nuisance variables zn, providing explicit access to discarded variability and a simplified decision-space readout.The architecture is bijective, while only the final projection removes nuisance variables from classification.
  • Invertible analysis: Metameric sampling forms xmet = F^-1(zs, ˜zn) by combining logits and nuisance variables from different inputs, enabling analytic inspection of semantic–nuisance dependencies.The method avoids expensive approximate optimization and produces samples that are visually close to natural ImageNet images.
  • Empirical vulnerability: On MNIST and ImageNet, nuisance variables can dominate visual appearance while the complete classifier feature vector remains unchanged.The result allows arbitrary semantic content to be attached to a fixed logit activation pattern.

3 OVERCOMING INSUFFICIENCY OF CROSSENTROPY-BASED INFORMATION-MAXIMIZATION

The paper argues that cross-entropy need not make classifiers explain all task-dependent variation, because task-relevant information can remain in nuisance variables. It uses mutual-information decompositions to formalize this insufficiency and motivate an objective that reduces excessive invariance.

  • Cross-entropy insufficiency: The cross-entropy objective does not necessarily encourage classifiers to explain all task-dependent variations of the data.The nuisance classifier on zn can use task-relevant information not captured by the logit classifier on zs.
  • Information-theoretic formulation: The classifier objective is formulated as maximizing mutual information I(y; zs) between labels and semantic features extracted by the network.Here zs denotes the logits used for classification.
  • Information-theoretic formulation: Under an adversarial distribution shift, the predictiveness of features encoded in zs may decrease while nuisance variables zn do not become more informative about y.The stated assumption is I_DAdv(zn; y) ≤ I_D(zn; y).
  • Information-theoretic formulation: For bijective networks, information preservation yields I(y; x) = I(y; zs, zn), which decomposes into label information in logits and conditional nuisance information.The chain rule gives I(y; zs) + I(y; zn|zs) = I(y; zn) + I(y; zs|zn).

1. Direct increase of I(y; zs)

The independence cross-entropy objective reduces label information in nuisance variables so semantic information remains stable under the adversarial distribution shift.

  • Information-theoretic motivation: Standard cross-entropy may achieve high classification performance while leaving class-specific information in nuisance variables, whereas iCE minimizes I(y; zn).The nuisance-classification component targets unused label information rather than only increasing I(y; zs).
  • Theoretical result: Assuming I_D(y; zn) = 0 and the stated distribution-shift conditions, the theorem gives I_DAdv(y; zs) = I_D(y; x).The result links nuisance independence during training to preserved semantic information after the shift.
  • Information-theoretic motivation: Under the training distribution, iCE minimizes I(y; zn), while cross-entropy already maximizes I(y; zs).The two objectives therefore differ mainly in their treatment of nuisance variables under training.
  • Theoretical result: Under DAdv, cross-entropy decreases I(y; zs), whereas iCE induces I(y; zn) = 0 and leaves I(y; zs) unchanged.Figure 6 visualizes the contrasting information changes under the two losses.
  • Objective design: A maximum-likelihood term is added to improve stability and further encourage factorization between semantic and nuisance variables.The log-determinant is computed exactly with negligible additional cost in the model.

4 APPLYING INDEPENDENCE CROSS-ENTROPY

Experiments show that independence cross-entropy preserves classification performance while reducing nuisance-based vulnerability across metameric attacks and adversarial distribution shifts.

  • Classification performance: Both losses perform approximately on par in test error, while iCE removes the train–test error gap, indicating less overfitting.The comparison uses the same architecture and settings, differing only by the additional iCE loss terms.
  • Metameric sampling attack: Holding logits constant, cross-entropy permits transformations between arbitrary classes, whereas iCE changes only digit style during nuisance interpolation.This metameric attack tests whether nuisance variables retain class-specific information.
  • Nuisance information: A classifier decodes class information well from cross-entropy nuisances but poorly from iCE nuisances, indicating little label information in zn after iCE training.The result is not attributed to readout difficulty because the metameric attack separately exposes the retained information.
  • shiftMNIST benchmark: shiftMNIST adds highly predictive features during training, then removes or randomizes them at test time to create adversarial distribution shifts.Binary shiftMNIST removes a location-based code, while textured shiftMNIST breaks the texture–label pairing.
  • shiftMNIST benchmark: Up to 87% shifted-test error occurs for CE-trained models despite zero training error, while iCE improves errors by up to almost 38% on binary and around 28% on textured shiftMNIST.Binary shiftMNIST removes only one pixel, leaving the image almost unchanged.

5 RELATED WORK

The paper connects its approach to unrestricted adversarial examples, reversible networks, and information-theoretic analyses of learned representations.

  • Adversarial examples: Prior adversarial-example research largely used ϵ-norm restrictions, while later work considered unrestricted examples that are harder to formalize beyond testing error.The paper positions invariance-based adversaries within this broader effort.
  • Network architectures: The analysis leverages reversible and bijective networks, whose representation behavior has been compared with that of ResNets and Glow-type networks.These architectures provide the invertible representations used in the paper’s analysis.
  • Information theory: The information-theoretic framing relates to information bottleneck, generative modeling, and mutual-information estimation research.The paper uses mutual information to analyze semantic and nuisance variables.

6 CONCLUSION

The paper attributes adversarial vulnerability to excessive invariance toward semantically meaningful variation and uses bijective networks plus an extended loss to control it.

  • Conclusion: The paper identifies excessive invariance to semantically meaningful variations as a major cause of adversarial vulnerability.It presents this as a reverse view of failures under distribution shift.
  • Conclusion: The issue persists across tasks and architectures, while fully invertible networks make invariance control tractable.The conclusion frames architecture and loss design as complementary parts of the approach.
  • Conclusion: Bijective networks reveal large adversarial subspaces on adversarial spheres, MNIST, and ImageNet.The paper then formalizes the associated distribution shifts using information theory.
  • Conclusion: The extended loss encourages separation between semantically meaningful and nuisance features and removes unwanted invariances in targeted distribution-shift experiments.The conclusion attributes the limitation of vanilla cross-entropy to its failure to capture all task-dependent input variations.

A SEMANTIC AND NUISANCE VARIATION ON ADVERSARIAL SPHERES

The spheres example separates semantic from nuisance perturbations and shows that a classifier can achieve perfect accuracy while remaining invariant to changes that alter class-specific semantics, causing chance-level performance under distribution shift.

  • Semantic perturbations change the sphere radius, whereas nuisance perturbations preserve the radius for the task of discriminating the two spheres.
  • The classifier’s pre-image can misalign with data semantics when it remains invariant to task-relevant changes.
  • The constructed classifier ignores changes in the last coordinate, making it invariant to a semantic radius change.
  • 100% train and validation accuracy still collapses to chance when invariant-coordinate perturbations move samples between the inner and outer spheres.

B APPROXIMATE GRADIENT-BASED METAMERIC SAMPLES

The paper generates metameric images by matching network representations, using gradient optimization for standard classifiers and information-theoretic analysis to characterize what semantic information remains in nuisance variables.

  • Metameric images are optimized to match a seed image’s logit percept by minimizing mean squared error in representation space.
  • The ImageNet attack targets all 1000 logit entries without an input-norm restriction, using Adam for 3000 iterations.
  • The invertible-network attack obtains an exact metameric solution with one inverse pass instead of thousands of gradient steps.
  • The nuisance classifier’s objective is analyzed through a variational mutual-information bound, with minimization reducing I(y; zn) and maximization tightening the bound.
  • The loss terms can be interpreted as maximum-likelihood estimation under a factorial prior that makes semantic and nuisance logits independent up to a constant shift.

C.1 PROOF OF THEOREM 6

The proof uses assumptions about adversarial distribution shifts, information-preservation properties, and the data processing inequality to relate adversarial information in the input to semantic and nuisance representations.

  • The argument assumes that the adversarial shift does not increase nuisance information about labels and excludes synergistic interaction effects.
  • Under these assumptions, adversarial label information decomposes into nuisance and conditional semantic information, bounded by information in the semantic representation.
  • The data processing inequality supplies the relation IDAdv(y; x) ≥ IDAdv(y; zs) because zs is a deterministic transform of the input.
  • The mutual-information objective has a fixed upper bound from the data-processing inequality, avoiding the need for gradient clipping or a bounded divergence substitution.
  • Experiments use fully invertible RevNet models with dataset-specific hyperparameters and frameworks.

D.1 SPHERES EXPERIMENTS

The spheres experiments use fully connected and convolutional invertible RevNets with nuisance classifiers, comparing vanilla cross-entropy against the proposed independence cross-entropy under controlled training settings.

  • The fully connected RevNet uses four invertible ReLU bottleneck blocks, no batch normalization, and a three-layer nuisance classifier with 1000 hidden units per layer.
  • The spheres are 100-dimensional with radii 1 and 10, trained on 500k samples and validated on 100k holdout samples.
  • 100% train and validation accuracy is achieved for both the logit and nuisance classifiers.
  • The convolutional RevNet adds actnorm, invertible 1x1 convolutions, squeezing, factor-out operations, and 16 RevNet blocks with 128 filters per layer.
  • Training compares vanilla cross-entropy with independence cross-entropy using Adamax, scheduled learning-rate decay, batch size 64, and l2 weight decay of 1e-4.

D.3 IMAGENET EXPERIMENTS

The experiments use an invertible RevNet architecture and compare cross-entropy with independence cross-entropy training. The reported results show comparable logit-classifier accuracy, while iCE yields less overfitting and lower nuisance-variable error.

  • Architecture: The model uses a four-stage fully invertible convolutional RevNet with invertible downsampling between stages.The first three stages use additive coupling layers, while the final stage uses affine coupling layers.
  • Results: CE and iCE achieve comparable logit-classifier accuracy, while CE has higher train error than test error, indicating less overfitting for iCE.The comparison is reported in Table 2 alongside two architectures from the literature.
  • Results: 27.70% error is achieved by an independently trained nuisance-variable classifier on iCE logits, lower than the corresponding CE nuisance-variable error.The table reports that the nuisance-variable classifier reaches even smaller error than the logits for CE.
  • Architecture: An orthogonal 2D DCT-II reads classes from low-pass components, providing an invertible form of global average pooling.This design makes the network more similar to ResNets, which apply global average pooling to final feature maps.
Loading 1811.00401v4…