PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning explanations that are hard to vary

Giambattista Parascandolo, Alexander Neitz, Antonio Orvieto, Luigi Gresele, Bernhard Schölkopf

arXiv:2009.00329v3cs.LGstat.ML

TL;DR

The paper asks how deep learning can identify invariant mechanisms when pooled gradient averaging may favor memorization and patchwork solutions. It formalizes consistency, develops a logical-AND alternative to gradient averaging, and reports improved invariant learning across synthetic and real-world settings, while noting scope limitations in how environments are defined.

  • Problem

    Pooled gradient averaging can discard environment-specific learning signals, favoring shortcuts or patchwork solutions instead of invariant mechanisms.

  • Method

    The paper formalizes consistency and Invariant Learning Consistency, then uses an AND-masked gradient approach based on a logical AND between environment signals.

  • Results

    The approach prioritizes invariances, generalizes out of distribution on a synthetic task, prevents convergence on CIFAR-10 with random labels, and is evaluated in behavioral cloning.

  • Takeaways & Limitations

    Studying explanations that are hard to vary provides a broader principle for learning invariances, while the AND-mask is one possible implementation rather than a universal algorithm.

  • Takeaways & Limitations

    The CIFAR-10 experiment treats every example as its own environment because the dataset has no standard environment partition, and the AND-mask is unlikely to suit all applications.

Abstract

from arXiv · show

In this paper, we investigate the principle that `good explanations are hard to vary' in the context of deep learning. We show that averaging gradients across examples -- akin to a logical OR of patterns -- can favor memorization and `patchwork' solutions that sew together different strategies, instead of identifying invariances. To inspect this, we first formalize a notion of consistency for minima of the loss surface, which measures to what extent a minimum appears only when examples are pooled. We then propose and experimentally validate a simple alternative algorithm based on a logical AND, that focuses on invariances and prevents memorization in a set of real-world tasks. Finally, using a synthetic dataset with a clear distinction between invariant and spurious mechanisms, we dissect learning signals and compare this approach to well-established regularizers.

1 INTRODUCTION

The paper argues that pooled loss can make distinct zero-loss minima look equally good even when only some solutions are shared across environments. It develops consistency as a way to favor invariant mechanisms over shortcuts and patchwork solutions.

  • Motivation: Two global minima can have identical pooled loss while differing in whether they are reachable from examples considered separately.One minimum overlaps across the separate loss surfaces, whereas the other appears only after pooling the examples.
  • Motivation: Averaging gradients can discard information about patterns shared across different environments.After averaging, the separate loss surfaces are no longer distinguishable from the pooled landscape.
  • Core principle: The paper hypothesizes that invariant mechanisms can be identified through explanations that are hard to vary across examples.The assumed invariant mechanism is shared across environments and generalizes out of distribution, unlike environment-specific shortcuts.
  • Core principle: Consistency measures whether a loss minimum appears only when data from different environments are pooled.Low-consistency minima are characterized as patchwork solutions that sew together different strategies.
  • Approach and scope: Invariant Learning Consistency changes the learning objective’s preference between fast pooled-loss reduction and invariant solutions.The paper reports out-of-distribution generalization on a synthetic task, prevention of random-label convergence on CIFAR-10, and a behavioral-cloning evaluation.
  • Illustration: A chess-puzzle analogy contrasts environment-specific shortcuts with learning to play chess, which is the strategy expected to generalize to new books.The example illustrates why training on only one book can hide shortcuts present in another.

2 EXPLANATIONS THAT ARE HARD TO VARY

The paper argues that standard gradient averaging can favor inconsistent, patchwork solutions because it prioritizes pooled-loss descent speed over agreement across environments. It formalizes consistency and proposes AND-based gradient masking to emphasize directions supported across environments.

  • Standard gradient-based optimization can stop after learning spurious patterns, before invariant mechanisms emerge.The paper identifies early low-loss stopping and differing learning rates for invariant and spurious patterns as barriers to invariance learning.
  • Pooled-loss gradients treat each example independently, so invariant learning depends on how environment-specific signals are aggregated.For a single example, every pattern can appear relevant; aggregation must reveal what remains shared across examples.
  • Arithmetic gradient averaging greedily maximizes learning speed but can retain conflicting environment-specific directions instead of trading speed for invariance.The pooled gradient averages environment signals, potentially discarding information about whether those signals arise from consistent landscapes.
  • A logical AND, implemented through geometric means or masking, suppresses directions lacking cross-environment agreement and targets harder-to-vary explanations.The AND-mask zeroes gradient components with fewer than t agreeing signs across environments.
  • Invariant Learning Consistency measures the expected consistency of the pooled-data solution, while low-consistency minima represent patchwork solutions that are not expected to generalize out of distribution.The formal notion evaluates whether a minimum appears only after examples from different environments are pooled.

3 EXPERIMENTS

Experiments use a synthetic dataset and real-world tasks to test whether the AND-mask favors invariant mechanisms over shortcuts and memorization. Across these settings, the AND-mask identifies the synthetic invariant mechanism, suppresses memorization signals, and prevents overfitting to random or noisy labels.

  • 3.1 THE SYNTHETIC MEMORIZATION DATASET: The synthetic task separates a shared, highly nonlinear mechanism from environment-specific shortcuts that are simple and perfectly accurate on pooled training data.The out-of-distribution test preserves the mechanism but randomizes shortcuts, so shortcut-only methods perform at chance.
  • 3.1 THE SYNTHETIC MEMORIZATION DATASET: The pooled shortcut boundary is not found from either environment alone, illustrating why pooled training can favor a patchwork solution over the invariant mechanism.In high dimensions, shortcuts also satisfy several classical simplicity criteria, making the invariant boundary difficult to discover.
  • 3.1 THE SYNTHETIC MEMORIZATION DATASET: The AND-mask was the only evaluated method to achieve perfect test accuracy on the synthetic dataset by fitting the spirals instead of the shortcuts.Combining the AND-mask with L1 or L2 regularization produced the most robust results overall.
  • 3.1 THE SYNTHETIC MEMORIZATION DATASET: At τ ≈ 1, most remaining gradient components contained mechanism signal, whereas increasing the threshold suppressed shortcut-driven memorization gradients.With τ = 0, the mechanism signal was present but smaller and drowned by memorization signal; the XOR-mask reduced mechanism correlation instead.
  • 3.2 EXPERIMENTS ON CIFAR-10: On CIFAR-10 with random labels, standard gradients memorized the data, while slightly increasing the AND-mask threshold quickly prevented memorization.The experiment treated each example as its own environment; the authors report that test accuracy was not hurt in the corresponding label-noise experiment.
  • 3.3 BEHAVIORAL CLONING ON COINRUN: CoinRun evaluates whether behavioral cloning generalizes from 64 training levels to 2000 test levels when physics and goals remain invariant despite procedural visual changes.The setup uses a pretrained policy’s state-action pairs and compares regular Adam with an AND-mask-trained ResNet-18.

4 RELATED WORK

The paper situates invariant mechanisms within covariate-shift, causal, and invariance-based approaches to out-of-distribution generalization.

  • Generalization and covariate shift: Covariate shift concerns test-time changes in covariate distributions, with reweighting methods requiring overlapping train and test supports.
  • Causal models and invariances: Causal models motivate invariant mechanisms as stable elements connecting causes and effects across interventions or changing external conditions.

5 CONCLUSIONS

The paper argues that gradient aggregation can produce patchwork solutions rather than invariances, and develops consistency-based analyses of this issue. Its geometric-mean perspective links landscape inconsistency to learning behavior, while the reported bounds and examples clarify both the approach and its limits.

  • 5 CONCLUSIONS: Averaging gradients can act as a logical OR of distinct patterns, leading to patchwork solutions.
  • 5 CONCLUSIONS: The paper introduces Invariant Learning Consistency to formalize how much a minimum appears only after pooling data from different environments.
  • A.1 A CLASSIC EXAMPLE OF A PATCHWORK SOLUTION: The neural-network example fits both environments with θ˚, yet modifying θ6 leaves environment A nearly unchanged while substantially worsening environment B, yielding low consistency.
  • A.2 SECTION 2.2: CONSISTENCY AS ARITHMETIC/GEOMETRIC MEAN OF LANDSCAPES: The arithmetic mean of Hessians performs a logical OR on dominant eigendirections, whereas the geometric mean reflects conflicting landscape geometries as a logical AND.
  • A.2 SECTION 2.2: CONSISTENCY AS ARITHMETIC/GEOMETRIC MEAN OF LANDSCAPES: The consistency proposition provides a pessimistic lower bound that is tight in two important limiting cases.
  • A.2 SECTION 2.2: CONSISTENCY AS ARITHMETIC/GEOMETRIC MEAN OF LANDSCAPES: For the illustrated Hessians, the geometric average preserves the original ellipse volume while the arithmetic-average landscape volume is 25 times larger, indicating inconsistency.

A.3 PROOF OF PROPOSITION 1

The appendix analyzes AND-masked gradient descent, which retains gradient components with sufficient cross-environment sign agreement. Under smoothness and a suitable learning rate, it guarantees a point with a small masked gradient.

  • A.3 PROOF OF PROPOSITION 1: AND-masked gradients set components to zero when fewer than t environments agree on their gradient signs.
  • A.3 PROOF OF PROPOSITION 1: With L-Lipschitz gradients and η ≤ 1/L, AND-masked gradient descent visits a point whose masked-gradient squared norm is O(1/k) after k iterations.
  • A.3 PROOF OF PROPOSITION 1: The proof obtains this result by showing that each iteration decreases the loss in proportion to the squared masked-gradient norm.

A.4 PROOF OF PROPOSITION 2

The appendix studies AND-masked gradients under independent Gaussian gradient signals and derives bounds on their expected magnitude. The analysis shows that sign agreement can make masked updates substantially smaller, especially at higher thresholds, while experiments use specified regularization ranges.

  • A.4 PROOF OF PROPOSITION 2: The Gaussian analysis assumes independent zero-mean gradients with covariance σ²I and studies how AND-masking reduces the gradient-step magnitude.
  • A.4 PROOF OF PROPOSITION 2: Without masking, the expected squared gradient norm is O(n/d), while masking yields a bound O(n/(cd)) for some c in (1,2].
  • A.4 PROOF OF PROPOSITION 2: The mask retains a direction only when at least t environments agree on its sign, with positive and negative agreement both accounted for.
  • A.4 PROOF OF PROPOSITION 2: The derivation uses conditional independence and bounds involving positive half-normal gradient components.
  • A.4 PROOF OF PROPOSITION 2: For t = d/2, the exponential rate is lost and the bound returns to O(n/d).
  • A.4 PROOF OF PROPOSITION 2: The experiments implement the paper in PyTorch, and Table 1 specifies that L1 and L2 regularization are selected separately rather than combined.

B.2 DATASET

The synthetic dataset separates invariant mechanisms from environment-specific Gaussian shortcuts, using two-dimensional spirals for the mechanism and 32 shortcut dimensions.

  • Each example combines a randomly sampled binary label with dS + dM features representing shortcuts and the mechanism.The experiments use dM = 2 and dS = 32.
  • The mechanism is implemented as two interconnected spirals, while shortcut vectors are sampled independently for each environment.Class 1 uses xS and class -1 uses -xS.
  • The test set samples shortcut components independently, so each test example effectively belongs to a different domain.
  • The training data uses 32 environments unless otherwise noted, with 1280 examples per environment.

B.3 EXPERIMENT

The experiments compare the AND-mask with regularized networks, DANN, and IRM using MLPs, cross-entropy training, and hyperparameter searches.

  • All networks are MLPs with LeakyReLU activations and cross-entropy loss, trained with scheduled learning-rate drops.Trials can stop early when training accuracy exceeds 97% while test accuracy remains below 60%.
  • The evaluated regularizers include L1, L2, dropout, and batch normalization, with hyperparameters selected from specified ranges.L1 and L2 are never combined.
  • DANN uses embedding, classification, and domain-discrimination networks, each implemented as a two-layer MLP.
  • IRM experiments use the authors’ PyTorch implementation with random hyperparameter search over the ranges in Table 2.

B.3.4 CURVES FOR ALL EXPERIMENTS

The paper presents training and test learning curves for the evaluated methods and specifies the setup used for additional gradient-correlation analyses.

  • Figure 14 reports learning curves for training and test accuracy across the different methods.
  • The correlation analysis uses a randomly initialized three-layer MLP with 256 hidden units, 16 environments, and batches of size 1024.Ten weight seeds produce ten least-squares regression lines, excluding zero gradients.
  • The AND-mask normalization sets c to 1 when all components are unmasked and to infinity when all are masked, retaining the result as zero.

B.4 FURTHER VISUALIZATIONS AND EXPERIMENTS

The further experiment measures how many environments are needed before the baseline changes its decision boundary from shortcuts to the mechanism.

  • 1024 environments are required before the baseline without AND-mask first switches its decision boundary from shortcuts to the mechanism.This result uses the same experimental conditions as the main paper.
  • Figure 15 plots the relationship between the number of training environments and test accuracy for the AND-mask method versus the baseline.It reports the best performance out of five runs under the main-text experimental settings.

B.5 SECTION 3.2: CIFAR-10 MEMORIZATION AND LABEL NOISE EXPERIMENTS

The experiments examine whether the AND-mask favors consistent signals over memorization and report its training, test, and label-noise behavior alongside implementation details.

  • CIFAR-10 memorization: The memorization experiment reports test performance with dashed lines and training performance with solid lines.The curves correspond to the CIFAR-10 memorization experiment.
  • CIFAR-10 memorization: As the threshold increases, test performance with standard labels decreases more slowly than training performance until both reach the same value.This behavior is consistent with selecting invariant patterns and pruning spurious signals.
  • Label noise: The label-noise experiment randomly shuffles 25% of training labels to test whether disagreeing gradients are masked out.The experiment compares the same ResNet with and without the AND-mask across three starting learning rates.
  • Training configuration: The best reported AND-mask threshold was 0.2, while the baseline worked best with a learning rate of 1e-3 and the AND-mask with 5e-3.The higher AND-mask learning rate was likely used to compensate for masked gradients.
  • Behavioral cloning on CoinRun: For CoinRun behavioral cloning, training uses 64 levels with 1000 examples per level, while testing uses 2000 unseen levels.A ResNet-18 is trained with KL divergence to a PPO target policy, and performance is compared between regular Adam and the AND-mask.
  • Optimization details: Applying weight decay as a separate update after the AND-mask improved performance, and the same switch also improved the Adam baseline.This addition was included in both hyperparameter searches for a fair comparison.

C.2 LEARNING INVARIANCES IN THE DATA

The paper contrasts its approach to IRM by framing invariance as stability across examples, whereas IRM seeks a predictor that is simultaneously optimal across environments.

  • Invariant Risk Minimization: IRM addresses settings where minimizing training error can absorb training correlations instead of recovering the relevant causal explanation.Its objective is to extract data representations supporting invariant predictors across environments.
  • Invariant Risk Minimization: An invariant predictor uses a representation Φ for which one classifier w is simultaneously optimal across all environments.The paper states this as the defining condition of invariant predictors in IRM.
  • Invariant Risk Minimization: IRM penalizes representations whose predictions can be made closer to label distributions by sharpening or softening a dummy scaling variable.The penalty is described as large when such rescaling improves the output distribution on an environment.
  • Comparison with the paper’s setting: In the cow–camel example, grass appears with cows in 80% of one dataset and 90% of the other, making its label correlation inconsistent.The example motivates ignoring grass as a feature under IRM.
  • Comparison with the paper’s setting: Under the paper’s invariance concept, treating each image as its own environment may suffice because grass predicts cows only in 80% of examples, whereas the cow appears in 100%.This differs from the IRM example’s reliance on two datasets to expose the inconsistency.
Loading 2009.00329v3…