On the Foundations of Shortcut Learning

• Because predictivity alone is insufficient to explain feature reliance, the paper introduces availability as a factor influencing whether models use a feature beyond what a purely statistical account predicts.Source: §1 · p. 2
• A generative framework quantifies feature predictivity and availability and defines shortcut bias by deviation from an optimal classifier’s feature reliance.Source: §1 · p. 2
• Nonlinear models show greater shortcut bias than linear models, and greater depth amplifies the bias.Source: §1 · p. 2
• A Neural Tangent Kernel account indicates that shortcut bias is an inevitable consequence of nonlinear architectures.Source: §1 · p. 2
• Vision architectures used in practice can be sensitive to non-core features beyond their predictive value, and availability manipulations can shift their feature reliance.Source: §1 · p. 2

• Prior work found that models preferentially represent one of two equally predictive image features, often the more linearly decodable at initialization, and can favor a less-predictive linear feature over a more-predictive nonlinear one (Hermann & Lampinen, 2020).Source: §2 · p. 2
• Vision models have been reported to favor texture over shape, local over global content, and backgrounds over foreground objects (Baker et al., 2018; Geirhos et al., 2018a; Hermann et al., 2020; Baker et al., 2020; Malhotra et al., 2020; Beery et al., 2018; Sagawa et al., 2020a; Xiao et al., 2020).Source: §2 · p. 2
• Figure 1 depicts synthetic datasets differing in shortcut-feature predictivity and an embedding procedure that manipulates availability through the mapping from z to x.Source: Figure 1 · p. 3
• Reducing background-texture opacity shifted classifications toward shape, while models preferred the larger-footprint object when images contained objects with opposing labels (Tartaglini et al., 2022; Wolff & Wolff, 2022).Source: §2 · p. 3

• The procedure maps latent shortcut and core features z = (zs, zc) to an input vector x and class label y.Source: §3 · p. 3
• Feature predictivity ρi is defined as the probability that the label matches the sign of latent feature zi, with Gaussian likelihoods setting the feature means to achieve that predictivity.Source: §3 · p. 3
• Availability is manipulated independently through embedding amplification αi and nesting ηi; nesting applies cascaded random tanh layers, making recovery increasingly complex as ηi increases.Source: §3 · p. 3
• The Figure 2 heatmap shows mean bias across shortcut-feature availability and predictivity, and its caption reports a model shortcut bias of 0.53.Source: Figure 2 · p. 4
• Shortcut bias is model reliance on the shortcut feature minus optimal LDA reliance, measured by comparing model and optimal-classifier decisions in latent space.Source: §3 · pp. 3–4

4 EXPERIMENTS MANIPULATING FEATURE PREDICTIVITY AND AVAILABILITY

• Sufficiently high availability lets models prefer a less-predictive shortcut over a more-predictive core feature.When predictivity is matched, models also prefer the more-available feature.Source: §4 · p. 4
• Greater model depth and nonlinear activations increase shortcut bias relative to shallower and linear models.Figure 3A varies width and depth with αs/αc = 64 and ρs = 0.85; Figure 3B compares activations at width 128, depth 2, matched predictivity, and αs/αc = 32.Source: §4 · p. 4, Figure 3 · p. 5
• ResNet-18 shows shortcut bias from object pixel footprint even when the shortcut feature is less predictive.The figure fixes αs/αc = 4 for the less-predictive-feature comparison.Source: Figure 4 · p. 5
• Increasing the relative nesting of feature representations also increases shortcut bias.These experiments manipulate nesting, ηc − ηs ≥ 1, as a second hypothesized availability factor beyond amplitude ratio.Source: §4 · p. 5

5 AVAILABILITY MANIPULATIONS WITH SYNTHETIC IMAGES

• The study treats large image footprint as a hypothesized source of availability, analogous to texture or background features.The authors suggest this may help explain why models can favor backgrounds over typically smaller foreground objects.Source: §5 · pp. 5–6
• Each latent feature is rendered as a colored circle or square, and availability is varied through object size in 224 × 224 images.Objects are randomly positioned without overlap.Source: §5 · p. 5
• ResNet18 is biased toward the more available shortcut feature even when it is less predictive than the core feature; Figure 4B considers equal predictivity (ρs = ρc = 0.9), while Figure 4C fixes αs/αc = 4 and varies ρs.Source: §5 · p. 6

6 THEORETICAL ACCOUNT

• In the large-width limit, the two-layer network is analyzed as kernel regression, and its kernel depends on architecture and activation.The analysis assumes a small covariance scale and uses a polynomial approximation for the ReLU kernel.Source: §6 · p. 6
• For tractability, the ReLU kernel is replaced by a quadratic approximation to analyze its spectral characteristics.Under the stated data distribution, the approximated kernel has two non-zero eigenvalues.Source: §6 · p. 7

6.1 SENSITIVITY ANALYSIS

• The analysis measures each feature's sensitivity as an mth-order derivative of prediction alignment with respect to its availability scaling.Comparing |ζi| across features with their availability indicates whether sensitivity favors the more- or less-available feature.Source: §6.1 · p. 7
• Linear networks have no availability bias, whereas ReLU networks can be biased toward more-available features.For ReLU networks, the first nonzero difference in feature sensitivities occurs at m = 9; it is zero for 1 ≤ m ≤ 8.Source: §6.1 · pp. 7–8
• For equally predictive features, ReLU networks are biased toward the more-available feature for any non-negative availability parameters.The paper relates feature predictivity to μi, with larger μi implying greater predictivity.Source: §6.1 · p. 8

7 FEATURE AVAILABILITY IN NATURALISTIC DATASETS

• Bird accuracy on incongruent Waterbirds probes increases when background availability is reduced through spatial extent, color, noise, or spatial-frequency manipulations.The pattern also occurs in ImageNet-pretrained models; background noise and spatial-frequency manipulations affect feature reliance.Source: §7 · p. 8
• On Waterbirds and CelebA, increasing non-core predictivity raises accuracy on congruent probes and lowers it on incongruent probes more than for a Bayes-optimal classifier.This pattern indicates that predictivity alone does not account for models’ non-core feature sensitivity.Source: §7 · p. 8, Figure 6 · p. 9

8 CONCLUSION

• Across vector and image tasks, deep nonlinear models exhibit shortcut bias, and theory predicts availability can outweigh predictivity.The account predicts bias for a single-hidden-layer nonlinear ReLU MLP but not a linear model.Source: §8 · p. 9
• In naturalistic vision datasets, non-core features influence models beyond their statistical value, while manipulated background properties alter shortcut propensity.The conclusion identifies spatial extent, color, and spatial frequency as properties that influence this propensity.Source: §8 · p. 9
• Understanding model behavior requires considering both feature predictivity and availability; future work will examine more domains and discover additional shortcut features.The generative framework is proposed as a basis for systematically investigating architectural manipulations.Source: §8 · pp. 9–10

A BROADER IMPACTS

• The work’s stated societal relevance is that shortcut features spuriously correlated with core features, such as gender or ethnicity and default likelihood, can intersect with fairness concerns.The authors describe the research as seeking a fundamental understanding of existing algorithms and state no downside risk.Source: A BROADER IMPACTS · p. 14

B SUPPLEMENTARY FIGURES

• Supplementary figures cover train and validation performance, alternative σsc settings, nesting, image availability manipulations, and attribution maps.The figures also include Waterbirds and pixel-footprint results, plus performance plots for ImageNet-pretrained models.Source: Figure B.3 · p. 15, Figure B.4 · p. 15, Figure B.7 · p. 16

C.1 ASSESSING BIAS

• Shortcut reliance is assessed by comparing a model’s feature reliance with that of an optimal classifier; hypothetical classifiers illustrate reliance patterns.The optimal classifier is fitted with Linear Discriminant Analysis, generally using the same embedded base inputs as the trained model.Source: §C.1 · p. 20, §C.2 · p. 20

C.3 VECTOR EXPERIMENTS

• The default vector-experiment model is an eight-layer, width-128 ReLU MLP trained with MSE loss and SGD for 100 epochs.Weights use Glorot Normal initialization, with batch size 64 and learning rate 10^-2.Source: §C.3 · p. 20
• With architecture fixed, models prefer the more-available feature and can favor it over a more-predictive feature when availability is sufficiently high.The vector experiments average bias across ten runs; the nesting study compares a shallowly embedded feature with a deeply embedded one.Source: §C.3 · p. 20

C.4 PIXEL FOOTPRINT EXPERIMENTS

• Pixel-footprint experiments vary shortcut-feature footprint from one to five times its 400-pixel base and compare model reliance with LDA.Bias is evaluated on image instantiations of probes against LDA fitted and probed on matching vector instantiations; five runs are used.Source: §C.4 · p. 20

C.5 FEATURE AVAILABILITY IN NATURALISTIC DATASETS

• The Waterbirds study manipulates background availability through bird size, patch removal, color, low-pass filtering, Perlin noise, and white noise.Bird predictivity is held at 0.99 while background predictivity varies; experiments report at least 500 model-training runs per perturbation type.Source: §C.5 · p. 21
• Bird accuracy on incongruent probes rises when background availability is reduced, while clearer, informative backgrounds encourage reliance on background features.Bird size increases, background noise, high-frequency filtering, and masking shift reliance toward the bird; the reported pattern also shows pretraining can modulate availability effects.Source: §C.5 · p. 22
• Attribution maps for probe images align with the reported shift from background focus at 40% bird size to foreground-bird focus at 80%.The analyses combine black-box methods, including Rise and Sobol indices, with gradient-based methods, including Saliency and Smoothgrad.Source: §C.5 · p. 22

D QUADRATIC APPROXIMATION OF H

• The quadratic approximation is chosen to resemble h over u in [-1, 1] by minimizing their L2 error with a parameterized quadratic.The appendix plots h and its approximation to illustrate approximation quality.Source: D QUADRATIC APPROXIMATION OF H · pp. 22–23

E PROOF OF RELU KERNEL THEOREM

• The ReLU kernel proof replaces h with bh to obtain an alternate kernel function that approximates the ReLU kernel.It computes the definite integral of (h(u) − bh(u))^2 by evaluating its indefinite integral at the boundaries.Source: E PROOF OF RELU KERNEL THEOREM · p. 23

E.1 EIGENFUNCTIONS

• For the eigenfunction derivation, the input distribution is a Gaussian mixture, and small component covariances permit a Laplace-type approximation.The resulting linear system implies that each solution a is an eigenvector of C; for two equal-weight, opposite-mean components, the eigenvectors are proportional to v+ + v− and v+ − v−.Source: §E.1 · pp. 25–26

E.2 EIGENVALUES

• The ReLU-kernel eigenvalues are found by substituting the eigenfunctions into their defining equation.Source: §E.2 · p. 28

F PROOF OF LINEAR KERNEL THEOREM

• For the linear kernel, the eigenfunction solutions reduce to eigenvectors of C under a small-covariance Laplace-like approximation.The proof obtains the eigenvalue by substituting the eigenfunction into the kernel eigenfunction equation.Source: F PROOF OF LINEAR KERNEL THEOREM · pp. 30–31

G OPTIMAL PREDICTION

• The optimal regression function is expressed using eigenfunctions of the kernel operator with nonzero eigenvalues.The proof specializes the setup to two small-covariance Gaussian class-conditional distributions, with targets determined by the positive or negative mixture component.Source: G OPTIMAL PREDICTION · pp. 31–32

H.1 LINEAR KERNEL

• In a linear network, |ζ1| − |ζ2| is zero for every m ≥ 1, regardless of a1 and a2.The proof evaluates the quantities at b1 = b2 = 1 and obtains ζ1 = ζ2 = 0.Source: §H.1 · pp. 32–33

H.2 RELU LINEAR KERNEL

• For a ReLU network, |ζ1| − |ζ2| = 0 for 1 ≤ m ≤ 8, and its first nonzero value occurs at m = 9.The passage states that the first nonzero value has a specified form, but the supplied text does not include that formula.Source: §H.2 · p. 33
• The derivatives of γ with respect to b1 and b2, evaluated at b1 = b2 = 1, yield ζ1 = ζ2 for every derivative order m ≤ 8.The passage begins to describe the order-9 result but does not provide it.Source: §H.2 · p. 34

I QUADRATIC APPROXIMATION VS RELU KERNEL

• The theory characterizes the predictivity–availability trade-off for a single-layer ReLU model, using asymptotically small covariance and a quadratic kernel approximation.The authors examine whether these tractability approximations alter the predictions.Source: I QUADRATIC APPROXIMATION VS RELU KERNEL · pp. 34–35
• Finite-data predictions with real covariance match the theory’s closed-form expression for the quadratic kernel.The left panel of Figure I.1 reports this comparison.Source: I QUADRATIC APPROXIMATION VS RELU KERNEL · p. 35, Figure I.1 · p. 35
• Using the actual ReLU kernel instead of its quadratic approximation does not affect the trend of predictions.The right panel of Figure I.1 supports the authors’ claim that little is lost by switching kernels.Source: I QUADRATIC APPROXIMATION VS RELU KERNEL · p. 35, Figure I.1 · p. 35