Deep Learning is Not So Mysterious or Different

Andrew Gordon Wilson

Source-linked AI summary

Deep Learning is Not So Mysterious or Different

Andrew Gordon Wilson

arXiv:2503.02113v2Mar 3, 2025cs.LGstat.ML

TL;DR

The paper addresses whether anomalous generalization behaviors make deep learning fundamentally different from other model classes. It uses simple models and long-standing generalization frameworks to analyze these behaviors, finding that they are reproducible and formally characterizable, while identifying soft inductive biases as a unifying principle and delimiting other distinctive features of deep learning.

Problem
The paper examines whether benign overfitting, double descent, and overparametrization require new generalization theory or are distinctive to neural networks.
Method
The paper uses simple model examples, PAC-Bayes and countable-hypothesis bounds, and soft inductive biases to analyze these generalization phenomena.
Results
The considered phenomena are reproducible with other model classes and consistent with long-standing generalization frameworks, including PAC-Bayes and countable-hypothesis bounds.
Takeaways & Limitations
Flexible hypothesis spaces with soft preferences for simpler solutions provide a unifying account of the considered phenomena, while representation learning, mode connectivity, and relative universality remain comparatively distinctive.

Abstract

from arXiv · show

Deep neural networks are often seen as different from other model classes by defying conventional notions of generalization. Popular examples of anomalous generalization behaviour include benign overfitting, double descent, and the success of overparametrization. We argue that these phenomena are not distinct to neural networks, or particularly mysterious. Moreover, this generalization behaviour can be intuitively understood, and rigorously characterized, using long-standing generalization frameworks such as PAC-Bayes and countable hypothesis bounds. We present soft inductive biases as a key unifying principle in explaining these phenomena: rather than restricting the hypothesis space to avoid overfitting, embrace a flexible hypothesis space, with a soft preference for simpler solutions that are consistent with the data. This principle can be encoded in many model classes, and thus deep learning is not as mysterious or different from other model classes as it might seem. However, we also highlight how deep learning is relatively distinct in other ways, such as its ability for representation learning, phenomena such as mode connectivity, and its relative universality.

1. Introduction

The paper argues that benign overfitting, double descent, and overparametrization are neither unique to neural networks nor mysterious, because simple models and established generalization frameworks can reproduce and explain them. It proposes soft inductive biases as a unifying perspective while acknowledging that deep learning remains distinctive in other respects.

1. Introduction: Benign overfitting, double descent, and overparametrization are not distinct to neural networks or particularly mysterious.The paper contrasts these phenomena with conventional beliefs about deep learning and generalization.
1. Introduction: PAC-Bayes and countable-hypothesis bounds formally characterize these phenomena, unlike VC dimension and Rademacher complexity in the paper’s comparison.The authors therefore argue that understanding deep learning does not require rethinking generalization.
1. Introduction: The paper does not claim that deep learning is fully understood or comprehensively survey explanations of its phenomena.It instead aims to connect existing progress and generalization frameworks across communities.
1. Introduction: Simple linear models can reproduce and explain benign overfitting and double descent.Figure 1 uses simple examples to demonstrate that these behaviors extend beyond neural networks.
1. Introduction: Soft inductive biases combine an arbitrarily flexible hypothesis space with preferences for some solutions over others.This framework is presented as a unifying intuition for the paper’s treatment of generalization behavior.
1. Introduction: Representation learning, mode connectivity, and broadly successful in-context learning are discussed as relatively distinctive features of deep neural networks.Scaling laws and grokking are outside the paper’s main focus.

2. Soft Inductive Biases

Soft inductive biases preserve a flexible hypothesis space while preferring simpler or otherwise favored solutions, offering an alternative to hard architectural restrictions. The paper illustrates this principle through complexity-aware bounds, regularized polynomials, equivariance preferences, and overparametrization.

2. Soft Inductive Biases: Restriction biases constrain the hypothesis space, whereas soft inductive biases retain flexible solutions while preferring some over others.The paper presents soft biases as preferences that need not exclude any solution entirely.
2. Soft Inductive Biases: Generalization bounds combine empirical risk with hypothesis compressibility, allowing flexible models to generalize when their solutions are compressible.Kolmogorov complexity and model filesize provide formalizations or upper bounds for compressibility.
2. Soft Inductive Biases: Soft biases can favor approximate translation or rotation equivariance without imposing those properties as hard constraints.Residual pathway priors provide one mechanism, and can outperform hard symmetry constraints when the data have approximate or absent symmetry.
2. Soft Inductive Biases: Overparametrization, Bayesian priors, marginalization, regularization, and architecture can all induce soft inductive biases.The paper treats preference among equally data-fitting solutions as the general concept underlying these mechanisms.
2. Soft Inductive Biases: A high-order polynomial with order-dependent regularization uses lower-order terms preferentially and higher-order terms only when needed.The resulting model remains flexible while expressing a simplicity bias through increasingly strong penalties on higher-order coefficients.
2. Soft Inductive Biases: Vision transformers can become more translation equivariant after training than convolutional networks, despite lacking the same architectural constraint.The paper connects this result to compression and soft inductive bias.
2. Soft Inductive Biases: Soft inductive biases are presented as a key prescription for building intelligent systems.This conclusion favors flexible hypothesis spaces with preferences over solutions rather than restriction alone.

3. Generalization Frameworks

The paper presents PAC-Bayes and countable-hypothesis bounds as frameworks that characterize generalization with flexible hypothesis spaces and preferences for simple, compressible solutions. These frameworks provide intuition and formal guarantees while effective dimensionality offers an additional geometric interpretation.

3. Generalization Frameworks: Flexible hypothesis spaces can generalize when paired with a simplicity bias rather than restricted outright.The paper frames compressibility as a formalization of complexity and combines low empirical risk with low-complexity solutions.
3. Generalization Frameworks: PAC-Bayes and countable-hypothesis bounds focus on which hypotheses are likely, making them suitable for large and overparametrized models.They can yield non-vacuous guarantees even when models have millions or billions of parameters.
3. Generalization Frameworks: A Solomonoff prior connects generalization bounds to prefix-free Kolmogorov complexity conditioned on the model architecture.The resulting complexity term is based on the shortest program producing the hypothesis.
3. Generalization Frameworks: Countable-hypothesis bounds measure complexity through the number of bits required to represent a hypothesis.A large model can still generalize strongly when it has low empirical risk and small compressed size.
3. Generalization Frameworks: PAC-Bayes replaces a point-hypothesis coding term with KL(Q ∥ P) by considering a distribution over desirable solutions.This distributional view can recover coding-style bounds while allowing uncertainty over selected hypotheses.
3. Generalization Frameworks: Effective dimensionality measures the number of relatively large eigenvalues and provides intuition about sharp directions in the loss landscape.Lower effective dimensionality corresponds to flatter solutions, although Hessian flatness is not parametrization invariant and is not itself a generalization bound.

4. Benign Overfitting

Benign overfitting is the ability to fit noise while still generalizing on structured data, and the paper argues that it is neither uniquely neural nor mysterious. Flexible hypothesis spaces, data fitting, and simplicity preferences explain the phenomenon intuitively and through established bounds.

4. Benign Overfitting: Benign overfitting occurs when a model fits noise without losing its ability to generalize on structured data.The phenomenon challenges conventional views that fitting noise necessarily implies poor generalization.
4. Benign Overfitting: Benign overfitting can be reproduced with other model classes and characterized by long-standing generalization frameworks.The paper specifically identifies PAC-Bayes and countable-hypothesis bounds as formal tools for this characterization.
4. Benign Overfitting: The intuitive recipe combines a flexible hypothesis space, a loss that fits the data, and a simplicity bias among data-consistent solutions.Lower-complexity solutions are preferred even when many solutions fit the data perfectly.
4. Benign Overfitting: Order-dependent regularization makes polynomial models use lower-order terms first and higher-order terms only when needed.This preference allows simple structured data to be fit with simple, compressible functions that generalize.
4. Benign Overfitting: PAC-Bayes bounds can provide non-vacuous guarantees for neural networks exhibiting benign overfitting.These frameworks also define how large neural networks can remain simple through Kolmogorov complexity.
4. Benign Overfitting: The paper concludes that PAC-Bayes and countable-hypothesis bounds distinguish learning from natural data and randomized data more effectively than parameter-count-based measures.It presents soft inductive biases and compression as the corresponding intuition.

5. Overparametrization

The paper argues that parameter count alone is a poor proxy for complexity: arbitrarily flexible models can generalize when they favor simple solutions. Increasing parameters can simultaneously enlarge the hypothesis space and strengthen compression or simplicity bias.

5. Overparametrization: Parameter counting is a poor proxy for model complexity because arbitrarily large models can generalize with a simplicity bias.The relevant issue is how parameters control the functions used to fit data, not the parameters in isolation.
5. Overparametrization: Gaussian processes can be more flexible than standard neural networks yet perform especially well on small datasets because of a strong soft simplicity bias.The paper presents this as another example of flexible models generalizing without restrictive hypothesis spaces.
5. Overparametrization: Recent generalization bounds can become tighter as the number of parameters increases.This contrasts with the common assumption that larger parameter counts necessarily weaken generalization guarantees.
5. Overparametrization: Larger models can have fewer effective parameters after training and stronger simplicity bias across settings and modalities.The paper relates these observations to Hessian effective dimensionality and lower Kolmogorov complexity.
5. Overparametrization: The reason larger models appear to have stronger compression bias remains an open question.Proposed intuitions include more low-norm, low-effective-rank solutions and exponentially greater volumes of flat solutions.
5. Overparametrization: Increasing parameters can improve generalization because flat, compressible solutions occupy a greater relative volume of the hypothesis space.Overparametrization can therefore increase both flexibility and an implicit preference for simple solutions.

6. Double Descent

Double descent describes generalization error decreasing, increasing, and then decreasing again as model parameters grow. The paper argues that this behavior is not specific to deep neural networks and can be explained using simple linear models, minimum-norm solutions, compressibility, and formal bounds.

6. Double Descent: Double descent is a decrease, increase, and second decrease in generalization error as the number of model parameters increases.The first change occurs in a classical regime, while the second descent is associated with an interpolating regime.
6. Double Descent: Further increasing flexibility can alleviate overfitting after the classical regime, making double descent appear to challenge conventional generalization wisdom.The paper treats this as a motivating question rather than evidence that deep learning requires entirely new principles.
6. Double Descent: Double descent predates modern deep learning and has been reproduced with random forests and random feature models.The phenomenon was introduced decades earlier than its recent prominence in deep neural networks.
6. Double Descent: A linear model exhibits double descent when its feature dimension increases relative to the number of data points.For Xw = y, the model can interpolate perfectly once d > n, producing infinitely many parameter settings that fit the data.
6. Double Descent: Among interpolating solutions, least squares selects the minimum-ℓ2-norm solution, favoring simpler models that rely on informative feature directions.The paper connects this simplicity preference to effective dimensionality and Hessian eigenvalues.
6. Double Descent: In the second descent, larger models can achieve similar empirical risk while being more compressible.Countable-hypothesis and PAC-Bayes bounds can formally track this behavior.

7. Alternative Views

The paper argues that benign overfitting, double descent, and overparametrization are not uniquely neural-network phenomena and can be characterized by established generalization frameworks. It presents soft inductive biases as a unifying alternative to restriction-based views of generalization.

The bias-variance trade-off need not produce a U-shaped relationship because some models can achieve both low bias and low variance.
PAC-Bayes and countable-hypothesis bounds formally describe these phenomena, whereas VC dimension and Rademacher complexity do not.
Benign overfitting, double descent, and overparametrization are not specific to neural networks or inherently mysterious.

8. What is Different or Mysterious?

The paper identifies representation learning, universality, and mode connectivity as relatively distinctive aspects of deep learning while emphasizing that none is entirely exclusive to neural networks. Neural networks learn adaptive, often non-stationary similarity structures, support broad transfer across problems, and exhibit connected low-loss solutions.

Representation Learning: Representation learning sets neural networks apart by learning task-specific basis functions or similarity metrics rather than relying solely on fixed representations.
Representation Learning: Learned representations can make distant inputs similar for a task, enabling accurate prediction of a 60 degree face rotation from only distant angles.
Representation Learning: Representation learning is not unique to neural networks because kernel learning and deep kernel learning also provide adaptive or learned similarity structures.
Representation Learning: Neural networks are relatively efficient at learning adaptive, non-stationary metrics in high-dimensional settings, although why this works is not entirely clear.
Universal Learning: Deep learning runs contrary to the conventional expectation that strong performance on one problem requires poor performance on others, with models and naturally occurring data biased toward low Kolmogorov complexity.
Universal Learning: In-context learning and pretraining support representations that transfer across real-world problems, including competitive zero-shot time-series forecasting from a standard text-pretrained LLM.
Mode Connectivity: Mode connectivity shows simple paths between independently trained solutions that maintain essentially zero training loss, even though points along the paths can make different test predictions.
Mode Connectivity: Mode connectivity is largely a deep-learning phenomenon but is not entirely unique to neural networks and applies to sophisticated non-convex loss landscapes.

9. Discussion

The discussion argues that overparametrization, benign overfitting, and double descent are consistent with established generalization frameworks and reproducible beyond neural networks. It presents soft inductive biases as a unifying explanation while distinguishing deep learning through other features and acknowledging computational boundaries.

Overparametrization, benign overfitting, and double descent are consistent with long-standing generalization frameworks and reproducible using other model classes.
The paper distinguishes deep learning through representation learning, mode connectivity, and relative universality, rather than through the discussed generalization phenomena.
Optimizer biases can improve generalization, but respectable generalization does not require stochastic optimization.
Soft inductive biases allow large models to generalize by preferring simple solutions without restricting the hypothesis space.
Restriction biases can offer practical computational benefits, although parameter sharing may be poor for compute-optimal scaling.

A. Common Misconceptions about PAC-Bayes

The paper addresses misconceptions about PAC-Bayes and countable hypothesis bounds by clarifying that their priors need not generate models, deterministic models are covered, and larger parameter counts do not inherently invalidate bounds. It also notes that compression-based and Solomonoff-prior bounds can be practically informative or empirically bounded.

Deterministically trained models are covered by point-mass PAC-Bayes posteriors and directly by countable hypothesis bounds.
Countable hypothesis bounds apply to finite-precision neural networks and can become tighter for larger models when likely hypotheses are favored.
Some PAC-Bayes and countable bounds avoid parameter-counting penalties and can tighten as larger models acquire stronger compression bias.
Recent non-vacuous bounds for billion-parameter language models are not necessarily compression bounds, while model filesize can upper-bound Solomonoff complexity.
PAC-Bayes priors need not generate, contain, or be used by the bounded hypothesis; misspecification instead produces a looser bound.

B. Other Generalization Frameworks

This section contrasts capacity-based frameworks with approaches that emphasize which hypotheses are likely. Rademacher, VC, and fat-shattering dimensions measure fitting or shattering capacity, while the paper summarizes their limitations for flexible hypothesis spaces.

Rademacher complexity measures a hypothesis space’s ability to fit uniform random noise and can yield uninformative bounds for spaces that fit such noise.
VC dimension measures the largest set size a hypothesis space can shatter with all binary labelings.
Fat-shattering dimension refines VC dimension by requiring label fitting within a margin, which becomes harder as the margin increases.
The paper provides a comparative summary of generalization bounds in Table 1.

C. Countable Hypothesis Bound

The countable hypothesis bound uses hypothesis-dependent deviations and a prior over a countable space to control expected risk, including for hypotheses selected from training data. Its proof applies concentration to fixed hypotheses and then extends the guarantee through a union bound.

The theorem assumes bounded risk, a countable hypothesis space, and a data-independent prior.
For a fixed hypothesis, Hoeffding’s inequality controls the deviation between expected and empirical risk.
The deviation threshold is chosen separately for each hypothesis according to its prior probability.
Although a training-selected hypothesis does not preserve independence, a bound holding for every hypothesis also holds for the selected hypothesis.
A union bound over hypotheses produces the simultaneous guarantee used by the countable hypothesis framework.

D. Experimental Details

The experiments combine polynomial regression, neural-network-related analyses, and conceptual figures. They evaluate generalization across varied model settings, including noisy-label replication and countable-hypothesis bounds for LLMs.

Figure 1(a)(b)(c) uses a 150th-order polynomial with order-dependent regularization on three regression datasets.The datasets are generated from sin(x) cos(x^2), x + cos(πx), and N(0, 1) noise.
Figures 1(d)(e) replicate a CIFAR-10 noisy-label experiment using a Gaussian process with an RBF kernel and a PreResNet-20.The setup includes an isotropic prior p(w) = N(0, α2I) and Laplace marginal likelihood.
Figure 2 evaluates countable-hypothesis bounds with an upper bound on Kolmogorov complexity for LLMs of various sizes.
Figure 5 compares two 15th-order polynomials and one 2nd-order polynomial using RMSE across training sample sizes from 10 to 100.For each training size, data are regenerated 100 times; test sets contain 100 samples, and shading represents RMSE standard deviation.
All other figures are conceptual figures.