Source-linked AI summary
A proof that rectified deep neural networks overcome the curse of dimensionality in the numerical approximation of semilinear heat equations
Martin Hutzenthaler, Arnulf Jentzen, Thomas Kruse, Tuan Anh Nguyen
TL;DR
The paper asks whether deep neural networks can rigorously avoid the curse of dimensionality when approximating solutions of high-dimensional nonlinear semilinear PDEs. It combines stability analysis with multilevel Picard approximations and DNN representations, proving polynomial parameter growth for semilinear heat equations with gradient-independent nonlinearities.
Problem
Rigorous evidence was missing for overcoming the curse of dimensionality in DNN approximation of nonlinear semilinear PDEs, despite empirical success and results for certain linear PDEs.
Method
The proof combines stability estimates for multilevel Picard approximations with constructions representing those approximations as deep neural networks.
Results
P(Ψd,ε) ≤ ηd^ηε^-η: for semilinear heat equations with gradient-independent nonlinearities, approximating DNN parameter counts grow polynomially in dimension d and reciprocal accuracy ε^-1.
Takeaways & Limitations
The paper rigorously establishes that DNN approximations can overcome the curse of dimensionality for the specified class of semilinear heat equations.
Takeaways & Limitations
The result is established for semilinear heat equations with gradient-independent nonlinearities under the theorem’s stated assumptions.
Abstract
from arXiv · showhide
Deep neural networks and other deep learning methods have very successfully been applied to the numerical approximation of high-dimensional nonlinear parabolic partial differential equations (PDEs), which are widely used in finance, engineering, and natural sciences. In particular, simulations indicate that algorithms based on deep learning overcome the curse of dimensionality in the numerical approximation of solutions of semilinear PDEs. For certain linear PDEs this has also been proved mathematically. The key contribution of this article is to rigorously prove this for the first time for a class of nonlinear PDEs. More precisely, we prove in the case of semilinear heat equations with gradient-independent nonlinearities that the numbers of parameters of the employed deep neural networks grow at most polynomially in both the PDE dimension and the reciprocal of the prescribed approximation accuracy. Our proof relies on recently introduced multilevel Picard approximations of semilinear PDEs.
1 Introduction
The paper addresses the gap between empirical evidence and rigorous guarantees for deep-learning approximation of high-dimensional semilinear PDEs. It proves polynomial parameter growth for rectified DNN approximations under specified assumptions.
- Empirical studies suggest that deep-learning methods avoid the curse of dimensionality for high-dimensional PDE approximation, but rigorous results for nonlinear PDEs were lacking.
- The main theorem concerns semilinear heat equations with gradient-independent nonlinearities.
- The approximating DNN parameter count grows at most polynomially in the PDE dimension d and reciprocal accuracy ε^-1.
- Under the theorem’s assumptions, networks Ψd,ε satisfy P(Ψd,ε) ≤ ηd^ηε^-η for all dimensions d and accuracies ε ∈ (0,1].
- The proof combines stability results for multilevel Picard approximations with their representation by deep neural networks.
2 A stability result for full history recursive multilevel Picard (MLP) approximations
This section establishes moment and stability estimates for solutions and recursive multilevel Picard approximations under Lipschitz and growth assumptions. These estimates support controlled perturbation and approximation analyses.
- The setting assumes continuous nonlinearities and terminal data, a Lipschitz condition on f, polynomial growth, and controlled terminal-data approximation.
- The probabilistic construction uses Brownian motions, independent uniform random variables, and randomized intermediate times Uθ = t + (T − t)uθ.
- Lemma 2.2 provides q-th moment bounds for the exact solution under the stated setting.
- Lemma 2.3 establishes stability estimates for solutions when the nonlinearities and terminal conditions are perturbed.
3 Deep neural network representations for MLP approximations
The section develops the neural-network framework needed to represent multilevel Picard approximations. Composition, summation, affine transformations, and rectified activations preserve representability while enabling explicit architectural control.
- The central result shows that multilevel Picard approximations can be represented by deep neural networks.
- The activation Ad applies the componentwise rectified map Ad(x) = (max{x1, 0}, …, max{xd, 0}).
- Composition of represented functions is implemented by concatenating their layer-dimension descriptions through the ⊙ operation.
- A network Φ represents a function R(Φ), while D(Φ) records its layer dimensions.
- These closure properties support network representations of affine transformations and the full multilevel Picard constructions.
4 Deep neural network approximations for PDEs
The section constructs deep neural network approximations for semilinear heat-equation solutions and proves that their parameter counts grow polynomially with dimension and inverse accuracy. The result is obtained by combining approximation assumptions, multilevel Picard constructions, and complexity bounds.
- Network construction: The neural-network model uses ReLU activations A_d(x) = (max{x_1, 0}, …, max{x_d, 0}) and represents network architectures through layer-width vectors D(Φ).The network class specifies affine transformations followed by coordinatewise ReLU activation and tracks parameter counts through P and dimension through D.
- Problem setting: The considered PDEs have unique continuous solutions u_d on [0,T] × R^d with polynomial growth and terminal data g_d.The solutions satisfy a semilinear heat equation with globally Lipschitz nonlinearity f and terminal condition u_d(T,x)=g_d(x).
- Complexity result: P(Ψ_{d,ε}) ≤ C_γ d^η ε^−(4+2α+β+γ) for all d ∈ N and ε,γ ∈ (0,1], establishing polynomial parameter growth.The exponent depends on approximation-growth parameters α, β and the arbitrarily selected γ, while η and C_γ are independent of d and ε.
- General rate: Under broader polynomial approximation assumptions, there exist networks satisfying P(Ψ_{d,ε}) ≤ η d^η ε^−η for all d ∈ N and ε ∈ (0,1].This general-rate corollary applies when the terminal data, their approximating networks, and the PDE solutions satisfy the stated polynomial growth conditions.