Source-linked AI summary
Deep splitting method for parabolic PDEs
Christian Beck, Sebastian Becker, Patrick Cheridito, Arnulf Jentzen, Ariel Neufeld
TL;DR
High-dimensional nonlinear parabolic PDEs are difficult to solve numerically because standard methods scale poorly with dimension. The paper combines operator splitting, local Feynman–Kac representations, and deep learning to create smaller recursive learning problems, producing accurate results across several PDE classes in high dimensions.
Problem
Nonlinear parabolic PDEs become difficult to solve numerically at large d, where classical methods have exponentially growing complexity and linear Monte Carlo approaches do not directly cover nonlinear cases.
Method
The method splits the differential operator into linear and nonlinear parts, solves successive short-time approximations using Feynman–Kac representations, and approximates the resulting minimization problems with neural networks.
Results
The method produced accurate results for HJB, nonlinear Black–Scholes, Allen–Cahn-type, semilinear heat, and sine-Gordon-type equations in high dimensions with short runtimes.
Takeaways & Limitations
Decomposing the PDE approximation into smaller learning problems supports application to extremely high-dimensional nonlinear parabolic PDEs.
Takeaways & Limitations
The method relies on simplifying regularity assumptions, and its comparisons note that deep-learning methods require hyper-parameter fine-tuning for good results.
Abstract
from arXiv · showhide
In this paper we introduce a numerical method for nonlinear parabolic PDEs that combines operator splitting with deep learning. It divides the PDE approximation problem into a sequence of separate learning problems. Since the computational graph for each of the subproblems is comparatively small, the approach can handle extremely high-dimensional PDEs. We test the method on different examples from physics, stochastic control and mathematical finance. In all cases, it yields very good results in up to 10,000 dimensions with short run times.
1 Introduction
The paper targets nonlinear parabolic PDEs that become difficult in high dimensions, introducing a deep-learning method that splits the operator and decomposes approximation into smaller learning problems.
- Nonlinear parabolic PDEs arise in physics, engineering, economics, and finance, but typically lack closed-form solutions and require numerical approximation.
- Nonlinearities caused by constraints, frictions, or control make high-dimensional PDEs more challenging than linear heat and Black–Scholes equations.
- Classical finite-difference and finite-element methods work well for d = 1, 2 and 3 but have complexity growing exponentially in d.
- The proposed deep-learning method splits the differential operator into linear and nonlinear parts, choosing the decomposition so the nonlinearity becomes small.
- The method iteratively solves short-time problems using local Feynman–Kac representations and deep-learning approximations of conditional expectations.
- Separate learning problems produce comparatively small computational graphs, enabling application to very high-dimensional PDEs.
2 Derivation of the proposed approximation algorithm
The derivation discretizes time, constructs a splitting-up approximation, and represents each local linear problem through Feynman–Kac expectations under stated regularity and integrability assumptions.
- Temporal discretization: The algorithm partitions [0, T] into N time intervals and uses a splitting-up method to obtain a semi-discrete approximation.
- Temporal discretization: The auxiliary function v approximates u at the grid points t_n under assumptions on the initial gradient, smoothness, and polynomial growth of derivatives.
- Feynman–Kac representation: A diffusion process Y and Itô’s formula yield the local Feynman–Kac representation used in the approximation.
- Feynman–Kac representation: The derivation uses conditional expectations and can be extended from time-independent to time-dependent f, µ, and σ.
2.3 Formulation as recursive minimization problems
The local approximation is recast as recursive minimization of squared-error objectives, with each time-slice function characterized by conditional-expectation L2 minimality.
- The L2-minimality property of conditional expectations justifies the recursive minimization formulation.
2.4 Deep artificial neural network approximations
The method approximates each recursive time-slice function with feedforward neural networks, using parameterized affine layers and an activation function whose gradients support optimization.
- The functions v(t_n, ·) are approximated by parameterized neural networks V_n at each time step.
- The activation function is applied componentwise, with ReLU used in the paper’s numerical examples and weak gradients used for optimization.
- The network architecture uses feedforward layers with d input neurons, k−1 hidden layers of width l, and one output neuron.
2.5 Stochastic gradient descent based minimization
The algorithm computes successive parameter vectors by recursively minimizing quadratic objectives, using stochastic gradient updates driven by simulated auxiliary processes.
- Parameter vectors θ1, θ2, …, θN are computed recursively so that Vn(θn, x) approximates v(tn, x).
- Each minimization starts from the previous parameter vector, with θ0 chosen arbitrarily, such as the zero vector.
- Standard stochastic gradient descent uses a stepsize γ and performs M iterative updates before setting θn to the final iterate.
- The updates use stochastic processes Ym generated from SDEs driven by Brownian motions and independent random initial variables.
2.6 Discretization of the auxiliary stochastic process Y
When the auxiliary SDE processes cannot be simulated exactly, the method replaces them with Euler–Maruyama approximations and incorporates those paths into an implementable training algorithm.
- Euler–Maruyama is used to numerically approximate the auxiliary processes Ym when exact simulation is unavailable.
- The discretization follows the time relation τn = T − tN−n and derives discrete process updates from the continuous SDE representation.
- The resulting approximations satisfy the relevant discrete-time identities at every time index.
- The implementable algorithm initializes parameter processes at zero or from the preceding terminal iterate, then updates them over M samples.
2.7 Description of the algorithm in a special case
The special-case algorithm uses feedforward neural networks, Euler–Maruyama paths, and plain stochastic gradient descent to approximate the PDE solution.
- The special case requires only a weak gradient of the initial condition, without growth conditions, for implementation.
- Feedforward neural networks are used to approximate the PDE solution, with stochastic gradient descent and quadratic loss functions defining the training procedure.
- The auxiliary paths Ym are constructed from independent Brownian motions and independent random variables.
- The parameters are updated using a constant learning rate γ and stochastic processes Θn over the prescribed training iterations.
- The resulting network outputs provide approximations to the PDE solution at the discretized times.
2.8 Description of the algorithm in the general case
The general framework extends the basic method to mini-batches, batch normalization, and sophisticated stochastic optimization while retaining neural-network approximations of the PDE solution.
- The functions Vn^(j,s) are parameterized by the extra indices and used to approximate the PDE solution at discretized times.
- Additional parameters j and s extend the framework to describe mini-batches and batch normalization.
- Quadratic loss functions train the parameterized functions, while their gradients specify the optimization updates.
- The stochastic processes Sn encode running means and standard deviations required for batch normalization.
- The processes Θn and Ξn encode general updating rules specified through functions Ψn, allowing sophisticated stochastic gradient optimization methods.
- The framework is built on Brownian motions and independent random variables used to construct the stochastic approximation scheme.
3 Examples
The deep splitting method is evaluated on five high-dimensional nonlinear PDE classes, using neural-network approximations and stochastic optimization. Across the examples, the experiments examine approximation values, errors, runtimes, dimensionality, and time-step dependence.
- Example classes: The experiments cover HJB, nonlinear Black–Scholes, Allen–Cahn-type, semilinear heat, and sine-Gordon-type equations.These examples span stochastic control, mathematical finance, and nonlinear evolution equations.
- Approximation procedure: The method approximates solutions with feedforward neural networks trained using mini-batches, batch normalization, and Adam optimization.The networks use four layers and ReLU activation, with mini-batches of size Jm = 256.
- Evaluation: The reported estimates include expectations, standard deviations, relative L1-approximation errors against reference values, and average runtimes over 10 independent runs.Reference solutions are obtained using alternative methods including Monte Carlo, deep BSDE, and multilevel Picard approaches.
- Hamilton–Jacobi–Bellman (HJB) equations: The HJB experiments report approximations at the origin across different d, T, and N, including a configuration with d = 10,000.The HJB reference values use a logarithmic transformation followed by standard Monte Carlo.
- Semilinear and related heat equations: For the semilinear heat equation at d = 100 and T = 0.3, the relative L1-error decreases approximately linearly as N increases.The paper also reports tables for Allen–Cahn-type, semilinear heat, and sine-Gordon-type equations across different dimensions.
4 Comparison with other methods
The deep splitting method is compared with deep BSDE and multilevel Picard approaches on high-dimensional PDE approximation. Results are comparable on one sine-Gordon example, while the methods differ in computational organization and practical limitations.
- For the sine-Gordon-type equation with d = 10 and T = 0.3, all three methods yield comparable estimated relative L1-approximation errors.The comparison uses the number of one-dimensional standard normal random variables as the varying resource.
- Deep splitting and deep BSDE require hyper-parameter tuning that depends on the PDE form and parameters.The reported figure reflects particular implementations rather than an implementation-independent ranking.
- The deep splitting method learns approximations across all spatial inputs while decomposing the computation into smaller tasks solved successively.This decomposition can make larger problems manageable even when total random-variable usage is similar to another approach.
- Multilevel Picard has theoretical computational-effort bounds for a target accuracy but becomes impractical for large t because it separately approximates different space-time points.
5 Conclusion
The paper develops a deep-learning numerical method for high-dimensional nonlinear parabolic PDEs by splitting the operator and solving successive linear approximations. Tests across five PDE classes produced accurate high-dimensional results with short run times.
- The method combines operator splitting, deep learning, and the Feynman–Kac formula to solve linearized approximations over small time intervals.This divides the PDE approximation task into smaller problems solved successively.
- The approach was tested on Hamilton–Jacobi–Bellman, nonlinear Black–Scholes, Allen–Cahn-type, semilinear heat, and sine-Gordon-type equations.
- Accurate results in high dimensions with short run times were obtained across all tested equation classes.