Source-linked AI summary
Machine learning approximation algorithms for high-dimensional fully nonlinear partial differential equations and second-order backward stochastic differential equations
Christian Beck, Weinan E, Arnulf Jentzen
TL;DR
High-dimensional fully nonlinear PDEs remain difficult because existing approximation methods often require low dimensions or restrictive PDE structures. The paper proposes the deep 2BSDE method, combining PDE–2BSDE connections, time discretization, neural-network spatial approximation, and stochastic optimization, and evaluates it on high-dimensional examples.
Problem
Existing approximation methods generally require low dimensions or restrictive structures, leaving practical algorithms for high-dimensional fully nonlinear parabolic PDEs unavailable.
Method
The deep 2BSDE method merges a fully nonlinear PDE with a 2BSDE, using time discretization, neural-network spatial approximation, and stochastic gradient descent.
Results
The numerical study evaluates runtime and relative L1-approximation errors for a 100-dimensional Black-Scholes-Barenblatt equation.
Takeaways & Limitations
The method extends earlier semilinear approaches to fully nonlinear PDEs and nonlinear expectations.
Takeaways & Limitations
The framework is supported only under suitable further assumptions, with sufficiently large discretization and network parameters and sufficiently small learning rate.
Abstract
from arXiv · showhide
High-dimensional partial differential equations (PDE) appear in a number of models from the financial industry, such as in derivative pricing models, credit valuation adjustment (CVA) models, or portfolio optimization models. The PDEs in such applications are high-dimensional as the dimension corresponds to the number of financial assets in a portfolio. Moreover, such PDEs are often fully nonlinear due to the need to incorporate certain nonlinear phenomena in the model such as default risks, transaction costs, volatility uncertainty (Knightian uncertainty), or trading constraints in the model. Such high-dimensional fully nonlinear PDEs are exceedingly difficult to solve as the computational effort for standard approximation methods grows exponentially with the dimension. In this work we propose a new method for solving high-dimensional fully nonlinear second-order PDEs. Our method can in particular be used to sample from high-dimensional nonlinear expectations. The method is based on (i) a connection between fully nonlinear second-order PDEs and second-order backward stochastic differential equations (2BSDEs), (ii) a merged formulation of the PDE and the 2BSDE problem, (iii) a temporal forward discretization of the 2BSDE and a spatial approximation via deep neural nets, and (iv) a stochastic gradient descent-type optimization procedure. Numerical results obtained using ${\rm T{\small ENSOR}F{\small LOW}}$ in ${\rm P{\small YTHON}}$ illustrate the efficiency and the accuracy of the method in the cases of a $100$-dimensional Black-Scholes-Barenblatt equation, a $100$-dimensional Hamilton-Jacobi-Bellman equation, and a nonlinear expectation of a $ 100 $-dimensional $ G $-Brownian motion.
1 Introduction
High-dimensional fully nonlinear PDEs are important in financial models but remain exceedingly difficult to solve numerically. The paper proposes a deep-learning algorithm based on combining PDE and 2BSDE formulations.
- Motivation: Financial models use nonlinear PDEs for derivative pricing, default-risk models, and portfolio optimization.The nonlinearities arise from effects such as borrowing and lending rates, issuer default risk, and untradable underlyings.
- Research gap: Most existing approximation methods require low dimension or restrictive PDE structures, such as small nonlinearities or semilinearity.The paper states that no practical algorithm was known for high-dimensional fully nonlinear parabolic PDEs.
- Contribution: The proposed method targets fully nonlinear PDEs and nonlinear 2BSDEs, including high-dimensional nonlinear expectations.It is specifically described as enabling sampling from Shige Peng’s nonlinear expectation in high space-dimensions.
- Contribution: The approach connects PDEs with 2BSDEs, merges their formulations, and converts the resulting problem into a neural-network learning task.The loss is based on the error between the prescribed terminal condition and a neural-network forward discretization of the 2BSDE.
- Organization: The article presents a simplified algorithmic framework before extending the method to a general framework supporting more sophisticated machine-learning techniques.The general framework is described as allowing techniques beyond the simplified presentation.
2 Main ideas of the deep 2BSDE method
The deep 2BSDE method approximates fully nonlinear PDE solutions by linking them to 2BSDEs, discretizing time, representing spatial quantities with neural networks, and minimizing a stochastic loss.
- PDE–2BSDE connection: The method begins with a connection between fully nonlinear second-order PDEs and 2BSDE systems.Under suitable smoothness and regularity hypotheses, the PDE solution and its spatial gradient correspond to components of the 2BSDE system.
- Merged formulation: A merged formulation combines the PDE and 2BSDE problems into a single representation used by the algorithm.The merged formulation is derived from the 2BSDE relations and is the basis for subsequent discretization.
- Temporal discretization: The merged system is discretized forward in time on a grid 0 = t_0 < t_1 < ... < t_N = T with sufficiently small mesh size.The forward discretization is developed for the merged PDE–2BSDE system.
- Target quantity and scope: The method approximates u(0, x), and the scalar presentation restricts attention to a terminal-value problem.The authors state that systems of fully nonlinear second-order parabolic PDEs can also be handled, but restrict the work to the scalar case for notational simplicity.
- Spatial approximation: Deep neural networks approximate spatial functions and quantities associated with the discretized 2BSDE.The construction uses rectifier activations and four-layer networks in the described implementations.
- Optimization: The parameters are optimized by stochastic gradient descent-type minimization of a loss function tied to the 2BSDE relation.For sufficiently large discretization and network parameters and sufficiently small learning rate, the iterates are treated as approximations of a local minimum and u(0, ξ).
- Specific framework: In the specific framework, Euler–Maruyama discretizes the system and plain stochastic gradient descent uses a constant learning rate.A more general framework can incorporate batch normalization or the Adam optimizer.
3 The deep 2BSDE method in the general case
The general deep 2BSDE scheme extends the approximation method to more general Itô-process dynamics and machine-learning techniques while retaining the merged PDE–2BSDE formulation. It discretizes time, approximates selected solution components with neural networks, and uses minimization to compute the PDE solution.
- Generalization: The general scheme extends the approximation framework from Brownian motion to more general Itô processes and permits more sophisticated machine-learning techniques.These extensions distinguish the general framework from the simplified scheme in Section 2.
- PDE–2BSDE connection: The method connects fully nonlinear second-order PDEs with 2BSDEs through a merged formulation of their solution systems.The connection identifies Y, Z, Γ, and A with the PDE solution, gradient, Hessian, and related differential term along the forward process.
- Temporal discretization: The approximation uses a forward time discretization on a partition 0 = t0 < t1 < . . . < tN = T with sufficiently small mesh size.The discretization propagates the forward process and approximates the associated 2BSDE quantities at the grid points.
- Spatial approximation: Neural-network functions approximate u(0, x), its gradient, and Hessian-related quantities, while intermediate-time u and gradient functions are not separately approximated.The construction also represents the discretized Y and Z processes through the selected parameterized functions.
- Optimization: A minimization algorithm selects the network parameters, with the general framework allowing optimizers such as Adam and techniques such as batch normalization.The framework therefore includes alternatives to plain stochastic gradient descent.
4 Examples
The examples apply the deep 2BSDE method to high-dimensional nonlinear PDEs, including Allen-Cahn, Black-Scholes-Barenblatt, and related benchmark problems. Experiments report approximation errors, loss functions, and runtimes across independent realizations.
- Examples: The examples use deep 2BSDE frameworks to approximate solutions of Allen-Cahn, Black-Scholes-Barenblatt, Hamilton-Jacobi-Bellman, and G-Brownian-motion problems.The section includes a 20-dimensional Allen-Cahn example and a 100-dimensional Black-Scholes-Barenblatt example.
- 4.1 Allen-Cahn equation: The 20-dimensional Allen-Cahn experiment evaluates relative L1-approximation errors, loss-function statistics, and runtime over 10 independent realizations.The reported quantities are evaluated across iteration counts m from 0 through 5000.
- 4.3 Black-Scholes-Barenblatt equation: The Black-Scholes-Barenblatt setup uses d=100, T=1, N=20, ε=10^-8, and alternating initial coordinates ξ=(1,1/2,1,1/2,...) .The model parameters include r=5/100, σmax=4/10, σmin=1/10, and σc=4/10.
- 4.3 Black-Scholes-Barenblatt equation: The 100-dimensional Black-Scholes-Barenblatt experiment evaluates estimates, relative L1 errors, empirical losses, and runtimes over 10 independent realizations.The iteration counts extend to m=400, and the reference value u(0,ξ) is set to 77.1049 from Lemma 4.2 for error calculations.
A.1 A Python code for the deep 2BSDE method used in Subsection 4.1
The code implements the deep 2BSDE method for the Allen-Cahn equation, combining neural-network approximations, forward discretization, and stochastic-gradient training.
- Python code 1 is a simplified implementation of the deep 2BSDE method for the Allen-Cahn equation.
- The implementation defines the PDE right-hand side and terminal-condition-related functions used by the solver.
- Time-dependent neural networks approximate the required solution components, including network outputs associated with Z and Gamma.
- The forward scheme updates the stochastic processes over discretized time steps using simulated Brownian increments and neural-network quantities.
- Training uses stochastic gradient descent with an exponentially decaying learning-rate schedule and records losses, estimates, learning rates, and runtimes.
- Python code 1 uses plain stochastic gradient descent and does not use batch normalization.
A.2 A Matlab code for the Branching diffusion method used in Subsection 4.1
The Matlab implementation provides a branching diffusion solver used to calculate the Allen-Cahn reference value and its empirical standard deviation.
- The solver evaluates branching processes with specified drift, volatility, branching, probability, time, state, terminal, and payoff inputs.
- The Monte Carlo branching evaluation returns a result and standard deviation for the computed PDE value.
- The branching process updates the state using drift, volatility, and a Gaussian random increment over a time step.
- Matlab code 2 implements the Branching diffusion method used in Subsection 4.1.
A.3 A Python code for the deep 2BSDE method used in Subsection 4.3
The code implements the deep 2BSDE method for the 100-dimensional Black-Scholes-Barenblatt equation with batch normalization and neural-network approximations.
- Python code 3 is the deep 2BSDE implementation used in Subsection 4.3 for the Black-Scholes-Barenblatt equation.
- The implementation defines coefficient and driver functions for the Black-Scholes-Barenblatt dynamics and PDE.
- Time-dependent neural networks approximate solution quantities, with separate handling for Gamma-related outputs and batch-normalized hidden layers.
- The forward discretization updates X and Y across time steps and uses Gamma-based terms in the stochastic updates.
- The training code computes gradients, applies training operations, tracks losses and Y0 values, and records learning rates and runtimes.
A.4 A Matlab code for the classical Monte Carlo method used in Subsection 4.4
The appendix identifies a slightly modified Matlab implementation of the classical Monte Carlo method used in Subsection 4.4.
- The code is described as a slightly modified version of the Matlab code in E, Han, and Jentzen [33, Subsection 6.3].
- Matlab code 4 is the classical Monte Carlo implementation used in Subsection 4.4.
A.5 A Matlab code for the finite differences method used in Subsection 4.6
This appendix provides Matlab code for the finite differences method used in Subsection 4.6, including a GBrownianMotion function and matrix constructions. The implementation defines a nonlinear driver and builds tridiagonal matrix components.
- The code is inspired by Matlab code from E et al., specifically MATLAB code 7 in Section 3.
- The code defines a GBrownianMotion function for the numerical implementation.
- The nonlinear driver uses a condition on gamma and sigma_max within its computational expression.
- The implementation constructs tridiagonal matrices with diagonal and off-diagonal entries involving ones and zeros.
- The appendix identifies the implementation as Matlab code for the finite differences method used in Subsection 4.6.