Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks

Sharmila Karumuri; Rohit Tripathy; Ilias Bilionis; Jitesh Panchal

Source-linked AI summary

Simulator-free Solution of High-Dimensional Stochastic Elliptic Partial Differential Equations using Deep Neural Networks

Sharmila Karumuri, Rohit Tripathy, Ilias Bilionis, Jitesh Panchal

arXiv:1902.05200v3Feb 14, 2019physics.data-anphysics.comp-ph

TL;DR

High-dimensional stochastic inputs make surrogate modeling difficult as input dimensions grow. The paper parameterizes elliptic SPDE solutions with a ResNet trained by a variational energy-based loss, avoiding a deterministic forward solver, and reports low solution and mean-estimation errors across tasks.

Problem
Traditional surrogate models become difficult to use as the number of input dimensions increases, limiting reliable prediction for high-dimensional problems.
Method
The method learns elliptic SPDE solutions with a deep residual network using a loss derived from the variational principle of the analogous deterministic PDE.
Results
Solution errors ranged from 3.86–5.3% for a 2D SBVP, while mean estimates had smaller than 1.35% relative L2 error across tasks.
Takeaways & Limitations
The approach provides a solver-free technique for uncertainty propagation in elliptic SPDEs without requiring a deterministic forward solver.
Takeaways & Limitations
The discussion notes that existing methods recover only linear manifolds in the input space, marking a limitation of the surrounding dimensionality-reduction approach.

Abstract

from arXiv · show

Stochastic partial differential equations (SPDEs) are ubiquitous in engineering and computational sciences. The stochasticity arises as a consequence of uncertainty in input parameters, constitutive relations, initial/boundary conditions, etc. Because of these functional uncertainties, the stochastic parameter space is often high-dimensional, requiring hundreds, or even thousands, of parameters to describe it. This poses an insurmountable challenge to response surface modeling since the number of forward model evaluations needed to construct an accurate surrogate grows exponentially with the dimension of the uncertain parameter space; a phenomenon referred to as the \textit{curse of dimensionality}. State-of-the-art methods for high-dimensional uncertainty propagation seek to alleviate the curse of dimensionality by performing dimensionality reduction in the uncertain parameter space. However, one still needs to perform forward model evaluations that potentially carry a very high computational burden. We propose a novel methodology for high-dimensional uncertainty propagation of elliptic SPDEs which lifts the requirement for a deterministic forward solver. Our approach is as follows. We parameterize the solution of the elliptic SPDE using a deep residual network (ResNet). In a departure from the traditional squared residual (SR) based loss function for training the ResNet, we introduce a novel physics-informed loss function derived from variational principles. Specifically, our loss function is the expectation of the energy functional of the PDE over the stochastic variables. We demonstrate our solver-free approach through various examples where the elliptic SPDE is subjected to different types of high-dimensional input uncertainties. Also, we solve high-dimensional uncertainty propagation and inverse problems.

1. Introduction

High-dimensional stochastic uncertainty makes surrogate construction expensive because response-surface evaluations grow rapidly with input dimension. The paper proposes a solver-free neural approach using variational physics-informed training for elliptic SPDEs.

Existing approaches: Monte Carlo convergence is independent of stochastic dimension, but its large sample requirements make uncertainty propagation expensive when each PDE evaluation is costly.The resulting computational burden can make Monte Carlo impractical for sophisticated PDE solvers.
Motivation: Traditional surrogate models do not scale successfully to high stochastic dimensions because response-surface construction suffers from the curse of dimensionality.Increasing input dimensionality requires exponentially more forward-model evaluations to preserve accuracy.
Existing approaches: Dimensionality-reduction methods such as KLE, PCA, and active subspaces reduce stochastic complexity but may retain restrictive linear or output-information assumptions.KLE and PCA recover only linear input manifolds and do not use model-output information.
Contribution: The paper removes the deterministic PDE-solver requirement by parameterizing elliptic SPDE solutions with a deep neural network and training it through a variational energy-functional loss.This replaces squared-residual minimization with an expectation-based physics-informed objective.
Contribution: The authors derive a variational principle and optimization objective, then demonstrate performance for high-dimensional uncertainty propagation, including out-of-sample examples and non-trivial random-field mixtures.They also compare the energy functional with integrated squared residuals and assess response-surface accuracy.

2. Methodology

The methodology formulates elliptic SPDEs variationally and represents their stochastic solutions in an appropriate function space. Under stated regularity and ellipticity assumptions, the solution is unique and minimizes an expected functional.

Problem formulation: The target is a stochastic process solving an elliptic PDE with random diffusion, reaction, forcing, and boundary data.The governing equation is posed over a spatial domain and probability space with Dirichlet and Neumann boundary conditions.
Variational formulation: The paper derives a variational formulation for the stochastic boundary-value problem as its main theoretical result.The theorem is stated under integrability, uniform ellipticity, and positivity assumptions.
Variational formulation: The stochastic PDE has a unique solution that minimizes the associated functional, with expectation taken over the stochastic parameter.The proof establishes uniqueness of the minimizer, equivalence with the weak formulation, and the converse stationary-point property.
Proof strategy: Coercivity and the Lax–Milgram theorem provide the existence and uniqueness argument for the variational minimum.The first variation yields the weak problem, and the proof concludes that the functional has a unique minimum.
Scope: The theorem can also be extended to the case C = 0 under an additional domain-dependent inequality.This extension is presented as a remark rather than the primary assumption set.

the H1 seminorm and the H1 norm are equivalent in H1

The method learns elliptic SPDE solutions from high-dimensional random-field inputs with a ResNet, using a variational energy functional and boundary-aware parameterization.

The random field is represented by a finite-dimensional vector Ξ(ω) formed by flattening or discretizing its pixel values.
The solution U(x, Ξ(ω)) is obtained by minimizing the energy functional J, recasting ResNet training as stochastic minimization.
The network approximates N(x, Ξ(ω)) from spatial coordinates and random-field inputs, producing the scalar output N(x, Ξ(ω); θ).
The ResNet uses an initial dense layer, K residual blocks with L dense layers, and a final linear layer producing one value.
Dirichlet conditions are imposed automatically through U(x, Ξ(ω)) = B(x) + K(x)N(x, Ξ(ω)), while Neumann conditions remain in the functional.
The resulting optimization is nonconvex in θ even though the original functional J[U] is convex.

3. Numerical examples

The numerical examples evaluate ResNet approximators for elliptic SBVPs using energy-functional and integrated squared-residual losses, across diverse input fields, out-of-distribution cases, uncertainty propagation, and inverse problems.

3.2. Comparison of Energy Functional (EF) loss function vs integrated Squared Residual (SR) loss function with a 1D SBVP: Energy-functional training produced significantly smaller median and interquartile relative RMS errors than integrated squared-residual training at every tested length-scale.As length-scale decreased, integrated squared-residual training failed and learned only the mean profile.
3.2. Comparison of Energy Functional (EF) loss function vs integrated Squared Residual (SR) loss function with a 1D SBVP: For 1D inputs with length-scale 0.03, EF-trained predictions had relative L2 error below 0.05 and R2 scores near 0.99 against FVM solutions.The predictions closely matched FVM solutions for the tested realizations.
3.3. Stochastic boundary value problem in 2D: A single DNN achieved 4.56% relative RMS error across four input datasets, except that channelized fields remained poorly predicted.The result indicates that one network can learn responses across multiple field types when supplied sufficient training data.
3.3. Stochastic boundary value problem in 2D: The ResNet generalized to unseen length-scales and out-of-distribution inputs, but accuracy decreased for very fine scales and channelized fields.For stratified out-of-distribution inputs, the model captured jumps at the stratification.
3.4. Uncertainty propagation and inverse problems: The trained DNNs were applied to uncertainty propagation and inverse problems, with comparisons of output statistics, errors, and computational times against FVM solutions.Posterior input-field samples produced solution responses representing distributions conditional on noisy observations.

4. Conclusion

The paper develops a solver-free DNN methodology for high-dimensional elliptic SPDEs by minimizing a variational energy functional, achieving low errors and out-of-distribution generalization while retaining several implementation and analysis limitations.

The approach does not require a classical numerical solver and was benchmarked across SBVPs with multiple high-dimensional random input fields.
The methodology solves elliptic SPDEs by recasting them variationally, minimizing the energy functional over DNNs that satisfy Dirichlet conditions, and optimizing its stochastic expectation.An unbiased estimator enables stochastic gradient descent while preserving the same solution.
Relative root mean square error across 2D SBVP test datasets was 3.86–5.3%, and trained DNNs generalized to stratified fields and unseen Gaussian random-field length-scales.The reported out-of-distribution inputs included lower and higher length-scales than those used in training.
The DNNs estimated uncertainty-propagation means with less than 1.35% relative L2 error and variances with less than 10% relative L2 error.
The authors identify training time, manual Dirichlet-boundary construction, stochastic-estimator variance, and limited analytical understanding of the energy loss as areas for improvement.They suggest multi-GPU parallelization, automatic boundary enforcement, lower-variance estimators, and further analysis.