Source-linked AI summary
Quantifying total uncertainty in physics-informed neural networks for solving forward and inverse stochastic problems
Dongkun Zhang, Lu Lu, Ling Guo, George Em Karniadakis
TL;DR
The paper addresses the lack of uncertainty quantification in PINNs for stochastic PDE solutions, including uncertainty from random parameters and DNN approximation. It introduces NN-aPC with dropout-based uncertainty and active sensor placement, enabling efficient prediction of new solution or parameter trajectories after training.
Problem
PINNs lack uncertainty quantification for solution randomness and approximation limitations when solving stochastic differential equations.
Method
NN-aPC combines aPC expansions with PINNs to learn stochastic modal functions, while dropout quantifies DNN approximation uncertainty and guides active sensor placement.
Results
After training on existing sensor data, the method predicts new trajectories of the solution or parameter at very small additional computational cost.
Takeaways & Limitations
The approach addresses both forward and stochastic inverse problems using sparse or partial sensor information.
Takeaways & Limitations
Directly using stochastic data as inputs was not chosen because it lacks explicit stochasticity expressions and becomes difficult as random-space dimension grows.
Abstract
from arXiv · showhide
Physics-informed neural networks (PINNs) have recently emerged as an alternative way of solving partial differential equations (PDEs) without the need of building elaborate grids, instead, using a straightforward implementation. In particular, in addition to the deep neural network (DNN) for the solution, a second DNN is considered that represents the residual of the PDE. The residual is then combined with the mismatch in the given data of the solution in order to formulate the loss function. This framework is effective but is lacking uncertainty quantification of the solution due to the inherent randomness in the data or due to the approximation limitations of the DNN architecture. Here, we propose a new method with the objective of endowing the DNN with uncertainty quantification for both sources of uncertainty, i.e., the parametric uncertainty and the approximation uncertainty. We first account for the parametric uncertainty when the parameter in the differential equation is represented as a stochastic process. Multiple DNNs are designed to learn the modal functions of the arbitrary polynomial chaos (aPC) expansion of its solution by using stochastic data from sparse sensors. We can then make predictions from new sensor measurements very efficiently with the trained DNNs. Moreover, we employ dropout to correct the over-fitting and also to quantify the uncertainty of DNNs in approximating the modal functions. We then design an active learning strategy based on the dropout uncertainty to place new sensors in the domain to improve the predictions of DNNs. Several numerical tests are conducted for both the forward and the inverse problems to quantify the effectiveness of PINNs combined with uncertainty quantification. This NN-aPC new paradigm of physics-informed deep learning with uncertainty quantification can be readily applied to other types of stochastic PDEs in multi-dimensions.
1. Introduction
The paper extends PINNs from deterministic to stochastic differential equations, targeting both parametric and DNN approximation uncertainty. It combines aPC, PINNs, dropout, and active learning for forward and inverse problems.
- Data-driven modeling can combine measurements with mathematical models to support inference and reduce required data.
- Existing PINN formulations addressed deterministic differential equations, motivating extensions to stochastic equations with uncertain parameters and data.
- The paper combines aPC with PINNs to learn individual stochastic-solution modes and predict new realizations from few measurements.
- Dropout is used both to regularize DNNs against over-fitting and to quantify uncertainty in their approximations.
- Numerical studies evaluate NN-aPC for forward and inverse stochastic diffusion problems, including active learning driven by dropout uncertainty.
2. Problem Setup
The setup covers forward problems with known stochastic parameters and inverse problems with incomplete parameter information but additional solution data. Both are data-driven through sensor measurements, with the study emphasizing inverse inference of stochastic parameter profiles.
- The stochastic differential equation uses a general differential operator, physical domain, random space, solution, and boundary-condition operator.
- The random parameter k(x; ω) is the source of parametric uncertainty and may be represented by random variables or a random process.
- Forward problem: Forward problems assume the distribution of k is known throughout D and treat u as the quantity of interest.
- Inverse problem: Inverse problems use incomplete information about k, additional knowledge of u, and sensor data to infer the full stochastic profile of k.
- Sensor configurations: Forward setups use boundary u-sensors and unrestricted virtual k-sensors, whereas inverse setups have limited interior u-sensors and limited k-sensors.
- The paper addresses both problem types but focuses more heavily on the inverse problem.
3. Methodology
The methodology extends PINNs to stochastic forward and inverse problems by learning aPC solution modes from sensor data and enforcing the governing equation through residual networks. Dropout supplies approximation uncertainty estimates that also guide active sensor placement.
- 3.1. Physics-informed neural network: A PINN represents the solution with a DNN and enforces observations and governing physics through a residual network constructed by automatic differentiation.The solution surrogate takes spatial coordinates as input, while the residual shares its parameters and is trained toward zero.
- 3.2. Stochastic problems: The stochastic extension uses simultaneous sensor snapshots to infer continuous random processes for both forward and inverse problems.Each snapshot is treated as corresponding to one random instance, despite measurements varying between snapshots.
- 3.2. NN-aPC: NN-aPC reduces the stochastic dimension, constructs an orthonormal polynomial basis from arbitrary random samples, and trains DNNs to approximate the resulting modal functions.The workflow comprises dimension reduction, aPC-basis construction, and mode-specific neural-network training.
- 3.2. NN-aPC: The trained NN-aPC predicts new QoI trajectories and statistics from newly collected sensor data.This enables efficient prediction after training.
- 3.2.3. Learning stochastic modes: Separate networks learn the modes of the solution and stochastic parameter, while residuals are computed by substituting their aPC reconstructions into the differential equation.The residual network is designed to reflect the aPC expansion rather than operate as an uninterpretable black box.
- 3.3. Dropout for uncertainty: Dropout produces stochastic test-time predictions whose Monte Carlo mean and variance estimate DNN approximation uncertainty and guide active learning.The method is planned for uncertainty estimation in NN-aPC and for placing additional sensors.
4. Numerical Examples
Numerical tests show that NN-aPC accurately solves forward and inverse stochastic problems, learns aPC modes, and supports efficient prediction from sensor data. Dropout reduces over-fitting and provides uncertainty estimates that guide active sensor placement, improving inverse predictions.
- Forward problem: stochastic Poisson equation: Sensor measurements are correlated, whereas the induced aPC basis functions are uncorrelated and therefore provide a valid random-space basis.The setup retains six principal random variables representing 99% of stochastic energy.
- Forward problem: stochastic Poisson equation: NN-aPC accurately predicts the mean, standard deviation, and aPC modes of the solution u(x; ω) against references from continuous samples.The trained model recovers test solutions well and requires one forward DNN evaluation for a prediction at any queried location.
- Forward problem: stochastic Poisson equation: Increasing the number of f-sensors improves predicted-solution accuracy, while the tested DNN configuration varies mode-network depth, width, and l2 regularization.The experiments compare averaged relative L2 errors under different network settings and sensor counts.
- Inverse problem: stochastic elliptic equation: The second-order aPC expansion improves lower-order mode accuracy and significantly improves mean, standard-deviation, and common-mode errors for k and u.It can also capture higher-order modes whose magnitudes are much smaller than those of lower-order modes.
- Dropout and active learning: Dropout reduces over-fitting and enables uncertainty-guided active learning: eight added k-sensors reduce k-prediction relative error from more than 5% to less than 1%.Sensors cluster where k has high curvature, and active learning also improves standard-deviation predictions for both k and u.
5. Summary
The paper combines aPC expansions with PINNs to quantify parametric uncertainty in stochastic forward and inverse problems, while dropout estimates approximation uncertainty and guides sensor placement. After training on sensor data, the model predicts new solution or parameter trajectories at very small additional computational cost, with numerical results supporting active learning for sensor placement and data use.
- aPC expansions represent stochastic solutions, while PINNs learn their modal functions from sensor measurements for forward and stochastic inverse problems.The inverse setting uses partial information about both the solution and the stochastic parameter.
- After training on historical sensor data, the model predicts new trajectories of the solution or parameter at very small additional computational cost.
- Dropout estimates PINN approximation uncertainty and supports iterative active learning to place new sensors and improve approximation accuracy.The strategy also improves the use of existing sensor data.
- The approach quantifies both parametric uncertainty from the stochastic equation and approximation uncertainty from PINN training and prediction.
- The aPC approach becomes harder to train as random-space dimensionality increases because it requires high-dimensional DNN outputs.Directly using stochastic data also lacks explicit QoI stochasticity expressions and may require dimension reduction.