PaperlayerView exact revision on arXiv

Source-linked AI summary

Adversarial Uncertainty Quantification in Physics-Informed Neural Networks

Yibo Yang, Paris Perdikaris

arXiv:1811.04026v1stat.MLcs.LGphysics.comp-ph

TL;DR

The paper targets uncertainty quantification for nonlinear physical systems when observations are noisy, incomplete, costly, and scarce. It combines latent-variable probabilistic physics-informed neural networks with adversarial inference and PDE constraints, demonstrating uncertainty propagation and constitutive-law discovery while noting adversarial tuning as a limitation.

  • Problem

    Uncertainty quantification is difficult for physical systems with costly, scarce, noisy, or incomplete data, especially when predictions must respect known governing laws.

  • Method

    The framework uses latent-variable probabilistic physics-informed neural networks trained by adversarial inference, with PDE residuals constraining generated predictions.

  • Results

    The approach demonstrates uncertainty propagation for nonlinear conservation laws and constitutive-law discovery for porous-media flow from noisy data.

  • Takeaways & Limitations

    PDE constraints regularize probabilistic deep-learning models for physical systems and support uncertainty characterization without repeatedly sampling expensive experiments or numerical simulators.

  • Takeaways & Limitations

    Adversarial inference still requires careful tuning of the generator–discriminator interplay, and robust training remains an open research problem.

Abstract

from arXiv · show

We present a deep learning framework for quantifying and propagating uncertainty in systems governed by non-linear differential equations using physics-informed neural networks. Specifically, we employ latent variable models to construct probabilistic representations for the system states, and put forth an adversarial inference procedure for training them on data, while constraining their predictions to satisfy given physical laws expressed by partial differential equations. Such physics-informed constraints provide a regularization mechanism for effectively training deep generative models as surrogates of physical systems in which the cost of data acquisition is high, and training data-sets are typically small. This provides a flexible framework for characterizing uncertainty in the outputs of physical systems due to randomness in their inputs or noise in their observations that entirely bypasses the need for repeatedly sampling expensive experiments or numerical simulators. We demonstrate the effectiveness of our approach through a series of examples involving uncertainty propagation in non-linear conservation laws, and the discovery of constitutive laws for flow through porous media directly from noisy data.

1. Introduction

The paper addresses uncertainty quantification for physical systems where data are costly and scarce, combining machine learning with governing PDE constraints. It proposes probabilistic physics-informed neural networks trained through variational inference to characterize uncertainty in partially observed nonlinear systems.

  • Quality, error-free experimental and simulation data are often prohibitively costly to obtain, making purely data-driven learning difficult in data-limited physical systems.
  • Purely data-driven machine learning methods may lack robustness, convergence guarantees, and quantified prediction uncertainty.
  • The proposed framework combines scalable machine-learning inference with strong prior information encoded by governing partial differential equations.
  • Probabilistic physics-informed neural networks provide posterior characterizations of uncertainty associated with predicted physical-system outputs.
  • A variational inference framework trains the models directly from noisy input/output data to predict partially observed nonlinear dynamical systems with quantified uncertainty.
  • The paper demonstrates the approach through uncertainty propagation in nonlinear conservation laws and constitutive-law discovery for porous-media flow.

2.1. Physics-informed neural networks

Physics-informed neural networks represent system states with differentiable neural networks and impose governing PDEs through residual-based soft constraints. Training combines scattered observations with collocation points to recover parameters while approximately encoding physical properties.

  • Physics-informed neural networks represent u(x, t) with a parameterized neural network fθ(x, t) and define a PDE residual using automatic differentiation.
  • The PDE residual is formed as rθ(x, t) := ∂fθ(x, t)/∂t + Nxfθ(x, t), where Nx is a nonlinear differential operator.
  • Training uses scattered observations of u(x, t) together with collocation points whose target residuals are zero.
  • The PDE residual enters the likelihood as a soft penalty constraint, regularizing learning from small data sets.
  • Predictions are encouraged to inherit physical properties imposed by the PDE, including conservation, invariance, and symmetries.

2.2. Probabilistic physics-informed neural networks

The paper extends physics-informed neural networks with latent-variable probabilistic representations for deterministic and stochastic physical fields. PDE constraints regularize the admissible solution space, supporting uncertainty quantification and learning in small-data regimes.

  • The probabilistic formulation models conditional system states as p(u|x, t, z), with random latent variables z, subject to the governing PDE.
  • The formulation covers deterministic and stochastic problems in which u(x, t) may be a multivariate field.
  • Domain-knowledge constraints limit admissible conditional distributions by discarding solutions that violate physical principles such as mass conservation.
  • This constrained probabilistic formulation enables training deep probabilistic models in small-data regimes and provides complete uncertainty characterizations with a-posteriori error estimates.
  • Physics-informed latent variables may support physically relevant low-dimensional representations for nonlinear model-order reduction and coarse-graining.

2.3. Adversarial inference for joint distribution matching

The framework trains a latent-variable generative model by matching generated and observed joint distributions through adversarial inference, while physics-informed penalties constrain generated states. Its objectives combine density-ratio estimation, entropy regularization, and PDE residual minimization, but training remains sensitive to model capacity and optimization choices.

  • Joint distribution matching: Reverse KL divergence matches the generated joint distribution pθ(x, t, u) to the observed distribution q(x, t, u), encouraging corresponding marginal and conditional distributions to match.The objective also balances entropy maximization against penalties for non-overlapping supports, limiting mode collapse when generated support remains within the observed support.
  • Density-ratio estimation: A binary classifier estimates the density ratio between generated and observed samples, enabling reverse-KL estimation directly from samples without specifying pθ analytically.Classifier probabilities for the two sample classes provide the ratio used in the adversarial inference procedure.
  • Entropy regularization: The latent inference model qφ(z|x, t, u) variationally approximates the intractable posterior and supplies an entropy bound for optimizing the generative model.With deterministic generation u=fθ(x, t, z), the entropy construction uses the latent-variable representation and an auxiliary inference model.
  • Adversarial training objective: The discriminator classifies generated versus observed samples, while the generator produces samples that fool the discriminator and the encoder regularizes training and encourages latent cycle-consistency.The encoder is also described as supporting disentangled, interpretable low-dimensional representations of observed data.
  • Physics-informed constraints: Physics-informed training extends the generator objective with PDE residual penalties evaluated at collocation points so generated samples satisfy the governing constraint.The residual is treated deterministically and penalized through a mean-square loss, while the same framework learns from scattered, potentially noisy observations.
  • Advantages and caveats: Robust uncertainty estimates and precise predictions can be obtained with enough PDE collocation points by tuning discriminator and generator capacities.Training dynamics require a discriminator with sufficient but not excessive capacity; adversarial optimization is sensitive to network capacity and update frequency.

3. Results

The experiments evaluate physics-informed generative models for uncertainty propagation in nonlinear differential equations, including deterministic and noisy settings. Results cover a pedagogical nonlinear equation, Burgers equation, and constitutive-law discovery for porous-media flow.

  • 3.1. A pedagogical example: The pedagogical example represents solutions probabilistically with a latent-variable generator constrained by the nonlinear differential equation.The model uses generator, encoder, and discriminator networks to represent pθ(u|x, z).
  • 3.1. A pedagogical example: 1.36 · 10^-3 relative L2 error was obtained for deterministic boundary data, with predictive variance serving as an a-posteriori error estimate.The deterministic case uses 20 realizations per boundary point and 100 collocation points.
  • 3.1. A pedagogical example: For random boundary conditions with 5% Gaussian uncorrelated noise, predictive uncertainty reflects both boundary randomness and epistemic neural-network uncertainty.The predictions are compared with a reference mean from 2,000 Monte Carlo samples, and marginal densities are examined at x = −0.5 and x = 0.5.
  • 3.2. Burgers equation: The Burgers-equation study tests sensitivity to initialization, data and collocation counts, architecture, and adversarial training, with additional studies reported in Appendix A.The equation is strongly nonlinear and develops shock discontinuities in finite time.
  • 3.2. Burgers equation: 4.1·10^-2 relative L2 error was measured for noise-free Burgers data, where predictive uncertainty estimates neural-network approximation error.Training used 150 input/output pairs and 10,000 collocation points.
  • 3.2. Burgers equation: With non-additive, non-Gaussian noisy initial conditions, the model captures propagated spatio-temporal uncertainty, which concentrates around the shock.The generative model provides a complete probabilistic characterization of the resulting non-Gaussian statistics.
  • 3.3. Discovery of constitutive laws for flow through porous media: For porous-media flow, the model predicts an unknown state-dependent diffusion coefficient K(u) from measurements of u(x), without observing data on K(u).Predictions include uncertainty from measurement noise and neural-network approximation error and are compared with STOMP reference solutions.

4. Conclusions

The paper presents probabilistic physics-informed neural networks that approximate conditional probability densities while approximately satisfying PDE constraints, and trains them from noisy, incomplete measurements. The adversarial framework supports uncertainty propagation and probabilistic computation in small-data regimes, but requires careful generator–discriminator tuning.

  • Probabilistic physics-informed neural networks approximate arbitrary conditional probability densities while generating samples that approximately satisfy given partial differential equations.
  • A regularized adversarial inference framework enables end-to-end training from noisy and incomplete measurements.
  • Latent variables capture uncertainty in system inputs and outputs and may support probabilistic model-order reduction for complex systems.
  • The developments support probabilistic computations, small-data generative modeling, complex noise handling, and uncertainty propagation without repeated experiments or numerical simulations.
  • The adversarial framework depends on carefully tuning the interplay between generator and discriminator networks, remaining a known limitation of adversarial algorithms.

Appendix A. Sensitivity studies

The sensitivity studies systematically examine how predictions respond to initialization, data and collocation counts, network architecture, and adversarial training using nonlinear Burgers’ equations as a prototype.

  • The studies quantify prediction sensitivity to neural network initialization, training and collocation points, architecture, and adversarial training procedure.
  • All sensitivity analyses use the nonlinear Burgers problem as a prototype.

Appendix A.1. Sensitivity with respect to the neural network initialization

The initialization study evaluates a fixed generator and discriminator architecture on noise-free data, with reported prediction-error uncertainty summarized as an interval.

  • The initialization study uses noise-free data with Nu = 150 training points and Nr = 10000 collocation points.
  • The generator has 4 hidden layers with 50 neurons each, while the discriminator has 3 hidden layers with 50 neurons each and hyperbolic tangent activation.
  • The reported relative L2 error is bounded by [4.7 × 10^-2 − 1.3 × 10^-2, 4.7 × 10^-2 + 1.3 × 10^-2].
  • The appendix identifies a separate study of sensitivity to the total numbers of training and collocation points Nu and Nr.

Appendix A.3. Sensitivity with respect to the neural network architecture

The architecture study varies generator, discriminator, and encoder designs while fixing the data size and coordinating discriminator depth with generator depth. The reported predictions are accurate, consistent with PDE-residual constraints regularizing training against over-fitting.

  • The architecture study varies the generator, discriminator, and encoder architectures using Nu = 150 and Nr = 10000 noise-free training and collocation points.
  • The discriminator is always assigned one fewer layer than the generator.
  • Table A.3 reports relative L2 prediction errors for feed-forward architectures across the generator, encoder, and discriminator.
  • Table A.4 reports relative L2 error for different generator and discriminator training counts within each epoch.
  • The reported accurate predictions indicate that the physics-informed PDE-residual constraint can regularize training and safeguard against over-fitting.

Appendix A.4. Sensitivity with respect to the adversarial training procedure

The study tests how the number of generator and discriminator training steps affects adversarial training performance, measured by relative L2 prediction error.

  • The experiment varies generator steps Kg and discriminator steps Kd within each stochastic gradient descent iteration.The number of noise-free training data is fixed at Nu = 150 and Nr = 10000, with the neural network architecture held constant.
  • Performance is evaluated using relative L2 prediction error across the tested adversarial training configurations.The results are reported in table A.4.
Loading 1811.04026v1…