Source-linked AI summary
Hidden Physics Models: Machine Learning of Nonlinear Partial Differential Equations
Maziar Raissi, George Em Karniadakis
TL;DR
The paper addresses learning nonlinear partial differential equations when scientific data are limited and expensive to acquire. It introduces hidden physics models that combine Gaussian processes with physics-informed covariance functions to learn PDE parameters from noisy observations. Across benchmark equations, the approach identifies parameters from very small subsets of available data, while its main limitation is cubic scaling with the number of training points.
Problem
Scientific experiments often provide small, noisy datasets, motivating data-efficient methods for learning, system identification, and discovery of nonlinear partial differential equations.
Method
Hidden physics models place Gaussian-process priors on latent solutions and encode governing physical laws in covariance functions whose parameters are learned from noisy snapshot data.
Results
Using only 140 of 25856 Burgers’ data points, 220 of 102912 KdV data points, and 500 points from two Navier–Stokes snapshots, the method identifies the correct parameter values up to relatively good accuracy.
Takeaways & Limitations
The framework demonstrates that explicitly encoding physical laws in Gaussian-process covariance functions can support PDE parameter identification from small data across multiple benchmark equations.
Takeaways & Limitations
The paper’s main limitation is cubic scaling with the total number of training data points, although recursive Kalman updates, variational inference, and parametric Gaussian processes are suggested as possible remedies.
Abstract
from arXiv · showhide
While there is currently a lot of enthusiasm about "big data", useful data is usually "small" and expensive to acquire. In this paper, we present a new paradigm of learning partial differential equations from {\em small} data. In particular, we introduce \emph{hidden physics models}, which are essentially data-efficient learning machines capable of leveraging the underlying laws of physics, expressed by time dependent and nonlinear partial differential equations, to extract patterns from high-dimensional data generated from experiments. The proposed methodology may be applied to the problem of learning, system identification, or data-driven discovery of partial differential equations. Our framework relies on Gaussian processes, a powerful tool for probabilistic inference over functions, that enables us to strike a balance between model complexity and data fitting. The effectiveness of the proposed approach is demonstrated through a variety of canonical problems, spanning a number of scientific domains, including the Navier-Stokes, Schrödinger, Kuramoto-Sivashinsky, and time dependent linear fractional equations. The methodology provides a promising new direction for harnessing the long-standing developments of classical methods in applied mathematics and mathematical physics to design learning machines with the ability to operate in complex domains without requiring large quantities of data.
1. Introduction
Scientific data may be abundant overall, but obtaining high-quality, error-free data for complex physical systems remains difficult and analyzing large datasets is costly.
- Scientific experiments can produce data affected by bifurcations, instabilities, hysteresis, and irreversible responses.
- Purely data-driven machine-learning approaches often achieve state-of-the-art performance only when trained with large amounts of data.
2. Problem Setup
The paper formulates PDE learning as estimating unknown operator parameters from two nearby noisy solution snapshots, assuming the time gap is small enough for backward Euler discretization.
- PDE learning asks which parameters λ best describe observed data, distinct from inferring hidden states when parameters are fixed.
- The setup observes two snapshots of the latent solution at consecutive times, separated by Δt = tn − tn−1.
- The main assumption is that the snapshot interval is small enough to apply the backward Euler time-stepping scheme.
- The nonlinear operator on the left-hand side is approximated by a linear operator before constructing the discretized equation.
3. The Basic Model
The hidden physics model places a Gaussian-process prior on the latent function and encodes the governing differential operator directly in multi-output covariance functions.
- A Gaussian-process prior is placed over the latent function hn(x), using the analytical closure of Gaussian outputs under linear systems.
- The framework uses squared exponential covariance functions with hyperparameters θ = (γ, w1, · · ·, wD).
- The Gaussian-process prior and the discretized operator capture the structure of Lλ in the resulting multi-output Gaussian process.
- Gaussian processes provide flexible, analytically tractable function priors and can be interpreted as infinite-width one-layer Bayesian neural networks.
- The squared exponential covariance function implies smooth approximations, while nonlinear input warping can represent more complex function classes such as discontinuities.
- The multi-output Gaussian process is called a hidden physics model because its covariance matrix explicitly encodes the governing physical laws.
4. Learning
The model learns operator and covariance parameters from noisy snapshot data by minimizing negative log marginal likelihood, which balances data fit against model complexity.
- The operator parameters λ and covariance hyperparameters θ are learned from two noisy snapshots using L-BFGS to minimize negative log marginal likelihood.
- The covariance matrix combines four cross-time covariance blocks for the two observed snapshots.
- The noise model assumes independent Gaussian observation errors with covariance σ2I, and σ2 is learned during optimization.
- Negative log marginal likelihood balances training-data fit through hTK−1h against model complexity through the log-determinant term log |K|.
5. Results
Across canonical PDEs, hidden physics models identify governing parameters from sparse snapshots and noisy measurements by encoding physical laws in Gaussian-process covariance functions. Performance improves with more data, less noise, and smaller snapshot gaps, while the framework also handles velocity-only Navier–Stokes data and fractional operators.
- Scope: The framework is applied to canonical PDEs spanning Navier–Stokes, Schrödinger, Kuramoto–Sivashinsky, and fractional equations.The dataset covers multiple scientific domains, and the manuscript makes its data and code publicly available.
- Burgers’ equation: Using 140 of 25,856 data points, the method identifies Burgers’ equation parameters up to relatively good accuracy.The two randomly selected snapshots contain 71 and 69 points, respectively, and are separated by Δt = 0.1.
- Sensitivity: More data, less noise, and a smaller gap Δt between snapshots enhance performance, while increased noise reduces confidence in estimated parameters.The same sensitivity pattern is reported for Burgers’, KdV, and Kuramoto–Sivashinsky experiments.
- KdV equation: Using 220 of 102,912 data points, the method identifies KdV equation parameters up to relatively good accuracy.The two snapshots contain 111 and 109 points, respectively, and are separated by Δt = 0.1.
- Kuramoto–Sivashinsky equation: Using 600 of 257,024 data points, the method identifies Kuramoto–Sivashinsky parameters up to relatively good accuracy.The two snapshots contain 301 and 299 points, respectively, and are separated by Δt = 0.4; the covariance functions explicitly encode the equation’s physical laws.
- Schrödinger and fractional equations: The method identifies nonlinear Schrödinger parameters from 100 data points and identifies a fractional order from two histograms with 100 bins each.For Schrödinger data, the covariance functions encode the nonlinear Schrödinger laws; for fractional equations, the inferred order can take real values between classical advection and diffusion behaviors.
- Navier–Stokes equations: For Navier–Stokes, two velocity snapshots totaling 500 data points identify parameters with relatively good accuracy without pressure or vorticity measurements.The method encodes the Navier–Stokes laws in covariance functions and uses a prior whose samples satisfy the continuity equation.
6. Summary and Discussion
The paper introduces a physics-informed structured learning machine for identifying parametric nonlinear partial differential equations from noisy, data-scarce observations. Its applications use much smaller datasets than related sparse-regression work, while cubic scaling remains a limitation addressed by proposed inference alternatives.
- The framework embeds underlying physical laws into a structured learning machine for data-efficient identification of general parametric nonlinear partial differential equations from noisy data.
- The approach was demonstrated on benchmark problems with different attributes, supporting its generality for nonlinear partial differential equation identification.
- Cubic scaling with the total number of training data points is identified as the work’s most pressing limitation.
- Recursive Kalman updates, variational inference, and parametric Gaussian processes are proposed as possible ways to address the cubic-scaling limitation.
- Compared with prior work using thousands of spatio-temporal measurements, this study uses hundreds of points and two system snapshots.