Source-linked AI summary
Deep Hidden Physics Models: Deep Learning of Nonlinear Partial Differential Equations
Maziar Raissi
TL;DR
The paper asks how physical laws represented by nonlinear partial differential equations can be learned from high-dimensional, scattered, and potentially noisy observations. It uses neural networks for both the unknown solution and nonlinear dynamics, avoiding numerical differentiation and fixed basis libraries, and evaluates the approach on benchmark equations with reported accurate identification and forecasting. The method also shows generalization limitations on some datasets and remains unsuccessful on a complicated Kuramoto-Sivashinsky problem.
Problem
Discovering physical laws expressed as partial differential equations from scattered space-time data is less well studied, despite the need for predictive mathematical models of physical systems.
Method
The approach represents the unknown solution and nonlinear dynamics with deep neural networks, using automatic differentiation for derivatives and black-box PINNs solvers for learned equations.
Results
Across benchmark problems, the framework accurately learns underlying dynamics and forecasts future states; for KdV, the reported relative L2-error is 6.28e-02.
Takeaways & Limitations
The framework supports discovery of nonlinear PDE dynamics without direct access to temporal or spatial derivatives or commitment to a particular basis-function family.
Takeaways & Limitations
Generalization can fail on unseen datasets, and the approach remains unsuccessful for a more complicated Kuramoto-Sivashinsky solution; more data may help in one reported case.
Abstract
from arXiv · showhide
A long-standing problem at the interface of artificial intelligence and applied mathematics is to devise an algorithm capable of achieving human level or even superhuman proficiency in transforming observed data into predictive mathematical models of the physical world. In the current era of abundance of data and advanced machine learning capabilities, the natural question arises: How can we automatically uncover the underlying laws of physics from high-dimensional data generated from experiments? In this work, we put forth a deep learning approach for discovering nonlinear partial differential equations from scattered and potentially noisy observations in space and time. Specifically, we approximate the unknown solution as well as the nonlinear dynamics by two deep neural networks. The first network acts as a prior on the unknown solution and essentially enables us to avoid numerical differentiations which are inherently ill-conditioned and unstable. The second network represents the nonlinear dynamics and helps us distill the mechanisms that govern the evolution of a given spatiotemporal data-set. We test the effectiveness of our approach for several benchmark problems spanning a number of scientific domains and demonstrate how the proposed framework can help us accurately learn the underlying dynamics and forecast future states of the system. In particular, we study the Burgers', Korteweg-de Vries (KdV), Kuramoto-Sivashinsky, nonlinear Schrödinger, and Navier-Stokes equations.
1. Introduction
The paper addresses the underexplored problem of discovering nonlinear partial differential equations from scattered, potentially noisy space-time observations. It replaces numerical differentiation and fixed candidate libraries with neural-network representations of the solution and nonlinear dynamics.
- Problem and contribution: The framework learns nonlinear functions governing partial differential equations directly from scattered and potentially noisy observations of a solution.It aims to uncover hidden physical laws governing the observed data's evolution.
- Existing approach: Sparse-regression alternatives construct candidate-term dictionaries and estimate coefficients to identify the right-hand-side terms contributing to the dynamics.Their main advantage is interpretability, but they require numerical derivatives and a sufficiently rich library.
- Existing approach: In higher-dimensional inputs or outputs, library-based methods require exponentially more terms and cannot estimate parameters inside nonlinear terms such as sin(αu(x)).The paper identifies these as limitations of the preceding library-based approach.
- Solution representation: A neural network prior represents the unknown solution, allowing derivatives to be evaluated through automatic differentiation instead of unstable numerical differentiation.This removes the need for direct derivative observations and supports noisy, scattered space-time data.
- Dynamics representation: A second deep neural network approximates the nonlinear dynamics N, avoiding commitment to polynomial, sine, or cosine basis functions.The richer approximator addresses cases where a chosen library is insufficient, though it reduces interpretability.
2. Solution methodology
The method represents both the solution and nonlinear dynamics with deep neural networks, trains them against observations and PDE constraints, and uses automatic differentiation to obtain derivatives. The learned black-box dynamics can then be evaluated with physics-informed neural networks as solvers.
- Automatic differentiation computes derivatives of the neural-network solution with respect to time and space using the chain rule.
- The network parameters are learned by minimizing squared error, with one term fitting observations and another enforcing the PDE at collocation points.
- Two deep neural networks approximate the solution u and the nonlinear function N, with approximating N being the work’s stated novelty.
- Training can occur simultaneously for both networks or sequentially by training the solution network first and the dynamics network second.
- Because the learned N is a black-box function, the authors use physics-informed neural networks rather than classical PDE solvers to solve the learned equations.
3. Results
The framework identifies and solves nonlinear partial differential equations across several benchmark systems, including successful extrapolation and cross-dataset tests. Performance depends on noise, derivative order, and whether the learned dynamics are made time- or space-dependent, while a challenging Kuramoto-Sivashinsky case remains unsolved.
- The framework is applied to Burgers’, KdV, Kuramoto-Sivashinsky, nonlinear Schrödinger, and Navier-Stokes equations.These problems span fundamentally different nonlinear partial differential equations and scientific domains.
- Burgers’ equation: 4.78e-03 relative L2-error is achieved when learning Burgers’ dynamics and extrapolating beyond t = 6.7.The training data cover roughly two-thirds of the domain; the training-region error is 3.89e-03.
- Burgers’ equation: Lower noise improves performance, while additional data may partly remedy the negative effects of stronger noise.The noise study keeps the number of observations and network architectures fixed, and the authors note that architecture choice is also crucial.
- Burgers’ equation: Making the nonlinear function depend on time or space increases the relative L2-error to 4.25e-01, 2.58e-01, or 1.46e+00 when both are included.The authors suggest overfitting to training dynamics as one possible explanation and report that more data may help.
- Burgers’ equation: The method is generally robust to derivative order, but higher-order derivatives slow computation and can be less accurate with float32.A single first-order derivative is insufficient to capture Burgers’ second-order dynamics; boundary information and validation studies can help select order.
4. Summary and Discussion
The paper presents a deep learning approach for extracting nonlinear partial differential equations from spatio-temporal data and validates its performance. It also identifies general-purpose PDE solvers and parameterized equations as important directions for further development.
- 4. Summary and Discussion: The proposed approach extracts nonlinear partial differential equations from spatio-temporal datasets using automatic differentiation and deep neural networks.The method targets infinite-dimensional dynamical systems.
- 4. Summary and Discussion: General-purpose PDE solvers remain underdeveloped, leaving black-box solvers necessary and requiring further work to reach the maturity of conventional numerical methods.The cited conventional methods include finite elements, finite differences, and spectral methods.
- 4. Summary and Discussion: Future work includes inferring parameterized PDEs from data collected across varying parameter values, including regimes that may undergo bifurcations.The Reynolds number for Navier-Stokes equations is given as an example parameter.