PaperlayerView exact revision on arXiv

Source-linked AI summary

Physics-informed learning of governing equations from scarce data

Zhao Chen, Yang Liu, Hao Sun

arXiv:2005.03448v3cs.LGphysics.comp-phphysics.data-anstat.ML

TL;DR

Discovering governing PDEs from scarce and noisy measurements remains difficult for nonlinear spatiotemporal systems. The paper combines physics-informed deep learning, automatic differentiation, sparse regression, and alternating optimization to achieve accurate and robust closed-form PDE discovery across varied systems and measurement conditions.

  • Problem

    Discovering governing PDEs from scarce and noisy data remains challenging, while existing sparsity-promoting methods scale poorly with data noise and scarcity.

  • Method

    The framework jointly trains an interpretable neural network and sparse PDE coefficients, using automatic differentiation and alternating direction optimization to identify governing equations.

  • Results

    The method accurately and robustly discovers diverse nonlinear PDE systems from scarce, noisy measurements under varied initial and boundary conditions.

  • Takeaways & Limitations

    The framework shows potential for closed-form physical-law discovery when large, accurate datasets are impractical to capture.

  • Takeaways & Limitations

    Applying the fully connected neural network to higher-dimensional systems creates computational and optimization bottlenecks because satisfactory accuracy may require many collocation points.

Abstract

from arXiv · show

Harnessing data to discover the underlying governing laws or equations that describe the behavior of complex physical systems can significantly advance our modeling, simulation and understanding of such systems in various science and engineering disciplines. This work introduces a novel physics-informed deep learning framework to discover governing partial differential equations (PDEs) from scarce and noisy data for nonlinear spatiotemporal systems. In particular, this approach seamlessly integrates the strengths of deep neural networks for rich representation learning, physics embedding, automatic differentiation and sparse regression to (1) approximate the solution of system variables, (2) compute essential derivatives, as well as (3) identify the key derivative terms and parameters that form the structure and explicit expression of the PDEs. The efficacy and robustness of this method are demonstrated, both numerically and experimentally, on discovering a variety of PDE systems with different levels of data scarcity and noise accounting for different initial/boundary conditions. The resulting computational framework shows the potential for closed-form model discovery in practical applications where large and accurate datasets are intractable to capture.

RESULTS

The proposed PiDL framework accurately discovers closed-form governing PDEs from scarce, noisy measurements across nonlinear systems and diverse initial/boundary conditions. Its DNN–sparse-regression synergy recovers solution fields, derivatives, active terms, and coefficients while ADO makes the optimization tractable.

  • Framework: PiDL combines deep neural networks, automatic differentiation, physics embedding, and sparse regression to identify governing PDE structure and parameters from limited noisy data.The DNN models latent solutions, derivatives, and candidate functions, while sparse PDE representations constrain the model and eliminate redundant terms.
  • Burgers’ Equation: ADO converges after the first alternating iteration in the Burgers’ example and recovers the correct PDE sparsity pattern from noisy measurements.For Burgers’ equation, the dynamical response is reproduced with a full-field ℓ2 prediction error of 2.02%.
  • Nonlinear Schrödinger Equation: 0.7% coefficient error and 1.87% relative ℓ2 prediction error are obtained for nonlinear Schrödinger equation discovery.Both candidate terms and coefficients are correctly identified within a few ADO iterations, and the predicted wave closely matches the exact solution.
  • Navier-Stokes Equation: Both sparse terms and associated coefficients are precisely identified for Navier–Stokes data with 10% noise, while predicted response snapshots remain close to ground truth.The result demonstrates robustness for high-dimensional PDE discovery under highly noisy measurements.
  • Multiple Initial/Boundary Conditions: The correct Burgers’ terms uux and uxx are identified across three distinct initial/boundary-condition cases, with redundant terms eliminated and a 2.24% stacked full-field ℓ2 error.All cases obey the same underlying PDE, ut = -uux + 0.0032uxx, despite distinct system responses.
  • Robustness: Increasing spatiotemporal measurement resolution mitigates noise-related difficulties, while the exact PDE form is successfully discovered even under 10% noise.The passage notes that coefficients can be further tuned or calibrated when additional data become available.

DISCUSSION

The discussion presents PiDL as an interpretable framework for discovering parsimonious closed-form PDEs from scarce, noisy data by combining neural networks, automatic differentiation, and sparse regression. It also highlights mutual DNN–PDE constraint and computational limitations for higher-dimensional systems.

  • Contributions: PiDL discovers parsimonious closed-form PDEs for multidimensional nonlinear spatiotemporal systems from scarce and noisy data.The framework combines DNN representation learning, automatic differentiation, and ℓ0 sparse regression.
  • Applications: For cell migration and proliferation, the discovered active terms were t∆ρ, ρ, and ρ2u across systems containing 14,000, 16,000, 18,000, and 20,000 cells.The corresponding coefficients and PDEs were reported for each cell-count case, with simulated cell densities evaluated at different time instants.
  • Contributions: Jointly training the DNN and sparse coefficients lets the network model solutions and derivatives while the PDE representation informs and constrains the DNN.The passage describes this synergy as improving the DNN’s generalizability.
  • Limitations: Applying the fully connected DNN directly to higher-dimensional systems can create computational bottlenecks and optimization challenges.The stated example is long/short-term response evolution in a 3D domain, where many collocation points may be needed to maintain satisfactory accuracy.

METHOD

PiDL integrates deep neural networks, physics embedding, automatic differentiation, and sparse regression to approximate system solutions, compute derivatives, and identify governing PDE terms and parameters from scarce, noisy data. Its architecture combines physics-constrained loss optimization with alternating updates of neural-network parameters and sparse PDE coefficients.

  • Network architecture: PiDL uses dense or root-branch fully connected networks to approximate latent solutions for single or multiple independent initial/boundary-condition datasets.The networks take spatiotemporal coordinates as input and use tanh or sin activation functions.
  • Physics reconstruction: Automatic differentiation evaluates solution derivatives at machine precision, while candidate-function libraries from branch networks are concatenated to enforce one unified physical law across datasets.The reconstructed PDE is represented through a sparse residual involving the time derivative, candidate library, and coefficient matrix.
  • Physics-constrained learning: The physics-constrained loss combines measurement-data loss, residual physics loss at randomly sampled collocation points, and sparsity regularization on PDE coefficients.For multiple datasets, measurements, derivatives, and candidate libraries are stacked or concatenated across initial/boundary conditions.
  • Alternating direction optimization: Because ℓ0-regularized optimization is NP-hard and ℓ1 relaxation can cause false-positive terms, ADO alternates sparse-coefficient and neural-network updates.STRidge updates the sparse PDE coefficients, while Adam plus L-BFGS updates the network parameters.
  • Training refinements: Optional ℓ1-regularized pre-training supplies an admissible initial guess, and optional post-training fine-tunes the network using the discovered nonzero PDE coefficients.The post-training step can improve discovery accuracy, although ADO is reported to provide a high-quality solution without it.

ADO Pretraining … B.1.1 Burgers’ equation

The paper presents an ADO-based physics-informed framework that combines neural-network training, automatic differentiation, and sparse regression to discover closed-form PDEs from scarce and noisy data. Its benchmark demonstrations include Burgers’ equation, where the method identifies the governing structure under 10% noise with low coefficient error.

  • ADO Pretraining: The supplementary document details the proposed algorithm, examples, and technical challenges for discovering closed-form PDEs from scarce and noisy data.The document frames the method around practical PDE discovery when large, accurate datasets are difficult to obtain.
  • A Alternating Direction Optimization (ADO): Algorithm: The ADO algorithm alternates neural-network training with sparse regression, using STRidge as a proxy for ℓ0 regularization.STRidge sequentially thresholds small coefficients to zero and updates remaining coefficients with ridge regression.
  • A Alternating Direction Optimization (ADO): Algorithm: ADO pre-training uses combined Adam and L-BFGS optimization before iterative coefficient identification and model validation.Measurement and collocation data are split into training-validation sets, and the network is pre-trained before alternating updates.
  • B Examples: The methodology discovers closed forms for Burgers’, Kuramoto-Sivashinsky, nonlinear Schrödinger, Navier-Stokes, and λ-ω Reaction-Diffusion equations from scarce, noisy time-series measurements.The demonstrations cover nonlinear, periodic, and/or chaotic physical systems using sensors at fixed locations from a single initial-boundary condition.
  • B.1 Discovery of Benchmark PDEs with Single Dataset: Burgers’ equation models a dissipative one-dimensional viscous shock system relevant to simplified fluid mechanics, nonlinear acoustics, gas dynamics, and traffic flow.The diffusion coefficient is denoted by ν, and the system describes a decaying stationary viscous shock after a finite period.
  • B.1.1 Burgers’ equation: For Burgers’ equation, the reconstruction uses 16 candidate functions containing polynomial terms, derivatives, and their multiplications, with a fully connected DNN of 8 hidden layers and width 20.The training uses 1 × 10^3 L-BFGS pre-training epochs and 20 ADO iterations; collocation points are sampled by a Sobol sequence.
  • B.1.1 Burgers’ equation: The discovered Burgers’ equation under 10% noise is u_t = −0.997u u_x + 0.098u_xx, with an aggregated relative identification error of 1.15±1.20%.Adaptive sparsity thresholding prunes redundant candidate components during the ADO stage.

B.1.2 Kuramoto-Sivashinsky equation

The PiDL framework discovers the closed-form Kuramoto–Sivashinsky equation from sparsely sampled, noisy data despite chaotic dynamics and difficult high-order derivative estimation. With 10% noise, it identifies the correct candidate terms and coefficients while accurately reproducing the full-field solution.

  • System and challenge: The KS equation’s reverse diffusion term drives blowup, while its fourth-order derivative generates chaotic patterns and makes it a demanding equation-discovery problem.Its nonlinear dynamics evolve a smooth initial condition toward an unstable laminar state.
  • System and challenge: PiDL successfully distils the closed form of the KS equation from subsampled sparse data with 10% noise, whereas existing SINDy methods fail under coarse, noisy measurements.The challenge is especially severe for approximating the high-order uxxxx derivative from poorly measured data.
  • Equation discovery: The discovered equation uses coefficients -0.992, -0.990, and -0.991 for uux, uxx, and uxxxx, respectively.The candidate terms and corresponding coefficients are identified close to the original parameters within a small number of ADO iterations.
  • Equation discovery: 0.71±0.06% is the average relative error for all non-zero coefficients in Λ.The coefficient evolution shows that both candidate terms and their coefficients are correctly identified.
  • Solution prediction: 1.87% is the relative ℓ2 error of the trained PiDL full-field wave prediction against the exact solution.Spatial and temporal snapshots of the predicted response match the ground truth seamlessly.

B.1.3 Nonlinear Schrödinger equation

The framework discovers the nonlinear Schrödinger equation for a complex field from sparse sensor measurements, using separate real and imaginary solution components and 40 candidate functions. Under 10% noise, it identifies coefficients close to the true values and predicts the full-field response with about 1% relative ℓ2 error.

  • Data and setup: The solution uses a Gaussian initial condition on 512 spatial points and 501 temporal steps, with measurements from 256 randomly chosen sensors over 375 time instants.These measurements produce scarce observations for discovering the governing equation.
  • Model formulation: The nonlinear Schrödinger equation is modeled for a complex field variable using separate real (uR) and imaginary (uI) DNN outputs assembled as u = uR + iuI.The candidate library contains 40 polynomial, absolute-value, derivative, and combined functions.
  • Discovery result: 2.31 ± 0.28% is the average relative error for non-zero coefficients.The sparse coefficients converge to the actual values, supporting accurate closed-form PDE identification.
  • Prediction accuracy: About 1% relative ℓ2 error characterizes the predicted full-field response for both real and imaginary parts compared with the exact solution.The comparison covers the spatiotemporal response of the complex solution.

B.1.4 Navier-Stokes equation

The framework is applied to discover the 2D Navier–Stokes vorticity equation for flow past a circular cylinder from noisy velocity measurements. With 10% noise, it recovers the vorticity equation and its coefficients with low aggregated relative identification error and precise sparsity.

  • Problem setup: The test case models incompressible 2D flow past a unit-diameter circular cylinder at Reynolds number 100 on a 499×199 spatial grid over 151 time steps.The input flow has unit velocity, and the full-field solution is obtained using the immersed boundary projection method.
  • Equation discovery: Under 10% noise, the discovered equation is w_t = -0.999uw_x - 0.994vw_y + 0.010w_xx + 0.010w_yy.The recovered derivative terms and coefficients are nearly identical to the ground-truth equation.
  • Identification accuracy: The aggregated relative identification error for all non-zero elements in Λ is 1.40 ± 1.83%.The 60 candidate functions converge quickly to the correct values with precise sparsity after the first ADO iteration.

B.1.5 λ-ω type Reaction-Diffusion equations

The framework discovers a two-dimensional λ-ω reaction-diffusion system of coupled PDEs governing spiral-wave pattern formation under periodic boundary conditions. With 10% noise, it recovers the sparse terms and coefficients with an average relative error of 4.78˘3.66% for all non-zero coefficients.

  • System and applications: The λ-ω system comprises two coupled PDEs for field variables u and v in a 2D domain.The equations describe local reactive transformation and global diffusion.
  • System and applications: The test equations display spiral waves under periodic boundary conditions and represent an activator-inhibitor system with wave-like and self-organized patterns.λ-ω equations are used in pattern formation, biological morphogenesis, and ecological invasions.
  • Equation discovery: For data with 10% noise, the reconstructed u- and v-equations closely match the original coefficient structure while retaining diffusion and nonlinear reaction terms.The recovered coefficients are given explicitly for both equations, including uxx, uyy, vxx, vyy, and nonlinear polynomial terms.

B.2 Comparison with SINDy · B.3 Discovery of PDEs with Multiple Independent Datasets

PiDL is evaluated against PDE-FIND across five canonical PDEs under varying data scarcity and noise, using relative errors in identified non-zero coefficients. Its framework combines neural representation learning, automatic differentiation, sparse regression, and collocation points to support closed-form PDE discovery from limited measurements.

  • B.2 Comparison with SINDy: PiDL and PDE-FIND are compared on five previously described PDEs across different data sizes and noise levels.Discovery error is defined as the average relative error of the identified non-zero PDE coefficients relative to ground truth.
  • B.2 Comparison with SINDy: PDE-FIND reports NA entries when correct identification of the sparse PDE coefficients is not applicable or fails.The table note defines NA as “not applicable,” including failure in correct identification of sparse PDE coefficients.
  • B.2 Comparison with SINDy: The canonical-model summary includes computational-cost entries alongside discovery-error results for PiDL and PDE-FIND.The supplied rows show costs such as „96K, „30K, „257K, „300K, „37.5K, and „150K.
  • B.2 Comparison with SINDy: The comparison reports PiDL results for Schrödinger and Navier–Stokes systems alongside corresponding PDE-FIND entries in the canonical-model summary.The supplied table fragments include Schrödinger PiDL and PDE-FIND rows, plus NS PiDL and PDE-FIND rows with discovery-error and computational-cost entries.
  • B.2 Comparison with SINDy: The reported noise is Gaussian white noise, with its level defined as the root-mean-square ratio between noise and exact solution.This definition applies to the synthetic response used in the comparison table.
  • B.2 Comparison with SINDy: PiDL combines deep neural networks, automatic differentiation, and ℓ0 sparse regression to represent nonlinear functions, calculate derivatives, and identify PDE structure.The approach also uses collocation points as additional pseudo datasets that compensate indirectly for scarce measurements and enrich PDE constraints.

B.3.1 Burgers’ equation with shock behavior

The framework discovers Burgers’ equation with a small viscosity parameter under shock-forming conditions and scarce, noisy measurements across three initial/boundary conditions. A root-branch DNN captures shared solution patterns and condition-specific differences, recovering the active PDE terms and viscosity coefficient accurately.

  • Shock-forming setup: A diffusion coefficient of 0.01/π ≈ 0.0032 creates localized shock formation with sharp gradients, challenging approximation and PDE discovery.The shock behavior motivates using a denser sensor grid than in the previous Burgers’ example.
  • Shock-forming setup: Data use three initial/boundary conditions, with 30 randomly deployed sensors measuring the wave for 500 time instants, or 7.5% of grid points.The simulated domain is [−1, 1]^200 × [0, 1]^1000.
  • Discovery results: The discovered PDE, u_t = −1.006u u_x + 0.0039u_xx, shows great agreement with the ground truth.The identified coefficient 0.0039 captures the small viscosity parameter associated with shock formation.
  • Discovery results: The method retains the correct library terms u u_x and u_xx while eliminating redundant candidate terms through ADO.The coefficients of the active terms are also accurately identified.

B.3.2 Fitzhugh-Nagumo type of Reaction-Diffusion equations

The study applies the physics-informed framework to a two-dimensional Fitzhugh–Nagumo reaction-diffusion system with periodic boundaries, known diffusion terms, and nonlinear reaction terms reconstructed from candidate functions. Under 10% noise and scarce measurements, it precisely recovers the PDE structure, including the unknown external stimulus, with generally accurate coefficients except for the less-sensitive v-component diffusion coefficient.

  • Problem setup: The Fitzhugh–Nagumo system contains two coupled PDEs for interactive components u and v on a 2D domain with periodic boundary conditions.The components can represent biological matter, and the system is commonly used to describe neuron activity under external stimulus.
  • Candidate library: The model uses known Laplacian terms while identifying their positive diffusion coefficients and reconstructing nonlinear reaction terms from 70 candidate functions.Candidates include up to third-order polynomials, derivatives, and mutual products; coefficient bounds are γu ∈ [0, 5] and γv ∈ [0, 150].
  • Optimization: The optimization combines Adam, L-BFGS, automatic differentiation, and STRidge within alternating discovery iterations, using sigmoid parameterizations to enforce diffusion-coefficient bounds.The procedure includes 10 ADO iterations, 10^4 Adam epochs per iteration, up to 10^4 L-BFGS epochs, and 10^5 post-training Adam epochs.
  • Results: Under 10% noise and very scarce, noisy measurements, the method precisely uncovers all correct active PDE terms, including the unknown external stimulus in the first equation.The identified non-zero coefficients have an error of 11.72±8.34%, while γv is comparatively less sensitive.

B.4 Experimental Discovery of Cell Migration and Proliferation · C Discussion · C.1 Selection of candidate functions

PiDL discovers a parsimonious cell-density PDE from scratch-assay measurements across initial cell densities, recovering a Fisher–Kolmogorov form. The discussion emphasizes that candidate-function libraries must balance diversity for term recovery against conditioning, complexity, and computational cost.

  • B.4 Experimental Discovery of Cell Migration and Proliferation: Scratch-assay imaging provides cell-density measurements at five time points across experiments with varied initial cell densities.The extracted distributions correspond to 0h, 12h, 24h, 36h, and 48h measurements.
  • B.4 Experimental Discovery of Cell Migration and Proliferation: The discovery task seeks a parsimonious PDE for cell density using four initial-cell scenarios from 14,000 to 20,000 cells.Each scenario uses the mean of three identically prepared experimental replicates.
  • B.4 Experimental Discovery of Cell Migration and Proliferation: The assumed cell-density dynamics combine diffusion-driven migration with a nonlinear reaction term for proliferation.The PDE is specified as ρ_t = γρ_xx + F(ρ), supplemented by eight additional candidate terms.
  • B.4 Experimental Discovery of Cell Migration and Proliferation: PiDL uses 10^4 Latin-hypercube collocation pairs and a three-hidden-layer tanh network with a softplus output to enforce positive cell density.The collocation domain spans 0–1900 μm and 0–48 h.
  • B.4 Experimental Discovery of Cell Migration and Proliferation: All four discovered equations share ρ_t = γρ_xx + λ_1ρ + λ_2ρ^2, matching the Fisher–Kolmogorov model.Identified migration and proliferation rates generally increase with cell number, while trained networks capture measurement patterns with little evidence of overfitting.
  • C.1 Selection of candidate functions: Candidate-function libraries should be diverse enough to recover governing terms but constrained using domain knowledge and statistical experience.Specialized libraries can reduce the search space and the complexity of PDE discovery.
  • C.1 Selection of candidate functions: Increasing library dimension improves the likelihood of uncovering exact terms, but very large libraries create poor conditioning, theoretical complexity, and computational burden.Balancing these concerns and establishing mathematical principles for efficient domain-specific libraries remains open.
  • C.1 Selection of candidate functions: Omitting essential candidate functions can produce false-positive parsimonious closed-form PDEs even when a best-of-fit expression is available.The passage identifies candidate-library design as a central limitation of reliable equation discovery.

C.2 Noisy measurements and collocation points

PiDL’s PDE-discovery accuracy depends on noisy measurement data and collocation-point sampling, while global collocation points substantially support robust reconstruction and identification, including under noisy or sparse measurements.

  • Collocation-point sensitivity: PiDL remains robust when using a larger number of collocation points, such as ≥ 8ˆ104.The study evaluates sensitivity to measurement noise and collocation-point quantity using Burgers’ equation and relative ℓ2-norm errors.
  • Collocation-point sensitivity: Collocation points need not correlate with measurements and can be sampled globally using Sobol sequences or Latin hypercube sampling.These methods create finer, more representative uniform partitions of the spatiotemporal domain, improving generalizability and PDE-discovery accuracy.
  • Extrapolation and noisy measurements: With clean enclosed-area measurements and global collocation points, PiDL reconstructs the full field and identifies sparse PDE coefficients effectively.This corresponds to Case 1 in Table S.2.
  • Extrapolation and noisy measurements: At 10% noise, response-prediction errors increase, but the PDE is still accurately discovered with global collocation points.This corresponds to Case 2 in Table S.2.
  • Extrapolation and noisy measurements: Without collocation points, response-prediction errors remain over 50% and the PDE is completely misidentified.This corresponds to Case 3 in Table S.2.
  • Extrapolation and noisy measurements: Doubling clean measurement points to 3ˆ103 improves interpolation and discovery but still yields poor full-field extrapolation.This corresponds to Case 4 in Table S.2.

C.3 Simultaneous identification of unknown source term

The framework incorporates source candidates into the discovery library to identify a governing PDE and reconstruct an unknown spatiotemporal source simultaneously. In a noisy, sparsely measured Burgers’ equation test, it accurately recovered both the PDE structure and source while revealing challenges for complex unknown sources.

  • C.3 Simultaneous identification of unknown source term: Unknown source candidates are incorporated into the library so the PDE and source can be discovered and reconstructed simultaneously.The corresponding sparse coefficients represent the field variable u and source p separately.
  • C.3 Simultaneous identification of unknown source term: The Burgers’ equation experiment used 20 randomly selected sensors, 50 time steps, and 10% noise, without measuring the source.The solution domain contained 201 spatial grid points over x ∈ [−5, 5] and 101 time steps over t ∈ [0, 10].
  • C.3 Simultaneous identification of unknown source term: The discovered equation recovered the coefficients 1.002 for uux, −0.088 for uxx, and 0.995 for sin(x)sin(t).The reconstructed expression was ut + 1.002uux − 0.088uxx = 0.995 sin(x)sin(t).
  • C.3 Simultaneous identification of unknown source term: 4.39% ± 7.03% was the average relative error for non-zero coefficients, with robust and quick convergence toward the ground truth.The sparse coefficients for both the PDE and source converged robustly despite scarce and noisy system-response measurements.
  • C.3 Simultaneous identification of unknown source term: Predicted spatial and temporal solution snapshots matched the exact solutions well, including near boundaries with scarce training data.The comparison is reported in Fig. S.12C.
  • C.3 Simultaneous identification of unknown source term: A completely unknown complex source may require an extraordinarily large candidate library, increasing computational complexity for accurate PDE discovery.The source may instead be approximated in some cases by combinations of continuous basis functions.

C.4 Other network architecture

The fully connected DNN in the PiDL framework enables analytical PDE-derivative approximation through automatic differentiation but has potential limitations for physical law discovery. Discrete spatiotemporal architectures such as ConvLSTM may help address these limitations, provided their differentiators are carefully designed.

  • C.4 Other network architecture: The fully connected DNN provides analytical approximation of PDE derivatives via automatic differentiation but retains potential limitations for physical law discovery.
  • C.4 Other network architecture: Discrete DNNs with spatiotemporal discretization, including ConvLSTM, have potential to help resolve these challenges.The paper states that this direction will be demonstrated in future work.
  • C.4 Other network architecture: Careful design of spatiotemporal differentiators is required, including high-order finite-difference filters and treatment of domain irregularity.
Loading 2005.03448v3…