Error estimates for physics informed neural networks approximating the Navier-Stokes equations

Tim De Ryck; Ameya D. Jagtap; Siddhartha Mishra

Source-linked AI summary

Error estimates for physics informed neural networks approximating the Navier-Stokes equations

Tim De Ryck, Ameya D. Jagtap, Siddhartha Mishra

arXiv:2203.09346v2Mar 17, 2022math.NAcs.LG

TL;DR

The paper addresses the lack of rigorous error guarantees for PINNs solving incompressible Navier–Stokes equations. It analyzes PINN and XPINN approximation, residual stability, and quadrature-based training, showing affirmative answers to the central theoretical questions under stated regularity assumptions.

Problem
Despite widespread PINN applications, rigorous error estimates for nonlinear PDE approximations remain scarce.
Method
The paper analyzes PINN and XPINN approximations using neural-network approximation, residual-based stability bounds, and midpoint-quadrature estimates linking training and generalization errors.
Results
The three central questions are answered affirmatively: tanh networks with two hidden layers can achieve arbitrarily small residuals, total error is bounded by residuals, and training error with sufficient quadrature points controls total error.
Takeaways & Limitations
For sufficiently regular Navier–Stokes solutions, small training error and enough quadrature points suffice for small total PINN or XPINN error within the established bounds.
Takeaways & Limitations
The estimates do not control training error unless optimization reaches a global minimum, and they rely heavily on regularity of the Navier–Stokes solution.

Abstract

from arXiv · show

We prove rigorous bounds on the errors resulting from the approximation of the incompressible Navier-Stokes equations with (extended) physics informed neural networks. We show that the underlying PDE residual can be made arbitrarily small for tanh neural networks with two hidden layers. Moreover, the total error can be estimated in terms of the training error, network size and number of quadrature points. The theory is illustrated with numerical experiments.

1. Introduction

The paper develops a rigorous error analysis for PINN and XPINN approximations of incompressible Navier–Stokes equations by addressing approximation, stability, and training-to-generalization questions.

Motivation: Rigorous justification for PINNs remains scarce despite their widespread use and rapid growth in scientific computing and PDE approximation.Prior analyses include linear and selected nonlinear PDE settings, while the present work targets a prototypical nonlinear equation.
PINN framework: PINNs seek neural networks whose pointwise PDE, boundary, and initial-condition residuals are simultaneously small.For the exact solution, all corresponding residuals vanish.
Open questions: The central theoretical questions ask whether networks can achieve small generalization and training errors, whether small residuals imply small total error, and whether training error controls generalization error.These questions connect the optimization objective to accuracy of the PDE approximation.
Contributions: The paper answers Q1, Q2, and Q3 affirmatively for incompressible Navier–Stokes equations.The analysis establishes existence of accurate networks, residual-to-total-error bounds, and training-to-generalization control.
Contributions: Neural networks and XPINNs can have arbitrarily small generalization and training errors, while total error is bounded through PDE residuals and quadrature-based training estimates.The bounds also quantify network size, weights, and the number of quadrature points.

2. Preliminaries

The preliminaries define the Navier–Stokes setting, neural-network and quadrature notation, and the PINN/XPINN residual and loss constructions used for error analysis.

Navier–Stokes equations: The incompressible Navier–Stokes system models velocity and pressure with viscosity, incompressibility, initial data, and periodic conditions on the torus D = T^d.The paper assumes D = [0,1)^d and periodic boundary conditions.
Regularity: Sufficiently regular divergence-free initial data yield a classical solution with Sobolev regularity needed for the approximation analysis.The stated results provide u ∈ H^k(T^d × [0,T]) and ∇p ∈ H^{k−1}(T^d × [0,T]) under stronger regularity.
Neural networks: The neural networks approximate velocity and pressure as functions of space and time, with parameters consisting of layer weights and biases.The input is z = (x,t), and hidden layers are the first L−1 layers.
Losses and quadrature: Generalization losses integrate squared residuals, whereas training losses replace these integrals with weighted numerical quadrature over space-time and boundary data.The trained network is obtained by minimizing the training loss, typically through a local optimization procedure.
PINNs: PINN residuals measure PDE, divergence, boundary, and initial-condition violations, all of which vanish for the exact Navier–Stokes solution.The PDE residual includes time evolution, convection, pressure gradient, and diffusion terms.
XPINNs: XPINNs partition the domain among subnetworks and require additional interface residuals beyond the standard PINN residuals.These interface terms enforce compatibility across subdomains.

3. Error analysis

The analysis establishes rigorous residual and L2-error bounds for PINN and XPINN approximations of incompressible Navier–Stokes solutions, including explicit network-size and training-set dependencies. Tanh networks with two hidden layers can achieve arbitrarily small residuals, while total error bounds expose regularity, network, quadrature, and interface requirements.

3.1. Bound on the PINN residuals.: Two-hidden-layer tanh networks can approximate sufficiently regular Navier–Stokes solutions with arbitrarily small interior, divergence, and initial-condition residuals.The construction provides explicit width and weight bounds in terms of approximation parameters and solution regularity.
3.2. Bound on the total error.: Theorem 3.4 bounds the PINN/XPINN L2-error in terms of the corresponding residuals for a classical Navier–Stokes solution.The bound includes a constant that can depend on solution norms and domain-related quantities.
3.2. Bound on the total error.: For PINNs, a small loss implies a small L2-error provided the network’s C1-norm remains controlled.The result links the residual components used in the PINN generalization error to the solution error.
3.2. Bound on the total error.: For XPINNs, the interface residual R∇u must be added to the original loss to theoretically guarantee a small L2-error.The other XPINN-specific residuals are already equivalent to residuals in the XPINN generalization error.
3.3. Bounds on the total error in terms of training error.: The total error can be bounded using training error, training-set size, network norms, and quadrature-related quantities, yielding a priori and a posteriori estimates.Under the stated approximation and optimization conditions, the training and total errors can be made arbitrarily small.
3.3. Bounds on the total error in terms of training error.: The constants in the training-error bounds grow polynomially with network width but exponentially with depth, favoring shallow networks within this analysis.The paper notes that the constants can be estimated from the network used in practice.

4. Numerical experiments

Numerical experiments use the Taylor–Green vortex to test PINN and XPINN error estimates, varying training settings, quadrature points, and network width. The computed bound tracks total-error behavior, while errors saturate as quadrature points increase.

Taylor–Green vortex setup: The Taylor–Green vortex provides an analytical, sufficiently smooth solution for evaluating PINN and XPINN L2-errors against the exact solution.The experiment uses a two-dimensional unsteady flow with decaying vortices.
Taylor–Green vortex setup: PINNs and XPINNs are trained on the Taylor–Green vortex, with XPINNs splitting the domain into two subdomains and using 300 interface points.The XPINN subdomains are divided along the x-axis, with separate neural networks on each side.
Quadrature-point study: The theoretical bound follows the same qualitative and quantitative behavior as total error across quadrature-point settings, although its amplitude is approximately one order of magnitude larger.The bound and total error have very similar decay rates but the bound is not quantitatively sharp.
Quadrature-point study: Both training and total errors decrease as quadrature points increase, then saturate around 27K quadrature points.The experiment varies the number of quadrature points and reports saturation beyond this level.
Network-width study: The experiments vary network width while keeping two hidden layers fixed, using 20 training runs with different parameter initializations.The networks use the hyperbolic tangent activation function, and the quadrature-point count is fixed at 64 K.

5. Discussion

The paper answers three theoretical questions for PINNs and XPINNs approximating incompressible Navier–Stokes equations, linking arbitrarily small residuals to bounded total error and training-error estimates. It also reports numerical agreement with the resulting bounds and identifies assumptions limiting the analysis.

The paper affirmatively answers Q1, Q2, and Q3 for incompressible Navier–Stokes PINNs and XPINNs.
Tanh networks with two hidden layers can achieve arbitrarily small PDE residuals under sufficiently smooth initial data.
The total L2 error is bounded by PDE residuals, while another bound relates total error to training error and quadrature-point count.
The theorems provide the first comprehensive theoretical analysis of PINNs and XPINNs for a prototypical nonlinear PDE, the Navier–Stokes equations.
Numerical experiments show qualitative and quantitative agreement between the rigorous bounds and empirical results.
The estimates do not control training error unless optimization reaches a global minimum, and they rely heavily on regularity of the Navier–Stokes solution.
The analysis considers only forward problems, leaving inverse Navier–Stokes problems and physics-informed operator learning for future work.

Appendix A. Notation and auxiliary results

Appendix A introduces the notation and auxiliary mathematical results used throughout the paper, including Sobolev-space foundations.

The appendix provides notation and recalls basic results on Sobolev spaces.

A.1. Multi-index notation.

This subsection introduces multi-index notation and the derivative notation used for the paper’s Sobolev-space statements.

Multi-indices encode derivative orders componentwise, with |α|, α!, and α ≤ β defined through their components.
The notation includes classical or distributional derivatives of functions.
The appendix begins defining Sobolev spaces W^k,p(Ω) using Lebesgue-space norms and derivatives indexed by multi-indices.

A.2. Sobolev spaces.

This subsection defines Sobolev norms and spaces, then collects embedding, trace, approximation, product, and composition results used in the analysis.

Sobolev seminorms aggregate L∞ norms of derivatives up to order k.
The appendix defines the relevant norms and identifies W^k,p(Ω) as a Banach space.
Hilbertian Sobolev spaces satisfy H^k(Ω)=W^k,2(Ω), and sufficiently high regularity makes H^k(Ω) a Banach algebra.
Sobolev embedding gives continuous representatives with additional differentiability under suitable dimension and regularity conditions.
A multiplicative trace inequality relates interior H^1 regularity to boundary traces on Lipschitz domains.
The auxiliary lemmas cover polynomial approximation, Leibniz products, and composition estimates for Sobolev-regular functions.

Appendix B. Function approximation by tanh neural networks

This appendix develops explicit tanh-network constructions for approximating Sobolev functions, emphasizing adaptable accuracy, width, and weight bounds. The construction uses polynomial and monomial approximation, domain partitioning, and finite-difference schemes.

Function approximation: For every f ∈ H^m(Ω), m ≥ 3, a tanh neural network with two hidden layers can achieve an H^2(Ω) approximation error of at most ϵ.The proof partitions the domain into cubes, approximates locally by polynomials, and combines the local approximations using an approximate partition of unity.
Construction: The proof adapts prior tanh-network approximation results to provide explicit upper bounds for Navier–Stokes-relevant function approximation.The authors use De Ryck et al. as a guideline because it provides explicit network-size bounds, while adapting its polynomial approximation proof.
Monomial approximation: The multivariate monomial construction is identified as the main modification relative to the original results, with subsequent updates following from it.The lemma constructs a shallow tanh network for multivariate monomials on [−M,M]^d.
Monomial approximation: An n-th-order finite-difference scheme generalizes the second-order construction for approximating univariate monomials in W^k,∞-norm.The resulting approximation controls the network width and weight growth through the freely chosen parameter n.
Network bounds: The resulting weights scale as O(N ln(N) + N^γ), with γ = max{m^2, d(2 + m + d)}/n.This makes the dependence on dimension, smoothness, discretization, and the width-control parameter explicit.

Appendix C. Bounds on the derivative of a neural network

This appendix establishes bounds on derivatives of neural networks under smoothness and network-size assumptions. The proof recursively represents network components and applies derivative bounds layer by layer.

Derivative bounds: For a neural network u_θ with at least two layers and sufficiently smooth activation σ, the appendix bounds derivatives of its output components.The assumptions include ∥σ∥_C^n ≥ 1 and positive bounds on depth, width, and weight magnitude.
Proof strategy: The proof defines scalar functions for each network layer and applies a prior derivative lemma to obtain the stated bounds.The argument proceeds through the layerwise representation of the network.
Derivative bounds: The derivative estimates depend on the network weight bound R and the activation-function norm ∥σ∥_C^n.The displayed intermediate bound includes the factor R^n∥σ∥_C^n.