Source-linked AI summary
Physics-Informed Deep Neural Operator Networks
Somdatta Goswami, Aniruddha Bora, Yue Yu, George Em Karniadakis
TL;DR
The paper reviews neural operators for learning general nonlinear input-output mappings, addressing the need for fast inference across varied physical conditions and sparse-data settings. It surveys DeepONet, Fourier and graph neural operators, their extensions, and physics-informed training, highlighting applications in computational mechanics. The review concludes that hybrid physics- and data-based models can support complex multi-physics problems, including problems solved in less than a second.
Problem
Condition-specific PINNs require expensive optimization during inference, while purely data-driven neural operators require large paired datasets that are costly to acquire.
Method
The paper reviews DeepONet, Fourier neural operator, and graph neural operator architectures, their extensions, and variants that incorporate governing equations into training.
Results
Hybrid physics- and data-based models can achieve real-time or sub-second solutions for complex multi-physics problems within the reviewed applications.
Takeaways & Limitations
Neural operators can serve as fast surrogates for design, uncertainty quantification, autonomous systems, and other applications requiring real-time inference.
Takeaways & Limitations
FNO requires lattice-grid input functions and is difficult to apply directly to sparse-point inputs or unstructured meshes.
Abstract
from arXiv · showhide
Standard neural networks can approximate general nonlinear operators, represented either explicitly by a combination of mathematical operators, e.g., in an advection-diffusion-reaction partial differential equation, or simply as a black box, e.g., a system-of-systems. The first neural operator was the Deep Operator Network (DeepONet), proposed in 2019 based on rigorous approximation theory. Since then, a few other less general operators have been published, e.g., based on graph neural networks or Fourier transforms. For black box systems, training of neural operators is data-driven only but if the governing equations are known they can be incorporated into the loss function during training to develop physics-informed neural operators. Neural operators can be used as surrogates in design problems, uncertainty quantification, autonomous systems, and almost in any application requiring real-time inference. Moreover, independently pre-trained DeepONets can be used as components of a complex multi-physics system by coupling them together with relatively light training. Here, we present a review of DeepONet, the Fourier neural operator, and the graph neural operator, as well as appropriate extensions with feature expansions, and highlight their usefulness in diverse applications in computational mechanics, including porous media, fluid mechanics, and solid mechanics.
1 Introduction
Neural operators extend physics-informed learning from condition-specific models toward fast inference across varied inputs. The section introduces DeepONet and related operator architectures, and frames physics-informed training as a way to incorporate governing equations.
- Physics-informed learning: PINNs solve physical-system problems with measurements and parameterized PDE information, including some ill-posed problems.Examples include missing thermal boundary conditions, defect discovery from sparse displacements, and failure-pattern estimation.
- Motivation: PINNs are trained for specific conditions and require expensive inference optimization, limiting their effectiveness across operating conditions and for real-time inference.The section motivates generalized models that handle varied boundary, initial, loading, and source conditions with little or no retraining.
- Neural operators: DeepONet and other neural operators learn input-output operators offline, enabling fast inference for new conditions inside the predefined input space.Out-of-distribution inputs require further training, potentially light when the input space is sufficiently sampled.
- DeepONet: DeepONet uses branch and trunk networks to approximate a nonlinear operator from input functions to outputs evaluated at specified locations.The branch and trunk can accept different input types, including graphs and spikes.
- Scope: The review covers DeepONet, Fourier neural operator, and graph neural operator variants, including extensions and physics-informed training approaches.The physics can be enforced through labelled input-output pairs or by incorporating governing equations into training.
2 DeepONet and its extensions
DeepONet combines branch and trunk networks to learn operators from discretized input functions and evaluate outputs at continuous coordinates. Its extensions incorporate multiple inputs, adaptive loss weighting, problem-specific features, and physics-based regularization for sparse-data settings.
- Conventional DeepONet: DeepONet encodes sampled input functions with a branch network and output coordinates with a trunk network to learn a solution operator.The learned output is a scalar-valued function evaluated at continuous spatio-temporal coordinates.
- Approximation theory: The universal approximation formulation permits diverse neural-network classes for branch and trunk networks, including fully connected, residual, and convolutional networks.The theorem applies to nonlinear continuous operators under compact-domain assumptions.
- Adaptive weighting: Self-adaptive weights update loss penalties during training so constraints such as initial and boundary conditions receive non-uniform emphasis.The modified objective jointly optimizes network parameters and self-adaptive penalty parameters through gradient descent/ascent.
- Feature expansions: DeepONet extensions add problem-specific information through trunk or branch feature expansions, including temporal dynamics, periodic boundaries, historical inputs, and precomputed POD modes.POD-DeepONet replaces the learned trunk basis with modes computed from centered training data, while branch networks learn their coefficients.
- Limitations: POD-based branch features can be challenging for discontinuous functions because they resemble global spectral approximations of non-smooth functions.This identifies a scope boundary for one feature-expansion strategy.
- Multiple-input operators: Multiple-input operators extend DeepONet to approximate solutions associated with multiple initial or boundary conditions simultaneously.Such architectures can use separate branch networks for different input functions.
- Physics-informed DeepONet: Physics-informed DeepONet incorporates known differential equations into the loss function alongside labelled data to train under sparse-data conditions.Automatic differentiation of outputs with respect to input coordinates enables the physics-based regularization mechanism.
3 FNO and its extensions
The Fourier neural operator learns convolutional operators in Fourier space, while extensions address differing domains, complex geometries, deep-network stability, and physics-informed training.
- Fourier neural operator: FNO lifts input functions into higher-dimensional representations, iteratively applies Fourier-space transformations with residual connections, and projects outputs back to grid dimensions.Each layer filters higher Fourier modes through a linear transform before returning to physical space.
- Feature expansion in FNO: FNO requires input functions on equally spaced lattice grids, complicating applications with sparse boundary or initial data and unstructured complex geometries.Feature expansions were proposed to address these domain and discretization constraints.
- Feature expansion in FNO: dFNO+ handles different input and output domains by adding temporal components, recurrent time marching, or zero-padding unavailable interior values.These strategies support mappings from initial or boundary conditions to solutions over spatial and temporal domains.
- Feature expansion in FNO: gFNO+ addresses non-Cartesian domains and non-lattice meshes by defining a bounding box and projecting the input into a suitable representation.It targets complex real-life geometries where discrete FFT assumptions are difficult to satisfy.
- Related extensions: Wavelet neural operators replace Fourier integrals with wavelet integrals to represent patterns with simultaneous frequency and spatial localization.The approach is described for spatial behavior and systems with complex boundary conditions.
- Implicit FNO: Vanilla FNOs use different trainable parameters in each Fourier layer, increasing parameter counts with depth and contributing to overfitting and vanishing-gradient difficulties.The cited discussion reports decreasing training error but substantially larger test error as depth increases.
- Implicit FNO: IFNO shares one parameter set across iterative layers, interpreting layer updates as a discretized autonomous differential equation and improving stability in deep networks.Its design also reduces memory cost and avoids parameter growth with depth.
- Implicit FNO: IFNO retains FNO’s Fourier-space kernel parameterization, FFT efficiency, and resolution independence while improving stability and accuracy in the deep-network limit.The review demonstrates IFNO for biological tissue modeling using digital image correlation measurements.
4 Graph Neural Operators
Graph neural operators extend graph message passing and integral operators to learn PDE solution mappings, with accuracy, stability, expressiveness, and computational cost shaped by their kernel design and interaction range.
- 4.1 Graph Neural Networks: GNN message passing gathers node-edge information, aggregates neighboring messages, and updates node representations while preserving graph symmetry.These operations form the core message passing layer and support functions such as mean, max, normalized sum, MLPs, and RNNs.
- 4.2 Integral Neural Operators through Graph Kernel Learning: GKNs interpret graph neural networks continuously by representing PDE solution operators with learned integral kernels over continuum nodes.The approach is motivated by Green’s functions and uses lifting, iterative kernel integration, and projection layers.
- 4.2 Integral Neural Operators through Graph Kernel Learning: Whole-domain interactions increase GKN expressiveness but make computation expensive, whereas restricting kernels to local neighborhoods improves efficiency with possible accuracy loss.For J discretization points, fully connected graphs have O(J^2) edges; Nyström, multipole, and multiwavelet methods reduce computational cost.
- 4.2 Integral Neural Operators through Graph Kernel Learning: NKNs were proposed because deeper GKNs can become unstable, and their alternative integration layers associate network depth with time-discretization steps.This design targets more complicated and general learning tasks while retaining lifting and projection structures shared with related neural operators.
- 4.2 Integral Neural Operators through Graph Kernel Learning: Under square-integrable non-negative kernels and positive bounded reaction functions, NKNs are stable in the deep-layer limit, unlike the instability indicated for GKNs.For the 2D Darcy-flow comparison, NKN solutions were visually consistent with ground truth, while GKNs and FNOs lost accuracy in specified regions.
5 Neural Operator Theory
Neural operator theory establishes approximation results for DeepONets, FNOs, and graph-based approaches, while also identifying efficiency and stability limits for general operator learning.
- 5 Neural Operator Theory: DeepONet theory extends universal operator approximation to deep neural networks, with exponential approximation rates established for linear advection-diffusion equations.The cited work also considers nonlinear cases.
- 5 Neural Operator Theory: FNOs are universal for continuous operators, but their worst-case size can grow super-exponentially with the desired approximation accuracy for general Lipschitz operators.Efficient ψ-FNO approximation was established for Darcy flow and incompressible Navier–Stokes equations.
- 5 Neural Operator Theory: Physics-informed operator learning has established error bounds for both DeepONets and FNOs.Other work analyzes spectral convergence rates for a broad family of graph Laplacians.
- 5 Neural Operator Theory: Graph Kernel Networks have been reported to achieve competitive approximation accuracy relative to classical and deep learning methods.
6 Applications
The applications evaluate data-driven and physics-informed neural operators across porous-media flow, fracture, cavity flow, and biological-tissue modeling. Results show strong accuracy in several in-distribution settings, while out-of-distribution loading remains challenging and can benefit from partial physics constraints.
- Evaluation scope: Neural operators are evaluated using relative L2 prediction error across data-driven and physics-informed computational-mechanics examples.The application section covers porous-media flow, complex-domain Darcy flow, cavity flow, brittle fracture, and biological tissue modeling.
- 6.1.2 Darcy flow in a complex domain: 1.0%, 2.02%, and 7.12% relative L2 errors were obtained by POD-DeepONet, DeepONet, and dgFNO+, respectively, for Darcy flow in a notched triangular domain.The comparison used unseen boundary conditions generated for the complex geometry.
- 6.1.3 Flow in a cavity: 0.33%, 0.63%, and 1.20% relative L2 errors were reported for POD-DeepONet, dFNO+, and DeepONet on unseen cavity-flow boundary conditions.Training data covered velocity fields across Reynolds numbers from 100 to 2080.
- 6.2.1 Brittle fracture in a plate loaded in shear: 2.16% prediction error on φ was reported for PI-DeepONet, compared with 26.2% using data-driven DeepONet trained with 11 samples and 3.12% with 43 samples.The data-driven model was reported unable to capture crack diffusion and generalize to complex fracture phenomena with limited datasets.
- 6.2.3 Biological tissue modeling from experimental measurements: 3.12% predictive error was obtained for PI-DeepONet in heterogeneous porous-media flow, whereas a residual-based DeepONet could not approximate h(x) for a given K(x).The physics-informed model was trained using the variational formulation without labelled input-output datasets.
- 6.2.3 Biological tissue modeling from experimental measurements: 16.78% and 16.80% prediction errors were observed for implicit FNO and the Fung-type model on out-of-distribution loading ratios, while partial physics improved extrapolative performance by around 1.5%.The neural-operator approach was less effective out of distribution and had comparable performance to constitutive modeling before physics infusion.
7 Summary and Outlook
The chapter reviews DeepONet, FNO, GNO, and their extensions, emphasizing neural operators as fast surrogate models for computational mechanics. It identifies physics integration, scalable training, multifidelity data, and continual learning as routes toward real-time, large-scale applications.
- Review scope: The chapter reviews DeepONet, FNO, the graph neural operator, their extensions, and representative applications.The review covers neural-operator roles across computational mechanics and related domains.
- Applications: Real-time forecasting applications include efficient control systems, fault detectors, and complex multiphysics problems solved in less than a second.These applications are presented as roles for neural operators in computational mechanics.
- Hybrid and multifidelity modeling: Hybrid physics- and data-based models can provide efficient, accurate, and generalizable models for accelerating time-dependent multiscale systems.The chapter also identifies transfer learning and multifidelity data as ways to reduce dependence on large paired datasets and speed training.
- Broader applications: Neural operators are discussed for life-science applications including patient-specific aortic dissection and aneurysm prediction, potentially supporting surgery planning.Other listed applications include climate-model correction using learned high-order terms.
- Future directions: Continual learning at the operator-regression level is identified as a promising direction for real-time inference in large-scale computational mechanics and digital twins.The chapter notes that training DeepONet with only high-fidelity data is computationally expensive and points to multifidelity approaches and multi-GPU scaling.
- Physics-informed scaling: Physics-informed neural operators can enhance accuracy and generalization, including for tasks outside the input distribution, while supporting high-dimensional real-time design problems.The chapter cites a wind-plant problem involving 64 turbines and uncertainty in wind speed, inflow direction, and yaw angle.