Learning the solution operator of parametric partial differential equations with physics-informed DeepOnets

• Parametric PDEs require learning operators between infinite-dimensional function spaces rather than ordinary finite-dimensional functions.Source: §1 · p. 2
• Traditional solvers require independent simulations for different geometries, parameters, or initial and boundary conditions, creating substantial computational cost.Source: §1 · p. 2
• Large paired input-output datasets can require expensive experiments or high-fidelity simulations, motivating training from PDE knowledge and initial or boundary conditions alone.Source: §1 · p. 2
• Physics-informed DeepONets use automatic differentiation to regularize outputs toward satisfying governing PDE constraints, even without latent-output training data.Source: §1 · p. 2
• 1–2 orders of magnitude reduction in predictive errors and up to 100% reduction in required examples are reported for physics-informed DeepONets.Source: §1 · p. 2
• The paper develops and evaluates the approach through subsequent architectural, technical, and numerical studies across parametric PDE settings.Source: §1 · p. 3

• Parametric PDEs allow domain shapes, initial or boundary conditions, coefficients, and source terms to vary over specified ranges.Source: §2 · p. 3
• A DeepONet uses a branch net for sampled input functions and a trunk net for continuous coordinates, combining their feature embeddings through a dot product.Source: §2 · p. 3
• Training data pair sampled input functions and output evaluations across multiple coordinates, producing tensors with dimensions determined by samples, locations, sensors, and output dimension.Source: §2 · p. 4
• The anti-derivative experiment trains on 10,000 Gaussian-process input functions and corresponding RK45 solutions, using 100 sensors and testing 1,000 random samples.Source: §2 · p. 4
• Automatic differentiation computes derivatives of the inferred solution with respect to input coordinates so residuals can be compared with the input function.Source: §2 · p. 5
• A conventional DeepONet can fit the solution while approximating its derivative residual with step functions, producing large approximation error.Source: §2 · p. 5
• Similar poor predictions with ReLU and tanh indicate that universal operator approximation does not ensure compatibility with the ground-truth solution operator.Source: §2 · p. 5

• Physics-informed DeepONets extend DeepONets by minimizing PDE residuals alongside the operator-learning loss.The approach uses automatic differentiation to impose physical constraints through regularization.Source: §3 · p. 5
• The trained physics-informed DeepONet closely matches the exact solution and derivative, while satisfying the underlying ODE constraint.The result is reported over a representative input and supported by mean relative L2 errors over 1,000 test realizations.Source: Figure 3 · p. 6, §3 · p. 6
• Collocation points are randomly sampled from each output domain to approximately enforce the governing physical constraints.Source: §3 · p. 6
• The anti-derivative demonstration uses five-layer branch and trunk networks with 50 hidden units per layer and tanh activations.Training minimizes the composite loss for 40,000 Adam iterations.Source: §3 · p. 6
• Conventional DeepONets require more than 10x training data to achieve the same accuracy as their physics-informed counterpart.This comparison uses mean relative L2 error across different numbers of input-function samples.Source: §3 · p. 6, Figure 4 · p. 7

4.1 The anti-derivative operator

• The anti-derivative example trains a physics-informed DeepONet using input-function measurements and a zero initial condition, without paired input-output data.The model samples 10,000 input functions and uses the zero initial condition s(0)=0 as the available solution information.Source: §4.1 · pp. 7–8
• 40,000 training iterations produced excellent agreement between physics-informed DeepONet predictions and ground-truth solutions for test inputs.The conventional DeepONet instead learned a degenerate map that fit s(0)=0 but produced erroneous predictions for x>0.Source: §4.1 · p. 8
• Fourier feature embeddings improved high-frequency solution learning, whereas a conventional fully connected trunk failed to accurately capture high-frequency oscillations.The Fourier-feature model used σ=50 and was trained for 300,000 Adam iterations.Source: §4.1 · p. 9
• 7.12e-03 was the relative L2 prediction error averaged over 1,000 out-of-distribution test examples sampled with length scale l=0.2.Training inputs for this case used length scale l=0.01.Source: §4.1 · p. 9

4.2 Diffusion-reaction systems

• In the diffusion-reaction benchmark, physics-informed DeepONet achieved an average relative L2 error of ∼0.45%, compared with ∼1.92% for conventional DeepONet.The conventional model used 100 randomly selected solution measurements per input function, while the physics-informed model used the PDE constraints.Source: §4.2 · p. 11

4.3 Burgers’ equation

• O(10^3) Burgers-equation solutions can be predicted in a fraction of a second, with inference taking ∼10ms and reaching up to three orders of magnitude speedup over a conventional spectral solver.The inference is trivially parallelizable.Source: §4.3 · p. 13

• Physics-informed DeepONets reduce predictive errors by up to 1-2 orders of magnitude while improving generalization performance.Source: §5 · p. 16
• Physics-informed DeepONets can reduce the number of training examples required by up to 100%.Source: §5 · p. 16
• The framework can solve parametric PDEs without paired input-output observations, using initial or boundary conditions instead.Source: §5 · p. 16
• Open questions include optimal feature embeddings, network architectures, loss weights, and algorithms for training accuracy and robustness.Source: §5 · p. 16

• Table 4 summarizes the main symbols and notation used throughout the work.Source: A Nomenclature · p. 21, Table 4 · p. 21

• Most benchmarks use Tanh activations in both branch and trunk networks, while the Eikonal airfoil benchmark uses ELU activations.Source: B Hyper-parameter settings · p. 22
• Physics-informed DeepONets are trained with mini-batch Adam optimization using batches of 10,000 and exponential learning-rate decay.The decay rate is 0.9 every 1,000 iterations.Source: B Hyper-parameter settings · p. 22
• Table 5 provides default hyper-parameter settings for each benchmark unless otherwise stated.Source: Table 5 · p. 22
• Tables 6 and 7 list the physics-informed and conventional DeepONet architectures used for each benchmark.Source: Table 6 · p. 22, Table 7 · p. 22

Training

• Physics-informed DeepONet training is generally slower than conventional DeepONet training because it requires additional computations.Source: C Computational cost · p. 23
• Table 8 reports computational costs in hours for training DeepONet and physics-informed DeepONet models on a single NVIDIA V100 GPU.Source: Table 8 · p. 23

Inference

• ~10ms is sufficient for a trained physics-informed DeepONet to predict the entire spatio-temporal Burgers solution.Source: C Computational cost · p. 23
• O(10^3) PDEs can be solved in a fraction of a second, with up to three orders of magnitude speedup over a traditional spectral solver.Figure 14 compares inference timing for trained physics-informed DeepONets with conventional spectral-solver timing.Source: C Computational cost · p. 23, Figure 14 · p. 23

• 40,000 Adam iterations compare training-loss convergence for conventional DeepONets with different activations and physics-informed DeepONets using Tanh activations.Source: Figure 15 · p. 24
• A physics-informed DeepONet solves a 1D parametric ODE without paired input-output data, using only the initial condition during training.Source: Figure 16 · p. 24
• Predicted solutions and corresponding ODE residuals are shown across three 1D parametric ODE test examples.Source: Figure 17 · p. 25
• For irregular input functions, training-loss convergence is compared between conventional fully connected and Fourier feature architectures over 300,000 Adam iterations.Source: Figure 18 · p. 25
• The irregular-input ODE experiments report predicted solutions and ODE residuals for four in-distribution examples with each architecture, plus four out-of-distribution Fourier-feature examples.Source: Figure 19 · p. 26, Figure 20 · p. 26, Figure 21 · p. 27
• For a parametric diffusion-reaction system, training-loss convergence is compared with and without paired input-output data, and predictions are shown for three test examples.Source: Figure 22 · p. 28, Figure 23 · p. 29

• Relative L2 prediction error is averaged over 1,000 test examples for physics-informed DeepONets trained with different batch sizes.Source: Figure 24 · p. 30
• Training-loss convergence is compared across physics-informed DeepONets trained using different batch sizes.Source: Figure 25 · p. 30

G.1 Modified full-connected neural network

• The modified MLP forward pass is introduced as the architecture used in the subsequent formulation.Source: §G.1 · p. 31
• U and V are independently computed by applying the activation function to affine transformations of the network inputs X.Source: §G.1 · p. 31
• At each layer, Z(k) is computed from H(k), then gates an element-wise mixture of U and V to produce H(k+1).Source: §G.1 · p. 31
• The architecture adds transformer-network weights and biases to the parameters otherwise used in a standard fully connected network.Source: §G.1 · p. 31

Burgers’ equation

• A modified MLP architecture is used to predict solutions for three Burgers’ equation test examples.Source: Figure 26 · p. 31

Parametric Eikonal equation

• For the parametric Eikonal equation, training-loss convergence is shown with Tanh and ELU activations, and signed distance functions are predicted for three test examples.Source: Figure 27 · p. 32, Figure 28 · p. 33