PaperlayerView exact revision on arXiv

Source-linked AI summary

Long-time integration of parametric evolution equations with physics-informed DeepONets

Sifan Wang, Paris Perdikaris

arXiv:2106.05384v1cs.LGphysics.comp-ph

TL;DR

Long-time ODE/PDE simulation remains difficult for machine-learning methods that may lose accuracy over extended horizons. This paper learns physics-informed DeepONet solution operators over short intervals without paired input-output data, then iterates them to obtain long-time predictions, reporting accurate simulations across several systems at a fraction of classical-solver cost.

  • Problem

    Existing machine-learning approaches can struggle to provide stable and accurate ODE/PDE predictions over long temporal horizons.

  • Method

    Physics-informed DeepONets learn operators mapping random initial conditions to short-time solutions without paired input-output observations, then iteratively reuse predictions as subsequent initial conditions.

  • Results

    The approach produces accurate long-time simulations across a range of ODE and PDE systems, including wave propagation, reaction-diffusion dynamics, and stiff chemical kinetics, at a fraction of classical-solver cost.

  • Takeaways & Limitations

    The proposed temporal domain decomposition uses one network trained over a short interval and a distribution of initial conditions instead of training multiple networks.

  • Takeaways & Limitations

    Approximation-error effects on stability and accuracy remain insufficiently understood, while classical numerical solvers remain the de-facto choice when guarantees are required.

Abstract

from arXiv · show

Ordinary and partial differential equations (ODEs/PDEs) play a paramount role in analyzing and simulating complex dynamic processes across all corners of science and engineering. In recent years machine learning tools are aspiring to introduce new effective ways of simulating PDEs, however existing approaches are not able to reliably return stable and accurate predictions across long temporal horizons. We aim to address this challenge by introducing an effective framework for learning infinite-dimensional operators that map random initial conditions to associated PDE solutions within a short time interval. Such latent operators can be parametrized by deep neural networks that are trained in an entirely self-supervised manner without requiring any paired input-output observations. Global long-time predictions across a range of initial conditions can be then obtained by iteratively evaluating the trained model using each prediction as the initial condition for the next evaluation step. This introduces a new approach to temporal domain decomposition that is shown to be effective in performing accurate long-time simulations for a wide range of parametric ODE and PDE systems, from wave propagation, to reaction-diffusion dynamics and stiff chemical kinetics, all at a fraction of the computational cost needed by classical numerical solvers.

1 Introduction

The paper motivates machine-learning approaches for simulating ODEs and PDEs, emphasizing the limitations of long-time prediction and proposing physics-informed DeepONets to learn short-time solution operators. These operators are trained without paired input-output data and iterated to produce long-time solutions.

  • Classical ODE/PDE solvers typically use finite-dimensional representations based on fixed features such as polynomials or trigonometric functions.
  • Physics-informed neural networks represent an entire spatio-temporal PDE solution with one network trained against observed initial-boundary conditions and PDE residuals.
  • The paper learns operators mapping random initial conditions to associated ODE/PDE solutions over short time intervals using physics-informed DeepONets.
  • The learned model is iteratively evaluated, using each short-time prediction as the next initial condition to construct long-time solutions across initial conditions.

2 Physics-informed neural networks

The section reviews physics-informed neural networks, which enforce differential-equation, initial-condition, and boundary-condition constraints through a composite loss. A gravity-pendulum example illustrates that conventional PINNs can fail over long horizons, with predictions collapsing after t = 10.

  • PINNs approximate unknown ODE/PDE solutions with neural networks and compute spatial or temporal derivatives using automatic differentiation.
  • The physics-informed loss combines residual, boundary-condition, and initial-condition terms evaluated at sampled points, with weights controlling their relative contributions.
  • The gravity-pendulum test uses a five-layer fully connected network to approximate a two-dimensional ODE solution through T = 20.
  • After 10^5 gradient-descent iterations, the conventional PINN predictions collapse to zero after T = 10, indicating inaccurate long-time integration.
  • Possible explanations include saturated activations from large coordinates and spectral bias against high-frequency or complex functions.

3 Methods

The method uses physics-informed DeepONets to learn solution operators from initial conditions to short-time ODE/PDE solutions, then recurrently composes those predictions for long-time integration. The approach combines branch and trunk networks with physics-based regularization and achieves low relative errors in a gravity-pendulum test.

  • 3.1 A primer on physics-informed DeepONets: DeepONets use a branch network for input-function features and a trunk network for coordinate features, merging them through a dot product to produce the output function.
  • 3.1 A primer on physics-informed DeepONets: Automatic differentiation of the continuously differentiable DeepONet output enables PDE-residual regularization for physics-informed training.
  • 3.2 Long-time integration of evolution equations: The long-time strategy trains one physics-informed DeepONet on t ∈ [0, Δt] across initial conditions, then feeds each prediction at Δt into the next evaluation.
  • 3.2 Long-time integration of evolution equations: The recurrent procedure infers short-time solutions, updates the input function with each prediction, and concatenates the inferred segments over the full temporal domain.
  • 3.2 Long-time integration of evolution equations: For the gravity pendulum, an eight-layer branch-trunk DeepONet trained on 5 × 10^4 initial conditions achieved relative L2 errors of 1.72% for s(1) and 1.63% for s(2).

4 Results

The results evaluate physics-informed DeepONets with temporal domain decomposition across inhomogeneous, stiff, and long-time dynamical systems. The approach achieves accurate predictions, including for varied initial conditions, while requiring relatively low inference cost.

  • Benchmark setup: The modified fully-connected physics-informed DeepONet architecture is used throughout the benchmarks.The architecture includes transformer sub-networks and is trained with Tanh activations, Glorot initialization, mini-batch Adam optimization, and exponential learning-rate decay.
  • Inhomogeneous ODEs: The inhomogeneous ODE requires a parametric formulation because the forcing term must be supplied as a varying input during temporal iteration.The modified DeepONet uses separate branch networks for the initial condition and forcing term, with a shared trunk network for input coordinates.
  • Inhomogeneous ODEs: The inhomogeneous ODE reaches a relative L2 error of 0.84% over T = 1000 and retains approximately 0.5% accuracy up to T = 100 across varied initial conditions.The model also performs well for initial conditions sampled from a different function space than the training inputs.
  • Stiff chemical kinetics: For stiff chemical kinetics, the scaled model obtains relative L2 errors of 0.38%, 0.56%, and 0.64% for s1, s2, and s3, respectively, while the unscaled model fails even at T = 1.The scaled model is evaluated to T = 500; the unscaled model produces errors of 3.05%, 37.01%, and 41.93% for s1, s2, and s3.
  • Computational cost: Inference takes O(10−3ms) per short-time query and typically ∼O(10−2) sec for a global solution requiring N ∼O(10)−O(103) model evaluations.The implementation can vectorize and parallelize simulations for multiple initial conditions on GPU hardware.
  • Long-time PDE prediction: For wave propagation, temporal domain decomposition produces a relative L2 error of 1.24e −02 over a long-time horizon, while the vanilla DeepONet error is 6.84e −02.The comparison reports that the physics-informed model is slightly better than the vanilla DeepONet for the evaluated traveling-wave problem.

5 Discussion

The paper proposes iterative physics-informed DeepONets for long-time integration, using short-time solution operators and temporal domain decomposition. Korteweg–De Vries results show lower relative L2 error for the physics-informed model than for a conventional DeepONet, while unresolved stability and chaotic-system questions remain.

  • Method: A single physics-informed DeepONet learns short-time solution operators and is iteratively evaluated to construct long-time ODE/PDE solutions across initial conditions.This temporal domain-decomposition strategy avoids training multiple networks and supports arbitrary spatio-temporal resolutions.
  • Motivation: The proposed framework targets long-time simulation bottlenecks arising in complex scientific and engineering processes.Applications mentioned include atmospheric and ocean chemical transport, combustion, and cardiovascular biophysics.
  • Korteweg–De Vries results: 1.24% relative L2 error is obtained for the Korteweg–De Vries equation up to T = 100 with the physics-informed DeepONet.The predicted and exact solutions are compared at t = 0, 50, and 100.
  • Korteweg–De Vries results: 6.84% relative L2 error is obtained for the conventional DeepONet on the same Korteweg–De Vries long-time integration task.The conventional model is evaluated at the same temporal horizon and snapshots as the physics-informed model.
  • Limitations: The approach remains at an early stage, with open questions about error-driven stability, convergence guarantees, chaotic systems, and turbulent Navier–Stokes flows.The paper notes that classical numerical solvers remain the de-facto choice where guarantees are required.

A Notations

The appendix identifies the work’s notation and summarizes its principal symbols in a nomenclature table.

  • Notation: Table 1 summarizes the main symbols and notation used throughout the work.The table provides the appendix’s nomenclature reference.

B Hyper-parameter settings

The appendix records the default hyper-parameter settings and DeepONet architectures used across the benchmark problems.

  • Hyper-parameters: Table 2 summarizes default hyper-parameter settings for each benchmark unless otherwise stated.These settings provide the common experimental configuration reference.
  • Architectures: Table 3 summarizes the DeepONet architectures used for each benchmark unless otherwise stated.The table provides the architecture reference accompanying the benchmark settings.

C Performance metrics

Performance is evaluated using the relative L2 norm, averaged over all examples in the test data set.

  • Error metric: The reported test error is the mean relative L2 error of a trained physics-informed DeepONet over the test examples.This is the error metric used throughout the numerical experiments.
  • Error metric: The relative L2 error compares predicted DeepONet outputs Gθ(u(i))(y) with ground-truth target functions G(u(i))(y) at equi-spaced domain points.N denotes the number of test examples and y is typically a set of equi-spaced points in G(u).

D Computational cost

Physics-informed DeepONets require more training computation than conventional DeepONets because automatic differentiation enlarges the computational graph. After training, however, the model supports rapid and highly parallelized long-time inference.

  • Physics-informed DeepONet training is generally slower than conventional DeepONet training.The additional cost comes from computing PDE residuals through automatic differentiation, which produces a larger computational graph.
  • Each trained-model query typically takes O(10^-3 ms) on a single Nvidia V100 GPU.
  • More than 1,000 initial conditions can be integrated in O(1) second, yielding a 10x-50x speedup over a traditional numerical solver.The implementation is vectorized and parallelized on GPU hardware.

E.1 Gravity pendulum

For the gravity pendulum, the figure compares training-loss convergence between a conventional PINN and a physics-informed DeepONet under Adam optimization.

  • The conventional PINN training-loss convergence is shown over 10^5 Adam gradient-descent iterations.
  • The physics-informed DeepONet training-loss convergence is shown over 3 × 10^5 Adam gradient-descent iterations.
  • The figure places conventional PINN convergence on the left and physics-informed DeepONet convergence on the right.

E.2 Inhomogeneous ODE

For the linear ODE, the trained physics-informed DeepONet is evaluated across three initial conditions, with relative L2 errors ranging from 0.52% to 3.30%.

  • The linear-ODE physics-informed DeepONet training-loss convergence is shown over 10^5 Adam gradient-descent iterations.
  • The trained model predicts solutions for initial conditions s(0) = 0.2, 0.4, and 0.6 using Algorithm 1.
  • Relative L2 errors are 0.52%, 1.79%, and 3.30% for the three tested initial conditions, respectively.

E.3 Stiff chemical kinetics

The figures assess physics-informed DeepONet training and predictions for stiff ODEs, while also showing training behavior for wave and diffusion-reaction systems.

  • Stiff chemical kinetics: Scaled and unscaled physics-informed DeepONet models for the stiff ODE are compared over 4 × 10^5 Adam iterations.
  • Stiff chemical kinetics: The stiff-ODE model produces predicted solutions for three different initial conditions using Algorithm 1.
  • Additional systems: Physics-informed DeepONet training-loss convergence is shown for the wave equation and diffusion-reaction system over 2 × 10^5 Adam iterations.
  • Additional systems: Predicted diffusion-reaction solutions are shown for three different initial conditions.
  • Stiff chemical kinetics: Stiff-ODE training-loss convergence compares a physics-informed DeepONet with a conventional DeepONet over 2 × 10^5 Adam iterations.
Loading 2106.05384v1…