Source-linked AI summary
PPINN: Parareal Physics-Informed Neural Network for time-dependent PDEs
Xuhui Meng, Zhen Li, Dongkun Zhang, George Em Karniadakis
TL;DR
Long-time PDE integration can make PINN training computationally prohibitive because of large spatiotemporal datasets. The paper proposes PPINN, which combines a serial coarse solver with parallel fine PINNs, and reports accurate convergence in a few iterations with speed-ups that can grow with time-domain decomposition when coarse solvers are efficient.
Problem
Long-time integration creates large spatiotemporal datasets for PINN training, making direct computation potentially prohibitive.
Method
PPINN splits a long-time problem into short-time subproblems, using a simplified-PDE coarse solver for prediction and parallel fine PINNs encoding the true PDE for correction.
Results
PPINNs converge in a few iterations for tested ODE and PDE problems, with reported speed-ups increasing with subdomain count when the coarse solver is efficient.
Takeaways & Limitations
PPINN can provide accurate and computationally faster long-time PDE integration by combining small-data fine-PINN training with parallel execution.
Takeaways & Limitations
The reported convergence criterion uses a user-defined tolerance set to 1%, and Burgers-equation experiments use specified viscosity and domain conditions.
Abstract
from arXiv · showhide
Physics-informed neural networks (PINNs) encode physical conservation laws and prior physical knowledge into the neural networks, ensuring the correct physics is represented accurately while alleviating the need for supervised learning to a great degree. While effective for relatively short-term time integration, when long time integration of the time-dependent PDEs is sought, the time-space domain may become arbitrarily large and hence training of the neural network may become prohibitively expensive. To this end, we develop a parareal physics-informed neural network (PPINN), hence decomposing a long-time problem into many independent short-time problems supervised by an inexpensive/fast coarse-grained (CG) solver. In particular, the serial CG solver is designed to provide approximate predictions of the solution at discrete times, while initiate many fine PINNs simultaneously to correct the solution iteratively. There is a two-fold benefit from training PINNs with small-data sets rather than working on a large-data set directly, i.e., training of individual PINNs with small-data is much faster, while training the fine PINNs can be readily parallelized. Consequently, compared to the original PINN approach, the proposed PPINN approach may achieve a significant speedup for long-time integration of PDEs, assuming that the CG solver is fast and can provide reasonable predictions of the solution, hence aiding the PPINN solution to converge in just a few iterations. To investigate the PPINN performance on solving time-dependent PDEs, we first apply the PPINN to solve the Burgers equation, and subsequently we apply the PPINN to solve a two-dimensional nonlinear diffusion-reaction equation. Our results demonstrate that PPINNs converge in a couple of iterations with significant speed-ups proportional to the number of time-subdomains employed.
1 Introduction
PINNs encode physical laws and PDE residuals into neural-network training, but long-time integration creates prohibitively large training datasets. PPINN addresses this by splitting long-time problems into smaller, parallelizable PINN tasks guided by a fast coarse solver.
- PINNs encode physical conservation laws and prior physical knowledge while relying partly on data and partly on PDE-described physics.
- PINNs avoid PDE discretization by using automatic differentiation and neural-network optimization, making them grid-free but shifting complexity into training.
- PINN training minimizes boundary or initial-condition mismatch together with PDE residuals evaluated at random time-space points.
- Long-time PDE integration produces large spatiotemporal training datasets that can make direct PINN computation prohibitively expensive.
- PPINN splits one long-time problem into independent short-time problems supervised by an inexpensive coarse-grained solver, enabling faster and parallelized fine-PINN training.
2 Parareal PINN
PPINN combines a serial coarse solver with parallel fine PINNs in an iterative prediction-correction scheme. Its speed-up improves with time-domain decomposition when coarse propagation is efficient, while convergence is assessed against a user-defined tolerance.
- Methodology: PPINN divides [0, T] into N equal time subdomains and uses serial coarse and parallel fine propagators.
- Methodology: The coarse solver encodes a simplified PDE, while fine PINNs encode the true PDE to correct coarse predictions through iterative prediction and refinement.
- Methodology: The coarse solver initializes the full time domain, fine PINNs solve subdomains in parallel, and coarse updates refine interfaces between adjacent subdomains.
- Methodology: PPINN repeats correction and refinement until the convergence criterion is met, with Etol set to 1% in the reported study.
- Speed-up analysis: The PPINN walltime includes coarse initialization, serial coarse propagation, and parallel fine propagation, whereas serial PINN walltime scales with the number of subdomains.
- Speed-up analysis: More subdomains can increase PPINN speed-up when the coarse solver is efficient, although inefficient PINN coarse solvers can make speed-up decline.
3 Results
PPINN is evaluated on deterministic and stochastic ODEs, Burgers equations, and a two-dimensional diffusion-reaction problem. Across these examples, it reaches accurate solutions in about two iterations, while speed-up depends strongly on the efficiency and quality of the coarse solver.
- Method: PPINN splits long-time problems into subdomains, using a coarse PINN prediction and parallel fine PINNs for iterative correction.The deterministic ODE example uses 10 subdomains, a simplified ODE for the coarse solver, and the exact ODE for fine PINNs.
- Computational efficiency: Using a PINN as the coarse solver can reduce speed-up as subdomains increase because the coarse-solver cost dominates for N ≥40.More than 90% of computational time is taken by the coarse solver when N ≥40, motivating more efficient coarse solvers.
- Deterministic ODE: 1.252×10^-5 l2 relative error is obtained after two iterations for the deterministic ODE with an analytic coarse solver.The analytic coarse solution has no computational cost, and its negligible coarse-solver time enables superlinear speed-up.
- Deterministic ODE: 1.257 × 10^-5 l2 relative error is obtained after two iterations when finite difference is used as the deterministic ODE coarse solver.The FDM uses 1,000 uniform elements and produces a superlinear speed-up similar to the analytic-solution case.
- Stochastic ODE: The stochastic ODE predictions converge to the exact solutions after two iterations for β = 0.108 and β = 0.090.The same qualitative agreement is reported for PINN and analytic coarse solvers, although those solutions are not shown.
- Burgers equation: The Burgers-equation solution differs slightly from the exact solution after the first iteration but shows little difference after the second.The coarse PINN is smoother, and interface velocity is initially discontinuous because subdomain initial conditions are inaccurate.
- Burgers equation: Three- and four-times-larger coarse-solver viscosities yield comparable computational errors but require more iterations for convergence.Selecting the coarse solver therefore involves a trade-off between coarse-solver accuracy and the number of PPINN iterations.
- Diffusion-reaction equation: The diffusion-reaction problem meets the 1% convergence criterion in two iterations and agrees well with reference solutions at representative times.The first iteration is most accurate at t = 0.4, while discrepancies increase at t = 0.8 and 1 before improving after the second iteration.
4 Summary and Discussion
PPINN uses an efficient coarse solver to initialize independently parallel fine PINNs, reducing training cost for long-time integration. The method converges in few iterations on ODE and PDE tests and extends naturally toward spatial decomposition and multi-fidelity inverse problems.
- Summary and Discussion: PPINN uses coarse-solver solutions as initial conditions for independently parallel fine PINNs, reducing their training cost.The coarse solver may use a simplified, reduced-order, or other surrogate PDE.
- Summary and Discussion: PPINN converges in a few iterations for deterministic and stochastic ODEs and for Burgers and nonlinear diffusion-reaction PDEs.The ODE tests use 10 uniform time subdomains, while the PDE coarse solvers use smoother or simpler surrogate equations.
- Summary and Discussion: Superlinear speed-ups are achieved for the deterministic and stochastic ODE tests and demonstrated for the two-dimensional case.The reported speed-up depends on parallel execution of the fine PINNs after serial coarse propagation.
- Summary and Discussion: Training on small datasets can make CPUs comparable to GPUs because data-transfer overhead may dominate computation.This provides an additional implementation consideration for the small-data fine-PINN subproblems.
- Summary and Discussion: PPINN could extend to spatial domain decomposition using a cheap CG PINN to supervise and iteratively connect subdomain PINNs.The paper frames this extension by analogy with multigrid and multiresolution methods.
- Summary and Discussion: Using different equations in coarse and fine models enables PPINN to address multi-fidelity modeling of inverse physical problems.The proposed extensions are identified as future work.