Source-linked AI summary
hp-VPINNs: Variational Physics-Informed Neural Networks With Domain Decomposition
Ehsan Kharazmi, Zhongqiang Zhang, George Em Karniadakis
TL;DR
The paper addresses how neural-network solvers for differential equations can combine nonlinear approximation with localized hp-refinement. It introduces hp-VPINNs using a global neural-network trial space and piecewise-polynomial test functions over decomposed sub-domains, and reports accuracy advantages in several numerical examples.
Problem
Neural-network differential-equation solvers require appropriate loss-function designs, while existing formulations do not provide the same combination of domain-decomposition h-refinement and high-order-polynomial p-refinement.
Method
hp-VPINNs use a global neural-network trial space with non-overlapping, piecewise-polynomial test functions on sub-domains to implement localized hp-refinement.
Results
The authors demonstrate improved accuracy or localized approximation behavior across function-approximation and differential-equation examples, including sharper interior accuracy under fine domain decomposition.
Takeaways & Limitations
hp-VPINNs provide a flexible framework for local and global approximations, accommodating singularities, steep solutions, and sharp changes while localizing network-parameter optimization.
Takeaways & Limitations
The loss-function weight coefficients may require manual or automatic tuning, and their optimal bound remains an open problem.
Abstract
from arXiv · showhide
We formulate a general framework for hp-variational physics-informed neural networks (hp-VPINNs) based on the nonlinear approximation of shallow and deep neural networks and hp-refinement via domain decomposition and projection onto space of high-order polynomials. The trial space is the space of neural network, which is defined globally over the whole computational domain, while the test space contains the piecewise polynomials. Specifically in this study, the hp-refinement corresponds to a global approximation with local learning algorithm that can efficiently localize the network parameter optimization. We demonstrate the advantages of hp-VPINNs in accuracy and training cost for several numerical examples of function approximation and solving differential equations.
1. Introduction
The paper develops hp-VPINNs for differential equations by combining neural-network trial functions with variational test functions and hp-refinement through domain decomposition and high-order polynomial projection.
- Motivation: Neural networks provide nonlinear trial approximations for differential equations through compositions of hidden layers and varied activation functions.Their nonlinear approximation space extends beyond conventional linear approximation spaces.
- Proposed framework: hp-VPINNs formulate neural-network differential-equation solvers as optimization problems using a sub-domain Petrov-Galerkin method.The neural network remains the trial space, while domain decomposition supplies h-refinement and high-order polynomial projection supplies p-refinement.
- Variational formulation: Residuals quantify violations of the governing, boundary, and initial equations, and their weighted projections onto test functions define the variational formulation.The resulting nonlinear system is solved through minimization of a weighted loss function.
- Open issue: The loss weights for governing, boundary, and initial residuals may be manually or automatically tuned, but their optimal bound remains an open problem.This leaves loss-weight selection as an unresolved aspect of the formulation.
- Related formulations: The framework distinguishes strong-form, collocation, least-squares, and variational formulations through different choices of trial and test functions.PINNs use Dirac delta tests at collocation points, DGM uses the residual as a test function, and VPINNs use orthogonal polynomial tests.
2. hp-Variational Physics-Informed Neural Network (hp-VPINN)
hp-VPINNs localize variational residual enforcement with non-overlapping polynomial test functions on decomposed sub-domains, while retaining a single neural network over the full domain.
- Localized test space: Localized test functions are supported on non-overlapping sub-domains and use polynomial functions whose order is selected in practice.This construction yields a sub-domain method with local variational residuals.
- Elementwise loss: The elemental variational residual is enforced for admissible local test functions, then aggregated into a variational loss over the test functions in each element.The formulation accounts for elementwise residual entries and boundary or initial residuals.
- Numerical considerations: Projection onto test functions introduces truncation and numerical-integration errors in addition to neural-network approximation and generalization errors.Increasing the number of test functions can complicate optimization, while deep networks generally require numerical quadrature.
- Numerical considerations: Gauss quadrature is used for integration, with quasi-Monte Carlo or sparse-grid approaches suggested for avoiding dimensionality problems in high-dimensional settings.The integration strategy is needed because deep-network variational integrals are generally not analytically computable.
- Parallelization boundary: Although domain decomposition creates local sub-domains, the current formulation still uses one global DNN and one associated loss function.Consequently, assigning optimization on each sub-domain to separate computer nodes may not be straightforward.
3. Variational Neural Networks (VNNs) for Function Approximation
VNNs use neural networks as trial functions and polynomial test spaces, with global, local, and multi-elemental formulations supporting localized approximation. Function-approximation examples show accurate smooth-function fitting, improved treatment of discontinuities through local decomposition, and depth- and activation-dependent convergence.
- VNN formulation: VNNs use neural networks as trial functions and piecewise polynomial test functions, with hp-refinement implemented through elementwise test-function choices.The formulation supports adaptive selection of the number of test functions in each element.
- Continuous function approximation: Using ℓ = 4, N = 20, K = 60, and Q = 80, global VNN obtains an L∞ error of O(10^-3) for a smooth continuous target.The domain is also divided into three equally spaced sub-domains for local approximation.
- Continuous function approximation: Local VNN slightly improves approximation within each sub-domain by focusing network optimization on the locally supported target region.The network parameters are optimized to capture the function within the selected sub-domain.
- Discontinuous function approximation: For a discontinuous target with a jump of magnitude 6, global VNN has L∞ error O(10^-1) near the discontinuity and misses the high-frequency component.It learns the low-frequency component but fails to capture the high-frequency index.
- Discontinuous function approximation: Local and multi-elemental VNNs capture the high-frequency component near the discontinuity more accurately, with multi-elemental VNN accurately representing the target in Fourier space.The multi-elemental formulation uses locally defined test functions across multiple sub-domains.
- Convergence by depth: Increasing network depth while holding width constant reduces error at activation-dependent rates, but the error saturates beyond a certain depth.The study compares sine, tanh, and ReLU activations; derivative learning is less successful with ReLU than with sine and tanh.
4. One-Dimensional Poisson’s Equation
The one-dimensional formulation applies localized, nonoverlapping polynomial test functions over decomposed elements to solve Poisson problems with steep, boundary-layer, and asymmetric sharp changes. Compared with PINN, VPINN achieves much smaller errors in steep and boundary-layer cases, while increasing elements helps capture asymmetric sharp changes.
- Formulation: The domain is divided into non-overlapping elements, with localized high-order polynomial test functions defining the elemental variational residual.Integrals are mapped from each element to the standard domain for computation.
- Implementation: VPINN uses a fully connected network with four hidden layers, 20 neurons per layer, sine activation, Legendre polynomials up to order 60, and 80 Gauss-Lobatto points per element.The loss is optimized with Adam and automatic differentiation is implemented through TensorFlow.
- Numerical results: For steep and boundary-layer solutions, VPINN errors are orders of magnitude smaller than PINN errors.PINN requires more residual points near sharp changes and a higher boundary-loss weight, while large force terms can sometimes cause optimization failure.
- Numerical results: Asymmetric sharp changes are poorly approximated with one symmetric-test-function element, but the network eventually captures the solution as the number of elements increases.The sharp-change location is not assumed known in advance and is obtained through successive domain decompositions.
5. Two-Dimensional Poisson’s Equation
The two-dimensional hp-VPINN formulation uses tensor-product localized test functions on structured, non-overlapping subdomains. Domain decomposition is ineffective for the smooth Poisson solution but improves interior accuracy for sharp or singular features, while sharp edges can remain the dominant error source.
- Formulation: The discrete test space uses tensor products of admissible one-dimensional functions, with K1 and K2 test functions in the two coordinate directions.The resulting variational residuals are assembled across elements.
- Formulation: The two-dimensional domain is partitioned into structured, non-overlapping rectangular elements using separate grids in the x and y directions.Variational integrals are mapped to the standard square element through affine transformations.
- Smooth solution: For the smooth homogeneous Poisson solution, PINN and hp-VPINN all achieve point-wise errors of order O(10^-4), and domain decomposition does not improve approximation.The decomposition may still support parallel computation by solving subdomains on separate computer nodes.
- Steep solution: For a solution with a steep change along x, error convergence is studied by successively increasing the number of subdomains in both coordinate directions.The experiments use five test functions per direction and 10 × 10 quadrature points per subdomain.
- Singular and sharp features: For the L-shaped-domain problem, fine decomposition decreases interior error, while the sharp-edge error remains dominant.A fine decomposition uses 35 subdomains; p1qR and p2qR formulations produce similar error levels in this case.
6. Advection Diffusion Equation (Inverse and Forward Problems)
The advection-diffusion formulation extends variational domain decomposition to space-time and estimates an unknown diffusivity from sparse sensor time series. After convergence of the diffusion coefficient, the remaining large point-wise error is concentrated near the right boundary.
- Forward formulation: The space-time domain is divided into structured, non-overlapping elements across temporal and spatial coordinates.Time is treated as another dimension in the variational residual formulation.
- Inverse problem: The inverse problem estimates the unknown diffusion coefficient κ from measured solution values available as time series at three sensor locations.The advection velocity is known and fixed at v = 1, while κ is initialized at one.
- Implementation: The VPINN uses Legendre test functions in space and time with five functions in each direction and 10 × 10 quadrature points.The reported setup uses the p1qR formulation, 160 boundary points, 80 initial points, and 15 observation points.
- Results: After convergence of the diffusion coefficients, the point-wise error has its largest magnitude near the right boundary at t = 1.The convergence study reports the mean diffusivity, its standard deviation, and the loss function over 10 randomly selected cases.
7. Summary
The paper develops hp-VPINNs as a sub-domain Petrov-Galerkin method for differential equations, combining neural-network trial functions with localized polynomial test functions. Numerical examples address approximation, nonsmooth solutions, and inverse parameter estimation, while long-time integration may benefit from a discrete-in-time variant.
- Summary: hp-VPINN uses neural networks as the trial space and localized, nonoverlapping high-order polynomials as the test space.The formulation follows weighted residuals and permits different choices of test functions.
- Summary: The method is evaluated on function approximation, differential equations with nonsmooth solutions, and inverse parameter estimation in an advection-diffusion equation.The paper develops the formulation and corresponding variational losses in one and two dimensions.
- Summary: The paper discusses hp-refinement and solution convergence for equations with nonsmooth solutions, alongside the efficiency and accuracy of hp-VPINNs.These properties are examined through several numerical examples and comparisons with other methods.
- Scope: For time-dependent problems and long-time integration, a discrete-in-time hp-VPINN may be more efficient.The paper identifies this as a possible development rather than part of the presented formulation.
B C D
Figure 13 presents the exact solution and hp-VPINN h-refinement results for a (1+1)-dimensional advection-diffusion equation across three subdomain counts. The experiments use a fixed fully connected tanh network and specified optimization parameters.
- Figure 13A shows the exact solution of the (1+1)-dimensional advection-diffusion equation.
- Figures 13B–D show hp-VPINN h-refinement with the p1qR formulation for Nelx = 1, 2, and 4.
- The experiments use a fully connected network with ℓ = 3, N = 5, tanh activation, Q = 10 × 10 points per subdomain, and 80 boundary points.Adam optimization uses learning rate 10^-3, with τb = 10 and K1 = K2 = 5.
A B C
Figure 14 examines diffusivity estimation for a (1+1)-dimensional advection-diffusion equation using an exact solution, sensor observations, VPINN predictions, errors, and training diagnostics.
- Figure 14A combines the exact solution with one realization of N* = 15 randomly selected points at sensor locations.
- Figures 14B and 14C present the VPINN prediction and its point-wise error, respectively.
- Figures 14D and 14E show diffusivity-coefficient convergence and loss values versus training iterations.