Source-linked AI summary
Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems
Jeremy Yu, Lu Lu, Xuhui Meng, George Em Karniadakis
TL;DR
PINNs can solve diverse forward and inverse PDE problems but often have limited accuracy even with many training points. The paper introduces gPINNs, which add PDE-residual gradient information to the loss, and combines them with RAR. Across forward and inverse problems, gPINNs outperform PINNs with fewer training points, while RAR further helps on steep-gradient solutions.
Problem
PINNs have limited accuracy even with many training points, motivating methods that improve their accuracy and efficiency.
Method
gPINNs add gradients of the PDE residual to the PINN loss and can be combined with residual-based adaptive refinement.
Results
Across forward and inverse PDE problems, gPINNs outperform PINNs with the same number of training points, and learn unknown parameters more accurately.
Takeaways & Limitations
RAR further improves gPINNs, particularly for PDEs whose solutions have steep gradients, allowing good performance with fewer residual points.
Takeaways & Limitations
gPINNs typically cost 2 to 3 times more computationally than PINNs because their additional loss terms involve higher-order derivatives.
Abstract
from arXiv · showhide
Deep learning has been shown to be an effective tool in solving partial differential equations (PDEs) through physics-informed neural networks (PINNs). PINNs embed the PDE residual into the loss function of the neural network, and have been successfully employed to solve diverse forward and inverse PDE problems. However, one disadvantage of the first generation of PINNs is that they usually have limited accuracy even with many training points. Here, we propose a new method, gradient-enhanced physics-informed neural networks (gPINNs), for improving the accuracy and training efficiency of PINNs. gPINNs leverage gradient information of the PDE residual and embed the gradient into the loss function. We tested gPINNs extensively and demonstrated the effectiveness of gPINNs in both forward and inverse PDE problems. Our numerical results show that gPINN performs better than PINN with fewer training points. Furthermore, we combined gPINN with the method of residual-based adaptive refinement (RAR), a method for improving the distribution of training points adaptively during training, to further improve the performance of gPINN, especially in PDEs with solutions that have steep gradients.
1. Introduction
PINNs offer a mesh-free approach for solving diverse forward and inverse PDE problems, but their accuracy and efficiency remain open problems. The paper develops gPINNs, which add PDE-residual gradient information, and evaluates them alongside adaptive refinement.
- 1. Introduction: PINNs solve PDEs by embedding the PDE into a neural-network loss using automatic differentiation.The formulation is mesh-free and applies to integro-differential, fractional, and stochastic PDEs.
- 1. Introduction: Improving PINN accuracy and efficiency remains an open problem involving sampling, weighting, loss balancing, domain decomposition, and network architecture.The introduction lists residual-based adaptive refinement and importance sampling among proposed improvements to training-point distribution.
- 1. Introduction: gPINNs add gradient information from the PDE residual to the loss function to improve PINN accuracy and training efficiency.The method also combines gPINNs with residual-based adaptive refinement.
- 1. Introduction: The study evaluates PINN, gPINN, PINN with RAR, and gPINN with RAR across eight function-approximation, forward-PDE, and inverse-PDE problems.The comparisons are presented in the numerical-results section.
2. Methods
The method approximates PDE solutions with neural networks trained on PDE, boundary, initial, and optional measurement constraints. gPINNs additionally penalize spatial derivatives of the PDE residual, while RAR adds points where the residual is largest.
- PINN formulation: PINNs approximate the PDE solution with a trainable neural network and minimize losses evaluated at interior and boundary points.For inverse problems, additional measurements of u support simultaneous learning of unknown parameters and the solution.
- PINN formulation: The PINN loss combines PDE, boundary, and initial-condition terms, with weights assigned to the respective losses.The study sets wf = wb = wi = 1 in its examples.
- gPINN formulation: gPINNs enforce derivatives of the PDE residual to be zero in addition to enforcing the residual itself to be zero.Derivative losses are defined for residual-point sets Tgi; in the study, Tgi is chosen to match Tf.
- gPINN formulation: gPINNs improve predicted solution and derivative accuracy while requiring fewer training points in the numerical examples.The claimed improvements motivate adding residual-gradient terms to the loss.
- gPINN with RAR: RAR adaptively adds residual points where the PDE residual is largest during training.The algorithm repeats training, residual evaluation, point addition, and stopping checks for a prescribed number of iterations or residual threshold.
3. Results
The experiments use tanh activations and problem-specific hyperparameters to evaluate the proposed methods across the paper’s numerical problems.
- 3. Results: All numerical examples use tanh as the activation function and specify the remaining hyperparameters by problem.The hyperparameters are listed in Table 1.
3.1. Function approximation via a gradient-enhanced neural network (gNN)
The function-approximation experiment compares a standard neural network with a gradient-enhanced network using sparse training data. Adding gradient information substantially improves both solution and derivative predictions.
- 3.1. Function approximation via a gradient-enhanced neural network (gNN): The experiment compares standard NN and gradient-enhanced gNN models for approximating a function and its derivative.The gradient-enhanced model adds a gradient loss to the standard training objective.
- 3.1. Function approximation via a gradient-enhanced neural network (gNN): The gNN accuracy is insensitive to tested gradient-loss weights of 1, 0.1, and 0.01, so the study reports wg = 1.These results are based on different training runs with the listed weights.
- 3.1. Function approximation via a gradient-enhanced neural network (gNN): With more training points, gNN achieves about one order of magnitude smaller L2 relative error for u than NN and is more accurate for du/dx.The comparison covers both the function and its derivative.
- 3.1. Function approximation via a gradient-enhanced neural network (gNN): With 15 training data points, NN has more than 10% error for u and du/dx, while gNN reaches about 1% error.The predictions are illustrated in Figure 1.
3.2. Forward PDE problems
Forward PDE experiments show that gPINN can substantially reduce solution, derivative, and residual errors versus PINN, often with fewer training points, although its performance depends on the gradient-loss weight and optimization settings.
- 3.2.1. Poisson equation: For the 1D Poisson problem, gPINN with w = 0.01 outperforms PINN in solution, derivative, and PDE-residual errors.With 20 residual points, gPINN’s solution error is about one order of magnitude smaller, while its derivative error is about two orders smaller.
- 3.2.1. Poisson equation: An improperly chosen gradient-loss weight can make gPINN worse than PINN, with an optimum around w = 0.01 in the Poisson example.When w is smaller than 1, gPINN outperforms PINN in the reported comparison.
- 3.2.2. Diffusion-reaction equation: In the diffusion-reaction system, gPINN is insensitive to w values of 0.01, 0.1, and 1 and outperforms PINN by up to two orders of magnitude.gPINN reaches 1% L2 relative error of u with 40 training points, whereas PINN requires more than 100.
- 3.2.2. Diffusion-reaction equation: With 50 residual points, gPINN achieves a largest absolute error around 0.007 and an L2 relative error of 0.2%, compared with about 100% PINN error.
- 3.2.2. Diffusion-reaction equation: A gPINN plateau near 130 training points is attributed to network optimization, while a smaller learning rate and more iterations reduce error below 0.01% with 140 points.The improved settings use a learning rate of 10^-6 and 5 × 10^6 iterations.
3.3. Inverse problems of PDEs
The paper evaluates gPINN for inverse PDE problems, including inferring effective viscosity and permeability from sparse or noisy velocity measurements and recovering a spatially varying reaction rate. Across these tests, gPINN improves parameter or function inference and solution accuracy relative to PINN, particularly with few training points or noisy data.
- Brinkman-Forchheimer model: When inferring both νe and K from 5 velocity measurements, both methods accurately recovered K, while gPINN converged to a much more accurate νe.
- Noisy measurements: With Gaussian noise of standard deviation 0.05 and 12 velocity measurements, gPINN learned u and νe well, while PINN struggled unless its PDE training points were doubled.PINN 2x performed well for νe but remained slightly worse than gPINN.
- Space-dependent reaction rate: For a diffusion-reaction system with unknown k(x), gPINN predicted the reaction-rate function and du/dx more accurately than PINN using 8 observations and 10 residual points.Both methods learned u well, although PINN slightly deviated from the exact solution near x = 0.8.
3.4. gPINN enhanced by RAR
For steep-gradient PDE solutions, RAR adaptively concentrates residual points in high-error regions, improving gPINN performance and reducing the points needed for accuracy.
- Burgers’ equation: gPINN reduces the Burgers-equation L2 relative error from PINN’s ∼10% to below 1%.The comparison uses the same problem and reports approximately one order of magnitude improvement.
- Burgers’ equation: RAR adds residual points near x = 0, where the Burgers solution has a steep gradient.The procedure begins with 1500 uniformly distributed residual points and adds 400 points during training.
- Burgers’ equation: gPINN with RAR consistently decreases solution and PDE-residual errors as adaptively added points increase.The method adds new points near x = 0, similarly to PINN with RAR.
- Burgers’ equation: With 100 extra training points, gPINN with RAR reduces error faster than PINN with RAR, although their final accuracies are similar.The combined method is reported to achieve the best performance overall.
- Allen–Cahn equation: For Allen–Cahn, gPINN reaches 1% error with around 2000 training points, whereas PINN requires around 4000.The equation’s solution contains multiple very steep regions.
- Allen–Cahn equation: RAR places added points in Allen–Cahn’s high-error regions around x = −0.5 and x = 0.5.The experiment starts with 500 uniformly distributed residual points and adds 3000 more during training.
4. Conclusion
The paper concludes that gradient-enhanced PINNs improve accuracy across forward and inverse PDE problems, while RAR further helps on steep-gradient solutions. These gains involve higher computational cost and may require tuning an additional loss-weight hyperparameter.
- Conclusion: gPINN outperforms PINN with the same number of training points on solution and derivative L2 relative errors across the examples.The examples include forward and inverse PDE problems, and gPINN learns unknown inverse-problem parameters more accurately.
- Conclusion: RAR allows gPINN to perform well with much fewer residual points for PDEs with especially steep gradients.The paper specifically identifies Burgers’ and Allen–Cahn equations as examples.
- Conclusion: gPINN typically costs 2 to 3 times more computationally than PINN because its additional loss terms use higher-order derivatives.In some cases, PINN with twice as many training points has similar performance, although gPINN remains better for Burgers’ equation.
- Conclusion: gPINN introduces an extra hyperparameter: the weight coefficient of the gradient loss.Some problems are insensitive to this weight, while others require tuning an optimal value for best accuracy.