Source-linked AI summary
Variational Physics-informed Neural Operator (VINO) for Solving Partial Differential Equations
Mohammad Sadegh Eshaghi, Cosmin Anitescu, Manish Thombre, Yizheng Wang, Xiaoying Zhuang, Timon Rabczuk
TL;DR
PDE simulation becomes costly across changing conditions, while neural operators struggle with generalization and data requirements. VINO minimizes PDE energy formulations using domain discretization and analytical derivatives and integrals. It outperforms existing approaches particularly as mesh resolution increases, with convergent fine-mesh behavior.
Problem
Neural operators struggle to generalize beyond training conditions, and physics-informed variants may still require paired data for sufficient accuracy.
Method
VINO incorporates PDE energy forms into neural operators, discretizing the domain and analytically handling derivatives and integrals.
Results
VINO outperforms existing approaches particularly as mesh resolution increases and exhibits convergent behavior on fine meshes.
Takeaways & Limitations
VINO provides a way to incorporate physical laws into neural operators while learning operators from physical data alone.
Takeaways & Limitations
VINO is currently configured only for PDEs with an available variational form.
Abstract
from arXiv · showhide
Solving partial differential equations (PDEs) is a required step in the simulation of natural and engineering systems. The associated computational costs significantly increase when exploring various scenarios, such as changes in initial or boundary conditions or different input configurations. This study proposes the Variational Physics-Informed Neural Operator (VINO), a deep learning method designed for solving PDEs by minimizing the energy formulation of PDEs. This framework can be trained without any labeled data, resulting in improved performance and accuracy compared to existing deep learning methods and conventional PDE solvers. By discretizing the domain into elements, the variational format allows VINO to overcome the key challenge in physics-informed neural operators, namely the efficient evaluation of the governing equations for computing the loss. Comparative results demonstrate VINO's superior performance, especially as the mesh resolution increases. As a result, this study suggests a better way to incorporate physical laws into neural operators, opening a new approach for modeling and simulating nonlinear and complex processes in science and engineering.
1 Introduction
Neural operators learn mappings between function spaces for PDE solution operators, but existing approaches struggle to generalize and often require paired data. VINO incorporates PDE energy forms into neural operators, analytically handling derivatives and integrals while improving performance on finer meshes.
- Neural operators: Neural operators learn mappings between input and solution functions across parametric PDE families.They map functions such as initial and boundary conditions to output solution functions.
- Limitations: Existing neural operators struggle to generalize beyond training conditions, while data generation may be costly, unavailable, or low resolution.Physics-informed variants still often require paired input-output data for sufficient accuracy.
- Limitations: Existing methods’ reliance on strong-form equations creates higher-order derivative and collocation-point challenges for purely physics-based learning.Weak-form approaches also add the challenge of evaluating integrals efficiently.
- VINO: VINO incorporates the energy form of physical laws into neural operators and discretizes the domain using shape functions.Analytical differentiation and integration address the associated computational challenges.
- VINO: VINO learns operators from physical data alone, uses consistent hyperparameters, and improves as mesh size increases, unlike PINO.The method exhibits converging behavior on finer meshes.
- Contribution: The study presents VINO as a way to incorporate physical laws into neural operators and makes supporting code and data publicly available.The repository is identified at https://github.com/eshaghi-ms/VINO.
2 Results
The results compare VINO with FNO and PINO across benchmark PDE examples and mesh resolutions. VINO’s error decreases and converges on finer meshes with constant hyperparameters, whereas PINO becomes less accurate and more variable at larger resolutions.
- Experimental design: The study evaluates VINO against FNO and PINO on three benchmark examples and three additional complicated cases.It also investigates how mesh resolution affects method accuracy.
- Darcy flow results: FNO maintains relatively consistent performance across mesh sizes, with a modest upward trend in median error and stable variability.Its boxplot variability remains relatively stable as resolution increases.
- Darcy flow results: PINO’s relative L2 error reaches its lowest median around the 32 × 32 mesh, then rises at 98 × 98 and 128 × 128.Its error variability also increases at larger mesh sizes.
- Darcy flow results: VINO’s median relative L2 error consistently decreases with finer meshes, while its error variability diminishes for smaller meshes.The results describe increasingly reliable accuracy improvements under mesh refinement.
- Darcy flow results: VINO exhibits converging behavior on finer meshes while using constant hyperparameters.This contrasts with the decreasing or inconsistent performance reported for other methods.
2.1 Comparative Numerical Results
VINO learns PDE solution operators by minimizing a variational loss without paired datasets, using domain discretization and elementwise analytical derivatives and integrals. Across benchmark problems and test functions, VINO generally achieves lower and more stable relative L2 errors than PINO, with similar training and computational demands.
- Method: VINO uses a variational loss derived from PDE functional minimization, eliminating the need for paired datasets.The method learns the solution operator directly through physics-based optimization.
- Method: Domain discretization and analytical elementwise derivatives and integrals allow VINO to operate across varying resolutions.On uniform rectangular grids, shared element matrices reduce computational and memory cost.
- Second-order anti-derivative: VINO exhibits smaller point-wise and relative L2 errors than competing methods on Gaussian-random-field, sinusoidal, and exponential test functions.The advantage is also reported for inputs that differ substantially from the training data.
- Comparative results: VINO generally has lower mean errors, smaller deviations, and narrower error distributions than PINO across benchmark problems.The reported training times and computational demands are approximately the same across methods.
- Poisson Equation: 0.00751 and 0.00631 are VINO and VINO+data mean L2 errors, respectively, for the Poisson equation benchmark.Data-enhanced VINO also has a standard deviation of 0.0026, while PINO has a deviation of 0.0192.
2.2 Practical Numerical Results
VINO is evaluated on porous structures and nonlinear hyperelasticity, addressing heterogeneous materials, large deformation, and geometric complexity. Across these examples, it produces stable convergence, agreement with reference solutions, and substantially faster post-training solution times than IGA.
- Experimental scope: Three examples test VINO on porous material structures, hyperelasticity, and plates with arbitrary holes, covering material heterogeneity, large deformation, and geometric complexity.The experiments are presented as evaluations of VINO’s versatility in engineering contexts.
- Porous material structure: VINO learns an operator mapping arbitrary traction and elasticity-modulus distributions to a porous beam’s displacement field.The beam supports and distributed loading define the porous-material setting, while the operator handles varying input functions.
- Porous material structure: 0.035 mean relative L2 error occurs on training data and 0.032 on test data, with standard deviations of 0.087 and 0.051, respectively.The reported distributions compare relative L2 errors for the train and test datasets.
- Porous material structure: 376 s of training enables approximately 3 ms solutions for arbitrary porous-beam functions, compared with 1.5 s for IGA.The timing comparison applies after training is complete.
- Hyperelasticity: In hyperelasticity, training and testing losses rapidly decrease and then stabilize, while predicted displacement agrees very well with the ground-truth solution.The reported evaluation uses 1000 iterations and a test-dataset sample.
- Hyperelasticity: 1282 s of training enables approximately 0.07 s solutions for arbitrary hyperelastic functions, compared with 40 seconds on average.The passage reports the post-training computational time against the conventional comparison.
3 Discussion
VINO incorporates variational physical laws into neural operators, enabling physics-only learning and stronger performance as mesh resolution increases. The approach analytically handles derivatives and integrals while remaining limited to PDEs with available variational forms and requiring future extensions for broader problem classes.
- VINO approach: VINO incorporates the variational form of physical laws into the learning process by discretizing domains and using shape functions.This enables analytical treatment of derivatives and integrals.
- VINO approach: VINO can learn operators solely from physical principles, removing the need for labeled datasets used by data-driven methods.The paper identifies costly or unavailable paired input-output datasets as a motivation.
- Performance: VINO outperforms existing approaches such as PINO, particularly as mesh resolution increases.Other methods show decreasing or inconsistent accuracy as mesh size grows, whereas VINO exhibits convergent behavior on fine meshes.
- Performance: VINO’s convergent behavior on fine meshes highlights robustness and scalability compared with methods whose performance changes adversely with mesh size.The comparison concerns accuracy behavior as mesh resolution increases.
- Limitations and future work: VINO is currently configured for PDEs with an available variational form, while future work targets weak-form extensions and broader PDE applicability.The study also identifies adaptations to other element types and more complex, multi-scale, nonlinear problems as future directions.
- Evaluation: For a porous-material beam, VINO’s train and test relative L2-error means are 0.035 and 0.032, respectively.The corresponding standard deviations are 0.087 for training and 0.051 for testing.
4 Methods
The methods formulate PDE solution operators and compare strong-form physics-informed learning with variational formulations. VINO combines neural operators and variational physics-informed training, using analytical element-level derivatives and integrals to avoid difficult loss evaluation.
- PDE and operator formulation: The PDE framework defines a solution operator G that maps PDE parameters to corresponding solutions over a bounded domain.Parameters may include coefficient or input functions and boundary conditions.
- Strong-form PINNs: PINNs approximate solutions with neural networks by minimizing a strong-form PDE loss using automatically differentiated derivatives.The loss is evaluated for fixed PDE parameters and includes boundary-condition terms controlled by hyperparameters.
- Strong-form PINNs: PINNs can struggle with high-frequency multi-scale features, nonlinear hyperbolic PDEs, large-scale implementations, and changing parameter sets.Changing parameters requires retraining the model for each set.
- VINO: VINO combines neural operators and PINNs in variational form to reduce dependence on labeled data and train by minimizing the variational functional.Domain discretization and shape functions permit analytical derivative and integral calculation during loss construction.
- Variational formulation: Variational losses represent functionals minimized by exact solutions rather than direct collocation equations, and they may carry physical meaning such as potential energy.Unlike squared-error losses, variational functionals can take negative values and have physical dimensions.
- Variational formulation: VPINN uses fewer hyperparameters and lower derivative order, improving computational efficiency and accuracy, but depends on accurate integration schemes and available variational forms.Not all PDEs possess a corresponding variational form.
- VINO evaluation: For the reported comparison, L2-errors are 0.1377, 0.1378, 0.1583, and 0.0181 for Midpoint, Trapezoidal, Simpson’s, and VINO methods, respectively.The passage states that VINO’s lower error indicates superior accuracy even on a coarse mesh.
Declaration of Competing Interest
The authors declare no known competing financial interests or personal relationships that could have influenced the reported work.
- The authors declare no known competing financial interests or personal relationships influencing the reported work.