Source-linked AI summary
A mixed formulation for physics-informed neural networks as a potential solver for engineering problems in heterogeneous domains: comparison with finite element method
Shahed Rezaei, Ali Harandi, Ahmad Moeineddin, Bai-Xiang Xu, Stefanie Reese
TL;DR
The paper addresses whether PINNs can solve engineering boundary value problems in heterogeneous domains without labeled data. It combines separate networks for a primary variable and its spatial gradients with strong-form, energy-form, and connection losses, and compares the resulting predictions with FEM. The reported results show good overall agreement for the tested heterogeneous elasticity and diffusion settings, while identifying computational-time and scope limitations.
Problem
PINNs require refinement for heterogeneous systems with sharp contrasts between material phases, while data-free engineering boundary-value-problem solutions remain valuable.
Method
The method uses separate neural networks for the primary variable and its spatial gradients, applying strong-form constraints to gradients, energy-form constraints to the primary variable, and an additional connection loss.
Results
The predictions generally agree with FEM; in one heterogeneous case, average relative differences were about 4% for displacement and 3.7% for stress.
Takeaways & Limitations
Properly designed mixed-formulation PINNs have potential to solve unknowns in heterogeneous domains without initial data from other sources.
Takeaways & Limitations
The demonstrated study is limited to isotropic elastic material in a two-dimensional setting, while PINNs required hours versus minutes for the reported problems using FEM.
Abstract
from arXiv · showhide
Physics-informed neural networks (PINNs) are capable of finding the solution for a given boundary value problem. We employ several ideas from the finite element method (FEM) to enhance the performance of existing PINNs in engineering problems. The main contribution of the current work is to promote using the spatial gradient of the primary variable as an output from separated neural networks. Later on, the strong form which has a higher order of derivatives is applied to the spatial gradients of the primary variable as the physical constraint. In addition, the so-called energy form of the problem is applied to the primary variable as an additional constraint for training. The proposed approach only required up to first-order derivatives to construct the physical loss functions. We discuss why this point is beneficial through various comparisons between different models. The mixed formulation-based PINNs and FE methods share some similarities. While the former minimizes the PDE and its energy form at given collocation points utilizing a complex nonlinear interpolation through a neural network, the latter does the same at element nodes with the help of shape functions. We focus on heterogeneous solids to show the capability of deep learning for predicting the solution in a complex environment under different boundary conditions. The performance of the proposed PINN model is checked against the solution from FEM on two prototype problems: elasticity and the Poisson equation (steady-state diffusion problem). We concluded that by properly designing the network architecture in PINN, the deep learning model has the potential to solve the unknowns in a heterogeneous domain without any available initial data from other sources. Finally, discussions are provided on the combination of PINN and FEM for a fast and accurate design of composite materials in future developments.
1. Introduction
The introduction motivates physics-informed neural networks as data-free boundary-value-problem solvers and proposes extending them for heterogeneous engineering domains by combining PINN and finite-element ideas.
- Motivation: Deep learning attracts engineering applications because it can recognize patterns in available data and offer fast predictions after successful training.The paper highlights potential speedups for multiscaling analysis, where transferring microscale material information is tedious.
- Learning strategies: Supervised learning requires sufficient offline calculations or measurements to provide labeled training data.The paper contrasts this with physics-based training that does not require labeled data from multiple boundary-value problems.
- Physics-informed learning: PINNs solve boundary value problems without labeled data by adding governing equations, initial conditions, and boundary conditions to the neural-network loss.The introduction compares this data-free setting with finite-element solvers.
- Related work: Prior PINN and deep-learning studies addressed PDEs, energy formulations, non-homogeneous media, coupled systems, and mixed-variable outputs across physics and engineering.The cited work includes applications spanning fluid mechanics, solid mechanics, heat transfer, electric conduction, and related problems.
- Research gap: Many existing contributions target homogeneous materials, whereas sharp phase contrasts in heterogeneous systems motivate further PINN refinements.The paper positions its extension as a direct solver for PDEs and boundary conditions in heterogeneous domains.
- Proposed direction: The proposed architecture combines weak and strong problem forms using separate networks for the primary variable and its spatial gradients, linked by an additional loss.The work presents this as an extension intended for heterogeneous engineering boundary-value problems.
2. Formulation of the problem
The paper formulates heterogeneous elasticity and steady-state diffusion problems through governing equations, boundary conditions, weak forms, and energy representations, then specifies heterogeneous geometries and loading conditions for evaluation.
- 2.1. Linear elasticity in heterogeneous solids: Linear elasticity is governed by kinematics, Hooke’s law, and mechanical equilibrium, with displacement and traction boundary conditions.The equilibrium equation is div(σ) = 0, while displacement is prescribed on ΓD and traction on ΓN.
- 2.1. Linear elasticity in heterogeneous solids: The elasticity tensor depends on spatial coordinates to represent material heterogeneity, under a two-dimensional plane-strain assumption.The formulation assumes isotropic behavior within each phase, while noting that anisotropic tensors could use the same formulation.
- 2.1. Linear elasticity in heterogeneous solids: The Galerkin weak form is obtained by multiplying equilibrium by a test function satisfying Dirichlet conditions, then applying integration by parts and Gauss’ theorem.The weak form provides the basis for an equivalent balance of internal and external mechanical energy.
- 2.1. Linear elasticity in heterogeneous solids: The mechanical energy representation balances internal and external energy and is added to the neural-network loss for the primary displacement variable.The energy formulation is imposed subject to the prescribed boundary conditions.
- 2.2. Diffusion Problem in heterogeneous solids: Steady-state diffusion is formulated for heat transfer using Fourier heat flux, position-dependent conductivity, conservation of flux, and Dirichlet or Neumann conditions.The same derivation is stated to apply to electrostatic, electric-conduction, and magnetostatic problems.
- 2.2. Diffusion Problem in heterogeneous solids: The thermal weak form is interpreted as an energy variation, yielding an energy-minimization formulation subject to the thermal boundary conditions.The loss term associated with this formulation is evaluated for the primary temperature variable T.
- 2.2. Diffusion Problem in heterogeneous solids: The mechanical case applies tensile loading with traction-free upper and lower sides, while the thermal case fixes temperatures at left and right edges and isolates the upper and lower sides.Model inputs and material parameters are summarized in Table 1.
3. New architecture for PINNs
The proposed PINN architecture uses feed-forward networks with position inputs and displacement/stress outputs, combining mixed variables, energy constraints, and reduced derivative order. Comparisons with FEM and alternative architectures show strong agreement overall, while errors concentrate near phase interfaces and depend on collocation-point placement.
- Architecture: The network uses 2D position coordinates as inputs and approximates displacement and stress through fully connected feed-forward neural networks.The output variables are computed with neural networks, while network layers pass information sequentially through weighted sums, biases, and nonlinear activation functions.
- Architecture: Separate networks for each output quantity, combined with the energy form, improve the solution of heterogeneous-domain problems.The proposed design uses multiple displacement and stress outputs; the authors report advantages for separated networks compared with combining all outputs.
- Loss functions: First-order derivatives suffice to construct the physical loss functions, reducing derivative-order demands relative to formulations requiring second-order derivatives.The authors associate lower derivative order with greater activation-function flexibility, lower computational cost, and easier solution finding for fixed iterations and collocation points.
- Mechanical comparison: The model achieved good overall agreement with FEM for displacement and stress, but maximum relative differences reached 44% and 77%, respectively, near critical regions.Average relative differences were about 4% for displacement and 3.7% for stress; shear stress predictions were relatively poor.
- Diffusion comparison: For the diffusion problem, average relative differences were about 0.5% for temperature and 0.6% for flux, with maxima of 12% and 9.6%.The largest differences occurred near the sharp phase transition, while the PINN and FE predictions otherwise showed close agreement.
- Error reduction: Adding collocation points near the interphase region significantly reduced the difference between mixed-formulation PINN and FE results.The study added about 800 points near the region with the highest observed error.
4. Conclusion and outlooks
The work extends PINNs with a mixed formulation for heterogeneous-domain boundary value problems and compares the approach with FEM. It identifies opportunities for combining DL, FEM, and transfer learning while noting current computational and scope limitations.
- 4. Conclusion and outlooks: The proposed mixed-formulation PINN adds spatial gradients as separate network outputs and constrains outputs through the governing PDE and its energy form.The approach predicts heterogeneous-domain solutions without initial data and is evaluated on elasticity and steady-state diffusion problems against standard FEM.
- 4. Conclusion and outlooks: PINNs require further study of how network hyperparameters and collocation-point distributions affect the final solution.The cited examples include layer depth, learning rate, and collocation-point location and distribution.
- 4. Conclusion and outlooks: DL training currently takes a few hours for the discussed problems, whereas FEM takes less than a few minutes once codes and models are available.The paper identifies computational time as a main current drawback and suggests optimizer or network-design improvements.
- 4. Conclusion and outlooks: Training DL methods across boundary value problems, loading conditions, and material properties may reduce computational cost as the networks learn new systems.The paper also discusses recycling data from different computational strategies and combining DL with FEM or transfer learning.
- 4. Conclusion and outlooks: The study is limited to isotropic elastic material in two dimensions, motivating future work on nonlinear geometry and materials, three-dimensional settings, and time-dependent multiphysics.These extensions are presented as future developments rather than results established by the current work.
5. Appendix A: ideas to enhance the learning rate based on the Newton-Raphson method
The appendix interprets loss minimization as a root-finding problem and relates gradient descent to a Newton-Raphson-based learning-rate design. It also notes that second-derivative computation and inversion can be costly.
- 5. Appendix A: ideas to enhance the learning rate based on the Newton-Raphson method: Training minimizes the total loss L with respect to network parameters θ through an iterative gradient descent process.The optimization is written as min_θ L(X; θ).
- 5. Appendix A: ideas to enhance the learning rate based on the Newton-Raphson method: The minimization can be interpreted as root finding, with the solution approached as L tends toward zero.The appendix applies the standard Newton-Raphson method to this formulation.
- 5. Appendix A: ideas to enhance the learning rate based on the Newton-Raphson method: A tensorial learning rate can weight components of the loss-gradient direction rather than using a single scalar step size.This design is proposed to accelerate convergence toward the minimum point.
- 5. Appendix A: ideas to enhance the learning rate based on the Newton-Raphson method: Second-derivative calculation and inversion may be nontrivial and costly when using the Newton-Raphson-based update.The appendix therefore motivates approximating this operation.
6. Appendix B: Influence of the derivation order, number of collocation points, and number of epochs on the obtained solution
The appendix tests how derivative order, collocation-point count, and training epochs affect ODE solutions under matched neural-network settings. Lower derivative order and more training resources produce more accurate and stable predictions.
- 6. Appendix B: Influence of the derivation order, number of collocation points, and number of epochs on the obtained solution: The desired function is differentiated once and twice to construct first- and second-order ODEs with prescribed boundary conditions.The resulting equations provide the two alternative physical constraints for reproducing y(x).
- 6. Appendix B: Influence of the derivation order, number of collocation points, and number of epochs on the obtained solution: Two separate neural networks with identical architecture and hyperparameters use losses derived from the respective ODEs and boundary conditions.This setup isolates the influence of derivation order on the predicted solution.
- 6. Appendix B: Influence of the derivation order, number of collocation points, and number of epochs on the obtained solution: Lower derivative order produces more accurate predictions, while increasing collocation points or epochs improves convergence toward the exact solution.The comparison uses first- and second-order ODE networks and plots extrapolation beyond the training zone.
- 6. Appendix B: Influence of the derivation order, number of collocation points, and number of epochs on the obtained solution: The MAE of first-order ODEs is always lower than that of second-order ODEs as collocation points increase from 10 to 100.All calculations use 1500 epochs, and training is repeated three times to assess variability from parameter initialization.
- 6. Appendix B: Influence of the derivation order, number of collocation points, and number of epochs on the obtained solution: Increasing collocation points reduces training randomness or instability, with both ODE formulations converging as sampling becomes denser.The appendix attributes part of the first-order advantage to supplying two boundary conditions when only one is required.
7. Appendix C: Details on the calculation of the loss functions
The appendix expands the elasticity and thermal loss functions by computing kinematics, constitutive responses, boundary contributions, temperature gradients, and fluxes. Boundary conditions and zero terms determine the final simplifications.
- 7. Appendix C: Details on the calculation of the loss functions: For the elasticity problem, strain components are computed from the network-predicted deformation through the kinematic relation.These strain components enter the subsequent constitutive and loss-function calculations.
- 7. Appendix C: Details on the calculation of the loss functions: Stress components are then obtained from the constitutive relations using the computed strain components and material properties.The appendix gives a shear-stress relation involving E(X), ν(X), and the predicted shear strain.
- 7. Appendix C: Details on the calculation of the loss functions: The loss expressions distinguish Dirichlet and Neumann boundary-condition terms across the left, right, top, and bottom boundaries.The notation Γ_L, Γ_R, Γ_T, and Γ_B denotes the corresponding boundary points.
- 7. Appendix C: Details on the calculation of the loss functions: Terms that are zero under the specified boundary conditions are omitted from the expanded loss functions.Traction-free top and bottom boundaries and fixed displacement on the left edge provide the stated simplifications.
- 7. Appendix C: Details on the calculation of the loss functions: For the thermal problem, temperature gradients are computed from the network-predicted temperature and then used to calculate flux components and the final loss terms.The appendix presents these operations as successive expansions of the thermal loss formulation.
- 7. Appendix C: Details on the calculation of the loss functions: The external-work contribution vanishes because traction T equals zero on the right boundary in the stated boundary value problem.This is a problem-specific simplification of the energy expression.