Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next

1 Introduction

• Deep learning is being applied to difficult PDEs because nonlinearities, convection dominance, shocks, and high dimensionality challenge standard numerical approaches.Neural networks offer universal approximation and expressivity, while machine-learning PDE solvers are identified as a promising scientific-computing direction.Source: §1 · p. 2

1.1 What the PINNs are

• PINNs train neural networks by minimizing a loss containing initial and boundary conditions plus PDE residuals at collocation points, providing mesh-free forward and inverse solvers.The residual encodes governing physics without requiring labelled simulation or experimental data, and the same optimization framework can address inverse problems.Source: §1.1 · pp. 3, 4, 6
• PINNs extend earlier neural-network PDE methods and coexist with alternatives and variants including DRM, DGM, hp-VPINN, CPINN, and PCNN.PCNNs enforce initial and boundary conditions through architecture, whereas PINNs impose them as soft constraints in the training loss; the review primarily focuses on collocation-based PINNs.Source: §1.1 · pp. 4–5

1.2 What is this review about

• The review examines how PINNs are built, applied, and evaluated, including architectures, learning theory, available tools, future directions, accuracy, convergence, and documented benefits and drawbacks.Its literature search also covers broader physics-guided, physics-informed, and physics-constrained neural-network and machine-learning studies, with customizations involving depth, layer size, activation functions, and transfer learning.Source: §1.2 · p. 6

• The governing formulation includes nonlinear differential operators, physical parameters, data functions, and Dirichlet, Neumann, Robin, or periodic conditions.Initial conditions can be treated as Dirichlet boundary conditions on the spatio-temporal domain.Source: §2 · p. 8
• PINNs address forward problems by predicting u for specified γ and inverse problems by estimating γ from data.Both problem types use the same general framework, although inverse problems require determining physical parameters.Source: §2 · p. 8
• PINNs represent the unknown solution u(z) with a neural-network approximation parameterized by θ.The network maps space-time variables to predicted field values.Source: §2 · p. 8
• Training minimizes a weighted loss combining differential-equation residuals, boundary conditions, and optionally known data.With only equations and boundary conditions, forward training is unsupervised; noisy or inverse-problem data makes it supervised.Source: §2 · p. 8
• PINNs comprise a neural network, a physics-informed network that computes derivatives and residuals, and feedback that updates θ with an optimizer and learning rate.Algorithmic differentiation commonly connects the predictive and physics-informed components during training.Source: §2 · p. 9, Figure 2 · p. 9

2.1.1 Feed-forward neural network

• Feed-forward networks dominate PINNs, but architecture selection remains difficult because insufficient size limits approximation while excessive depth or width increases training difficulty and cost.Studies vary layers, neurons, and activations according to the PDE, including deeper networks for discontinuous solutions and constrained boundary conditions.Source: §2.1 · p. 10, FFNN architectures · pp. 11–12

Multiple FFNN

• Multiple feed-forward networks partition equations, phases, domains, or operator inputs, enabling specialized representations such as subdomain models and DeepONet branch–trunk networks.Examples include separate networks for Stefan-problem interfaces and temperatures, one network per subdomain, and branch and trunk networks for input and output-function spaces.Source: Multiple FFNN · p. 12

Shallow networks

• Shallow and sparse PINNs reduce training cost or parameter count through ELM-based single-layer networks, hybrid PINN–ELM methods, and interpretable kernel architectures.ELM updates only output-layer weights, while SPINN uses kernels such as RBF, softplus hat, and Gaussian functions.Source: Shallow networks · p. 13

Activation function

• Activation functions materially affect PINN convergence, accuracy, regularity, and derivative evaluation, making smooth choices such as tanh or sigmoid important for PDE residuals.Cheng and Zhang (2021) report that Swish improves convergence rate and accuracy, while second-order derivatives and PDE rescaling motivate cautious activation selection.Source: Activation function · p. 13

CNN architectures

• CNN-based PINNs address irregular-domain and image-like problems using coordinate transformations, finite-volume or mesh/graph convolutions, boundary-aware padding, and physics-constrained encoder–decoder architectures.PhyGeoNet transforms irregular physical domains to rectangular reference domains and strictly enforces boundary conditions; Zhu et al (2019) report greater processing efficiency than FC-NN methods for stochastic fluid flows.Source: CNN architectures · p. 14, Convolutional encoder-decoder network · p. 15

RNN architectures

• Recurrent PINNs exploit hidden-state memory for dynamical systems, including cells implementing Euler or other numerical integration schemes and LSTM architectures connected by differentiators.Viana et al (2021) embed Euler, Riemann, or Runge–Kutta integration in recurrent cells, while Zhang et al (2020) use PhyLSTM 2 and PhyLSTM 3 for physics-augmented sequence-to-sequence representation learning.Source: §2.1.3 · p. 15, RNN architectures · p. 16, LSTM architectures · p. 16

Bayesian Neural Network

• Other PINN architectures include Bayesian networks for posterior uncertainty and PI-GANs that embed stochastic differential equations through automatic differentiation.Yang et al (2021) find HMC more appropriate than mean-field variational inference for B-PINN posterior estimation, while KL is faster but less suitable for high-dimensional settings; DNF is more expensive than HMC but extracts independent samples more effectively after training.Source: Bayesian Neural Network · p. 17, GAN architectures · pp. 17–18

Multiple PINNs

• Multiple-PINN systems decompose domains or route inputs among expert networks, supporting parallelization, distinct subdomain architectures, interface conditions, and conditional computation.cPINN patches subdomain solutions using interface conditions, Mortar PINN connects non-overlapping domains, and GatedPINN combines expert outputs through a gating network.Source: §2.1.4 · p. 17, Multiple PINNs · p. 18

Derivative computation

• Automatic differentiation evaluates neural-network derivatives analytically from an operation graph, avoiding the approximation errors and restrictions of symbolic or numerical differentiation.It can compute numerical derivatives in either forward or reverse mode after constructing the graph of operations.Source: §2.2 · p. 18

Alternative implementation

• AD is the main technique used across PINN implementations, whereas Fang (2021) uses local fitting to approximate the differential operator and verify convergence.The local-fitting approach replaces automated differentiation rather than differentiating the network directly.Source: §2.2 · pp. 19–20

Physical-law injection

• AD incorporates PDE residuals and boundary or initial-condition residuals into the PINN loss to assess how well the network satisfies the governing problem.For the exact solution, both residuals are zero.Source: §2.2 · p. 20

Computational formulation

• Chain-rule differentiation across network layers creates a cumulative computational graph that effectively approximates the PDE equations.Mathews et al (2021) further normalize embedded loss functions into dimensionless form, while repeated differentiation enlarges the graph.Source: §2.2 · p. 20

Fractional operators

• When the basic chain rule is unsuitable, Pang et al (2019) replace fractional differential operators with discrete versions incorporated into the PINN loss.This provides a distinct treatment for fractional operators within the physics-informed framework.Source: §2.2 · p. 20

2.3 Model estimation by learning approaches

• PINNs recover network parameters θ by minimizing a combined loss over PDE residuals, boundary or initial conditions, and observed data, using scattered observations alongside many collocation points.Losses may use summed or integral formulations, and the optimization can typically use stochastic gradient descent with mean-square error.Source: §2.3 · p. 21

Observations about the loss

• Automated differentiation computes PDE derivatives without numerical discretization, while PDE-loss weighting can represent model fidelity and regularize training when parameters outnumber measurements.Removing the PDE loss eliminates governing-equation knowledge, whereas enforcing physics across the domain increases cost; both the number and distribution of residual points affect the loss.Source: Observations about the loss · pp. 22–23

Soft and hard constraint

• Boundary conditions can be enforced softly through penalty losses or hard-coded into network design, but soft enforcement may violate conditions and lacks theory for selecting loss weights.Variational or energy-based losses can be simpler to minimize and more computationally efficient, although they are more sensitive to training-sample size and resolution than PDE-based losses.Source: Soft and hard constraint · pp. 23–24

Optimization methods

• PINN training commonly combines Adam with L-BFGS, but gradient methods can become trapped among multiple local minima, making global minimization NP-hard for networks with many parameters.For noisy data, Mathews et al (2021) found that increasing sample size and training only with L-BFGS achieved optimum learning.Source: Optimization methods · p. 24
• Optimization can be adapted to problem structure: cPINN selects depth, width, activation, and optimizer by subdomain regularity, while two-step Adam then L-BFGS-B training benefits cases with little data or few residual points.Smooth regions may use shallow networks, whereas complex regions may use deeper ones; L-BFGS-B is reported to converge faster with reduced computing cost in sparse-data settings.Source: Optimization methods · p. 24
• Initialization also affects convergence, and Pang et al (2019) use a preliminary low-fidelity inverse solution to initialize the optimized parameters.The preliminary solution may be inaccurate because of discretization, sampling, and optimization errors, but can still provide a suitable starting point.Source: Optimization methods · pp. 24–25

2.4 Learning theory of PINN

• Learning theory for PINNs remains early-stage, framing convergence and error analysis through physical laws, data, and statistical learning concepts.The framework addresses convergence, optimization error, generalization error, and approximation error.Source: §2.4 · p. 25

2.4.1 Convergence aspects

• PINN theory seeks convergence of computed predictors to PDE solutions; results prove convergence for widening tanh networks and strong convergence for selected linear elliptic and parabolic PDEs.De Ryck et al. (2021) prove ˆuθ(n) − u converges to zero as network width tends to infinity, while Shin et al. (2020a) establish strong convergence of minimizers for specified PDEs.Source: §2.4.1 · pp. 25–26

2.4.2 Statistical Learning error analysis

• Statistical error analysis decomposes PINN performance into approximation, optimization, and generalization errors, while nonconvex training may fail to find a global minimum.Approximation depends on architecture, and common optimizers include Adam and L-BFGS; finding global minima remains open (Shin et al, 2020a).Source: §2.4.2 · pp. 26–27

2.4.3 Error analysis results for PINN

• Generalization bounds are available for several PDE classes, but many estimates suffer from the curse of dimensionality, requiring exponentially more points and larger networks for reduced error.Results cover linear elliptic, parabolic, and hyperbolic problems, abstract forward and inverse problems, and Navier–Stokes equations.Source: §2.4.3 · p. 28
• For a Kolmogorov-type PDE, De Ryck and Mishra (2021) prove an inequality independent of dimension, while another reported result has dimensional dependence only through a logarithmic factor.Source: §2.4.3 · pp. 28–29

Scope and organization

• Vanilla PINNs were demonstrated for both forward and inverse problems involving complex nonlinear PDEs, then applied across ODEs, PDEs, fractional PDEs, IDEs, and SDEs.The section organizes this literature by equation form, from ODEs and steady-state PDEs through dynamical systems and uncertainty-aware stochastic problems.Source: §3 · p. 29
• The review groups prior PINN research by equation class, covering ODEs; elliptic, steady-state diffusion, and Eikonal equations; Navier–Stokes; heat transport; advection-diffusion-reaction; hyperbolic and Euler equations; and quantum harmonic oscillators.It concludes by discussing Bayesian and stochastic formulations for addressing uncertainty.Source: §3 · p. 29

3.1 Ordinary Differential Equations

• Neural ODEs learn governing dynamics by combining physics-informed and unknown-discrepancy terms, tested on noisy spring-mass and negative-stiffness structural data (Lai et al, 2021).The framework uses a neural network f parameterized by θ to represent the dynamical system and supports initial-value problems.Source: §3.1 · pp. 29–30
• Deep LSTM networks are applied to nonlinear structural systems under seismic excitation, including steel moment-resisting frames and the rate-dependent Bouc–Wen model (Zhang et al, 2020).The approach addresses nonlinear equations of motion for these structural systems.Source: §3.1 · p. 30
• Directed graph models implement ODEs directly as deep neural networks while using an Euler recurrent neural network for numerical integration (Viana et al, 2021).Source: §3.1 · p. 30
• Nascimento et al (2020) provide a Python tutorial for integrating ODEs with recurrent neural networks.Source: §3.1 · p. 30
• Symplectic Taylor neural networks use two subnetworks and symplectic integrators instead of Runge–Kutta, and are tested on Lotka–Volterra, Kepler, and Hénon–Heiles systems (Tong et al, 2021).The original ODE-net approach used residual blocks computed with the Euler method.Source: §3.1 · p. 30

Partial Differential Equations

• PINN applications are examined across different typologies of partial differential equation models.These equations form the building blocks of many physics models, while established numerical strategies have a substantial theoretical framework for stability and convergence.Source: §3.2 · p. 30

3.2.1 Steady State PDEs

• PINNs outperform maximum a posteriori estimation for unsaturated-flow conductivity inference and use capillary-pressure data to estimate pressure-conductivity from finite pressure measurements.The inverse problem involves an unknown diffusion coefficient and limited measurements of water pressure.Source: §3.2.1 · p. 31
• VPINN lowers differential-operator order through integration by parts and is tested on steady Burgers, two-dimensional Poisson, and elliptic reaction-diffusion Schrödinger Hamiltonians.The VPINN approach was introduced by Kharazmi et al. (2019) and later used for Schrödinger Hamiltonians by Grubisic et al. (2021).Source: §3.2.1 · p. 31
• A nonlocal PDDO-PINN framework captures stress and strain concentrations with global functions in two-dimensional linear-elastic and elastoplastic mechanics.The problem is difficult because localized deformation and strong solution gradients complicate the boundary-value problem.Source: §3.2.1 · pp. 31–32
• SPINN addresses non-smooth solutions of linear elliptic PDEs on regular and irregular domains by reinterpreting meshless PDE-solution representations.The architecture is described as partially interpretable and sparse.Source: §3.2.1 · p. 32
• Steady-state PINNs extend to Laplace–Beltrami equations on complex and high-dimensional 3D surfaces, cardiac and seismic Eikonal problems, and Helmholtz equations, with sampling, regularization, and architecture choices affecting applicability.Applications include fiber-orientation learning from electroanatomical maps, uncertainty-aware cardiac activation mapping, and high-frequency or weakly inhomogeneous Helmholtz media.Source: §3.2.1 · pp. 32–33
• PINNs outperform first-order fast sweeping in Eikonal accuracy tests, especially for anisotropic models, while EikoNet predicts travel-time fields in heterogeneous 3D structures.EikoNet is limited to a single fixed velocity model, so even slight velocity changes require retraining.Source: §3.2.1 · p. 32

3.2.2 Unsteady PDEs

• PINNs reliably solve time-evolving physical phenomena, offering a flexible methodology for unsteady PDEs.Source: §3.2.2 · p. 33

Diffusion Problems

• Adaptive loss weighting improves PINN accuracy for composite-material heat transfer and enables correct prediction of the maximum internal temperature.Amini Niaki et al. (2021) use two disconnected subnetworks with sequential training that automatically adapts loss weights.Source: Diffusion Problems · pp. 33–34
• Dynamic weights significantly reduce inverse Stefan-problem prediction error after basic PINNs fail to identify thermal diffusivity because training reaches a local minimum.The result attributes improvement to mitigating stiffness in gradient-flow dynamics and emphasizes the importance of loss coefficients.Source: Diffusion Problems · pp. 34–35

Advection Problems

• For two-dimensional advection-dispersion, PINNs outperform typical discretization methods and ELMs, with boundary errors linked to stronger boundary and initial-condition weighting.Source: Advection Problems · p. 35

3.2.2.2 Flow Problems

• PINNs provide mesh-free approaches for fluid problems, while applications infer turbulent three-dimensional fields from limited two-dimensional plasma observations.The review identifies mesh-free solvers as a way to address computational and spatio-temporal discretization difficulties in fluid simulations.Source: §3.2.2.2 · p. 36, Navier-Stokes Equations · p. 36

Navier-Stokes Equations

• Navier–Stokes PINNs use varied strategies, including active learning, hard or soft boundary enforcement, residual stabilization, and alternative velocity-pressure or vorticity-velocity formulations.Res-PINN improves network stability with ResNet blocks, although predicted pressure magnitude can differ from the actual magnitude despite matching pressure distributions.Source: Navier-Stokes Equations · pp. 36–38

Hyperbolic Equations

• Hyperbolic PINNs capture discontinuous Euler flows from sparse data, with localized training and characteristic-form equations improving accuracy around shocks and high-gradient regions.Mao et al. (2020) infer states without initial or boundary conditions, and characteristic-form results outperform conservative-form results.Source: Hyperbolic Equations · pp. 38–39

3.2.2.3 Quantum

• For quantum problems, PINN approximations can match state-of-the-art spectral solvers, while gating networks and multi-stage training extend applicability to coupled nonlinear wave equations.Stiller et al. (2020) combine a gating network with multiple MLPs; Mo et al. (2022) use a pre-fixed multi-stage algorithm for vector solitons.Source: §3.2.2.3 · p. 39

3.3 Other Problems

• PINNs have been applied beyond classical differential problems, particularly to fractional PDEs and uncertainty estimation.These applications broaden PINN use to problems involving fractional operators, noisy or scarce observations, and stochasticity.Source: §3.3 · p. 39

3.3.1 Differential Equations of Fractional Order

• Fractional PINNs combine automatic differentiation for integer-order operators with numerical discretization, such as the L1 scheme, for fractional operators.Pang et al (2019) introduced fPINN for identifying unknown coefficients and operators in fractional PDEs and analyzed fractional advection-diffusion convergence; Kharazmi et al (2021a) extended the approach to time-dependent fractional orders using separate networks for each order.Source: §3.3.1 · p. 40

3.3.2 Uncertainty Estimation

• B-PINNs combine physical principles with scattered noisy observations to predict solutions and quantify aleatoric uncertainty in forward and inverse nonlinear PDE problems.Yang et al (2021) use Bayesian neural networks as priors and estimate posteriors with Hamiltonian Monte Carlo or variational inference, testing forward, inverse, and high-dimensional diffusion-reaction systems.Source: §3.3.2 · p. 40
• Uncertainty-focused PINN studies also model stochastic processes and probability distributions under limited sensor measurements.Yang et al (2020) approximate stochastic solution, forcing, and diffusion processes while varying observations to transition between forward, mixed, and inverse problems; Cavanagh et al (2021) fit PDFs, diffusivity, and landscapes for cell-drug morphodynamics using Bayesian computation.Source: §3.3.2 · p. 41

3.4 Solving a differential problem with PINN

• The experiment solves a 1D nonlinear Schrödinger problem with periodic boundary conditions and complex-valued solutions, initialized by ψ(x, 0) = 2 sech(x) over T = π/2.The benchmark reference is generated by integrating the equation with Chebfun and conventional spectral methods.Source: §3.4 · pp. 41–42
• The PINN uses inputs (t, x) and two outputs for the real and imaginary parts, minimizing initial and boundary losses plus the Schrödinger residual without extra data loss.Training uses initial data at t = 0, boundary data, and interior collocation points.Source: §3.4 · pp. 42–43
• The best tested PINN achieves an average MSE of 5,17·10^-04, but struggles near (x, t) = (0, π/4) and on symmetric later-time intervals.The predicted and exact solutions are compared at three temporal snapshots; Figure 4 also reports a local peak error of 10^-01.Source: §3.4 · p. 43, Figure 4 · p. 44

N0 Nb Nc Training loss relative L2 MAE MSE

• Increasing initial or boundary training points decreases errors, with doubling initial-condition points having a stronger effect than doubling spatial boundary points in this setup.Table 2 evaluates convergence using relative L2, MAE, and MSE across ten runs.Source: §3.4 · p. 43, Table 2 · p. 45
• The vanilla PINN has difficulty maintaining strong performance across the full space-time domain, particularly at longer times.The study varies boundary and initial data and examines error growth as the solution advances through time.Source: §3.4 · p. 43, Table 2 · p. 45

4 PINNs: Data, Applications and Software · 4.1 Data

• PINNs use training points alongside embedded equations; their distribution and quantity affect approximation, while arbitrary sampling supports complex geometries without fixed meshes or grids.Increasing training points improves approximation, but location can be crucial; lattice-like, Sobol, and Latin hypercube sampling are discussed.Source: §4.1 · p. 46, §4 · p. 46

Hemodynamics

• PINNs support hemodynamics by inferring velocity, pressure, flow patterns, wall stresses, and wave propagation from images or non-invasive MRI while enforcing governing equations.Applications include hidden fluid mechanics for intracranial aneurysms and patient-specific Windkessel parameter estimation from noisy measurements.Source: Hemodynamics · p. 47

Flows problems

• PINNs infer unobserved flow dynamics from partial data, but vanilla PINNs perform poorly on temporal extrapolation; applications include plasma turbulence, aerodynamic inverse problems, and Buckley–Leverett flow.Almajid and Abu-Al-Saud (2022) report that an ANN without physical loss cannot predict Buckley–Leverett solutions from only early-time saturation profiles.Source: Flows problems · pp. 47–48

Optics and Electromagnetic applications

• PINN applications in electromagnetics include hybrid CNN-based solvers, inverse scattering and permittivity retrieval, eigenvalue problems in magnetic materials, and surface PDEs on meshes.These studies address photonic metamaterials, nano-optics, cloaking, microelectromagnetic nucleation fields, and power-system swing equations.Source: Optics and Electromagnetic applications · pp. 48–49

Molecular dynamics and materials related applications

• PINNs and related architectures address molecular and materials problems by estimating nanofluid properties, reconstructing 3D nanoparticle shapes from scattering images, and forecasting bubble growth across four spatial-temporal orders of magnitude.The reported quantities include viscosity, energy per atom, pressure, diffusion coefficients, and experimentally consistent silver nanocluster geometries.Source: Molecular dynamics and materials… · p. 49

Geoscience and elastostatic problems

• Geoscience and elastostatic applications use PINNs for solid-mechanics stress and displacement fields, FvK loading cases, and earthquake hypocenter inversion through an Eikonal-equation forward model.The FvK tests include nonuniform in-plane tension, central-hole tension, out-of-plane deflection, and compression buckling.Source: Geoscience and elastostatic problems · p. 49

Industrial application

• Industrial PINN applications extend to settings with incomplete physical models, including lubricant-damage assessment from visual inspections and fatigue-related degradation studies.Yucesan and Viana (2021) used monthly inspections of 120 wind turbines over six months and reported an accurate, error-free model for optimizing regreasing intervals.Source: Industrial application · p. 50
• Parametric PINN simulations can accelerate industrial design searches by varying geometry, unlike traditional solvers limited to single-geometry simulations.NVIDIA Modulus varied heat-sink fin thickness, length, and height to construct a design space for channel heat transmission.Source: Industrial application · p. 50

4.3 Software; Other packages

• PINN libraries rely on feed-forward networks and automatic differentiation, while hard versus soft boundary constraints alter loss evaluation and complicate hyper-parameter tuning.The broader ecosystem also includes Neural Tangents and GPyTorch, which represent functions with kernel or Gaussian-process methods rather than standard neural networks.Source: §4.3 · p. 50, Other packages · p. 54

DeepXDE

• DeepXDE supports diverse boundary conditions, complex CSG geometries, and residual-based adaptive refinement that adds points where PDE residuals are largest.Its initial release demonstrated five applications, including Poisson, Burgers, Volterra integrodifferential, Lorenz inverse, and diffusion-reaction problems.Source: DeepXDE · p. 51

NeuroDiffEq; PyDENs; Nangs

• NeuroDiffEq and PyDENs impose initial or boundary conditions through neural-network construction, making them PCNNs, but NeuroDiffEq’s explicit constraints limit unsupported arbitrary bounds.PyDENs development appears to have stopped in 2020, while NeuroDiffEq and Nangs outperformed PyDENs on higher-dimensional PDEs.Source: NeuroDiffEq · p. 51, PyDENs · pp. 52–53, Nangs · p. 53

Modulus

• Modulus supports multiplicative-filter networks, gradient aggregation, CSG and tessellated geometries, integral losses, strong or weak PDE forms, and diverse neural architectures.It has been applied to multiphysics heat transfer, blood flow, data assimilation, inverse flow problems, and faster heat-transport solving.Source: Modulus · p. 52

SciANN

• SciANN provides a high-level Keras PINN wrapper with reproducible examples spanning elasticity, structural mechanics, vibration, flood simulation, and Eikonal problems.It also supports nonlocal PINNs and multi-network solid-mechanics models, although one flood-study implementation avoided extra-library overhead by separating the feedforward step.Source: SciANN · p. 52

NeuralPDE.jl; ADCME; TensorDiffEq; IDRLnet; Elvet

• Other packages broaden PINN capabilities: ADCME handles elasticity, Stokes, Burgers, and stochastic inverse problems; TensorDiffEq targets distributed scalability; IDRLnet supports noisy inverse, variational, and integral-differential problems.Elvet solves coupled ODE/PDE systems and variational problems, while NeuralPDE.jl integrates scientific machine learning and differential-equation modeling in SciML.Source: TensorDiffEq · p. 53, IDRLnet · p. 53, NeuralPDE.jl · p. 53

5 PINN Future Challenges and directions

• Overall, PINNs remain promising for high-dimensional scientific problems but can struggle with multi-scale, chaotic, turbulent, or otherwise complex PDE phenomena.Resolving these limitations requires advances spanning convergence, architecture design, boundary conditions, optimization, and general PINN methodology.Source: §5 · p. 54, §5.3 · p. 61

5.1 Overcoming theoretical difficulties in PINN

• PINNs lack established convergence and stability theory, including explanations of dimensionality scaling, bias-variance behavior, PDE stiffness, and optimization under soft constraints.Future analyses should examine network topology, frequency bias, initialization, collocation selection, dynamic loss weighting, and physics-informed hypothesis spaces.Source: §5.1 · pp. 56–57

5.2 Improving implementation aspects in PINN

• Implementation research should expand PINN architectures, improve activation and differentiation choices, resolve loss weighting and boundary-condition limitations, and develop stronger optimization and error-estimation methods.Non-FFNN architectures, adaptive loss strategies, alternatives to automatic differentiation, learning-rate effects, gradient normalization, and PDE-level error bounds remain open directions.Source: §5.2 · pp. 58–59

5.3 PINN in the SciML framework

• Within SciML, PINNs require deeper integration with scientific problems because high coefficients, dimensionality, training cost, and information propagation can undermine their replacement of traditional numerical methods.Causality-aware losses improved accuracy and enabled applications to chaotic Lorenz, Kuramoto–Sivashinsky, and turbulent Navier–Stokes systems, while safety-critical and multi-scale domains remain underdeveloped.Source: §5.3 · pp. 59–61

5.4 PINN in the AI framework

• PINNs may serve as components of broader AI systems, including reinforcement-learning agents that embed sensor, actuator, and physical-law knowledge.The framework also connects deep learning with symbolic intelligence and may help investigate causality in hybrid forward-inverse problems.Source: §5.4 · pp. 61–62

• Most analyzed studies customize PINNs through activation functions, gradient optimization, neural-network structures, or loss-function structures.Other directions use minimal physical-model information, embed initial or boundary-condition validity in the network, or propose frameworks for multi-physics systems.Source: §6 · p. 62
• Raissi et al (2017c,d) established PINNs by approximating unknown functions with neural networks under physical constraints and extending this to a hybrid data–equation approach.The review places this development within earlier physical-constraint optimization and non-neural approximations, including kernel approaches and PDE-constrained optimization.Source: §6 · pp. 62–63
• PINNs enforce physical information through data values or physical equations, with collocation points forcing the neural network to obey the physical model.Data are usually supplied as initial or boundary values, while physical equations constrain the network at collocation points.Source: §6 · p. 63
• The review covers vanilla PINNs and variants including soft and hard VPINNs, examining neural networks, physical-loss construction, feedback, applications, and available packages.Soft VPINNs include initial and boundary conditions in the loss, whereas hard VPINNs encode boundary conditions in the neural-network structure.Source: §6 · p. 63
• Further development remains possible, especially for unresolved theoretical issues, optimal PINN training, and solving multiple equations.Only a few studies examined alternatives to automatic differentiation or convergence problems, including Fang (2021) and Wang et al (2021b, 2022b).Source: §6 · pp. 62–63