Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications

Zhongkai Hao; Songming Liu; Yichi Zhang; Chengyang Ying; Yao Feng; Hang Su; Jun Zhu

Source-linked AI summary

Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications

Zhongkai Hao, Songming Liu, Yichi Zhang, Chengyang Ying, Yao Feng, Hang Su, Jun Zhu

arXiv:2211.08064v2Nov 15, 2022cs.LGcs.AIcs.CVmath.NA

TL;DR

Physics-informed machine learning addresses how to build models that remain useful when physical laws, sparse or noisy data, and high-dimensional systems matter. This survey formalizes PIML as combining empirical data with physical knowledge, reviews its tasks, representations, and integration methods, and concludes that substantial opportunities remain for improved optimizers, inference methods, and applications.

Problem
Purely data-driven models can lack robustness, interpretability, and physical adherence, while scientific data are often sparse and noisy and traditional solvers face high-dimensional computational costs.
Method
The survey organizes PIML by machine-learning tasks, physical-prior representations, and methods that integrate priors into models, and presents a unified latent-variable framework.
Results
PIML methods, including PINNs and DeepONets, are reported across scientific simulation and inverse problems, with many works achieving better inverse-problem results than traditional methods.
Takeaways & Limitations
PIML provides a broad framework for combining physical knowledge and data across neural simulation, inverse problems, computer vision, language processing, and control.
Takeaways & Limitations
The survey concludes that substantial potential remains for improving PIML methods and extending physical priors into optimizers and inference algorithms.

Abstract

from arXiv · show

Recent advances of data-driven machine learning have revolutionized fields like computer vision, reinforcement learning, and many scientific and engineering domains. In many real-world and scientific problems, systems that generate data are governed by physical laws. Recent work shows that it provides potential benefits for machine learning models by incorporating the physical prior and collected data, which makes the intersection of machine learning and physics become a prevailing paradigm. By integrating the data and mathematical physics models seamlessly, it can guide the machine learning model towards solutions that are physically plausible, improving accuracy and efficiency even in uncertain and high-dimensional contexts. In this survey, we present this learning paradigm called Physics-Informed Machine Learning (PIML) which is to build a model that leverages empirical data and available physical prior knowledge to improve performance on a set of tasks that involve a physical mechanism. We systematically review the recent development of physics-informed machine learning from three perspectives of machine learning tasks, representation of physical prior, and methods for incorporating physical prior. We also propose several important open research problems based on the current trends in the field. We argue that encoding different forms of physical prior into model architectures, optimizers, inference algorithms, and significant domain-specific applications like inverse engineering design and robotic control is far from being fully explored in the field of physics-informed machine learning. We believe that the interdisciplinary research of physics-informed machine learning will significantly propel research progress, foster the creation of more effective machine learning models, and also offer invaluable assistance in addressing long-standing problems in related disciplines.

1 INTRODUCTION

Physics-informed machine learning combines empirical data with physical prior knowledge to address limitations of purely data-driven models in scientific and real-world tasks. The survey formalizes this paradigm, reviews its methods and applications, and identifies open challenges.

The reviewed paradigm spans applications including scientific modeling, computer vision, natural language processing, reinforcement learning, and control.
Data-driven machine learning has advanced many domains, but its models can lack robustness, interpretability, and adherence to physical constraints.
Scientific and engineering data are often sparse and noisy, causing generalization errors when physical laws govern the underlying systems.
PIML constructs models that use both empirical data and prior physical knowledge for tasks involving physical mechanisms.
The survey reviews PIML across machine-learning tasks, physical-prior representations, algorithms, theory, applications, and future challenges.

2 PROBLEM FORMULATION

PIML represents physical knowledge through differential equations, symmetry constraints, and intuitive physics, then integrates these priors into machine-learning models. The section frames these representations and their roles across data, architectures, losses, optimization, and inference.

Differential Equations: PIML considers PDEs, ODEs, and SDEs as strong physical priors for describing governed physical systems.
Symmetry Constraints: Symmetry is a weaker inductive bias than differential equations and includes translation, rotation, reflection, scale, permutation, and topological invariance.
Problem Formulation: Figure 1 organizes physical-prior integration across data, model architecture, loss functions, optimizers, and inference algorithms.
Differential Equations: Physical systems are described by state variables over spatial or spatial-temporal domains, with governing equations, parameters, and initial or boundary constraints.
Intuitive Physics: Intuitive physics encodes human common-sense knowledge about object dynamics, including object permanence, gravity, motion laws, and conservation laws.
Intuitive Physics: These intuitive principles can constrain machine-learning models in computer vision, robotics, and language processing, but their mathematical formalization remains challenging.

3 NEURAL SIMULATION

Neural simulation uses neural networks to model physical systems governed by differential equations. It comprises neural solvers for individual equations and neural operators for learning solution maps of parametric equations.

Neural simulation applies neural-network methods to physical systems governed by PDEs, ODEs, or SDEs.
Neural solvers address individual PDEs or ODEs, whereas neural operators learn solution maps for parametric PDEs or ODEs.

3.1 Challenges of Traditional ODEs/PDEs Solvers

Traditional ODE/PDE solvers offer high precision, stability, and mature theoretical foundations, but their computational cost grows sharply with dimensionality and geometric complexity. They also poorly accommodate experimental data or partially unknown governing equations.

Traditional ODE/PDE solvers discretize differential equations into linear systems and solve them on regular or irregular meshes.
Traditional solvers provide high precision, good stability, and complete theoretical foundations.
O(dn^r) time complexity creates a curse of dimensionality, with nonlinear or geometrically complex problems potentially requiring weeks or months.
Traditional solvers have difficulty incorporating experimental data and handling partially unknown governing equations, including inverse-design settings.
Neural networks may learn lower-dimensional mappings from high-dimensional data, motivating their use as alternative simulation methods.

3.2 Neural Solver

Neural solvers use neural networks to represent and solve physical-system states, with PINNs incorporating governing equations, data, and constraints into training. The survey covers flexible loss formulations and architectural variants while identifying unresolved efficiency, accuracy, convergence, and applicability challenges.

3.2.1 Neural Solver: Neural solvers replace traditional FEM and FVM methods by representing physical-system states with neural networks when governing physical laws are known.They may support imperfect knowledge, limited data, and high-dimensional PDE representations, but existing methods still face computational-efficiency, accuracy, and convergence drawbacks.
3.2.2 Framework of Physics-Informed Neural Networks: PINNs incorporate PDE knowledge into neural-network architectures to solve forward and inverse PDE problems using physical constraints alongside observed data.The framework combines PDE residual, initial-condition, boundary-condition, and regular data losses.
3.2.2 Framework of Physics-Informed Neural Networks: PINN loss terms can be omitted when corresponding data or initial and boundary constraints are unavailable, while their relative weights are controlled by hyperparameters.Automatic differentiation computes high-order derivatives, and Monte Carlo sampling approximates the integral terms used in the loss.
3.2.3 PINN Variants: Loss reweighting and learning-rate annealing address unequal convergence speeds and conflicts among PINN loss terms, including pathological dominance of PDE residuals.The surveyed annealing scheme is reported as efficient for systems containing multiple frequencies, while Inverse-Dirichlet Weighting alleviates vanishing gradients and catastrophic forgetting in multi-scale modeling.
3.2.5 Novel Optimization Objectives: Variational formulations reduce PDE derivative order to first-order terms and naturally absorb natural boundary conditions, avoiding high-order derivative calculations and extra penalty terms.The survey notes that which equivalent PDE formulation best benefits PINN training remains largely unexplored.
3.2.6 Novel Neural Architectures: Architectural variants use multiple neural networks, smooth activations, and Fourier-feature embeddings to address high-dimensional, multi-scale, and high-frequency physical problems.Hard boundary-condition encoding remains unresolved for arbitrary forms on irregular domains, despite progress for Dirichlet, Neumann, and Robin conditions on complex geometries.

3.3 Neural Operator

Neural operators learn mappings from physical-system parameters or control functions to solution states, extending neural solvers from individual equations to whole parameterized PDE/ODE classes. The section reviews direct, Green’s-function, grid-based, and graph-based approaches, along with their benefits, limitations, and open challenges.

Direct Methods: Direct methods, including DeepONet, parameterize operator mappings with neural networks that take physical parameters and spatial coordinates as inputs.DeepONet uses branch and trunk networks, whose architectures may be instantiated with FNNs, ResNets, or other designs.
Overview: Neural operators approximate mappings from parameters or control functions to solutions across a class of differential equations, rather than solving only one physical-system instance.This distinguishes neural operators from neural solvers and motivates their use as surrogate models in science and engineering.
Direct Methods: DeepONet produces solutions for any instance in a parameterized PDE class with one forward pass and uses mesh-independent inputs and outputs, but usually requires costly, large training datasets.Its data requirement becomes especially restrictive for complex PDEs because training data often come from expensive simulations or experiments.
Direct Methods: Physics-informed DeepONet combines PINNs with DeepONet to reduce DeepONet’s data demand and PINNs’ weaker approximation of complex PDE solutions.Related variants include architectures that merge parameter and coordinate information earlier and MIONet for multiple-input operators.
Green’s Function Learning: Green’s-function learning embeds stronger physical structure for data efficiency and flexible architectural constraints, but is restricted to special PDE classes and can have doubled spatial input dimension.DeepGreen extends this approach to nonlinear boundary-value problems, although its theoretical underpinning is not rigorous for certain functions.
Grid-based Operator Learning: Fourier neural operators use spatial- and frequency-domain features and outperform architectures such as U-Net experimentally, while remaining grid-based and unable to overcome the curse of dimensionality.Attention-based neural operators may model complicated latent-operator properties but require more parameters and training samples.

3.4 Theory

This section reviews theoretical results for physics-informed machine learning, covering operator expression ability and convergence analyses for PINNs. It also identifies the difficulty of studying convergence and expected-loss behavior in these methods.

Expression Ability: DeepONet can approximate any nonlinear continuous operator under the stated compactness and continuity assumptions.The theorem establishes existence of a neural network approximation for every prescribed error tolerance.
Expression Ability: Deep, narrow networks can approximate certain nonlinear continuous operators with depth 2k^3 + 8 and width Θ(1), whereas constrained wide, shallow networks cannot approximate them effectively.The comparison concerns networks with depth ≤ k and total nodes ≤ 2k.
Expression Ability: Theorem 3 gives a neural-network operator approximation with width at most m + n + 5 and a depth bound of O(M + N + L) under its assumptions.The result provides explicit upper bounds on both width and depth.
Convergence: PINN convergence properties remain insufficiently studied because the relevant physical equations and PDE stability properties are complicated.Existing work takes an initial step toward convergence analysis for time-independent PDEs under conditions such as unique classical solutions.
Convergence: Under stated assumptions, regularized empirical PINN losses can bound expected PINN loss with high probability, and minimizing Hölder-regularized loss can converge to the ground truth.The analyses use residual and boundary sample counts and establish general convergence results, with further conclusions for linear elliptic and parabolic PDEs.

Then we have following results

The reviewed results provide convergence and error bounds for PINNs and DeepONet, while emphasizing that theoretical justification, approximation, generalization, and convergence remain preliminary areas of PIML research.

Convergence: PINN convergence analyses provide high-probability and almost-sure guarantees under stated assumptions, including results for linear elliptic and parabolic PDEs.Theorem 5 presents general convergence analysis, while related work studies convergence for other physics-informed methods.
Error Estimation: DeepONet approximation error can be decomposed into encoding, approximation, and reconstruction errors, each of which can be analyzed separately.The decomposition supports bounds for the total approximation error under mild assumptions.
Error Estimation: Theorem 7 provides a bound on DeepONet generalization error under stated assumptions.The review also notes related error analyses for PINNs and FNO.
Open Challenges: Theoretical properties of representative methods such as PINNs and DeepONet remain insufficiently explored across expression ability, convergence, and error estimation.The survey identifies these analyses as valuable for guiding the design of better algorithms and architectures.

3.5 Application

Physics-informed machine learning is applied across fluid dynamics, material science, optimal control, scientific discovery, and other scientific and engineering domains. Examples include neural operators for multiphase flows and PINNs for material characterization and diverse physical processes.

Application Scope: PIML applications span fluid dynamics, material science, optimal control, and scientific discovery.The survey presents these as important application areas for methods that incorporate physical information.
Fluid Dynamics: U-FNO combines a Fourier neural operator with a U-Net structure to improve high-frequency representation for parametric multiphase flow problems.The survey also reports PINN effectiveness in inverse flow problems in reviewed work.
Material Science: Physics-informed methods model optical, electrical, and mechanical material properties and structures such as cracks, fractures, and defects.PINNs have been used for nonhomogeneous materials, metamaterials, and elastic-viscoplastic materials.
Other Domains: Additional applications include heat transfer, waves, nuclear physics, traffic, electricity and magnetics, medicine, geography, and industry.Examples include epidemiological modeling, groundwater contamination evaluation, civil engineering, smart manufacturing, and metal additive manufacturing.

4 INVERSE PROBLEM

The survey frames inverse design as constrained optimization for identifying or controlling physical systems, then reviews physics-informed neural approaches that reduce reliance on costly numerical optimization. It also highlights scalability, convergence, and large-parameter challenges that remain open.

Problem formulation: Inverse design seeks an optimal physical-system configuration that achieves a desired performance while satisfying physical constraints.Examples include airfoil shape optimization for drag minimization and heater placement for temperature management.
Problem formulation: Inverse-design problems can be formulated as PDE-constrained optimization or, more generally, constrained optimization, depending on the physical-process representation.The state variables and design objective depend on parameters θ, while the physical process may be represented by PDEs or other constraints.
Traditional methods: Traditional all-at-once and black-box methods become costly for large systems because they repeatedly solve original and adjoint PDEs and handle many optimization degrees of freedom.The estimated FEM complexity is O(dn^r), and continuous-function parameters, nondifferentiability, and nonuniqueness add difficulty.
Physics-informed methods: Physics-informed algorithms impose PDE knowledge as soft loss constraints, enabling inverse design through joint objective-and-PDE optimization or PINN surrogate solvers.PINN variants also use augmented losses, hard constraints, penalty methods, and augmented Lagrangians to address conflicts among loss terms.
Neural surrogate models: Physics-informed DeepONets demonstrate PDE-constrained optimization on Poisson, heat, and Stokes-flow problems with lower computational cost than numerical solvers and adjoint methods.The survey identifies accurate solution of complex PDE systems as the bottleneck for extending this approach.
Open challenges and future work: Large-scale inverse design remains challenging because many parameters create curse-of-dimensionality, scalability, and computational-complexity issues.The survey identifies AI-assisted acceleration and improved use of data and physical knowledge as promising directions.

5 COMPUTER VISION AND GRAPHICS

The survey reviews ways to incorporate physical knowledge into computer vision and graphics through data, architectures, losses, and inference. It emphasizes that physically grounded representations and symmetry-aware methods remain underdeveloped.

Scope: Computer vision and graphics address perception, generation, and interaction with environments governed by physical rules.Graphics must satisfy physical laws to produce realistic scenes.
Motivation: Purely data-driven vision models can face stability, reliability, security, out-of-distribution, and adversarial-example challenges.Physical knowledge can constrain predictions to follow laws and improve their stated precision.
Integration methods: Physical priors enter vision models through physically structured data, symmetry-aware architectures, physics-based losses, and inference-time correction.Examples include observational bias, rotation equivariance, Hamiltonian objectives, differentiable rendering, and simulator-based inference.
Data: Existing physical datasets support learning object-centric dynamics, Newtonian scenarios, and intuitive concepts such as continuity, persistence, and solidity.The VIND dataset contains more than 6000 videos and more than 4500 still images with ground-truth dynamics.
Open challenges and future work: Future work includes enforcing underlying symmetries, learning physically meaningful representations, and formalizing intuitive physics for learning frameworks.Mapping image-based dynamics to equation-based dynamics could support simulation, dynamics discovery, and physical-variable extraction.

6 REINFORCEMENT LEARNING

The survey organizes physics-informed reinforcement learning around policy training, model training, and exploration guidance. Physical knowledge is used to accelerate learning, improve environment models, and constrain unsafe actions, but generality, scalability, and guarantees remain difficult.

Foundations: Reinforcement learning agents learn policies through trial and error to maximize cumulative reward in an environment modeled by states, actions, transitions, and rewards.The survey notes sequential interaction and the absence of ground-truth training signals as core difficulties.
Integration framework: The survey categorizes physical-knowledge integration into policy training, environment-model training, and exploration guidance.These correspond to designing tasks and rewards, modeling physical dynamics, and constraining unsafe actions.
Policy training: Physical knowledge can design task rewards and guidance, including adaptive mesh refinement, quantum-state measurement, and chip placement.Reported examples include high error reductions, high average fidelity in hundreds of iterations, and superhuman placements in under 6 hours.
Model training: Physics-based environment models can outperform black-box models in offline model-based reinforcement learning, probably because of better extrapolation power.Object-centric prediction and planning also outperformed a state-of-the-art video prediction model in the cited work.
Exploration guidance: Safety methods constrain exploration using logic filters, Gaussian-process uncertainty, Hamilton–Jacobi–Isaacs policies, or generalized control barrier functions.The cited approaches aim to avoid unsafe actions, provide probabilistic or robust safety, or reduce constraint violations.
Open challenges and future work: Open challenges include generic task modeling, high-dimensional sample efficiency, and safety guarantees in complex uncertain environments.The survey states that probabilistic guarantees may require strong assumptions and can remain compromised even with runtime checking.

7 CONCLUSION

The survey defines PIML by organizing physical priors and their integration across machine-learning components and tasks. It concludes that the field has broad promise but needs stronger benchmarks, theory, scalable algorithms, and improved task-specific methods.

Conclusion: PIML uses empirical data and prior physical knowledge, including PDEs/ODEs/SDEs, symmetry constraints, and intuitive physics, to improve models for physical-mechanism tasks.These priors can be embedded in data, architectures, loss functions, optimization methods, and inference algorithms.
Conclusion: Most existing work focuses on neural simulation and inverse design for systems governed by PDEs or ODEs, while the survey also covers applications across machine-learning tasks.The review summarizes methods, challenges, and future directions from a machine-learning perspective.
Methodological directions: Standardized datasets and benchmarks are lacking because physical priors are diverse, limiting fair comparison across algorithms.The survey calls for realistic benchmarks containing one or multiple levels of physical prior.
Methodological directions: Optimization and inference algorithms that incorporate physical priors receive less attention than model architectures and remain valuable research targets.Inference methods may help ensure that outputs satisfy physical laws after pretraining.
Task directions: Future applications require scalable algorithms combining large data with real-world physical priors for computer vision and robotic control.The survey also identifies a persistent speed-and-accuracy gap between PINNs and specialized numerical methods such as FEM, FVM, FDM, and spectral methods.
Methodological directions: Theoretical understanding of convergence, generalization, and the benefits of physical priors remains at an early stage.The survey explicitly states that the theoretical benefits of introducing physical priors are not yet known.
Conclusion: The survey concludes that PIML is a fundamental and essential AI topic with substantial potential for improving current methods.This conclusion is paired with continued open challenges across methods and applications.