PaperlayerView exact revision on arXiv

Source-linked AI summary

When Physics Meets Machine Learning: A Survey of Physics-Informed Machine Learning

Chuizheng Meng, Sungyong Seo, Defu Cao, Sam Griesemer, Yan Liu

arXiv:2203.16797v1cs.LGstat.ML

TL;DR

PIML addresses limited training data and the need for physically plausible, generalizable machine-learning models by integrating prior physics knowledge. This survey synthesizes recent work by examining PIML motivations, physics knowledge, and integration methods, while identifying challenges and research directions. It concludes that physics knowledge is integrated through data, architectures, and optimization, with reusable computation-graph biases prominent for several general physics principles.

  • Problem

    Limited training data can leave useful task information unavailable or difficult to extract, while machine-learning models also require physical plausibility and generalization.

  • Method

    The paper surveys recent PIML research across motivations, physics knowledge, and methods for integrating knowledge through data enhancement, architecture design, and physics-informed optimization.

  • Results

    Existing work integrates analytical physics knowledge into data, models, and optimization, while energy conservation, symmetry, PDE methods, and Koopman theory mainly enter computation graphs.

  • Takeaways & Limitations

    The survey provides a framework for selecting physics knowledge and integration methods and for identifying gaps and promising research directions in PIML.

  • Takeaways & Limitations

    PIML lacks comprehensive benchmarks for evaluating diverse knowledge-integration methods, partly because its physics and engineering problems are complex and heterogeneous.

Abstract

from arXiv · show

Physics-informed machine learning (PIML), referring to the combination of prior knowledge of physics, which is the high level abstraction of natural phenomenons and human behaviours in the long history, with data-driven machine learning models, has emerged as an effective way to mitigate the shortage of training data, to increase models' generalizability and to ensure the physical plausibility of results. In this paper, we survey an abundant number of recent works in PIML and summarize them from three aspects: (1) motivations of PIML, (2) physics knowledge in PIML, (3) methods of physics knowledge integration in PIML. We also discuss current challenges and corresponding research opportunities in PIML.

1 Introduction

PIML integrates validated physics knowledge with data-driven models to address limited training data and improve data efficiency, generalization, and physical plausibility. The survey organizes recent work around motivations, physics knowledge, and integration methods.

  • Prior knowledge can improve data efficiency, generalization, and the plausibility of machine-learning models when training data are limited.It provides information that may not exist in, or be easy to extract from, limited training data.
  • PIML incorporates theoretically and empirically validated physics knowledge into machine-learning models.Physics knowledge summarizes natural phenomena and human behaviours, but its special properties require dedicated integration designs.
  • The survey reviews PIML through motivations, physics knowledge, and methods for integrating that knowledge.Its motivation categories distinguish machine learning for physics-domain tasks from physics-informed models for real-world tasks.
  • Integration methods are categorized by their location in the pipeline: data enhancement, neural-network architecture design, and physics-informed optimization.This categorization describes where physics knowledge enters the machine-learning process.
  • The paper discusses challenges and potential future research directions after surveying PIML motivations, knowledge, and integration methods.The paper’s organization includes dedicated sections on challenges, future directions, and an overall summary.

2.1 ML for Physics: Enhancement of Physics Models via Data-Driven Methods

One PIML direction uses machine learning to enhance physics models, including simulation, PDE solving, parameterization, reduced-order modeling, and causal analysis. These methods target flexibility, generalizability, physical insight, or lower computational cost while retaining physics-related structure.

  • Machine-learning models can serve as surrogates for expensive numerical simulations across turbulence, climate, and particle-system domains.Simulation-trained surrogates can predict dynamics accurately within a shared architecture and run orders of magnitude faster than simulation methods.
  • Data-driven PDE methods infer solution mappings from spatial-temporal inputs and enforce PDE structure through derivatives and physics-based losses.Some approaches avoid discretization by learning PDE solutions directly from data.
  • Machine learning supports parameterization by approximating unknown dynamics or process parameters in complex physical models.Applications include geological models, subsurface flow, suspended sediment, and meteorological heat, moisture, and convection processes.
  • Neural networks can learn hidden spaces and mappings for reduced-order models, including trainable encoder-decoder pairs for Koopman representations.Neural Koopman models address the difficulty of designing hand-crafted mappings for complex dynamics and can achieve comparable or higher performance.

2.2 Physics for ML: Improvement of Data-Driven Models from External Knowledge

A second PIML direction injects physics knowledge into data-driven models for physics-related and real-world tasks. The reviewed approaches address varied data structures and integrate physics through model constraints, architectures, and optimization to improve plausibility, efficiency, and generalization.

  • Pure neural-network optimization is highly non-convex, motivating physics-informed inductive biases for data-driven modeling.PIML incorporates such biases into training data, model architectures, or optimization processes.
  • PIML is applied to object-centric, spatio-temporal, and manifold data arising in multi-agent systems, sensors, traffic, weather, electricity, brain activity, images, and scans.These data types reflect diverse physical-system structures and geometries.
  • Physics-informed models are reported to provide greater physical plausibility, data efficiency, and generalization than pure neural-network models.These properties are important for modeling control inputs and state transitions in dynamic systems.
  • Physics-informed dynamics models have succeeded in model predictive control and model-based reinforcement learning.Examples enforce stability, Lagrangian or Hamiltonian dynamics, compositional Koopman operators, and PDE-based constraints.

3 Physics Knowledge in PIML

The survey organizes general physics knowledge used in PIML into classical mechanics, symmetries, and numerical methods for PDEs. These principles constrain learned models through conservation laws, transformation structure, and discretizations of continuous dynamics.

  • Classical Mechanics and Energy Conservation Laws: Lagrangian mechanics represents dynamics with L(q, q̇) = T(q, q̇) − V(q), the difference between kinetic and potential energy.The physical trajectory satisfies stationary action, from which the Euler–Lagrange equation is derived.
  • Classical Mechanics and Energy Conservation Laws: Time-independent Lagrangians satisfy energy conservation through time-translation symmetry, while generalized forces extend the formulation to non-conservative systems.The extension uses generalized forces τ, which can also serve controllers such as PD controllers.
  • Classical Mechanics and Energy Conservation Laws: Hamiltonian mechanics uses spatial coordinates q and momenta p, with H(q, p) equal to total energy T + V for classical mechanical systems.The resulting Hamiltonian dynamics satisfy dH/dt = 0, expressing conservation of energy.
  • Symmetry, Invariant and Equivariant Functions: Symmetry-aware networks encode domain transformations by stacking layers that are invariant or equivariant under a symmetry group.The survey discusses translation, rotation, reflection, Euclidean, and permutation symmetries, and categorizes architectures using coordinate treatments or group representations.
  • Numerical Methods for Partial Differential Equations (PDEs): Finite difference, finite volume, and finite element methods incorporate PDE structure through derivative approximations, fluxes over cell boundaries, or weak formulations on spatial elements.Finite volume evolution computes analytical fluxes between grid cells, while finite elements solve a Galerkin system Au = f in a finite-dimensional function space.
  • Numerical Methods for Partial Differential Equations (PDEs): Neural networks can learn undetermined coefficients in discretized PDE schemes when some constraining equations are relaxed, combining prior structure with stronger expressivity.This extension follows spatial derivative approximation and numerical integration of the PDE.

4 Methods of PIML

Physics knowledge can be integrated into data, model, and optimization components of machine-learning solutions. The survey classifies these methods by knowledge form and integration location, finding reusable computation-graph inductive biases advantageous for several general physics principles.

  • Physics knowledge can be integrated into data, model, or optimization, and these integration techniques are not mutually exclusive.
  • Table 2 classifies existing PIML works by the form of physics knowledge and the method used to integrate it.
  • Analytical domain knowledge is integrated across data, model, and optimization, whereas energy conservation, symmetry, numerical PDE methods, and Koopman theory mainly enter computation graphs.
  • For general physics knowledge, reusable computation-graph inductive biases have advantages over data augmentation and physics-based loss functions in prediction performance and data efficiency.The survey attributes this to broad applicability and simpler forms expressible with limited differentiable operators.

4.1 Physics-Informed Data Enhancement

Physics-informed data enhancement generates or augments training information from physics-based simulations. These data can support surrogate modeling, pretraining, auxiliary tasks, and synthetic-to-real transfer, while distribution differences require transfer-learning techniques.

  • Training data generated from physics knowledge provide a direct way to incorporate that knowledge into expressive neural networks.The approach is motivated by the universal approximation theorem and is most useful when data are abundant and training is adequate.
  • Simulation-trained surrogate models can match numerical solutions competitively while reducing computation costs and predicting particle-system dynamics orders of magnitude faster.The cited examples cover global weather simulation and message-passing graph neural networks for particle systems.
  • Physics-based simulators provide large, high-quality labeled datasets that can pretrain models for real-world tasks with limited data or difficult labeling.Because simulation and target distributions differ, transfer learning is used to mitigate the gap.
  • Domain randomization varies simulator parameters such as lighting, pose, and textures to encourage object detectors to learn essential features.Other synthetic-data systems combine simulator rendering with pixel-level adaptation between synthetic and real images.
  • Synthetic simulation data can define auxiliary tasks for multitask or meta-learning, such as predicting interatomic forces alongside molecular energy.

4.2 Physics-Informed Neural Network Architecture Design

Physics-informed architecture design embeds physical behavior into neural computation graphs or combines neural modules with physics-based solvers. The surveyed designs cover conservation laws, symmetries, numerical PDE methods, Koopman theory, and higher-level hybrid models.

  • Physics-informed architectures replace difficult-to-estimate variables in physics-based solutions with neural-network outputs or allow fixed parameters to adapt to data.
  • Energy Conservation Laws: Lagrangian and Hamiltonian neural architectures encode energy conservation while retaining trainability or expressive density modeling.DeLaN can use standard optimizers while maintaining physical plausibility, whereas Neural Hamiltonian Flow provides invertibility, volume preservation, and avoids trace-of-Jacobian computation.
  • Symmetry: Symmetry-aware architectures use invariant coordinates, irreducible representations, or regular representations to incorporate group symmetries.Distance-based messages enforce E(3) equivariance, while Lie-group methods reduce the effort of adding new symmetry groups.
  • Numerical Methods: Numerical PDE methods are incorporated by learning constrained differential operators or coefficients within finite-difference, finite-volume, and finite-element-inspired architectures.Examples include CNN-generated finite-volume coefficients, GNN-based finite-element analysis, and wavelet-derived kernel constraints.
  • Koopman Theory: Koopman-based models learn nonlinear coordinates in which dynamics become globally linear, combining deep-learning flexibility with interpretable embeddings.
  • Hybrid Models: Hybrid models let standard neural-network modules and physics rules interact at a higher level while exposing inputs and outputs to one another.Examples combine physics-derived state prediction with GNN residuals or ConvLSTM backbones with model-driven computation.

4.3 Physics-Informed Optimization

Physics-informed optimization incorporates task-specific equations or physical regularizations into training objectives. These losses reshape the optimization space toward physically plausible solutions, including PDE-consistent predictions and stability-constrained control models.

  • Physics-based loss functions and regularizations reshape the optimization space and encourage convergence toward physically plausible solutions.
  • Task-Specific Knowledge Based Loss Terms: PDE-informed losses penalize governing-equation residuals and boundary violations alongside prediction error.Applications include PDE-solution modeling, turbulent-flow super-resolution, traffic-flow prediction, and lake-temperature prediction.
  • Drone control training minimizes an upper bound on the Lipschitz constant, represented by layer spectral norms, together with prediction error to guarantee stability.
  • Control-signal norms can be minimized when learnable signals augment physics- or rule-based predictors to mitigate prior-knowledge approximation errors.The regularization originates from the least action principle.

5 Challenges and Future Directions

The survey identifies three major PIML challenges: selecting physics knowledge manually, evaluating methods without comprehensive benchmarks, and adapting neural architectures and optimization to physics-informed objectives.

  • 5.1 Challenge 1: Handcrafted Selection of Physics Knowledge for Incorporation: Existing PIML methods rely on domain expertise to choose appropriate physics knowledge, and this choice remains heuristic even within the same physical system.For turbulence, one study uses the Rayleigh-Bénard governing equation while another uses a derived Hybrid RANS-LES coupling method.
  • 5.1 Challenge 1: Handcrafted Selection of Physics Knowledge for Incorporation: Automatic identification of suitable physics knowledge is proposed as a middle ground between domain-specific knowledge and purely data-driven approaches.NAS could search among pre-selected physics-knowledge combinations, while modular architectures could organize physics-specific components.
  • 5.2 Challenge 2: Lack of Benchmarks and Evaluations of PIML Methods: PIML lacks comprehensive benchmarks because tasks and datasets are heterogeneous, difficult to formalize, and often require domain expertise, hindering fair comparisons.Ideal benchmarks should provide organized public datasets, standard machine-learning tasks, and coverage of PIML, data-driven, and pure physics-based methods.
  • 5.3 Challenge 3: Suboptimal Existing Neural Network Architectures and Optimization Methods for PIML: Conventional neural-network architecture and optimization findings may not transfer to PIML because gradients and high-order derivatives create substantially different optimization spaces.The survey cites insufficient Kaiming and Xavier initialization and ineffective ReLU activation in physics-informed settings.
  • 5.3 Challenge 3: Suboptimal Existing Neural Network Architectures and Optimization Methods for PIML: The survey calls for novel PIML designs that preserve gradient information and address conflicts between task objectives and physics-informed constraints.Suggested directions include periodic activations and dynamic pulling to align boundary-constraint and approximation-loss update directions.

6 Summary

The paper surveys PIML motivations, physics knowledge, and integration methods, then discusses challenges and future research directions. It aims to guide physics-knowledge and integration-method selection while identifying gaps and opportunities.

  • 6 Summary: The survey organizes existing PIML research around motivations, physics knowledge, and methods for integrating that knowledge.It also discusses current challenges and corresponding future research directions.
  • 6 Summary: The paper is intended to help PIML users select appropriate physics knowledge and integration methods.This guidance is presented alongside a research-oriented identification of gaps and promising directions.
  • 6 Summary: The survey aims to help PIML researchers identify existing gaps and promising research directions.
Loading 2203.16797v1…