PaperlayerView exact revision on arXiv

Source-linked AI summary

Physics-Guided Deep Learning for Dynamical Systems: A Survey

Rui Wang, Rose Yu

arXiv:2107.01272v6cs.LG

TL;DR

Complex physical systems can be costly to simulate, partly unknown, and challenging for data-driven models to predict consistently. This survey organizes approaches that integrate physical knowledge into DL for learning dynamical systems. It concludes that such integration can improve physical consistency, accuracy, data efficiency, generalization, and interpretability, while identifying theoretical analysis as a challenge.

  • Problem

    Complex dynamics can be expensive to simulate and partly unknown, while DL models may violate physical laws or generalize poorly across systems.

    Source: §1 · p. 2

  • Method

    The survey reviews four ways to integrate physical constraints into DL for dynamical systems: loss regularizers, model design, hybrid models, and symmetry.

    Source: §8 · p. 18

  • Results

    The survey concludes that integrating physical principles can yield better physical consistency, higher accuracy, increased data efficiency, improved generalization, and greater interpretability.

    Source: §8 · p. 18

  • Takeaways & Limitations

    Physics-guided DL is a promising but still early-stage field, with challenges and opportunities for future research in learning physical dynamics.

    Source: §8 · p. 18

    Theoretical analysis of generalization remains lacking, and common i.i.d. assumptions do not fit many dynamical-system observations that are correlated across time and space.

    Source: §8.4 · p. 19

Abstract

from arXiv · show

Modeling complex physical dynamics is a fundamental task in science and engineering. Traditional physics-based models are sample efficient, and interpretable but often rely on rigid assumptions. Furthermore, direct numerical approximation is usually computationally intensive, requiring significant computational resources and expertise, and many real-world systems do not have fully-known governing laws. While deep learning (DL) provides novel alternatives for efficiently recognizing complex patterns and emulating nonlinear dynamics, its predictions do not necessarily obey the governing laws of physical systems, nor do they generalize well across different systems. Thus, the study of physics-guided DL emerged and has gained great progress. Physics-guided DL aims to take the best from both physics-based modeling and state-of-the-art DL models to better solve scientific problems. In this paper, we provide a structured overview of existing methodologies of integrating prior physical knowledge or physics-based modeling into DL, with a special emphasis on learning dynamical systems. We also discuss the fundamental challenges and emerging opportunities in the area.

1 Introduction

Physics-based simulation can be computationally intensive, difficult to simplify accurately, and incomplete for real-world systems; data-driven DL is efficient but may violate physical laws and generalize poorly. The survey reviews approaches that combine physical knowledge with DL for learning dynamical systems.

1 Introduction

  • Numerical methods for complex physical dynamics are computationally intensive, simplified models can be difficult to make sufficiently accurate, and many real-world systems are only partly known.Physical equations may not fully represent the true system states.Source: §1 · p. 2
  • DL can learn high-dimensional spatiotemporal dynamics efficiently, but without explicit constraints it may make physically implausible forecasts and struggle under distribution shift.For dynamics, shifts can arise from changing system parameters, external forces, and boundary conditions.Source: §1 · p. 2
  • Neither DL alone nor purely physics-based approaches are sufficient for complex dynamical systems, motivating methods that blend physical concepts with neural networks.The paper notes potential benefits including accurate simulation, scientific validity, reduced sample complexity, and improved generalization.Source: §1 · p. 2
  • The survey structures its review around four learning problems and four ways of combining physics and DL, then discusses challenges and future opportunities.The learning problems are differential-equation solving, dynamics forecasting, residual learning, and equation discovery; the combination categories are loss functions, architectures, hybrid models, and symmetry-based models.Source: §1 · p. 2

2 Significance of Physics-Guided Deep Learning

The section presents physics-guided DL as a way to accelerate simulation, improve scientific validity and generalization, and discover governing equations. It reviews motivations and examples across these four aims.

2 Significance of Physics-Guided Deep Learning

  • Physics-guided DL can potentially accelerate data simulation, build scientifically valid models, improve generalizability, and support governing-equation discovery.The section frames these as four potential benefits of incorporating physical principles, equations, mathematical modeling, and domain knowledge into DL.Source: §2 · p. 3

2.1 Accelerate Data Simulation.

  • Numerical simulation can be prohibitively expensive for complex dynamics because stability constraints require small discretization steps, while DL has demonstrated success streamlining compute-intensive scientific workflows.The section describes simulation as useful for analyzing, optimizing, and designing real-world processes.Source: §2.1 · p. 3

2.2 Build Scientifically Valid Models.

  • Scientific predictions may violate governing laws without physical constraints, so the section reviews methods that incorporate physical principles into models.Examples include conservation requirements in fluid dynamics and thermodynamic or quantum-mechanical principles in materials science.Source: §2.2 · pp. 3–4

2.3 Improve the generalizability of DL models

  • Models trained on particular datasets or systems may fail under distribution shifts, motivating approaches intended to generalize across systems and parameter domains.The section notes that physical laws remain constant across data domains and systems, and discusses symmetry-based and other physics-guided approaches.Source: §2.3 · p. 4

2.4 Discover Governing Equations

  • Governing-equation discovery is difficult when systems are nonlinear, variables interdependent, and data noisy or incomplete, while machine-learning methods can automate aspects of the search.Reviewed approaches include sparse basis-function identification, neural-network-augmented dictionaries, EQL, and PDE-Nets; scalability, overfitting, and reliance on high-quality data remain concerns.Source: §2.4 · pp. 4–5

3 Problem Formulation

The section formulates four learning problems for dynamical systems, distinguished by whether governing equations are known, partially known, or unknown and by what the model is asked to predict.

3 Problem Formulation

  • The section introduces these four problems as the principal research efforts motivated by physics-guided deep learning.They cover differential-equation solving, residual learning, forecasting, and governing-equation discovery.Source: §3 · p. 5

3.1 Solving Differential Equations

  • When governing equations are known but difficult to solve, neural networks approximate the solution while a residual constraint penalizes violations of the differential equations.The objective combines data misfit with a weighted equation-residual constraint.Source: §3.1 · p. 5

3.2 Learning Dynamics Residuals

  • When equations are partially known, hybrid models learn physics-model errors and combine the numerical physics prediction with a neural-network residual prediction.Training may jointly optimize equation and network parameters, or fit the equations first and then train on residuals.Source: §3.2 · p. 5

3.3 Dynamics Forecasting

  • Dynamics forecasting learns a map from a sequence of historic states to future states when equations are unknown or numerical solution is too costly.The input length is k, the output length is q, and the model is optimized using state prediction errors.Source: §3.3 · p. 6

3.4 Search for Governing Equations

  • Equation discovery seeks a governing function from candidate functions, selecting a sparse coefficient vector that approximates the system dynamics.The candidate library can include polynomials and trigonometric functions, with fitting based on observed data.Source: §3.4 · p. 6

4 Physics-Guided Loss Functions and Regularization

Physics-guided loss functions impose physical knowledge as soft constraints during training, supporting equation solving, prediction, and data generation. The section reviews PINNs and other applications, alongside optimization and constraint trade-offs.

4 Physics-Guided Loss Functions and Regularization

  • Physics-guided losses add differentiable soft constraints that can encourage physically consistent patterns and reduce the parameter search space.The section groups applications around differential-equation solving, prediction improvement, and data generation.Source: §4 · p. 6

4.1 Case Study: Physics-informed Neural Networks

  • PINNs approximate differential-equation solutions with neural networks trained on both measurement errors and equation-residual errors.Their loss also measures mismatch with boundary conditions and measured data at defined points.Source: §4.1 · pp. 6–7
  • PINNs have shown efficiency and accuracy on simple differential equations, but can struggle with complex systems because PDE regularization makes optimization difficult.The section also reports sensitivity to noisy or limited data and difficulty generalizing beyond the training space-time domain.Source: §4.1 · pp. 6–7

4.2 Solving Differential Equations

  • Other equation-solving approaches address PINN optimization, boundary errors, uncertainty, or specific physical constraints through training schedules, smoothing, Bayesian hybrids, and regularizers.Examples include gradually increasing constraint coefficients, stepwise time prediction, boundary-condition smoothing, uncertainty quantification, and divergence-free regularization.Source: §4.2 · p. 7

4.3 Improving Prediction Performance

  • Physics-guided losses encode constraints such as density ordering, thermal-energy conservation, atomic forces and energies, and model or boundary-condition violations to improve prediction consistency.The section also reviews Hamiltonian and Lagrangian neural networks, which use energy-related constraints to learn dynamics.Source: §4.3 · pp. 7–8

4.4 Data Generation

  • For data generation, physics-guided losses impose constraints including covariance, temporal coherence, energy spectra, snow water equivalent, divergence-free motion, and Navier–Stokes residuals.The cited applications include GAN-based simulation and flow super-resolution.Source: §4.4 · p. 8

4.5 Pros and Cons

  • Loss-based constraints are generally soft rather than guaranteed on new datasets, and PDE regularization can complicate optimization or trade off physical consistency against prediction error.For turbulent-flow prediction, too much divergence-free regularization may smooth small eddies and increase prediction error.Source: §4.5 · p. 8

5 Physics-Guided Design of Architecture

Physics-guided architectures encode physical properties as hard constraints in neural-network structure, with TF-Net combining scale separation and learned interactions for turbulent-flow prediction. The section surveys convolutional, graph, and MLP designs, and notes that hard inductive biases can constrain representation when data or assumptions make them unsuitable.

5.1 Case Study: Turbulent-Flow Net

  • TF-Net replaces fixed spectral filters with trainable convolutions, assigns three flow scales to specialized U-nets, and uses a shared decoder to combine their interactions.The design targets both large eddies and subgrid-scale motions, which direct MSE training may overlook.Source: §5.1 · p. 9, Figure 1 · p. 9
  • TF-Net consistently outperforms baselines on divergence and energy-spectrum metrics, and its predictions most closely match target velocity motion in shape and frequency.The component ablation indicates that each small U-net outputs flow at a different scale, supporting the model’s multi-scale behavior.Source: §5.1 · p. 9, Figure 2 · p. 10, Figure 3 · p. 10

5.2 Convolutional architecture

  • Physics-guided convolutional designs encode constraints or structure in CNNs, recurrent units, hybrid solver architectures, and continuous-filter layers for physical prediction tasks.PDE-Net also uses convolutions to approximate differential operators and a symbolic network to recover governing equations [105; 106].Source: §5.2 · p. 10

5.3 Graph Neural Networks

  • Graph networks support dynamics on irregular meshes through learned message passing, with adaptive remeshing and training strategies addressing scale resolution, instability, and error accumulation.Neural Operators and Fourier Neural Operators learn function-space mappings across approximations and grids [95; 96].Source: §5.3 · p. 11

5.4 Multilayer Perceptron

  • MLPs can learn finite-dimensional Koopman representations by embedding observed dynamics into a latent space where a linear layer approximates the operator.Koopman Neural Forecaster uses predefined measurement functions and a time-varying operator for highly non-stationary time series, outperforming state-of-the-art methods on M4, cryptocurrency returns, and sports-player trajectories.Source: §5.4 · pp. 11–12

5.5 Pros and Cons

  • Architectural physics constraints can provide theoretically guaranteed properties, but hard inductive biases may restrict representation when data suffice to learn the biases or those biases are not strict.The design challenge is to generalize well without diminishing neural networks’ representation power.Source: §5.5 · p. 12

6 Hybrid Physics-DL Model

Hybrid physics-DL models combine learned components with physics-based models, while Neural ODEs represent hidden-state dynamics continuously and use numerical solvers. The survey describes residual and intermediate-variable approaches and cautions that neural networks can dominate a hybrid model, weakening its interpretability and generalizability.

6.1 Case Study: Neural Differential Equations

  • Neural ODEs model hidden-state change continuously with a neural network and use a differential-equation solver to map encoded initial states to decoded predictions.The adjoint method computes gradients by solving an adjoint differential equation backward from the final time.Source: §6.1 · p. 12

6.2 Residual Modeling

  • Residual modeling trains deep learning to correct errors or bias in physics-based models, with DeepGLEAM combining epidemic simulation and a recurrent network for one-week-ahead COVID-19 death forecasts.Other examples use learned corrections for coarse-grid fluid solvers, subgrid modeling, and reduced-order models.Source: §6.2 · p. 13

6.3 Intermediate Variable Modeling

  • Intermediate-variable modeling replaces difficult or unknown physics-model components with learned components, including solver replacement, closure corrections, and neural estimates of velocity.These learned intermediate quantities are then used within the remaining physics-based computation.Source: §6.3 · p. 13

6.4 Pros and Cons

  • Hybrid models risk having neural networks capture most of the dynamics, leaving physics-based components with little contribution and weakening interpretability and generalizability.The intended balance is for neural networks to model only information the physical prior cannot represent.Source: §6.4 · p. 14

7 Invariant and Equivariant DL Models

The survey reviews how symmetry can be built into or learned by neural networks for dynamical systems. Equivariance links transformed inputs to correspondingly transformed outputs, while invariance leaves outputs unchanged; these structures can support physical consistency and robustness to distribution shifts.

7.1 Case Study: Equivariant Deep Dynamics Models

  • In fluid-dynamics forecasting, equivariant ResNets and U-Nets outperform non-equivariant counterparts on simulated RBC and real-world ocean-current data, with lower energy-spectrum errors and favorable sample complexity versus data augmentation.RBC tests apply random symmetry transformations; ocean tests use different time ranges and domains from training.Source: §7.1 · p. 15, Figure 5 · p. 16

7.2 Equivariant Convolutional Neural Networks

  • Approximate equivariant networks relax strict weight sharing so data can determine how much symmetry to preserve; continuous convolution models encode geometric structure through continuous inputs and equivariant kernels.Approximate equivariant models perform well on turbulence data with no, approximate, or perfect symmetry; ECCO uses polar-coordinate kernel weight sharing for rotational equivariance and outperforms baselines on Argoverse and TrajNet++.Source: §7.2 · p. 16

7.3 Equivariant Graph Neural Networks

  • Equivariant graph networks encode molecular and geometric symmetries, while some approaches address the limits of enforcing all permutations by using locally canonicalized graph neighborhoods.The reviewed applications include molecular-property prediction, conformation generation, and prediction of wavefunctions and electronic densities.Source: §7.3 · p. 17

7.4 Symmetry Discovery

  • Symmetry-discovery methods learn weight-sharing patterns or Lie-algebra representations from data, enabling neural networks to discover symmetries rather than rely only on predefined ones.The survey describes approaches based on meta-learning, Lie Algebra Convolution, and adversarial training.Source: §7.4 · p. 17

7.5 Pros and Cons

  • Equivariance can support generalization across input transformations and physical consistency, but imposing too many symmetries may constrain representation power and slow training and inference.Real-world dynamics may exhibit only partial or approximate symmetries, motivating models that can learn the appropriate degree of symmetry.Source: §7.5 · pp. 17–18

8 Discussion

The discussion reviews physics-guided DL for dynamical systems, outlining its potential benefits and identifying challenges and opportunities for future research. It highlights needs spanning generalization, forecasting robustness, non-Euclidean dynamics, theory, causal inference, law discovery, and computation.

8 Discussion

  • Physics-guided DL may improve physical consistency, accuracy, data efficiency, generalization, and interpretability, but the field remains at an early stage.The survey identifies emerging challenges and opportunities for future studies.Source: §8 · p. 18

8.1 Improving Generalization

  • Models trained on particular systems often struggle with distribution shifts; future models should extrapolate across parameters, external forces, and boundary conditions.DyAd generalizes across heterogeneous fluid-dynamics domains for interpolated parameters but not beyond its training range.Source: §8.1 · p. 18

8.2 Improving Robustness of Long-term Forecasting

  • Long-horizon forecasts face error accumulation and input-perturbation instability; proposed remedies include noise training, multi-step autoregression, Lyapunov regularization, and online normalization.These techniques have been trained on abundant simulation data, while obtaining real-world data such as jet-flow experiments can be expensive.Source: §8.2 · pp. 18–19

8.3 Learning Dynamics in Non-Euclidean Spaces

  • Many spatiotemporal phenomena occupy non-Euclidean spaces, so models may need geometric concepts and gauge equivariance rather than ordinary Euclidean representations.The discussion notes that much geometric deep learning work has focused on static graphs.Source: §8.3 · p. 19

8.4 Theoretical Analysis

  • Theory remains limited: dynamical observations can be correlated, and existing generalization bounds may be loose; tighter bounds should reflect dynamics such as order and Lyapunov exponents.The discussion also calls for lower generalization bounds to identify best-performance scenarios.Source: §8.4 · p. 19

8.5 Causal Inference in Dynamical Systems

  • Open causal-inference questions include using causality to improve DL, disentangling multiple treatments, controlling dynamics through environment design, and predicting responses to interventions.Climate-policy interventions are given as a setting where such predictions could inform government decisions.Source: §8.5 · p. 19

8.6 Search for Physical Laws

  • DL-based physical-law discovery remains preliminary, with current approaches constrained by large search spaces, limited high-quality experimental data, overfitting, and reliance on synthetic data for symmetry discovery.The discussed symmetry-discovery works identify known symmetries.Source: §8.6 · pp. 19–20

8.7 Efficient Computation

  • Physics-guided DL workflows need more efficient training, testing, and deployment on large datasets and HPC systems, with scalable, transferable tools integrated into simulation platforms.The discussion envisions a complete pipeline for simulation and analysis of dynamical systems.Source: §8.7 · p. 20

Related papers

Loading 2107.01272v6…