Integrating Scientific Knowledge with Machine Learning for Engineering and Environmental Systems

1 INTRODUCTION

• Scientific problems with incompletely understood processes or computationally infeasible mechanistic models motivate ML, but black-box ML can demand extensive data, violate physical consistency, and generalize poorly out of sample.The survey distinguishes integrating knowledge into the ML framework from feature engineering or post-processing.Source: §1 · pp. 1–2
• The field spans diverse disciplines and objectives, with publication counts for “physics-informed ML” rising from 2 in 2017 to 63 in 2020.The passage also reports 8 publications in 2018 and 27 in 2019.Source: §1 · p. 2
• The survey focuses on environmental and engineering systems where scientific knowledge is available as mechanistic models, theories, or laws, and proposes a taxonomy spanning objectives and methods.It distinguishes this scope from general knowledge integration and domain-specific surveys.Source: §1 · p. 2
• The paper organizes application-centric objectives, physics-guided losses, initializations, architectures and hybrid models, then considers cross-fertilization and research gaps.It categorizes surveyed work by application objective and integration method.Source: §1 · p. 2

2 APPLICATION-CENTRIC OBJECTIVES OF PHYSICS-ML INTEGRATION

• The section groups objectives into improving mechanistic-model effectiveness, increasing resource efficiency, and discovering knowledge such as physical parameters or governing equations.The generic problem represents dynamic inputs and static system characteristics feeding a model that predicts target variables.Source: §2 · pp. 3–4
• Resource-efficiency objectives include downscaling, ML parameterization, reduced-order models, and data-driven PDE solutions when physical models are too slow or coarse.The section also identifies data generation for limited-data settings and uncertainty quantification of predicted outputs.Source: §2 · pp. 3–4
• The objectives differ in their needs for real-world observations, synthetic samples, or knowledge of governing equations.Table 1 summarizes these requirements.Source: §2 · p. 4, Table 1 · p. 4

2.1 Replacing or improving state-of-the-art physical models

• Combining physics-based modeling and ML aims to leverage their complementary strengths, improving system dynamics and process understanding; an early lake-temperature example reported better accuracy with fewer samples and out-of-sample generalizability.Mechanistic models can be biased by incomplete process knowledge and limited parameter observations, while black-box ML may require extensive labels, yield inconsistent results, and generalize poorly.Source: §2.1 · p. 5

2.2 Downscaling

• Downscaling uses empirical models to map coarse-resolution inputs to finer-resolution variables or ML surrogates to approximate costly high-resolution regional simulations.Ensuring physical-law consistency and improved overall simulation performance remains a challenge.Source: §2.2 · pp. 5–6

2.3 Parameterization

• ML-based parameterization replaces complex or missing physical processes with learned dynamic or statistical representations, reducing computation time relative to traditional physical simulations.Integrating physical principles into these models may improve robustness and generalization while reducing training-sample needs.Source: §2.3 · p. 6

2.4 Reduced-Order Models

• ML can construct reduced-order representations, approximate dimensionality-reduction mappings, or learn residuals; physics principles may help address ROM robustness and data needs.ML-based Koopman operator embeddings approximate dominant modes for dimensionality reduction, while current methods tend to lack generalizability and interpretability.Source: §2.4 · pp. 6–7

2.5 Forward Solving Partial Differential Equations

• Neural-network surrogates can approximate PDE solutions in a forward pass, but slow convergence, limited applicability, and physical-law unawareness can reduce accuracy.The section also describes a neural Fourier operator learning mappings from parametric functions to solutions for families of PDEs.Source: §2.5 · p. 7

2.6 Inverse Modeling

• Inverse modeling uses system outputs to infer intrinsic physical parameters, and physics-guided constraints may improve data efficiency and help address ill-posed problems.The survey focuses on physical parameters in mechanistic engineering and environmental systems.Source: §2.6 · pp. 7–8

2.7 Discovering Governing Equations

• Equation-discovery methods search for governing equations when dynamical systems lack formal analytic descriptions, using approaches such as symbolic regression and sparse regression.Sparse identification methods seek concise equations by selecting only a few terms from candidate functions and derivatives.Source: §2.7 · pp. 8–9

2.8 Data Generation

• Generative ML learns data distributions in unsupervised settings and may generate novel data beyond the limits of physics-based simulations or experiments.Traditional simulation and experimentation can be time-consuming and restricted by what physics-based models can produce.Source: §2.8 · p. 9
• GANs can reduce data-generation computation time and reproduce complex phenomena; physics constraints such as conservation laws or energy spectra can further improve scientific generative models.A physics-based morphology constraint in a VAE also makes generated material samples match authentic morphology distributions, reducing the material design space.Source: §2.8 · p. 9
• GANs have high sample complexity, motivating efforts to incorporate physical laws and invariances into their design.Source: §2.8 · p. 9

2.9 Uncertainty Quantification

• UQ characterizes the full response distribution p(y|x), rather than only a point prediction, supporting quantile, skew, acceptability, and input-sensitivity analyses.Source: §2.9 · p. 10
• Monte Carlo UQ for physics-based models can be infeasible because it requires many forward evaluations, while Gaussian Processes scale as O(N^3) and do not scale well to higher dimensions or larger datasets.Source: §2.9 · p. 10
• Deep-learning surrogates are being fitted to numerical simulations for faster, more scalable UQ, using approaches including probabilistic dropout, Bayesian neural networks, and neural-network ensembles.Source: §2.9 · p. 10
• Physics integration may improve uncertainty characterization by reducing physically inconsistent surrogate predictions and potentially reducing Bayesian neural networks’ data needs and computational cost.Source: §2.9 · p. 10

• Physics-ML methods are categorized as physics-guided loss functions, initialization, architecture design, and hybrid modeling.Source: §3 · p. 10
• Method choice depends partly on the availability and performance of existing mechanistic models and the computational objectives to be addressed.Source: §3 · p. 10
• The three general objectives are improved prediction performance, sample efficiency, and interpretability.Prediction performance includes accuracy, out-of-sample generalizability, or physical consistency; sample efficiency reduces observation needs or search space.Source: §3 · p. 10
• The objectives can overlap: physical consistency can reduce the solution search space, and sample efficiency can improve prediction performance by extracting more value from each observation.Source: §3 · p. 11

Replacing or improving over physical models

• For lake-temperature prediction, a monotonic-density penalty and temporal energy-conservation penalties encode physical relations; the PGRNN also uses physics-derived energy and temperature outputs in its loss.The recurrent model adds nodes for physical variables calculated from physics-based equations; similar structure can encode other laws such as mass conservation.Source: §3.1 · p. 11, Figure 2 · p. 12
• Physics-based stability penalties improved fluid-flow and sea-surface-temperature predictions and generalizability, while physics-guided loss also improved Lorenz-system echo-state predictions and reduced convergence time.Source: §3.1 · p. 12

Inverse modeling

• In inverse modeling, physics-based loss terms improved accuracy and out-of-sample robustness for estimating forces from sparse velocity data and locating underwater obstacles from acoustic measurements.Source: §3.1 · p. 12

Parameterization

• Physics-based loss terms support physically consistent parameterization, with energy conservation improving cloud-process emulation and atomic-energy models incorporating force and energy terms.Source: §3.1 · p. 13

Downscaling

• PDE-based constraints in super-resolution can recover fluid quantities more efficiently, while constraints in turbulence GANs improved generalization and extrapolation.Source: §3.1 · p. 13

Uncertainty quantification

• Physics-guided probabilistic generative models use PDE structure for UQ and uncertainty propagation, with constraints serving as regularization when data are costly and sparse.Source: §3.1 · p. 13
• Physics-guided Bayesian neural networks use PDE constraints as priors, with physics-based regularization supporting more accurate predictions under significant noise.Source: §3.1 · p. 13

Generative models

• PDE-based GAN loss functions have been used to solve stochastic elliptic PDEs up to 30 dimensions and to reduce data and training time for turbulence solutions.Related work applies them to generating microstructures with specified physical properties.Source: §3.1 · p. 13

• Physics or contextual knowledge can inform model initialization, and transfer learning pretrains on a related task before fine-tuning with limited task-specific data.A physics-informed starting state may accelerate training and require fewer training samples.Source: §3.2 · pp. 13–14
• Pretraining lake-temperature PGRNNs on physics-model simulations and fine-tuning with limited observations reduced training-data needs, even when the simulator’s parameters were incorrect.Combining physics-guided initialization with a physics-guided loss also improved generalization to unseen scenarios relative to traditional physics-based models.Source: §3.2 · p. 14
• Simulation-based pretraining in robotics and autonomous driving can reduce real-world data needs, including a reported 50-fold reduction for object grasping.Source: §3.2 · p. 14
• Chemical-process transfer approaches adapt Gaussian-process or neural-network models from similar tasks, with similarity based on shared process characteristics and behaviors.Source: §3.2 · p. 14
• Self-supervised initialization can train hidden representations using physics-simulated intermediate variables; a river-network graph neural network used upstream variables simulated by PRMS-SNTemp.Source: §3.2 · p. 14

Intermediate Physical Variables

• Physically meaningful intermediate variables can yield consistent predictions and uncertainty estimates, improve drag-force prediction over physics-based models, and expose interpretable representations to domain scientists.The lake-temperature architecture uses a monotonicity-preserving structure and dropout; the drag-force model inserts physics-constrained variables into a CNN.Source: §3.3 · p. 15

Encoding invariances and symmetries

• Symmetry-aware designs encode physical invariances in neural networks, while symmetry or separability can also narrow symbolic-regression searches for governing equations.Examples include rotationally invariant tensor-basis networks, customized symmetry-enforcing convolutions, and a recursive method that uses neural networks to identify simplicity in data.Source: §3.3 · pp. 15–16

Additional architecture and learning structures

• Other designs use atom-wise networks constrained by element, wavelet-based layers for electrical impedance tomography, continuous-depth networks for differential equations, auxiliary learning tasks, Gaussian-process kernels, and physics-informed architecture search.The survey notes that most domain-knowledge-encoded architectures are manually developed, motivating searches with physically meaningful nodes and operations.Source: §3.3 · pp. 16–18

Architecture modifications for differential equations

• For PDE-related tasks, hyperbolic CNNs preserve system energy, and Fourier neural operators extend neural-network solvers to functional spaces and families of PDEs.The cited Fourier neural operators achieve greater accuracy than previous ML-based solvers.Source: §3.3 · p. 16

Hamiltonian architectures

• Hamiltonian-based networks predict a system’s Hamiltonian and reintegrate it, learning energy conservation and, in later work, abstract phase space.A further extension adds differential-equation integration steps based on derivatives approximated by the Hamiltonian network.Source: §3.3 · p. 17

Encoding other domain-specific physical knowledge

• Architectures can encode domain-specific structure through informed layers, discriminators, or probabilistic dependencies, including for bearing faults, brain segmentation, fluid-flow generation, and computer-memory devices.Examples include tempoGAN’s added discriminator and loss terms to preserve temporal coherence in physics-based fluid simulations.Source: §3.3 · p. 17

3.4 Hybrid Physics-ML Models

• Hybrid physics-ML models make mechanistic and ML models operate together, either by replacing a mechanistic model or replacing one of its components.They are also described as ML-enhanced physical models.Source: §3.4 · p. 18

3.4.1 Residual modeling.

• Residual modeling trains an ML model to predict a physics model’s errors and correct its predictions, but does not enforce physical constraints on the modeled quantities.Applications include correcting reduced-order-model truncation errors and reducing computational cost and time-discretization error in reduced-order modeling.Source: §3.4.1 · p. 19

3.4.3 Replacing part of a physical model with ML.

• Replacing physics-model components with ML can estimate missing turbulence-closure variables, substitute data-driven equations, or embed neural-network layers in a power-grid solution framework.The power-grid approach reduces computational cost through fast neural-network evaluation while retaining physical-model information and constraints.Source: §3.4.3 · p. 20

3.4.4 Combining predictions from both physical model and ML model.

• Hybrid models can combine physics and ML predictions with weights that depend on process scale or prediction circumstances.Examples include using physics for long-range interactions and ML for short-range interactions, and selecting a model using estimator quality.Source: §3.4.4 · p. 20

3.4.5 ML informing or augmenting physics-model for inverse modeling.

• For inverse modeling, hybrid approaches can refine physics-based direct-inversion predictions with deep learning or use a CNN as an image prior within a physics-based forward model.The cited applications include computed-tomography reconstruction and MRI.Source: §3.4.5 · p. 21

Requirements and benefits

• Method requirements vary: residual modeling needs only an operational mechanistic model, while physics-guided losses need domain expertise and architecture design needs both domain and ML expertise.Physics-guided initialization can use domain expertise to select synthetic data while leaving the ML process-agnostic.Source: §3.5 · p. 21

Applicability boundaries

• Hybrid methods cannot reduce computational complexity for PDE solving when the existing solver must still produce the output.Residual and hybrid methods also may not fit governing-equation discovery when no known physical model is available, or data generation whose purpose is to simulate a distribution.Source: §3.5 · p. 22

Combining methods

• Physics-guided losses, architectures, and initialization can be combined in a single ML model.The survey cites combinations of loss and initialization, loss and hybrid modeling, and initialization and architecture.Source: §3.5 · p. 22

Overall objectives

• Physics-ML methods target resource efficiency while maintaining predictive performance, sample efficiency, and interpretability relative to traditional ML approaches.The survey describes reduced computation for PDE solving and downscaling, and fewer observations and less training time for governing-equation discovery.Source: §3.5 · p. 22

Taxonomy and cross-fertilization

• The taxonomy is intended to organize existing work and identify knowledge gaps and cross-disciplinary method crossovers for future research.The table also includes papers not discussed in the survey because of space limitations.Source: §4 · p. 22

Cross-fertilization examples

• Examples of cross-fertilization include extending physics-guided loss methods from PDEs to inverse modeling and adapting a lake-temperature architecture based on prior loss-function and hybrid-model work.The survey also suggests physics-informed graph-network research on hidden-variable interpretability and dynamic graph structures.Source: §4 · p. 22

Taxonomy gaps

• The taxonomy exposes sparse or empty method-objective combinations, including opportunities to integrate physical principles into ML-based climate and weather parameterization and downscaling.Although super-resolution methods are being applied to climate downscaling, most cited approaches do not incorporate physics; the survey identifies scope for methods from other objective areas and for physics-guided initialization.Source: §4 · pp. 22–23

Additional research directions

• Potential directions beyond the taxonomy include forecasting with data assimilation and using scientific knowledge with ML to inform environmental or engineering decisions.Examples include predicting physical-model parameters from time series, estimating epidemiological model parameters, reservoir releases, and power-plant control.Source: §4 · p. 23

5 CONCLUDING REMARKS

• The application-centric categories help researchers relate physics–ML methods across disciplines, and the survey hopes this will accelerate cross-pollination.Table 3 classifies literature by objective and method.Source: §5 · p. 23, Table 3 · p. 24
• Physical constraints in ML loss functions can support fairness in predictive models and realism in GAN-generated data, while novel architectures can incorporate prior domain information and improve interpretability.Source: §5 · p. 23
• The survey focuses primarily on engineering and environmental systems traditionally modeled mechanistically, while noting related work in control and analysis of physics-based simulations.Source: §5 · p. 23
• The authors expect crossover among domain-knowledge methods used in geometric deep learning across chemistry, physics, hydrology, geostatistics, and neuroscience to expedite research.Source: §5 · p. 24