Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling

Arka Daw; Anuj Karpatne; William Watkins; Jordan Read; Vipin Kumar

Source-linked AI summary

Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling

Arka Daw, Anuj Karpatne, William Watkins, Jordan Read, Vipin Kumar

arXiv:1710.11431v3Oct 31, 2017cs.LGcs.AIcs.CVphysics.data-anstat.ML

TL;DR

The paper addresses the difficulty of using data-driven models when labeled scientific data are limited and physical consistency matters. It introduces PGNN, which combines physics-based simulations with neural networks and trains them using physics-based loss functions. In lake temperature modeling, the framework is reported to improve generalizability and physical meaningfulness compared with black-box data science methods, while its demonstrated hybrid construction remains simple and lake-specific calibration can be labor-intensive.

Problem
Black-box models depend on limited labeled data and may produce predictions inconsistent with known physics, while physics-based models rely mainly on available physics and approximations.
Method
PGNN combines physics-based model simulations with neural-network inputs and uses physics-based loss functions to guide learning toward physically consistent solutions.
Results
PGNN shows better generalizability and physically meaningful results than black-box data science methods in the lake temperature modeling study.
Takeaways & Limitations
Physics-based learning objectives can extend neural networks beyond task performance by incorporating physical relationships among model outputs and other variables.
Takeaways & Limitations
The study uses a simple hybrid design, and calibrating the baseline lake model for each lake is labor- and computation-intensive.

Abstract

from arXiv · show

This paper introduces a framework for combining scientific knowledge of physics-based models with neural networks to advance scientific discovery. This framework, termed physics-guided neural networks (PGNN), leverages the output of physics-based model simulations along with observational features in a hybrid modeling setup to generate predictions using a neural network architecture. Further, this framework uses physics-based loss functions in the learning objective of neural networks to ensure that the model predictions not only show lower errors on the training set but are also scientifically consistent with the known physics on the unlabeled set. We illustrate the effectiveness of PGNN for the problem of lake temperature modeling, where physical relationships between the temperature, density, and depth of water are used to design a physics-based loss function. By using scientific knowledge to guide the construction and learning of neural networks, we are able to show that the proposed framework ensures better generalizability as well as scientific consistency of results. All the code and datasets used in this study have been made available on this link \url{https://github.com/arkadaw9/PGNN}.

1 Introduction

The paper frames scientific knowledge discovery as a spectrum between data-driven and physics-based approaches, then introduces PGNN to combine their complementary strengths. PGNN uses physics-based simulations and equations to improve generalizability and physical consistency in lake temperature modeling.

Motivation: Black-box data science models depend on limited labeled data and may learn relationships inconsistent with known physics.Such models can appear accurate on small training and test sets yet generalize poorly outside the available labeled data.
Motivation: Physics-based models provide explainable relationships grounded in scientific principles but rely mainly on available physics and may use approximations for poorly understood processes.The paper identifies numerical simulations and physics-based rules or equations as two common forms of physical knowledge.
Physics-guided neural networks: PGNN combines neural networks with physics-based models through hybrid architectures and physics-based loss functions that guide predictions toward physically consistent solutions.The framework is demonstrated on lake temperature prediction using the relationship among temperature, density, and depth.
Physics-guided neural networks: The framework targets lake temperature at varying depths and times using input drivers, physics-based simulations, and a loss function based on temperature, density, and depth.This illustrative problem shows how scientific relationships can be incorporated into neural-network learning.

2 Physics-guided Neural Networks

PGNN combines physics-based simulations with neural networks and adds physics-based loss functions to training. This design addresses complementary deficiencies of physics and data models while allowing physical consistency to be evaluated on unlabeled instances.

Framework: The generic PGNN framework has two steps: constructing hybrid-physics-data models and using scientific knowledge as physics-based loss functions.These steps combine model outputs with data-driven learning and constrain predictions using physical relationships.
Constructing hybrid-physics-data models: HPD modeling combines physics-based and neural-network models to overcome their complementary deficiencies and leverage information from both physics and data.Physics-based outputs may be incomplete because of simplified or missing physics, while observations can be limited and expensive.
Constructing hybrid-physics-data models: Hybrid-physics-data models augment input drivers with physics-based model outputs so a neural network can produce the final target predictions.The augmented model can learn corrections when physics-based simulations contain systematic biases.
Using physics-based loss functions: Physics-based loss functions measure violations of known relationships among predictions and physical variables, including algebraic or differential equations.The paper requires these equations to be continuous and differentiable so they can be used in optimization.
Using physics-based loss functions: Unlike observation-based losses, the physics-based loss can be evaluated on unlabeled instances because it does not require the target observations.The complete objective balances empirical error, physical inconsistency, and model complexity through a physics-weighting hyperparameter.
Using physics-based loss functions: Neural networks are suited to this framework because automatic differentiation can compute gradients of the physics-based loss with respect to model parameters.The paper notes that stochastic gradient descent and its variants can minimize the resulting objective.

3 PGNN for Lake Temperature Modeling

Lake temperature modeling combines GLM simulations and meteorological observations in a neural network, while physical relationships among temperature, density, and depth guide learning toward consistent predictions.

3.1 Background: Lake Temperature Modeling: Accurate lake-temperature modeling is needed because observations are incomplete at broad spatial scales, while temperature affects aquatic ecosystems and thermal habitat.The 1D task predicts water temperature at a specified depth and time; GLM models relevant heating, mixing, radiation, and evaporation processes.
3.1 Background: Lake Temperature Modeling: GLM requires lake-specific calibration of vertical mixing, wind inputs, and water clarity, creating a trade-off between accuracy and scalability.Calibration searches parameter combinations for maximum agreement with observations and is labor- and computation-intensive.
3.2 Proposed PGNN Formulation: PGNN uses GLM temperature simulations together with physical input drivers as the neural network input X = [D, Y_PHY].Drivers include solar radiation, wind speed, air temperature, depth, and day of year.
3.2 Proposed PGNN Formulation: The model uses a multilayer perceptron with mean squared error and L1/L2 weight regularization to predict temperature from the combined inputs.The architecture maps input features through hidden layers to an output temperature prediction.
3.2.1 Temperature–Density Relationship:: PGNN adds a physics-based loss that penalizes violations of the known temperature–density relationship and the monotonic increase of density with depth.The density–temperature relationship is nonlinear, with water maximally dense at 4°C; violations are measured across consecutive depth pairs and time steps.
3.2.2 Density–Depth Relationship:: Combining physics-based, empirical, and regularization losses makes temperature predictions consistent with density and depth relationships as well as temperature observations.The resulting objective is intended to provide a more complete solution to the physical problem.

4 Evaluation

The evaluation compares PGNN with physics-based, black-box, and hybrid baselines across two lakes, varying data availability and the physics-loss weight. PGNN combines low test RMSE with substantially better physical consistency, including under smaller training sets.

Data and experimental design: The study evaluates lake-temperature models on Mille Lacs Lake and Lake Mendota using observational drivers, GLM simulations, and large unlabeled instance sets.The datasets contain 7,072 and 13,543 labeled observations, respectively, with GLM outputs used as physics-based inputs for unlabeled instances.
Baselines: PGNN is compared with PHY, black-box SVM, LSBoost, and NN models, plus PGNN0, which uses physics-based inputs without the physics-based loss.The comparison isolates the contributions of hybrid inputs and physics-guided training.
Overall results: 0.73 RMSE and physical inconsistency close to 0 are achieved by PGNN on Mille Lacs Lake, compared with 1.69 RMSE for PHY.NN reaches 1.18 RMSE but has inconsistent density-depth relationships at almost 73% of time-steps; PGNN0 remains inconsistent for roughly 72%.
Overall results: 1.79 RMSE is achieved by PGNN on Lake Mendota, versus 2.07 for NN and 1.93 for PGNN0, while PGNN remains physically consistent.The physics-based model has a test RMSE of 2.77 on this more stratified lake.
Effect of varying training size: At training sizes of 1250 and 1500, PGNN provides smaller RMSE values than all baseline methods and increases more gradually as training data decrease.At size 800, PGNN and PGNN0 become similar because little information remains in the data for RMSE improvement.
Physical consistency and sensitivity: PGNN remains physically consistent across all time-steps, whereas PGNN0 and NN violate the density-depth relationship more than 50% of time-steps on average.Increasing λPHY imposes a more stringent physics constraint and produces more physically consistent predictions without significantly affecting test RMSE.
Density-profile analysis: PGNN produces density profiles closer to observations while preserving the monotonic density-depth relationship, unlike PGNN0’s violations beyond observed depths.On Lake Mendota, PGNN again produces density estimates closest to observations, while PGNN0 and NN show larger discrepancies.

5 Discussion on Alternate HPD Model Designs

The paper compares three hybrid-physics-data designs: basic HPD, residual modeling, and HPD-Res, which combines residual prediction with physics-model outputs as inputs. Without physics-based loss, experiments on two lakes and varying training sizes isolate how design choice affects generalization.

Residual Model: Residual Model predicts corrections to physics-based outputs instead of directly predicting the target variable.It models Y_Res = Y − Y_PHY and adds the predicted residual to Y_PHY.
HPD-Res Model: HPD-Res predicts physics-model residuals using both input drivers and physics-model outputs.The residual predictions are added to Y_PHY to obtain final target predictions.
HPD-Res Model: HPD-Res combines residual modeling with the basic HPD architecture.Unlike the basic residual model, it uses Y_PHY as an additional input; unlike basic HPD, it predicts residuals rather than the target directly.
Experimental comparison: The three designs were compared on Lake Mendota and Mille Lacs Lake across varying training sizes.Physics-based loss was excluded so the experiments evaluated HPD design effects on generalization, with basic HPD corresponding to the PGNN0 baseline.

6 Conclusions and Potential Future Work

The paper concludes that PGNN combines physics-model simulations and physics-based loss functions to guide neural networks toward generalizable and physically meaningful predictions. It positions this approach as a basis for broader physics-informed learning, while identifying several directions for extending the lake-temperature formulation.

Conclusions: PGNN combines physics-model simulation outputs with physics-based loss functions to guide neural networks toward physically consistent solutions.The framework uses both hybrid model inputs and scientific knowledge in the learning objective.
Conclusions: The proposed framework shows better generalizability and more physically meaningful results than black-box data science methods.This conclusion is stated for the paper’s lake-temperature modeling application.
Potential future work: PGNN is presented as a stepping stone toward physics-based learning objectives in other scientific and engineering disciplines.The paper suggests exploiting known physical relationships as physics-based loss functions beyond lake temperature modeling.
Potential future work: Future work includes exploiting spatial and temporal dependencies in lake-temperature test instances with recurrent neural network architectures.The paper also identifies tighter physics–data coupling, theoretical analyses, interpretability, and additional physical relationships as research directions.