Source-linked AI summary
Physics Guided RNNs for Modeling Dynamical Systems: A Case Study in Simulating Lake Temperature Profiles
Xiaowei Jia, Jared Willard, Anuj Karpatne, Jordan Read, Jacob Zwart, Michael Steinbach, Vipin Kumar
TL;DR
Physics-based models and black-box ML each have important limitations for modeling physical systems, especially when observations are scarce and physical consistency matters. The paper develops PGRNNs that combine recurrent learning with physical constraints and physics-model pre-training, and reports improved accuracy, physical consistency, generalization, and performance with fewer observations in lake-temperature modeling.
Problem
Physics-based models require difficult calibration and scarce observations, while black-box ML can require much data, violate physical laws, and generalize poorly to unseen conditions.
Method
PGRNN integrates standard recurrent processing with energy conservation, optional density-depth constraints, and pre-training on simulated physics-model data.
Results
PGRNN reduced or eliminated energy-conservation and depth-density inconsistencies while improving model accuracy, and increasingly outperformed RNNs as fewer observations were used for training.
Takeaways & Limitations
The framework can incorporate additional physical laws and task-specific constraints and may extend beyond lake temperatures to scientific and engineering problems using mechanistic models.
Takeaways & Limitations
Energy conservation is modeled only during ice-free periods because ice and snow dynamics are outside the study’s scope.
Abstract
from arXiv · showhide
This paper proposes a physics-guided recurrent neural network model (PGRNN) that combines RNNs and physics-based models to leverage their complementary strengths and improve the modeling of physical processes. Specifically, we show that a PGRNN can improve prediction accuracy over that of physical models, while generating outputs consistent with physical laws, and achieving good generalizability. Standard RNNs, even when producing superior prediction accuracy, often produce physically inconsistent results and lack generalizability. We further enhance this approach by using a pre-training method that leverages the simulated data from a physics-based model to address the scarcity of observed data. The PGRNN has the flexibility to incorporate additional physical constraints and we incorporate a density-depth relationship. Both enhancements further improve PGRNN performance. Although we present and evaluate this methodology in the context of modeling the dynamics of temperature in lakes, it is applicable more widely to a range of scientific and engineering disciplines where mechanistic (also known as process-based) models are used, e.g., power engineering, climate science, materials science, computational chemistry, and biomedicine.
1 Introduction
Physics-guided recurrent neural networks combine data-driven temporal modeling with physical constraints to address the limited data, physical inconsistency, and poor generalization of black-box models. The approach uses energy conservation, optional density-depth constraints, and physics-model pre-training, with improvements demonstrated for Lake Mendota temperature modeling.
- Scope: The methodology is presented for lake temperature dynamics but is applicable to scientific and engineering domains using mechanistic models, including climate science, materials science, and biomedicine.The paper identifies additional applications such as power engineering and computational chemistry.
- Motivation: Physics-based models can be limited by simplified process representations, difficult parameter selection, scarce observations, and computationally expensive calibration prone to over-fitting.These limitations motivate combining mechanistic knowledge with data-driven methods.
- Motivation: Black-box ML models may require substantial training data, violate physical laws, and fail to generalize beyond forcing-variable combinations seen during training.The paper gives future warmer climate scenarios as an example of an unseen setting.
- Method: PGRNN combines a standard RNN flow for observed temporal dependencies with an energy flow that regularizes temporal progression toward physical consistency.The two parallel recurrent structures are designed to leverage complementary strengths of data fitting and physical modeling.
- Method: The proposed pre-training method uses simulated physics-model data to initialize learning when observed data are scarce, despite simulation inaccuracies.The paper reports that this improves performance with fewer observations used for training.
- Results: In Lake Mendota, energy conservation improved learning performance and generalization, while adding density-depth consistency produced highly accurate and physically meaningful predictions.The evaluation uses a real-world lake system with extensive observed data.
2 Problem Formulation
The problem formulation represents lake temperature as a depth- and time-indexed prediction driven by physical variables governing lake thermodynamics. Inputs include meteorological surface measurements, and the target is water temperature at each depth and time step.
- Problem Formulation: The task is to simulate lake-water temperature at every depth d and date or time step t.The formulation explicitly treats both depth and time as dimensions of the prediction problem.
- Problem Formulation: The input drivers X = {x_d,t} are physical variables governing lake-temperature dynamics at each depth and time step.These drivers are selected as primary determinants of lake thermodynamics.
- Problem Formulation: The target Y = {y_d,t} is water temperature, where y_d,t denotes temperature at depth d and time step t.Examples of inputs include solar radiation, wind speed, and air temperature at the water surface.
3 Preliminaries
The preliminaries describe GLM as a physics-based lake-temperature model and introduce LSTM components used for temporal regression. They also present the PGRNN flow, which integrates energy conservation into recurrent modeling.
- General Lake Model: GLM simulates lake-temperature processes including surface heating, radiation attenuation, layer mixing, and heat loss.Heat loss occurs through evaporation or long-wave radiation.
- General Lake Model: GLM parameters are calibrated by searching parameter combinations that minimize model error on available training data.The process can be labor- and computationally-intensive and may perform poorly on test data without expert knowledge.
- Long-Short Term Memory Networks: LSTM cells combine current inputs with information from previous time steps through a memory cell and gating mechanisms.The forget and input gates filter inherited information and candidate cell-state information, respectively.
- Long-Short Term Memory Networks: The LSTM predicts continuous temperature values at each time step through a linear combination of hidden units.The model is applied separately for each depth.
- Physics-Guided Recurrent Neural Network: PGRNN integrates energy conservation into the recurrent process so environmental changes and predicted temperatures conform to energy conservation.The recurrent component captures temporal dependencies while the energy-flow model constrains the physical evolution.
4 Method
PGRNN integrates recurrent prediction with an energy-conservation flow, physics-based pre-training, and a density-depth constraint. Its energy calculations use modeled temperatures and input drivers, while the conservation loss is applied only during ice-free periods and tolerates small discrepancies.
- Energy conservation: PGRNN uses parallel standard RNN and energy flows to fit observed temporal variation while regularizing predictions toward physical consistency.The energy flow explicitly represents heat-energy transfer across time.
- Energy conservation: The modeled energy balance includes incoming terrestrial long-wave and short-wave radiation and outgoing back-radiation, sensible, and latent evaporative heat fluxes.Smaller sediment and advected-energy terms are ignored.
- Energy conservation: The energy-conservation loss combines with the standard recurrent objective for ice-free periods, excluding winter ice and snow dynamics from this study.Lake Mendota’s ice-covered periods are not included in this constraint.
- Energy conservation: The loss penalizes energy-balance discrepancies only beyond a threshold, with λEC controlling the trade-off between recurrent and conservation losses.The threshold is set from the largest daily-average GLM discrepancy, accounting for omitted factors and observation errors.
- Energy calculation: Heat fluxes are estimated from input drivers and modeled surface temperature, while total lake thermal energy uses predicted temperatures across depth layers and cross-sectional areas.Temperature is simulated every 0.5 m, and the energy-flow calculation requires no true labels or observations.
- Pre-training: PGRNN is pre-trained on GLM-simulated temperatures to address scarce observations and initialize a physically consistent model.The simulated temperatures cover every depth and day for given input drivers.
- Density-depth constraint: A density-depth constraint transforms predicted temperature into density and penalizes consecutive depth pairs whose density decreases with depth.ReLU counts only inverse-density violations toward the penalty.
5 Experiment
Experiments on Lake Mendota compare RNN, PGRNN, GLM, and pretrained variants under varying observed-data availability. PGRNN improves accuracy and physical consistency, while pretraining strengthens performance, convergence, and generalization.
- Experimental setup: Lake Mendota observations span multiple sources, with meteorological drivers available daily and 8,037 observations used for training.The training periods cover April 02, 1980–October 31, 1991 and June 01, 2003–December 30, 2014.
- Prediction and generalization: PGRNN consistently outperforms standard RNN, with the advantage especially apparent when only small subsets of observed data are used.The reported explanation is that energy conservation regularizes the model toward physically consistent behavior.
- Prediction and generalization: PGRNN trained with 2% observed data outperforms the fully calibrated GLM trained with 100% of the data.PGRNN trained with all data and pretrained variants achieve still lower RMSE than GLM.
- Pretraining: Pretraining with GLM simulations improves prediction performance and reduces convergence from roughly 150–200 epochs to 30–50 epochs.The improvement is especially evident when few observed data are available.
- Generalization: Florida-pretrained models perform poorly before refinement on Lake Mendota, but fine-tuning with Mendota observations substantially improves them, especially for PGRNN.Direct application gives RMSE=9.010 for RNN˜p and 8.657 for PGRNN˜p.
- Physical consistency: PGRNN better aligns heat-flux imbalance with lake-energy change than RNN, although GLM produces even closer alignment.The remaining PGRNN mismatch is attributed to omitted minor flux terms and soft regularization.
- Physical consistency: Across the full test period, PGRNN significantly reduces both prediction RMSE and energy inconsistency relative to RNN and GLM.Energy inconsistency is defined as the average gap between the two curves during ice-free periods.
- Density-depth constraint: Adding the density-depth constraint reduces density inconsistency to 0.0021 and yields higher accuracy with physically meaningful density predictions.Without the constraint, PGRNN improves accuracy over RNN but retains high density inconsistency.
6 Related Work
Related work incorporates physical knowledge through features, output constraints, or residual modeling, but the paper positions PGRNN as combining complex physical constraints with physics-based pretraining.
- Existing approaches: Feature selection and construction are common ways to incorporate physical knowledge, but they cannot train models without labels.
- Existing approaches: Output constraints can support training with unlabeled data by relying on physical principles.
- Existing approaches: Residual modeling predicts physics-based-model errors but does not provide the same capability to train ML models with unlabeled data using physical principles.
- PGRNN distinction: PGRNN is presented as a framework for spatial and temporal processes that can incorporate complex laws such as energy and mass conservation and can be pretrained with physics-based models.
8 Conclusion
The conclusion presents PGRNN as a recurrent model that integrates physical constraints to improve lake-temperature modeling. It reports improved accuracy and physical consistency, particularly with scarce observations, and identifies broader application scope.
- Conclusion: PGRNN integrates energy conservation and a density-depth constraint into recurrent neural networks for dynamical-system monitoring.
- Conclusion: PGRNN reduces or eliminates energy-conservation and depth-density inconsistencies while improving model accuracy over the evaluated alternatives.
- Conclusion: With fewer observations, PGRNN increasingly outperforms RNN after pretraining on simulated data from a poorly parameterized physics-based model.
- Broader scope: The approach can be adjusted for other physical laws, including mass conservation, and applied to additional scientific and engineering problems.Examples include nutrient exchange, crop production, and vehicle refueling design.