Source-linked AI summary
From calibration to parameter learning: Harnessing the scaling effects of big data in geoscientific modeling
Wen-Ping Tsai, Dapeng Feng, Ming Pan, Hylke Beck, Kathryn Lawson, Yuan Yang, Jiangtao Liu, Chaopeng Shen
TL;DR
Geoscientific models require calibration of spatially varying parameters, but traditional approaches are inefficient and produce non-unique solutions. The paper introduces differentiable parameter learning, which learns parameter mappings with deep learning integrated into process-based models. As training data increases, dPL shows improved performance, physical coherence, and generalizability, alongside orders-of-magnitude computational savings.
Problem
Traditional calibration of geoscientific model parameters is highly inefficient and produces non-unique solutions.
Method
dPL uses deep neural networks to transform inverse parameter calibration into a big-data learning problem integrated with process-based models.
Results
dPL exhibits improved performance, physical coherence, and generalizability with increasing training data, while achieving orders-of-magnitude computational savings.
Takeaways & Limitations
dPL provides a generic, automatically adaptive, highly efficient approach for integrating deep learning with process-based models.
Takeaways & Limitations
Process-based model structural deficiencies impose an upper bound on achievable performance, and interpreting deep-learning results requires contextual grounding.
Abstract
from arXiv · showhide
The behaviors and skills of models in many geosciences (e.g., hydrology and ecosystem sciences) strongly depend on spatially-varying parameters that need calibration. A well-calibrated model can reasonably propagate information from observations to unobserved variables via model physics, but traditional calibration is highly inefficient and results in non-unique solutions. Here we propose a novel differentiable parameter learning (dPL) framework that efficiently learns a global mapping between inputs (and optionally responses) and parameters. Crucially, dPL exhibits beneficial scaling curves not previously demonstrated to geoscientists: as training data increases, dPL achieves better performance, more physical coherence, and better generalizability (across space and uncalibrated variables), all with orders-of-magnitude lower computational cost. We demonstrate examples that learned from soil moisture and streamflow, where dPL drastically outperformed existing evolutionary and regionalization methods, or required only ~12.5% of the training data to achieve similar performance. The generic scheme promotes the integration of deep learning and process-based models, without mandating reimplementation.
Main Text Introduction
Geoscientific models depend on difficult-to-observe parameters whose calibration is costly, non-unique, and unable to exploit information shared across locations. The paper proposes dPL, which integrates deep learning with process-based models to learn parameters from large datasets and evaluates it across hydrologic case studies.
- Motivation: Geoscientific model behavior depends strongly on unobservable, underdetermined parameters, making modeled results less certain.Parameter uncertainties limit confidence in modeled outcomes, including regional impacts of increasing CO2 levels.
- Motivation: Thousands of model runs may be required to calibrate only a dozen parameters, making traditional calibration repetitive and inefficient.Previously calibrated parameters may need readjustment when resolution, other parameters, model version, or input data change.
- Motivation: Site-by-site calibration cannot exploit process commonalities across locations and can overfit sparse data, producing non-physical and non-unique parameter sets.Different parameter sets can yield similar evaluation metrics, while neighboring areas may receive disparate parameters.
- Prior approaches: Existing regionalization methods constrain parameters using transfer functions, but human-designed rigid forms limit efficacy, scalability, and input flexibility.These methods struggle with many transfer-function parameters and are restricted to simple input attributes.
- Prior approaches: Data-driven deep learning predicts observable variables with sufficient data but does not directly provide unobserved prognostic variables, which still rely on manually calibrated process-based models.Examples of unobserved variables include evapotranspiration, groundwater recharge, and root carbon storage.
- Contribution: The proposed dPL framework uses deep neural networks to transform inverse parameter calibration into a big-data learning problem integrated with process-based models.The framework is presented with two versions for different geoscientific use cases and leverages modern deep-learning infrastructure.
- Evaluation: The study evaluates dPL using soil moisture, streamflow, and spatial extrapolation, comparing it with evolutionary and parameter-regionalization methods.Experiments include VIC with SMAP soil moisture, CAMELS streamflow, and a global streamflow dataset for prediction in ungauged basins.
Results
Across soil-moisture and streamflow experiments, dPL matched or exceeded comparison methods while generalizing better and producing more spatially coherent parameters. Its advantages increased with training data and required substantially less computation or data in several comparisons.
- Soil-moisture calibration: dPL achieved equivalent or slightly better ending soil-moisture performance than SCE-UA.The comparison was made across the tested sampling densities and training durations.
- Optimization efficiency: 9.9 minutes versus 253 minutes was the time required at 1/82 sampling density for dPL and SCE-UA, respectively.At 1/42 density, the corresponding times were 11 and 1,004 minutes.
- Generalization and coherence: dPL generalized better in space, with almost no degradation from training locations to test periods and lower error on neighboring untrained gridcells than SCE-UA.The authors also report spatially coherent, better-constrained parameter sets as data increased.
- Uncalibrated-variable performance: dPL-produced ET was closer to MOD16A2 than ET from NLDAS-2 or SCE-UA, with NSE 0.55 for dPL (gz), 0.38 for SCE-UA, and 0.44 for NLDAS-2.The comparison used models trained at 1/82 sampling density and evaluated an uncalibrated variable.
Discussion
The discussion attributes dPL’s advantages to data scaling, yielding simultaneous gains in performance, efficiency, robustness, and physical coherence. It presents dPL as a generic, adaptive approach whose benefits extend across geoscientific models while cautioning that comparisons must account for training-data amount.
- Generality: dPL’s scaling advantages are expected to hold across other geoscientific domains and varied problem formulations.The authors connect this expected generality to shared process patterns and differences revealed by large data.
- Scaling effects: Data scaling produced simultaneous benefits in performance, efficiency, robustness, and physical coherence over traditional methods.The paper describes these gains as previously undemonstrated in geosciences for physical coherence.
- Efficiency: Orders-of-magnitude computational savings over SCE-UA occurred even at moderate sampling density (1/82).
- Caveat: Interpretations of deep-learning results should be grounded in training-data amount because small-data comparisons may have limited validity.
- Applications: dPL is especially helpful when full simulation histories involving unobserved variables are needed for interpretable narratives or diagnostics.The discussion gives climate-change assessment as an example application.
- Generality: The same gA and gz network components handled three examples with different configurations and little customization.The framework avoids explicitly specified transfer functions, allowing the deep-learning algorithm to adapt automatically.
Methods
The methods use hydrologic process models together with LSTM surrogates and differentiable parameter learning. VIC simulates multiple hydrologic variables from meteorological and non-meteorological inputs, while the LSTM equations update states through gated recurrent computations.
- Process-based model: VIC simulates evapotranspiration, runoff, soil moisture, and baseflow using conceptualized bucket formulations.
- Process-based model: VIC inputs include daily meteorological forcings, non-meteorological data, and parameters to be determined.Meteorological forcings include precipitation, air temperature, wind speed, pressure, vapor pressure, and longwave and shortwave radiation.
- Surrogate modeling: An LSTM was trained to reproduce VIC behavior as closely as possible.
- Computational workflow: Using a surrogate avoids repeated CPU-based VIC simulations; without it, SCE-UA would require O(10^2) more expensive evaluations.
- Surrogate modeling: The surrogate model achieved median correlations of 0.91 for soil moisture and 0.92 for ET against VIC simulations.
- LSTM architecture: LSTM recurrence combines input transformations, gated updates, cell and hidden states, and an output layer to process sequential inputs.The cell state supports long-term memory, while gates determine information retention and forgetting across time steps.
The network
The network uses gradient descent for training, with hyperparameters adjusted to accommodate larger training datasets. Its parameterization includes infiltration and baseflow-related quantities that shape VIC model behavior.
- The network: Gradient descent trains the whole network as a first-order optimization scheme.
- The network: dPL hyperparameters were manually tuned using hidden sizes, batch size, test-period RMSE, and train-test RMSE differences.
- The network: Higher sampling densities produce larger training datasets that can be handled with larger hidden sizes.
- The network: VIC calibration includes INFILT, Dsmax, Ds, and the soil-moisture threshold for nonlinear baseflow.
- The network: Larger INFILT reduces infiltration rate and increases runoff when other parameters and saturated fraction remain unchanged.
Satellite-based estimates of ET
MOD16A2 provides an independent satellite-based estimate of evapotranspiration for validating calibration. Agreement with this product is interpreted as evidence of better overall physics modeling, although it is not used as a learning target.
- Satellite-based estimates of ET: MOD16A2 is an 8-day composite evapotranspiration product at 500-meter resolution.
- Satellite-based estimates of ET: MODIS radiation, albedo, meteorological, leaf-area-index, land-cover, and biome information supports the Penman-Monteith evapotranspiration estimate.
- Satellite-based estimates of ET: MOD16A2 was used for validation rather than as a learning target.
- Satellite-based estimates of ET: Better agreement with independent MODIS observations should indicate better overall physics modeling, despite imperfections in MOD16A2.
Shuffled Complex Evolution (SCE-UA) for comparison
SCE-UA serves as a reference method for comparing dPL's parameter-estimation module. Its population-based search evolves multiple complexes through competitive operations and was evaluated using an epoch definition based on average forward simulations.
- Shuffled Complex Evolution (SCE-UA) for comparison: SCE-UA was selected because it is a well-established and widely applied reference method.
- Shuffled Complex Evolution (SCE-UA) for comparison: SCE-UA ranks parameter populations by objective function and partitions them into multiple subpopulations called complexes.
- Shuffled Complex Evolution (SCE-UA) for comparison: Each complex undergoes reflection, contraction, and random trials before complexes are shuffled and redivided.
- Shuffled Complex Evolution (SCE-UA) for comparison: For comparison with dPL, an SCE-UA epoch was defined as approximately one forward simulation on average.
- Shuffled Complex Evolution (SCE-UA) for comparison: Seven complexes were used, while uneven simulations across discrete iterations made fixed-epoch objective comparisons inappropriate.
CAMELS streamflow test
The CAMELS streamflow test compares dPL with prior VIC calibration and regionalization work using the same dataset and basins where possible. The cited prior VIC result provides a median NSE benchmark of 0.30.
- CAMELS streamflow test: The regionalization study used 531 CAMELS basins across the conterminous United States.
- CAMELS streamflow test: That study restricted basins to fewer than 2,000 km2 and trained and tested on the same basins, assessing temporal generalization.
- CAMELS streamflow test: Its MPR calibration and validation periods were 1 October 1999–30 September 2008 and 1 October 1989–30 September 1999, respectively.
- CAMELS streamflow test: The comparison with Mizukami et al. used the same dataset, basins, and training and test periods.
- CAMELS streamflow test: Mizukami et al. reported a median NSE of 0.30 for VIC across their training and test periods.
Global PUB test
The global PUB test compares dPL with Beck20’s regionalization scheme using headwater-basin streamflow data and spatial cross-validation. The comparison preserves Beck20’s setup while also testing dPL with an expanded attribute set.
- Global PUB test: Beck20 used 4,229 headwater basins and grouped them into tropical, arid-and-temperate, and cold-polar climate categories.
- Global PUB test: The regionalization benchmark used eight raw attributes to learn linear transfer functions for 12 free HBV parameters.The attributes included humidity, precipitation, potential evaporation, NDVI, open water, slope, sand, and clay.
- Global PUB test: The dPL comparison matched Beck20’s 2000–2016 training and test periods and used the same 10-fold cross-validation procedure.
- Global PUB test: Beck20 reported a median KGE coefficient of 0.48 for its temperate catchment group, which served as the comparison target.
- Global PUB test: dPL was evaluated with both eight comparable raw attributes and the complete set of 27 attributes, demonstrating extensibility beyond the benchmark configuration.
Evaluation metrics
The evaluation uses several complementary metrics for model simulations, reported over the test period. These metrics distinguish systematic bias, random error, correlation, and overall efficiency.
- Evaluation metrics: The reported metric set includes bias, correlation, ubRMSE, and NSE, with KGE additionally calculated for comparability with Beck20.
- Evaluation metrics: Bias measures the mean difference between modeled and observed results, while ubRMSE measures random error after systematic bias is removed.
- Evaluation metrics: Correlation evaluates whether simulations capture the observations’ seasonality, whereas NSE incorporates bias and can be negative for poor models.NSE equals 1 for a perfect model.
- Evaluation metrics: All evaluation metrics were reported for the test period, including when gz was evaluated at training locations.
- Evaluation metrics: KGE combines correlation with bias and variability ratios, where the ratios compare simulated and observed runoff means and standard deviations.The indices s and o denote simulations and observations.
Supplementary Information
The supplementary information documents sampling, surrogate-model performance, parameter maps, basin-outlet evaluations, and ensemble evaluation metrics. Its figures cover both spatial sampling and comparisons with process-based model outputs.
- Supplementary Information: Figure S1 illustrates the sampling strategy and training and test sites at density s8.
- Supplementary Information: Figure S2 evaluates the LSTM surrogate for soil moisture and evapotranspiration using test metrics, spatial correlation maps, and gridcell time-series comparisons.The comparisons include LSTM-sur against VIC outputs and soil moisture against VI.
- Supplementary Information: Figure S3 provides maps of parameters other than INFILT.
- Supplementary Information: Figure S4 shows basin-outlet locations for CAMELS and the global headwater catchment database, colored by model-efficiency metrics.The panels use NSE for VIC parameters estimated by gz and KGE for HBV parameters estimated by gA.
- Supplementary Information: Table S1 reports means and standard deviations of CONUS-scale ensemble evaluation metrics for default NLDAS-2, SCEUA-derived, and dPL-derived parameters at density 1/8^2.NLDAS-2 has no ensemble standard deviation.