Source-linked AI summary
Rainfall-Runoff Prediction at Multiple Timescales with a Single Long Short-Term Memory Network
Martin Gauch, Frederik Kratzert, Daniel Klotz, Grey Nearing, Jimmy Lin, Sepp Hochreiter
TL;DR
Rainfall–runoff forecasting must serve both coarse and sub-daily timescales, but separate models and long high-resolution sequences are computationally costly. The paper proposes multi-timescale LSTMs that share processing for older inputs and branch into target-specific recent inputs. These models maintain accurate and consistent predictions while requiring less computational overhead than individual per-timescale models, with scope bounded by fixed target timescales and reanalysis inputs.
Problem
Operational forecasting needs multiple target timescales, but fixed-timescale alternatives can require separate models and daily resolution can be too coarse for short-range hydrologic information.
Method
Two multi-timescale LSTM architectures process long-past inputs coarsely and recent inputs at each target resolution, with regularization to reduce cross-timescale inconsistencies.
Results
The results show that LSTM advantages over process-based models on daily predictions extend to sub-daily predictions, while multi-timescale models have much lower overhead than individual models per timescale.
Takeaways & Limitations
One model can provide accurate and consistent predictions at multiple timescales with lower training and inference overhead than separate per-timescale models.
Takeaways & Limitations
The models operate in a reanalysis setting rather than performing actual forecasting, and the approach targets a fixed set of timescales rather than arbitrary granularity.
Abstract
from arXiv · showhide
Long Short-Term Memory Networks (LSTMs) have been applied to daily discharge prediction with remarkable success. Many practical scenarios, however, require predictions at more granular timescales. For instance, accurate prediction of short but extreme flood peaks can make a life-saving difference, yet such peaks may escape the coarse temporal resolution of daily predictions. Naively training an LSTM on hourly data, however, entails very long input sequences that make learning hard and computationally expensive. In this study, we propose two Multi-Timescale LSTM (MTS-LSTM) architectures that jointly predict multiple timescales within one model, as they process long-past inputs at a single temporal resolution and branch out into each individual timescale for more recent input steps. We test these models on 516 basins across the continental United States and benchmark against the US National Water Model. Compared to naive prediction with a distinct LSTM per timescale, the multi-timescale architectures are computationally more efficient with no loss in accuracy. Beyond prediction quality, the multi-timescale LSTM can process different input variables at different timescales, which is especially relevant to operational applications where the lead time of meteorological forcings depends on their temporal resolution.
1 Introduction
Rainfall–runoff forecasting needs multiple temporal resolutions because daily inputs can obscure short-lived discharge extremes, while separate timescale models increase computational demand and may disagree. The paper introduces LSTM architectures that jointly predict multiple timescales using coarser processing for older inputs and finer processing for recent inputs.
- Motivation: Daily forecasts can be too coarse for actionable short-range information because they mute diurnal discharge variations such as evapotranspiration and snowmelt.Short but extreme flood peaks may therefore be missed when discharge is represented only daily.
- Problem: Using independent models for different timescales multiplies computational demand and can produce inconsistent predictions where forecast ranges overlap.The National Water Model is described as using separate hourly and longer-range forecast setups.
- Related work: Prior hierarchical recurrent architectures reduce long-sequence burdens through multiple update frequencies, but high-frequency units may still process the full time series, slowing training.Other approaches also target irregular inputs or non-physical sequence tasks, creating mismatches with regular rainfall–runoff data.
- Novelty: Unlike prior single-output applications, multi-timescale rainfall–runoff prediction produces one discharge output for each target timescale, with outputs linked by conservative temporal aggregation.For example, 24 hourly predictions should average or sum to one daily prediction, depending on the units.
- Approach: The proposed architectures process long-past inputs at coarser resolution and reserve high-resolution inputs for recent steps, shortening sequences while supporting daily and hourly predictions.The models are benchmarked against separate per-timescale LSTMs and the US National Water Model.
- Results: All LSTM solutions achieve significantly higher Nash–Sutcliffe efficiency than NWM at all timescales, while the naive model has much higher computational overhead.A regularization scheme reduces cross-timescale inconsistencies and slightly improves predictions overall; the model can also ingest forcings separately by target timescale.
2 Data and Methods
The study uses CAMELS basin data, NLDAS-2 forcings, LSTM baselines, and two multi-timescale LSTM architectures for simultaneous daily and hourly prediction. The architectures extend long historical context while avoiding extremely long hourly sequences and support flexible inputs across timescales.
- Data: The experiments use 516 CAMELS basins with hourly USGS streamflow data averaged to hourly and daily time steps.Models were trained from 1990–2003, validated from 2003–2008, and tested from 2008–2018.
- Data: Hourly forcings come from the NLDAS-2 product, spatially averaged for each basin alongside CAMELS static catchment attributes.Each model uses eleven forcing variables concatenated with 27 static attributes at every time step.
- Baselines: The baselines include a daily LSTM, a naively adapted hourly LSTM, and the operational US National Water Model.The naive hourly LSTM ingests 4320 hours, while the National Water Model is process-based and based on WRF-Hydro.
- Motivation: Hourly LSTM modeling is computationally difficult because half a year exceeds 4300 time steps, and extremely slow training limits hyperparameter search.Long sequences also make long-term dependencies harder to learn because the forget gate can reintroduce vanishing gradients.
- Multi-timescale architectures: The shared multi-timescale LSTM generates daily and hourly predictions through two forward passes using a large daily look-back and a recent hourly sequence.The hourly branch begins from a transferred state corresponding to the shared temporal history and predicts 24 hourly values aligned with the daily prediction.
- Multi-timescale architectures: The MTS-LSTM generalizes to more timescales and permits different input variables at different timescales, while multi-timescale predictions remain accurate across target timescales.The architecture can use hourly forcings in the hourly branch and supports operational settings with timescale-dependent inputs.
- Design considerations: State transfer at branch points requires careful timescale selection because choosing a discrete step can leave data unprocessed or process one hour twice.This issue arises when the timescales do not align cleanly at the branching point.
2.4 Evaluation Criteria
Evaluation combines standard metrics, hydrologic signatures, cross-timescale consistency, and ensemble averaging across independently initialized LSTMs. The evaluation covers predicted-versus-observed behavior and agreement between daily predictions and aggregated hourly predictions.
- Metrics and signatures: The models are benchmarked using metrics and hydrologic signatures established in prior CAMELS studies.Hydrologic signatures summarize hydrographs rather than directly comparing predicted and observed time series.
- Metrics and signatures: Pearson correlation compares observed and predicted hydrologic signatures across the 516 CAMELS basins.This evaluates agreement between signature statistics across basins.
- Cross-timescale consistency: Cross-timescale consistency is measured by the root mean squared deviation between daily predictions and hourly predictions aggregated to daily values.The measure directly quantifies disagreement between the two prediction resolutions.
- Evaluation procedure: Reported LSTM hydrographs average predictions from ten independently trained models to account for random weight initialization.This averaging procedure reduces dependence on a single initialization.
3 Results
Across 516 basins, the multi-timescale LSTM models generally outperform the National Water Model on NSE and peak timing while requiring substantially less computation than naive hourly LSTMs. sMTS-LSTM achieves the strongest overall results, and regularization improves cross-timescale consistency and daily NSE.
- All LSTM models outperform NWM on NSE at both daily and hourly timescales.
- The sMTS-LSTM achieves the best median NSE for both daily and hourly predictions among the evaluated LSTM models.
- sMTS-LSTM reduces median peak-timing error versus NWM from over six to around three and a half hours hourly, and from 0.5 to 0.3 days daily.
- NWM has better flow-bias metrics and the highest correlations for several high-flow, low-flow, zero-flow, and flow-duration signatures.
- Cross-timescale regularization further reduces inconsistencies between daily and day-aggregated hourly predictions compared with unregularized and naive approaches.
- Regularization increases median daily NSE from 0.755 to 0.762, while the sMTS-LSTM and MTS-LSTM remain far faster than naive hourly training and inference.
- For 10 years of hourly predictions across 516 basins, naive inference takes around 9 hours versus around 13 minutes for both multi-timescale models.
- Using different inputs by timescale improves performance: daily NSE rises by 0.045 from 0.766 to 0.811, while hourly NSE rises by 0.036 from 0.776.
4 Discussion and Conclusion
The study extends LSTM rainfall–runoff modeling to multiple timescales with architectures that improve computational efficiency while preserving accurate predictions. MTS-LSTM also supports timescale-specific forcings, but operational deployment still requires further development.
- Contribution: MTS-LSTM and sMTS-LSTM generalize LSTM rainfall–runoff modeling to multiple timescales.Both architectures process long-term information before branching into more granular timescale predictions.
- Results: LSTM advantages over process-based models on daily predictions extend to sub-daily predictions.The comparison includes the National Water Model across 516 basins.
- Efficiency: MTS-LSTM has much smaller computational overhead than a naive hourly LSTM, while the naive hourly model retains high accuracy.The naive hourly model must process 4320 input sequence steps for each prediction.
- Interpretation: sMTS-LSTM’s summary state between daily and hourly components contains as much information as the naive model extracts from the full hourly sequence.This is presented as an intuitive assumption because high-resolution forcing information diminishes farther into the past.
- Operational relevance: MTS-LSTM can use distinct input variables at each timescale, including lower-resolution remote-sensing products without interpolation.Mixed-resolution forcings also improved accuracy beyond the best single-forcings model in the reported experiments.
- Limitations: Operational use remains bounded by reanalysis inputs, missing multi-objective optimization, and an implementation that generates hourly predictions only with low-resolution steps.Variable-length input sequences could instead produce one hourly prediction each hour.
Appendix A: A Peak-Timing Error Metric
The peak-timing metric evaluates whether high-resolution predictions capture not only flood magnitude but also the timing of observed peaks.
- Metric definition: The peak-timing metric quantifies the lag between observed and predicted flood peaks.It is motivated by the importance of timing for high-temporal-resolution predictions.
- Peak extraction: Observed peaks are filtered by prominence and minimum separation before matching predictions.Peaks below the observed standard deviation are discarded, and remaining peaks are separated by at least 100 steps.
- Peak matching: For each observed peak, the largest prediction within a one-day hourly or three-day daily window is selected.The peak-timing error is the mean absolute difference between observed and predicted peak times.
Appendix B: Negative Results
Exploratory experiments found LSTMs to be resilient across many multi-timescale designs, although the final MTS-LSTM models performed best among the tested approaches.
- Exploratory findings: Many alternative multi-timescale LSTM approaches worked reasonably well, but not as well as the presented MTS-LSTM models.The authors characterize LSTMs as a highly resilient architecture across their experiments.
- Caveat: The reported negative results come from exploratory experiments with few seeds and no extensive hyperparameter tuning.This limits how strongly the alternative-model comparisons should be interpreted.
B1 Delta Prediction
The authors tested delta-based hourly prediction designs intended to link hourly outputs to daily predictions. These designs reduced accuracy or increased architectural complexity relative to the final approach.
- Delta prediction: Feeding the daily prediction into the hourly branch and predicting hourly deviations slightly deteriorated accuracy while making the architecture more complicated.The design predicted each hour’s deviation from the daily mean.
- Weight-based alternative: Predicting 24 daily weights to distribute streamflow across hours produced clearly worse results.Its intended advantage was guaranteed consistency between mean hourly and daily predictions.
- Interpretation: Near-consistent timescale results could be achieved through regularization rather than enforcing equality.The authors suggest learning hourly weights and daily streamflow simultaneously may be harder than predicting the same streamflow at two timescales.
B2 Cross-Timescale State Exchange
The ResNet-multi-timescale LSTM exchanges daily and hourly hidden states after each day to provide the daily branch with finer recent information. However, daily NSE remained roughly unchanged while hourly predictions became much worse.
- Cross-Timescale State Exchange: The ResNet-multi-timescale LSTM combines hourly and daily hidden states after each day and uses them to initialize the next day's states.This skip-connection-inspired design was intended to transfer fine-grained recent information into the daily branch.
- Cross-Timescale State Exchange: Daily NSE remained roughly the same, but hourly predictions became much worse.
B3 Multi-Timescale Input, Single-Timescale Output
The models were trained with daily and hourly inputs but optimized only for hourly discharge, allowing focus on one output timescale. Nevertheless, hourly-only predictions were worse than combined multi-timescale predictions, possibly because the daily branch received no training signal for daily outputs.
- Multi-Timescale Input, Single-Timescale Output: The models ingested daily and hourly data while training only on hourly discharge predictions.This setup was intended to let the models focus fully on hourly prediction rather than conflicting multi-timescale objectives.
- Multi-Timescale Input, Single-Timescale Output: Hourly-only predictions were worse than combined multi-timescale predictions.
- Multi-Timescale Input, Single-Timescale Output: One possible explanation is that the daily branch passes a poorer state summary to the hourly branch without training signal for daily outputs.
Appendix C: Time-Continuous Prediction with ODE-LSTMs
The appendix compares ODE-LSTMs, which adjust hidden states to time-step size, with MTS-LSTM predictions transformed to desired resolutions. Across initial ten-basin experiments, aggregation or disaggregation of MTS-LSTM predictions was usually better than using an ODE-LSTM.
- ODE-LSTM Design: ODE-LSTMs post-process each normal LSTM hidden state with an ODE whose learned weights adjust it to the time-step size.
- Operational Scope: For operational streamflow prediction, the authors consider MTS-LSTM better suited because ODE-LSTMs cannot directly process different input variables at different target timescales.
- Experimental Comparison: The experiments compared time-continuous ODE-LSTM prediction with MTS-LSTM predictions aggregated or disaggregated to target resolutions.MTS-LSTM hourly predictions were obtained from 12-hourly outputs, and daily predictions from aggregated three-hourly outputs.
- Experimental Scope: The initial comparison used ten basins, with models trained on two target timescales and evaluated on a third.
- Experimental Comparison: Across almost all cases, transforming MTS-LSTM predictions was better than using an ODE-LSTM.
Appendix D: Hyperparameter Tuning
The appendix describes two-stage hyperparameter tuning for multi-timescale models and constrained tuning for the naive hourly baseline. Extremely long hourly sequences increased training time and limited the naive model's searchable parameter grid.
- Hyperparameter Tuning: Multi-timescale models were tuned in two stages, first for architectural parameters and then for learning rate and batch size.
- Hyperparameter Tuning: The first-stage schedule used 30 epochs, batch size 512, and a learning rate decreasing from 0.001 to 0.0001 by epoch 20.
- Baseline Tuning: Naive LSTM architectural parameters were not retuned because the architecture had already been extensively tuned in prior work.
- Baseline Tuning: The naive hourly model had a 24-times-larger training set, was trained for only one epoch, and allowed only a relatively small parameter grid because long sequences greatly increased training time.
- Selection Criterion: Configurations were selected using the average of daily and hourly median NSE, with three seeds averaged for each parameter combination.