Source-linked AI summary
Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting
Xingjian Shi, Zhourong Chen, Hao Wang, Dit-Yan Yeung, Wai-kin Wong, Wang-chun Woo
TL;DR
Precipitation nowcasting requires precise, timely local rainfall predictions, but machine-learning approaches have been limited. This paper introduces ConvLSTM for spatiotemporal sequence forecasting and finds it consistently outperforms FC-LSTM and ROVER.
Problem
Precipitation nowcasting requires precise, timely local rainfall predictions, while its high-dimensional, chaotic spatiotemporal sequences remain challenging for machine-learning methods.
Method
The paper extends FC-LSTM with convolutional input-to-state and state-to-state transitions, stacking ConvLSTM layers in an end-to-end encoding-forecasting network.
Results
ConvLSTM captures spatiotemporal correlations better and outperforms both FC-LSTM and the optical-flow-based ROVER algorithm for precipitation nowcasting.
Takeaways & Limitations
The results support applying end-to-end deep-learning models to the challenging precipitation nowcasting problem.
Takeaways & Limitations
The model uses zero-padding that assumes no prior knowledge about the outside world at hidden-state boundaries.
Abstract
from arXiv · showhide
The goal of precipitation nowcasting is to predict the future rainfall intensity in a local region over a relatively short period of time. Very few previous studies have examined this crucial and challenging weather forecasting problem from the machine learning perspective. In this paper, we formulate precipitation nowcasting as a spatiotemporal sequence forecasting problem in which both the input and the prediction target are spatiotemporal sequences. By extending the fully connected LSTM (FC-LSTM) to have convolutional structures in both the input-to-state and state-to-state transitions, we propose the convolutional LSTM (ConvLSTM) and use it to build an end-to-end trainable model for the precipitation nowcasting problem. Experiments show that our ConvLSTM network captures spatiotemporal correlations better and consistently outperforms FC-LSTM and the state-of-the-art operational ROVER algorithm for precipitation nowcasting.
1 Introduction
The paper frames precipitation nowcasting as a challenging spatiotemporal sequence forecasting problem with past radar maps as input and future radar maps as output. It proposes ConvLSTM, extending FC-LSTM with convolutional input-to-state and state-to-state transitions for modeling spatiotemporal relationships.
- Problem: Precipitation nowcasting aims to predict local rainfall intensity precisely and promptly over short horizons such as 0–6 hours.These forecasts support emergency rainfall alerts, airport weather guidance, and integration with longer-term NWP models.
- Related work: Existing approaches use either computationally complex NWP simulations or faster radar-echo extrapolation methods.Operational systems often favor extrapolation-based methods because they are faster and more accurate at nowcasting timescales.
- Problem formulation: The paper formulates precipitation nowcasting as sequence-to-sequence forecasting, mapping past radar-map sequences to multiple future radar maps.The high dimensionality of spatiotemporal sequences makes this learning problem nontrivial.
- Contribution: The proposed ConvLSTM extends FC-LSTM with convolutional structures in both input-to-state and state-to-state transitions.The model is designed to capture spatiotemporal relationships within an end-to-end sequence forecasting framework.
2 Preliminaries
Precipitation nowcasting forecasts future radar maps from observed radar echo sequences over local spatial regions and short operational horizons. The task is formulated as spatiotemporal sequence forecasting, motivating sequence models that capture both spatial and temporal structure.
- 2 Preliminaries: Nowcasting uses radar echo sequences to forecast 6–60 future frames, with maps sampled every 6–10 minutes for horizons of 1–6 hours.The forecasting region may be local areas such as Hong Kong, New York, or Tokyo.
- 2 Preliminaries: The problem predicts a length-K future sequence from the previous J observations, including the current observation, in an M × N spatial grid with P measurements per cell.Each observation is represented as a tensor X ∈ R^P×M×N.
- 2 Preliminaries: For precipitation nowcasting, each timestamp is a 2D radar echo map, and tiled non-overlapping patches provide the measurements for spatiotemporal sequence forecasting.Pixels within each patch are treated as measurements.
- 2 Preliminaries: Unlike one-step time-series forecasting, the target contains spatial and temporal structures, with possible prediction complexity up to O(MK NK PK).Exploiting structure in the space of possible predictions can reduce dimensionality and make forecasting tractable.
- 2 Preliminaries: LSTM is a stable, powerful RNN structure for long-range dependencies, using a memory cell and self-parameterized gates to access, write, and clear state information.Multiple LSTMs can be stacked and temporally concatenated into more complex sequence-modeling structures.
3 The Model
The model extends FC-LSTM with convolutions in both input-to-state and state-to-state transitions, preserving spatial information in spatiotemporal sequences. Stacked ConvLSTM layers form an encoding-forecasting network for precipitation nowcasting.
- ConvLSTM design: ConvLSTM replaces FC-LSTM’s fully connected transitions with convolutional structures to reduce spatial redundancy and encode spatial information.The design applies convolutions in both input-to-state and state-to-state transitions.
- ConvLSTM design: The ConvLSTM recurrence combines convolutional input and hidden-state transformations with Hadamard products for cell-state and gate updates.The equations use ∗ for convolution and ◦ for the Hadamard product.
- ConvLSTM design: Transitional kernel size controls motion sensitivity: larger kernels capture faster motions, whereas smaller kernels capture slower motions.FC-LSTM is described as a special case of ConvLSTM when feature dimensions are 1 × 1.
- Implementation details: Zero-padding preserves the inputs’ row and column dimensions, while zero-initialized states represent total ignorance before the first input.Padding is applied before convolution so the states retain the same spatial dimensions as the inputs.
- Encoding-forecasting network: The precipitation-nowcasting model stacks ConvLSTM layers in encoding and forecasting networks, copying the encoder’s final states and cell outputs into the forecaster.The encoder compresses the input sequence into a hidden-state tensor, while the forecasting network unfolds it to produce predictions.
4 Experiments
Experiments compare ConvLSTM with FC-LSTM on Moving-MNIST and with ROVER on precipitation nowcasting, showing advantages from convolutional state transitions and deeper models. The precipitation study uses radar data and standard nowcasting metrics to evaluate these methods.
- Experimental overview: ConvLSTM outperforms FC-LSTM on Moving-MNIST and ROVER on precipitation nowcasting.The experiments assess ConvLSTM against FC-LSTM on synthetic data and against the state-of-the-art ROVER algorithm on radar echo data.
- Moving-MNIST: State-to-state kernels larger than 1 are essential for capturing spatiotemporal motion patterns.The experiments vary ConvLSTM layer counts and kernel sizes to study how convolutional structure affects performance.
- Moving-MNIST: Deeper ConvLSTM models can produce better results with fewer parameters.The Moving-MNIST models include one-, two-, and three-layer variants with different hidden-state sizes.
- Moving-MNIST: The 3-layer model achieves an average cross-entropy error of 6379.42 on out-of-domain sequences containing three previously unseen moving digits.The test sequences use a different, non-overlapping subset of 500 MNIST digits and introduce a three-digit system absent from training.
- Precipitation nowcasting: The precipitation dataset comprises radar intensities from 97 rainy days in Hong Kong collected from 2011 to 2013.Radar maps are transformed to gray-level pixels, cropped to a central 330 × 330 region, and evaluated using Rainfall-MSE, CSI, FAR, POD, and correlation.
- Precipitation nowcasting: FC-LSTM performs poorly because fully connected layers contain redundant connections and fail to capture strong local spatial consistencies in radar maps, while ConvLSTM outperforms ROVER.The reported results are presented in Table 2 and Fig. 5.
5 Conclusion and Future Work
The paper applies deep learning to precipitation nowcasting by formulating it as spatiotemporal sequence forecasting and introducing ConvLSTM, an extension of LSTM for this task.
- Conclusion and Future Work: The study formulates precipitation nowcasting as a spatiotemporal sequence forecasting problem and proposes ConvLSTM as a new LSTM extension.The approach applies machine learning, especially deep learning, to a precipitation nowcasting problem that had not benefited from sophisticated machine learning techniques.
Appendix
The appendix documents ROVER’s optical-flow parameters and compares ConvLSTM, FC-LSTM, and ROVER across quantitative and illustrative precipitation-nowcasting evaluations. It also shows ConvLSTM’s faster loss decrease than FC-LSTM in an online training setting.
- Table 3 reports the best parameters for the optical flow estimator used in ROVER.
- Figure 7 compares different models over time using four precipitation-nowcasting metrics.
- Figure 8 presents two prediction examples comparing ConvLSTM and ROVER2 against ground truth at three-frame sampling intervals.
- Figure 9 illustrates in-domain predictions from FC-LSTM and multiple ConvLSTM architectures with varying filter sizes and layer counts.
- Figure 10 shows an out-domain run with ground truth and predictions from the 3-layer network.
- In online training, ConvLSTM’s average mini-batch cross entropy decreases faster than FC-LSTM’s as the number of data cases increases.Each iteration generates a new training-sample set; the x-axis starts at 25600 data cases.