Source-linked AI summary
Deep Learning for Post-Processing Ensemble Weather Forecasts
Peter Grönquist, Chengyuan Yao, Tal Ben-Nun, Nikoli Dryden, Peter Dueben, Shigang Li, Torsten Hoefler
TL;DR
Global ensemble weather forecasts provide uncertainty estimates but require substantial computation, motivating methods that retain forecast quality with fewer trajectories. This paper combines reduced ensembles with deep-neural-network post-processing and reports over 14% CRPS improvement, including stronger improvements in selected extreme-weather cases.
Problem
Ensemble weather prediction is computationally costly, while reliable uncertainty quantification is especially important for extreme weather events.
Method
The paper combines a subset of ensemble trajectories with CNN- and LCN-based post-processing for uncertainty quantification, bias correction, and PDF calibration.
Results
The mixed models achieve over 14% relative CRPS improvement globally, and selected extreme-weather cases show especially pronounced improvements.
Takeaways & Limitations
Using fewer trajectories can produce forecast skill comparable to or better than larger ensembles while reducing post-processing computation.
Takeaways & Limitations
Probabilistic scores for individual extreme-weather events are difficult to interpret, and single-event improvements do not establish general superiority.
Abstract
from arXiv · showhide
Quantifying uncertainty in weather forecasts is critical, especially for predicting extreme weather events. This is typically accomplished with ensemble prediction systems, which consist of many perturbed numerical weather simulations, or trajectories, run in parallel. These systems are associated with a high computational cost and often involve statistical post-processing steps to inexpensively improve their raw prediction qualities. We propose a mixed model that uses only a subset of the original weather trajectories combined with a post-processing step using deep neural networks. These enable the model to account for non-linear relationships that are not captured by current numerical models or post-processing methods. Applied to global data, our mixed models achieve a relative improvement in ensemble forecast skill (CRPS) of over 14%. Furthermore, we demonstrate that the improvement is larger for extreme weather events on select case studies. We also show that our post-processing can use fewer trajectories to achieve comparable results to the full ensemble. By using fewer trajectories, the computational costs of an ensemble prediction system can be reduced, allowing it to run at higher resolution and produce more accurate forecasts.
1 Introduction
The paper targets the computational cost and limited skill of global ensemble weather prediction by combining reduced ensembles with deep-neural-network post-processing. Across uncertainty quantification, bias correction, and PDF calibration, the approach improves forecast skill while using fewer trajectories.
- Motivation: Ensemble simulations estimate forecast uncertainty by running perturbed weather-model trajectories in parallel, but their computational cost limits prediction systems.Perturbations include initial conditions, parameter values, and stochastic forcing.
- Contributions: Over 16% relative RMSE improvement is achieved for temperature uncertainty quantification using five of ten ensemble members.The network predicts an ensemble-like spread from a small fraction of the trajectories.
- Contributions: A 7.9% relative RMSE improvement is achieved for temperature through point-wise neural-network bias correction.The correction accounts for local trends in weather patterns.
- Contributions: Over 14.5% forecast-skill improvement is achieved for PDF calibration using only half the trajectories of a full ensemble.The paper evaluates this improvement using CRPS-based calibration.
- Scope and novelty: Deep neural networks are applied to post-process operational medium-range ensemble forecasts, extending prior neural-network work beyond station-specific or global-interpolation applications.The paper focuses on global predictions and extreme weather forecasts.
2 Data
The study uses ECMWF re-forecast ensembles with ERA5 ground truth on a global latitude/longitude grid, then preprocesses the data with local standardization. LAS improves spread-prediction results by around 15% on validation sets.
- Data sources: ECMWF re-forecast data provide a ten-member ensemble and an unperturbed control experiment for training and evaluating the weather-forecast networks.The re-forecast experiments estimate the forecast model’s climate for each date and support drift removal and generic skill measurement.
- Data sources: ERA5 supplies ground-truth weather conditions, while ENS10 and ERA5 are interpolated to a global 0.5-degree latitude/longitude grid.The regular grid simplifies the models but oversaturates gridpoints near the poles.
- Preprocessing: The preprocessing pipeline converts GRIB data to floating point, standardizes features, and uses the first five ENS10 trajectories as model inputs.Different trajectory selections showed no significant differences in average means or spreads.
- Preprocessing: Local Area-wise Standardization uses moving mean and standard-deviation filters to preserve local spatial differences in meteorological variability.The resulting maps are padded at boundaries and upscaled with Gaussian filtering.
- Preprocessing results: Around 15% relative improvement is observed for spread prediction on validation sets with LAS versus one global standardization.LAS also gives faster convergence, while no difference is observed for locally connected networks.
3 Neural Networks
The paper develops distinct neural networks for uncertainty quantification and output bias correction, combining convolutional and locally connected architectures with CRPS-based evaluation and calibration.
- Model architectures: Separate neural networks address uncertainty quantification and output bias correction, using a residual Inception-style network and a U-Net with an LCN, respectively.The models are designed for different tasks within the post-processing pipeline.
- Uncertainty Quantification Model: The uncertainty network uses ten Inception-style modules with residual connections and parallel dilated convolutions to learn differently sized receptive fields.Post-processed NWP output is concatenated into each Inception-style layer to prioritize between forecast lead times and network outputs.
- Output Bias Correction Model: The output bias correction network predicts the difference between the NWP forecast and ground truth rather than directly approximating the identity mapping.This residual target is motivated by the difficulty of learning an identity function through several nonlinear layers.
- Output Bias Correction Model: A three-level U-Net extracts features, while a locally connected final layer models regional gridpoint-wise dependencies without shared spatial filters.The U-Net captures long-range dependencies, allowing a single 1 × 1-kernel LCN to produce the final bias correction.
- Metrics and Calibration: CRPS evaluates probabilistic forecast skill by integrating the squared difference between the prediction CDF and the ground truth, while CRPSS expresses relative improvement over the raw ensemble.The networks are initially evaluated with RMSE for regression tasks, then combined predictions are assessed with CRPS.
- Metrics and Calibration: PDF calibration combines uncertainty and bias correction networks and optimizes their output to minimize CRPS under a Gaussian forecast assumption.The uncertainty target is replaced by the difference between ground truth and the output bias-corrected forecast, ΔP.
4 Results
The experiments evaluate uncertainty quantification, bias correction, and PDF calibration on global T850 and Z500 forecasts. Deep-learning post-processing generally improves reduced-ensemble forecasts, with especially strong gains in probabilistic skill and selected extreme-weather cases.
- Evaluation: The models are evaluated primarily on 48-hour T850 forecasts and additionally on Z500, using global RMSE, CRPS, and linear-regression baselines.The experiments use ENS10 and ERA5 ground truth, with E5 and E10 as raw ensemble baselines for CRPS.
- Uncertainty quantification: The uncertainty network consistently outperforms reduced-ensemble baselines as the number of trajectories varies, although relative gains show diminishing returns.The model retains an advantage even when using half of the full ensemble.
- Input ablation: Using multiple input fields produces more accurate uncertainty quantification and bias correction than using only the predicted field.This conclusion is supported by the input-field ablation study.
- Probabilistic skill: Both PDF-calibration models outperform the full-ensemble CRPS for T850 and Z500 despite using fewer ensemble members.The reported improvement is attributed primarily to reducing extreme outliers, especially for T850.
- Baseline comparison: 14.5% improvement is reported for B5U5C on T850, compared with 5.5% for EMOS over the test set.This comparison uses CRPS improvement over the test set.
- Extreme-weather cases: The case studies show more than 25.5% CRPSS improvement over the affected region during the Asian cold wave, while hurricane cases show spatially concentrated gains.The authors caution that single-event probabilistic scores are difficult to interpret generally.
5 Conclusion
The paper concludes that informed deep-learning model construction improves global ensemble weather prediction while allowing reduced ensembles to surpass a ten-member ensemble in CRPS. It also identifies broader testing across fields, grids, lead times, ensemble sizes, and NWP applications as future work.
- Conclusion: Deep learning improves global ensemble weather-prediction skill through uncertainty estimation and locally adaptive output bias correction.The combined models improve a five-member ensemble to surpass the forecast skill of a ten-member ensemble in CRPS.
- Conclusion: The combined models show especially pronounced improvements for hard-to-predict extreme-weather cases such as tropical cyclone Winston.The conclusion also reports that global post-processing runs within tenths of a second on heterogeneous hardware.
- Future work: Future research should test the architectures on other NWP applications, physical fields, forecast lead times, native unstructured grids, and larger ensembles.The paper specifically mentions data assimilation, equations of motion, transfer learning, recurrent networks, graph neural networks, and operational 50-member ensembles.
A CRPS derivation
The appendix introduces the normal-distribution CDF and then defines CRPS through an integral involving the forecast CDF and the observation threshold. Subsequent steps evaluate the integral by splitting its bounds and summing the resulting terms.
- Normal CDF: The normal-distribution CDF is introduced using Φ, identified as the error function.The appendix presents this CDF before deriving the CRPS expression.
- CRPS definition: CRPS is defined as an integral over the squared difference between the forecast CDF and the observation indicator.The derivation then proceeds by evaluating contributions from the observation and infinite bounds.