Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models

Jonathan A. Weyn; Dale R. Durran; Rich Caruana; Nathaniel Cresswell-Clay

Source-linked AI summary

Sub-seasonal forecasting with a large ensemble of deep-learning weather prediction models

Jonathan A. Weyn, Dale R. Durran, Rich Caruana, Nathaniel Cresswell-Clay

arXiv:2102.05107v1Feb 9, 2021physics.ao-phcs.LG

TL;DR

S2S forecasting is challenging at two-week-to-two-month lead times, and computational limits constrain large NWP ensembles. This paper develops a global cubed-sphere DLWP ensemble with recursive CNN forecasts, finding modestly competitive probabilistic skill at weeks 4–6 while ECMWF performs better at shorter leads.

Problem
S2S forecasting at lead times from roughly two weeks to two months remains challenging, while computational resources limit large NWP ensembles.
Method
The paper develops a global DLWP ensemble that recursively predicts atmospheric fields with CNNs on a cubed-sphere grid, adding T850 and TCWV at approximately 1.4° resolution.
Results
The DLWP ensemble is only modestly inferior to ECMWF over land at weeks 4–6, while ECMWF performs better at shorter lead times.
Takeaways & Limitations
DLWP may supplement NWP-based S2S forecasts where those systems are weakest, particularly in the tropics and spring and summer months.
Takeaways & Limitations
The DLWP model does not forecast precipitation and is worse than operational NWP models for deterministic synoptic-scale weather prediction.

Abstract

from arXiv · show

We present an ensemble prediction system using a Deep Learning Weather Prediction (DLWP) model that recursively predicts key atmospheric variables with six-hour time resolution. This model uses convolutional neural networks (CNNs) on a cubed sphere grid to produce global forecasts. The approach is computationally efficient, requiring just three minutes on a single GPU to produce a 320-member set of six-week forecasts at 1.4° resolution. Ensemble spread is primarily produced by randomizing the CNN training process to create a set of 32 DLWP models with slightly different learned weights. Although our DLWP model does not forecast precipitation, it does forecast total column water vapor, and it gives a reasonable 4.5-day deterministic forecast of Hurricane Irma. In addition to simulating mid-latitude weather systems, it spontaneously generates tropical cyclones in a one-year free-running simulation. Averaged globally and over a two-year test set, the ensemble mean RMSE retains skill relative to climatology beyond two-weeks, with anomaly correlation coefficients remaining above 0.6 through six days. Our primary application is to subseasonal-to-seasonal (S2S) forecasting at lead times from two to six weeks. Current forecast systems have low skill in predicting one- or 2-week-average weather patterns at S2S time scales. The continuous ranked probability score (CRPS) and the ranked probability skill score (RPSS) show that the DLWP ensemble is only modestly inferior in performance to the European Centre for Medium Range Weather Forecasts (ECMWF) S2S ensemble over land at lead times of 4 and 5-6 weeks. At shorter lead times, the ECMWF ensemble performs better than DLWP.

1 Introduction

S2S forecasting is difficult because deterministic forecasts lose skill after about two weeks, while ensemble systems remain computationally expensive. The paper develops a global DLWP ensemble with improved resolution and variables to provide six-week forecasts and evaluates it against ECMWF products.

Motivation: S2S forecasting spans roughly two weeks to two months, a challenging interval between deterministic weather and seasonal forecasting.Deterministic forecast skill is limited by chaotic error growth, whereas seasonal skill benefits from slowly evolving ocean-land forcing.
Motivation: Computational costs constrain the size of NWP-based S2S ensembles, with some operational systems using only four members and ECMWF providing 51 members to 46 days.Machine learning is presented as a potential route to lower-cost S2S ensemble forecasting.
Prior foundation: The preceding global DLWP model could run for at least one year from a single initialization while capturing the seasonal cycle with reasonable accuracy.This result motivated further model improvements for extended S2S forecasting.
Approach: The improved DLWP model adds two prognostic fields, increases resolution to 1.4°, and generates 320-member forecasts through six weeks for comparison with ECMWF.Forecasts are verified against ERA5 data.
Evaluation: The paper evaluates the DLWP ensemble alongside operational ECMWF S2S products, with the study organized around model improvements, ensemble construction, deterministic forecasts, and S2S verification.The paper focuses on forecasts at lead times relevant to S2S applications.

2 The DLWP model

The DLWP model maps atmospheric and prescribed fields onto a cubed sphere and uses CNN operations to predict atmospheric states. The updated model adds T850 and TCWV, increases grid resolution, and retains a detailed CNN architecture with auxiliary inputs.

Model structure: The baseline model forecasts Z1000, Z500, τ300−700, and T2 on a cubed-sphere grid using prescribed radiation, topography, and land-sea fields.CNNs use shared horizontal filters on equatorial cube faces and separate filters on polar faces.
Model extensions: The update adds T850 and TCWV as forecast fields, supporting representation of large-scale weather patterns and tropical convective systems.TCWV is the vertically integrated gas-phase water above each grid cell and relates to phenomena including tropical cyclones and the MJO.
Resolution and data: Increasing each cube face from 48 × 48 to 64 × 64 cells yields approximately 1.4° resolution at the equator.ERA5 data at 1° latitude-longitude resolution are remapped for training, validation, and testing.
Architecture specification: Table 1 specifies the CNN as a sequence of layers, with v denoting input fields, t input time steps, and c auxiliary prescribed inputs.The architecture includes operations such as concatenation of states supplied to later layers.

3 Designing an ensemble of DLWP models

The ensemble combines perturbed initial conditions with stochastic variation in CNN model weights, while comparing alternative spread-generation strategies and controlling model training. Retrained models provide better ensemble calibration than initial-condition perturbations, with bias correction improving longer-lead performance.

Ensemble construction: The ensemble combines perturbed ERA5 initial conditions with variations in the DLWP model representation, following operational NWP ensemble practice.Initial-condition perturbations use ERA5’s 10 perturbed ensemble members, while model uncertainty is represented through differently trained CNNs.
Initial condition uncertainty: The initial-condition ensemble is under-dispersive because its perturbations do not strongly project onto the fastest-growing modes, while doubling them improves spread only imperfectly.The IC×2 ensemble doubles differences from the control ERA5 initial state to better match spread to ensemble-mean RMSE.
Ensemble construction: Randomly initialized CNN weights are exploited by retraining models with different seeds, producing approximately equally skillful but statistically independent ensemble members.The stochastic-perturbation ensemble ultimately contains 32 models selected from eight training cycles.
Uncertainty in the representation of the atmosphere: The stochastic-perturbation ensemble outperforms IC and IC×2, with spread roughly 80% of RMSE at 8 days and 95% at 14 days.At 14 days, its RMSE remains slightly better than climatology, whereas IC and IC×2 exceed climatology between 7 and 8 days.
Ensemble comparison: A 320-member grand ensemble is only slightly better than the stochastic-perturbation ensemble at 14 days but becomes clearly superior at longer leads, especially after bias correction.The grand ensemble applies 32 differently weighted DLWP models to each of 10 initial-condition perturbations.
The control member: The control model uses Adam optimization with progressively reduced learning rates to obtain weights that better minimize the loss function and produce good forecasts.The learning rate begins at 10^-3, is reduced fivefold when validation loss stalls, and has a minimum of 10^-6.
Correcting model bias: The DLWP ensemble develops warm biases over Northern Hemisphere land and subtropical marine-stratocumulus regions, plus a cold Antarctic bias, which amplify with lead time.Although spatial-mean bias decreases globally at longer leads, the spatial patterns become more pronounced.

4 Skill of the DLWP at short lead times

Short-lead evaluation finds that DLWP forecasts represent major atmospheric structures reasonably well at 4.5 days, while quantitative skill remains below ECMWF but above climatology for key fields through two weeks.

4.5-day DLWP forecasts reasonably represent Hurricane Irma and several major mid-latitude circulation features, despite errors in storm position, intensity, and another cyclone.The forecast uses 1.4° resolution and six prognostic variables, but is not equal to a state-of-the-art operational NWP forecast.
208 twice-weekly forecasts were initialized from January 2017 through the end of 2018 to match available ECMWF S2S initialization times.
Through 14 days, the DLWP grand ensemble remains superior to climatology for Z500 RMSE, while ECMWF remains more accurate overall.Beyond 9 days, DLWP outperforms the ECMWF control, but ECMWF ACC remains superior to the DLWP ensemble until almost day 13.
For daily-averaged T2, the DLWP and ECMWF ensemble RMSEs remain below climatology through 14 days, with ECMWF producing the best results.
For T850, ensemble RMSEs again remain below climatology for 14 days, although ECMWF performs best and persistence loses skill more rapidly.Persistence ACC falls below 0.4 in just two days, making T850 more difficult to forecast than Z500 or T2.

5 Extending the forecasts to the S2S range

The DLWP ensemble provides skillful subseasonal forecasts, approaching ECMWF performance at longer lead times and over land, while remaining limited by its lack of ocean coupling.

5.1 Ensemble-mean anomaly correlations: At weeks 3–4 and 5–6, DLWP ensemble T2 forecasts outperform persistence and the ECMWF control, although ECMWF’s ensemble performs best.ECMWF’s averaged ACC is roughly 0.5 for T2 at 3–4 weeks.
5.1 Ensemble-mean anomaly correlations: At 5–6 weeks, the best individual forecasts reach about 0.8 ACC for T2 and about 0.6 for T850 in both ensembles.
5.1 Ensemble-mean anomaly correlations: The DLWP grand ensemble exceeds 0.5 ACC for 3–4-week T2 forecasts roughly 25% of the time, compared with 50% for ECMWF.
5.1 Ensemble-mean anomaly correlations: ECMWF outperforms DLWP over North America and captures a developing El Niño that DLWP misses, consistent with ECMWF’s ocean coupling.DLWP lacks oceanic data, making the missed El Niño unsurprising.
5.2.1 Continuous ranked probability score: At global T850 CRPS, ECMWF leads at week 2 and remains slightly better at week 3, while DLWP catches up and essentially ties ECMWF at weeks 4 and 5–6.Climatology outperforms both control forecasts at lead times of three weeks or longer.
5.2.2 Ranked probability skill score: DLWP’s relative performance improves over land: ECMWF’s RPSS advantage shrinks because its skill drops after week 2, while DLWP remains similar to its global performance.The ensembles are most similar during NH extratropical summer, when DLWP ties ECMWF in CRPS at weeks 3, 4, and 5–6.

6 Conclusions

The DLWP ensemble combines refined global deep-learning forecasts with randomized model weights to produce large, computationally efficient S2S ensembles. It shows useful skill beyond two weeks, approaches ECMWF performance at longer leads over land, and remains limited by missing precipitation and weaker deterministic skill than operational NWP.

6 Conclusions: 1.4° resolution and added 850-hPa temperature and total column water vapor enabled tropical-cyclone generation and a reasonable four-day deterministic Hurricane Irma forecast.The model recursively forecasts six-hour-resolution atmospheric states and remains computationally efficient.
6 Conclusions: About 3 minutes produce a 320-member six-week ensemble using 32 DLWP models with different learned convolutional filter coefficients across 10 initial conditions.The initial conditions were drawn from ERA5 rather than generated using singular-vector information.
6 Conclusions: 14 days: DLWP ensemble-mean RMSE for Z500, T850, and 2-m temperature remained below climatology, while ACC remained above 0.6 for 7–8 days.The ensemble mean significantly improved on a single control member over the 2017–2018 test set.
6 Conclusions: At weeks 4 and 5–6, DLWP and ECMWF had essentially the same CRPS, while ECMWF was better at shorter lead times and both exceeded climatology and persistence.RPSS showed modest skill relative to climatology at all lead times, with larger ECMWF advantages except in northern extra-tropical summer.
6 Conclusions: DLWP skill was strongest in northern extra-tropics and comparable to ECMWF in summer, but weaker in winter and over tropical oceans, likely because the model lacks SST data.The eastern equatorial Pacific failure during the onset of a weak 2018 El Niño event supports this proposed explanation.
6 Conclusions: The model does not forecast precipitation and is worse than operational NWP models for deterministic synoptic-scale weather prediction.The authors suggest DLWP may supplement NWP-based S2S forecasts where they are weakest, including the tropics and spring and summer months.