PaperlayerView exact revision on arXiv

Source-linked AI summary

GenCast: Diffusion-based ensemble forecasting for medium-range weather

Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Tom R. Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, Matthew Willson

arXiv:2312.15796v2cs.LGphysics.ao-ph

TL;DR

Weather forecasts are uncertain, so decisions require distributions of possible scenarios rather than single trajectories. GenCast is a probabilistic machine-learning weather model that samples forecast trajectories and achieves greater accuracy and decision-making value than ECMWF’s ENS, while pointing toward broader use of generative AI in weather forecasting.

  • Problem

    Weather forecasts remain uncertain because observations and atmospheric models are imperfect, making distributions of possible future scenarios important for critical decisions.

  • Method

    GenCast uses probabilistic machine-learning weather prediction to generate sampled weather trajectories, with a graph-transformer processor operating on a refined icosahedral mesh.

  • Results

    GenCast achieves greater accuracy and decision-making value than ECMWF’s ENS across variables, vertical levels, lead times, and application settings.

  • Takeaways & Limitations

    The results support a broader role for generative AI in capturing complex weather distributions accurately and reliably for crucial decision-making applications.

  • Takeaways & Limitations

    Operational deployment may require higher resolution and training on operational inputs, while diffusion sampling is computationally more expensive than equivalent deterministic MLWP.

Abstract

from arXiv · show

Weather forecasts are fundamentally uncertain, so predicting the range of probable weather scenarios is crucial for important decisions, from warning the public about hazardous weather, to planning renewable energy use. Here, we introduce GenCast, a probabilistic weather model with greater skill and speed than the top operational medium-range weather forecast in the world, the European Centre for Medium-Range Forecasts (ECMWF)'s ensemble forecast, ENS. Unlike traditional approaches, which are based on numerical weather prediction (NWP), GenCast is a machine learning weather prediction (MLWP) method, trained on decades of reanalysis data. GenCast generates an ensemble of stochastic 15-day global forecasts, at 12-hour steps and 0.25 degree latitude-longitude resolution, for over 80 surface and atmospheric variables, in 8 minutes. It has greater skill than ENS on 97.4% of 1320 targets we evaluated, and better predicts extreme weather, tropical cyclones, and wind power production. This work helps open the next chapter in operational weather forecasting, where critical weather-dependent decisions are made with greater accuracy and efficiency.

1. Introduction

Weather forecasts support high-stakes decisions but remain uncertain because observations are incomplete, models are imperfect, and small initial errors can rapidly grow. Existing ML weather-prediction methods improve deterministic forecasting but have paid limited attention to calibrated probabilistic ensembles and can produce blurry states.

  • Motivation: Forecast uncertainty arises from partial observations, imperfect weather models, and nonlinear error growth that expands possible future scenarios.Decisions such as cyclone evacuation, renewable-energy planning, and heatwave preparation require more than a single probable forecast.
  • Existing forecasting: NWP ensemble forecasts generate multiple possible weather scenarios, but remain error-prone, slow to run, and time-consuming to engineer.ECMWF’s ENS is identified as the state-of-the-art NWP-based ensemble system within ECMWF’s Integrated Forecast System.
  • ML weather prediction: MLWP methods have improved accuracy and efficiency for deterministic forecasts but have generally focused on predicting the mean rather than uncertainty.These methods are typically trained with mean squared error, which can produce blurry forecast states, especially at longer lead times.
  • ML weather prediction: Prior MLWP ensemble attempts used initial-condition perturbations but did not resolve blurring or rival operational ensembles such as ENS.The passage distinguishes these attempts from Neural GCM, a hybrid NWP-MLWP method.

2. GenCast

GenCast is a conditional diffusion model that samples weather trajectories from noisy candidate states conditioned on recent atmospheric states. It autoregressively generates stochastic global ensembles over a 15-day horizon at 12-hour steps and 0.25° resolution.

  • Overview: GenCast generates global 15-day ensemble forecasts at 0.25° resolution, taking about 8 minutes for one forecast on a Cloud TPUv5 device.Ensemble forecasts can be generated in parallel.
  • Trajectory modeling: GenCast factors the trajectory distribution into successive conditional predictions of the next weather state given the current and previous states.Each successive state is sampled autoregressively.
  • State and horizon: The model represents six surface and six atmospheric variables at 13 pressure levels on a 0.25° latitude-longitude grid, with 30 twelve-hour forecast steps.The forecast trajectory is denoted X_1:T, where T=30.
  • Diffusion sampling: A conditional diffusion process iteratively refines a noise-initialized candidate state while conditioning on the previous two atmospheric states.Different initial noise samples produce different ensemble trajectories.
  • Architecture: Each refinement stage uses an encoder, graph-transformer processor, and decoder operating through a six-times-refined icosahedral mesh.The encoder maps from the latitude-longitude grid to the learned mesh representation, and the decoder maps back.
  • Training and initialization: GenCast is trained on 40 years of ERA5 reanalysis data with a single-step diffusion denoising objective, then rolled out autoregressively for 15-day forecasts.Evaluation initialization combines ERA5 reanalysis with perturbations derived from the ERA5 Ensemble of Data Assimilations.

3. Realism of GenCast samples

GenCast produces sharp, realistic forecast samples whose spectra closely match ERA5, while its ensemble mean is appropriately blurrier. Its cyclone forecasts express broad uncertainty at long lead times and narrower uncertainty near landfall.

  • Sample realism: GenCast samples remain sharp and spectrally similar to ERA5 at both one- and 15-day lead times, unlike blurrier deterministic or ensemble-mean states.GraphCast and the GenCast ensemble mean lose power at high frequencies relative to individual GenCast samples.
  • Derived quantities: GenCast samples avoid the large negative wind-speed bias seen in GraphCast by computing nonlinear quantities directly from samples.The issue arises because a nonlinear function of an average need not equal the average of that nonlinear function.
  • Tropical cyclones: At seven days before Typhoon Hagibis landfall, GenCast predicts a wide range of cyclone trajectories, while shorter lead times show narrower uncertainty.The narrowing spread reflects greater confidence about landfall timing and location.

4. Baselines

GenCast is evaluated against ENS and GraphCast-Perturbed using matched 50-member ensembles and corresponding analyses. The comparison includes resolution, initialization, perturbation, and verification choices that affect interpretation.

  • Baselines: ENS forecasts are regridded from 0.2° to 0.25°, and both ENS and GenCast evaluations use 50-member ensembles.The comparison is limited to variables and tropospheric pressure levels for which all 50 ENS members are available.
  • Baselines: The 2018 validation and 2019 test years precede ENS’s mid-2023 resolution change from 0.2° to 0.1°.Thus, the reported baseline comparison uses the earlier ENS resolution.
  • Baselines: GraphCast-Perturbed derives ensemble spread from initial-condition differences and added Gaussian Process perturbations because GraphCast itself is deterministic.The perturbations are added to improve ensemble dispersion and skill.
  • Verification: Models are evaluated against corresponding analyses: operational ENS against HRES-fc0 and ERA5-trained ML models against ERA5.This follows the evaluation protocol described for GraphCast.
  • Verification: ENS receives up to two hours of additional assimilation-window look-ahead over ML models for most variables, giving it a small evaluation advantage.The exception is sea surface temperature.
  • Verification: Using a single deterministic analysis as ground truth can favor under-dispersed ensembles at short lead times.Near zero lead time, scores are optimized by setting all initial conditions equal to the deterministic analysis, ignoring initial-condition uncertainty.

5. Ensemble skill

GenCast’s probabilistic forecasts are significantly more skillful than ENS across most evaluated variable, lead-time, and pressure-level combinations, while its ensemble mean also matches or exceeds ENS on most targets.

  • 97.4% of 1320 variable, lead-time, and vertical-level combinations significantly favor GenCast over ENS in CRPS (p< 0.05).At lead times greater than 36 hours, the advantage rises to 99.8% of targets.
  • GenCast’s largest CRPS improvements often occur at shorter lead times, across surface variables and higher-pressure-level temperature and specific humidity.For these higher-pressure-level variables, improvements range from 10-30% better.
  • 99.6% of targets have better GenCast CRPS than GraphCast-Perturbed, while GraphCast-Perturbed outperforms ENS on only 27% of targets.GraphCast-Perturbed’s gains over ENS are concentrated at shorter lead times.
  • GenCast’s ensemble-mean RMSE is as good as or better than ENS on 96% of targets and significantly better on 82%.RMSE evaluates how the ensemble mean matches ground truth but does not account for forecast uncertainty.

6. Ensemble calibration

Calibration tests indicate that GenCast represents forecast uncertainty at least as well as ENS, whereas GraphCast-Perturbed is consistently overconfident.

  • GenCast generally has calibration as good as or better than ENS according to spread-skill ratios and rank histograms.Its spread-skill scores are usually close to 1, the value associated with a perfect ensemble forecast.
  • GenCast’s spread-skill scores are usually close to 1, indicating ensemble spread commensurate with forecast errors on average.Values below 1 indicate under-dispersion, while values above 1 indicate over-dispersion.
  • GenCast tends to produce flat rank histograms, consistent with ensemble members being statistically indistinguishable from ground truth values.A flat rank histogram is the ideal pattern for this diagnostic.
  • GraphCast-Perturbed is consistently overconfident, with spread-skill scores below 1 and U-shaped rank histograms.These patterns indicate under-dispersed ensembles whose uncertainty is too narrow.

7. Extreme weather events

GenCast provides greater decision value than ENS for rare extreme events, including heatwaves and tropical cyclones, across multiple cost-loss settings and forecast lead times.

  • 7.1. Local surface extremes: 1.9× better REV than ENS is achieved for a 7-year-high temperature tomorrow at a cost/loss ratio of 0.05.GenCast retains a consistent REV advantage across other cost/loss values and at 5- and 7-day lead times.
  • 7.1. Local surface extremes: The evaluation measures local temperature extremes, cyclone presence, spatially pooled temperature, and aggregated wind power using REV or relative CRPS.Figure 4 varies lead time for event-based REV and pooling-region size for spatially aggregated CRPS.
  • 7.1. Local surface extremes: GenCast significantly improves Brier skill over ENS for extreme temperature and wind events across tested percentile thresholds and lead times.The exception is low mean sea-level pressure, where some improvements are not significant.
  • 7.2. Tropical cyclones: GenCast yields 11%, 38%, and 106% higher REV than ENS for cyclone preparation at 1-, 3-, and 5-day lead times, respectively.The comparison uses a cost/loss ratio of 0.01 and GenCast maintains greater REV across other ratios and lead times up to 7 days.
  • 7.2. Tropical cyclones: GenCast’s extreme-weather forecasts can provide substantial value when deciding how to prepare for tropical cyclones.The supported application is preparation decisions involving possible cyclone landfall.

8. Skill in predicting the joint distribution

GenCast is evaluated on whether its probabilistic forecasts capture coherent spatial weather structures and regional wind-power uncertainty. It generally outperforms ENS across pooled spatial targets and wind-power forecasts, while the simplified energy experiment retains a stated scope limitation.

  • Spatially pooled evaluation: GenCast improves max-pooled 2 m temperature CRPS by approximately 25% at short lead times, then approaches ENS performance after about 8 days.The improvement is consistent across spatial scales.
  • Spatially pooled evaluation: 98.1% of average-pooled and 97.6% of max-pooled CRPS targets favor GenCast over ENS across 5400 pooled verification targets.The targets span variables, levels, lead times, and spatial scales, with relative performance increasing at larger scales.
  • Spatially pooled evaluation: GraphCast-Perturbed outperforms ENS on only 23.9% of average-pooled and 7.0% of max-pooled CRPS targets.Its relative performance also degrades as pooling size increases.
  • Regional wind power forecasting: The wind-power experiment converts interpolated 10 m wind speeds into power at 5344 wind farms and aggregates farms within 120 km, 240 km, and 480 km regions.Power is computed using an idealised power curve and nominal capacity.
  • Regional wind power forecasting: Regional wind-power forecasts improve ENS CRPS by approximately 20% through 2 days and 10–20% from 2–4 days.Improvements remain statistically significant at p<0.05 through 10 days.

9. Conclusion

The results support MLWP-based probabilistic forecasting as a more skillful and faster alternative to ECMWF’s ENS, while GenCast produces stable long-range samples and calibrated uncertainty. Operational deployment still requires addressing resolution, training-data, computational, and data-assimilation constraints.

  • Conclusion: GenCast produces more skillful and faster probabilistic forecasts than ECMWF’s ENS, according to the reported results.The conclusion frames this as evidence for a broader role for MLWP in weather forecasting.
  • Conclusion: GenCast generates stable, skillful 15-day global forecasts despite training directly only on 12-hour prediction steps.The authors interpret this as suggesting realistic samples of weather trajectories.
  • Conclusion: Operational use requires higher-resolution scaling, training and evaluation on operational inputs, efficiency improvements, and continued NWP-based data assimilation.The diffusion sampler is computationally more expensive than an equivalent deterministic MLWP architecture.
  • Conclusion: GenCast’s results suggest generative AI can model high-dimensional, complex temporal distributions accurately and reliably enough for decision-making applications.The stated applications span weather-dependent settings rather than a single operational task.

A. Task definition and general approach

The paper formulates probabilistic weather prediction as modeling a joint future atmospheric-state distribution, then approximates it with an MLWP forecast model and NWP-based state inference. GenCast autoregressively samples 15-day trajectories from ERA5-based initial conditions.

  • Task definition: Probabilistic forecasting models the joint distribution of future atmospheric states conditioned on observations available at initialization.The forecast horizon is represented by T, while atmospheric states are defined on a spatial grid or mesh.
  • General approach: The paper’s central innovation is an MLWP-based forecast model combined with traditional NWP-based state inference.This separates learned future-state prediction from the procedure used to represent the current atmospheric state.
  • State inference: Initial atmospheric-state uncertainty is represented using ECMWF ERA5 and its Ensemble of Data Assimilations, rather than a newly learned state-inference system.The standard state-inference stage uses data assimilation to generate analysis-state ensembles.
  • State representation: GenCast represents weather with 84 × 720 × 1440 arrays containing surface and atmospheric variables on a 0.25° latitude–longitude grid.The atmospheric variables occupy 13 vertical pressure levels.
  • Trajectory sampling: GenCast samples trajectories autoregressively for T=30 steps, conditioning each state on the previous two states and repeating the process for multiple ensemble members.The 30 steps correspond to 15 days at 12-hour intervals, and repeated trajectories support Monte Carlo estimates.
  • Data and evaluation: The dataset covers 1979–2019, with 1979–2018 used for final training and 2019 reserved for evaluation after model choices were frozen.The development phase used 1979–2017 for training and 2018 for validation.

B.2. Data preprocessing.

The preprocessing and ensemble-initialization pipeline standardizes ERA5-based inputs, handles missing values and resolutions, and explores perturbations for ensemble spread. The added Gaussian-process perturbations are crude and can reduce diffusion-model skill.

  • Data handling: One ENS initialization for 2019-10-17 00:00 UTC was omitted because surface-variable forecasts could not be downloaded.Neighboring information was used for paired statistical tests.
  • Data handling: ERA5 SST NaNs over land are replaced during preprocessing with the minimum globally observed SST in a selected ERA5 subset.This creates finite SST inputs for land locations.
  • Data handling: ERA5 EDA perturbations are bilinearly interpolated from 0.5° to the model’s 0.25° grid before being added to ERA5 inputs.The interpolation aligns the initial-condition perturbations with GenCast’s forecast grid.
  • Variable specification: Table B1 distinguishes static, time-varying surface, and atmospheric variables, and marks which variables are predicted versus used only as input context.ECMWF names, short names, and parameter IDs provide the variable metadata.
  • Ensemble initialization: Deterministic ERA5 analyses collapse initial-condition uncertainty into paired point estimates, yielding no dispersion for deterministic models.This initialization worked relatively well for the diffusion model but is described as a crude approximation.
  • Gaussian-process perturbations: Gaussian-process perturbations use a 1200 km horizontal decorrelation scale and independently perturb selected variables, while applying the same perturbation across levels and input times.The perturbation scale was selected through CRPS sweeps over scale factors and decorrelation lengths.
  • Gaussian-process perturbations: The additional perturbations are not flow-dependent or designed to preserve physical invariants, and adding them generally reduces diffusion-model skill except at the smallest scale factors.They provide no additional diffusion-model skill over EDA perturbations alone.
  • Diffusion model: GenCast models p(X_t|X_t−1, X_t−2) with a diffusion network, samples a normalized residual, and reconstructs the weather state by adding the rescaled residual to the latest state.This learned conditional model supplies samples for the autoregressive forecast process.

D.4.1. Resolution training schedule

This section defines the evaluation metrics and decision framework used to assess probabilistic forecasts, including spread/skill, rank histograms, Brier scores, and relative economic value.

  • Resolution training schedule: The training procedure used a two-stage schedule: 2 million pre-training steps at 1° resolution followed by 64,000 fine-tuning steps at 0.25° resolution.Both stages used 32 TPUv5 instances; pre-training took a little over 3.5 days and fine-tuning just under 1.5 days.
  • Probabilistic scores: CRPS evaluates how well forecast marginal distributions represent the ground truth, with smaller values indicating better probabilistic forecasts.The traditional estimator is used because the evaluated NWP ensembles have finite size.
  • Spread/skill evaluation: Spread is the root mean estimate of ensemble variance, while skill is the ensemble-mean RMSE.These definitions support the spread/skill ratio used to diagnose ensemble dispersion.
  • Spread/skill evaluation: Under the perfect-forecast assumption, the spread/skill ratio is expected to be approximately 1; values below or above 1 indicate under- or over-dispersion when bias is small.Forecast bias can confound this interpretation.
  • Decision-based evaluation: Relative economic value compares expense savings from forecasts against climatology with the savings achievable under a perfect forecast.The cost-loss framework evaluates decisions across users with different preparation costs and event losses.

E.5.1. Cyclone evaluation

The cyclone evaluation uses probabilistic tracking and decision-based scoring to assess whether GenCast predicts cyclone existence, termination, and location risks. Comparisons account for tracker consistency and differing ground-truth cyclone rates.

  • Cyclone evaluation: The probabilistic cyclone evaluation captures cyclogenesis and extracts strike probabilities from forecast cyclone tracks.Strike probability measures the chance that a location experiences a cyclone passing within a specified distance at a given time.
  • Cyclone evaluation: The same TempestExtremes tracker is applied to ENS and GenCast, with ENS downsampled from 6-hourly to 12-hourly resolution for comparison.Forecast tracks are converted into 1° probability heatmaps based on the fraction of ensemble members predicting a cyclone centre in each cell.
  • Ground-truth differences: 23% more cyclones are tracked in HRES-fc0 than ERA5, but relative economic value normalizes against climatology and perfect-forecast expenses.This normalization is intended to support comparisons across the differing ground truths.
  • Cyclone results: GenCast significantly outperforms ENS across many cost-loss ratios at 1-day lead time and at smaller cost-loss ratios through 7 days.The comparison uses tracks evaluated against each model’s own ground truth.
  • Cyclone results: Against the shared HRES-fc0 ground truth, GenCast still significantly outperforms ENS at small cost-loss ratios and 3–5-day lead times.This setting gives ENS an advantage and introduces distribution-shift penalties for GenCast.

F.9.2. Pooled evaluation

Pooled evaluations show GenCast consistently outperforming ENS across spatial scales, while initialization choices and ensemble perturbations affect interpretation of comparative skill and dispersion.

  • Pooled CRPS: 98.1% of average-pooled CRPS cases favored GenCast over ENS, compared with 23.9% for GraphCast-Perturbed.The comparison covered 5400 pooled verification targets across variables, lead times, and spatial scales.
  • Pooled CRPS: 97.6% of max-pooled CRPS cases favored GenCast over ENS, compared with 7.0% for GraphCast-Perturbed.GraphCast-Perturbed performed especially poorly on max-pooling due to blurring.
  • Pooled CRPS: GenCast substantially outperformed ENS and GraphCast-Perturbed in surface-only 0.25° pooled evaluation, often improving as pooling size increased.This pattern was observed for surface variables evaluated at 0.25° resolution.
  • Regional wind power: GenCast’s regional wind-power CRPS advantage over ENS was statistically significant through 7 days for all pool sizes and through 10 days with some exceptions.Statistical significance was defined as p<0.05.
  • Initialization sensitivity: Removing EDA-perturbed initial conditions had little effect on CRPS and RMSE, but generally reduced GenCast’s dispersion quality after approximately 2–3 days.Deterministic ERA5 initialization improved spread-skill ratios during the first one or two forecast steps, which the authors expect is an evaluation artifact.
  • Initialization sensitivity: Using 00/12 UTC rather than 06/18 UTC initialization systematically improved metrics because ERA5 provided a longer assimilation look-ahead.The main evaluation used 06/18 UTC initialization as the more conservative approach.
Loading 2312.15796v2…