PaperlayerView exact revision on arXiv

Source-linked AI summary

Skillful joint probabilistic weather forecasting from marginals

Ferran Alet, Ilan Price, Andrew El-Kadi, Dominic Masters, Stratis Markou, Tom R. Andersson, Jacklynn Stott, Remi Lam, Matthew Willson, Alvaro Sanchez-Gonzalez, Peter Battaglia

arXiv:2506.10772v1cs.LGphysics.ao-ph

TL;DR

Probabilistic weather forecasts are needed to represent uncertain future conditions and support decisions involving both likely and dangerous scenarios. FGN uses learned model perturbations and constrained model ensembles, trained with CRPS, and substantially outperforms GenCast across marginal and spatial-structure forecast evaluations while producing skillful ensemble forecasts.

  • Problem

    Weather forecasting needs probabilistic ensembles because complex dynamics and partial observations prevent precise forecasts, yet many ML models have been deterministic.

    Source: §1 · p. 1

  • Method

    FGN models weather uncertainty with ensembles of independently trained, appropriately constrained models and learned parameter perturbations, optimizing per-location CRPS.

    Source: §1 · p. 2, §2.2.3 · p. 4

  • Results

    FGN outperforms GenCast on 99.9% of marginal CRPS targets and captures spatial dependencies, with better pooled CRPS in 99.9% of average-pooled and 99% of max-pooled cases.

    Source: §5.1 · p. 9, §5.2 · p. 9

  • Takeaways & Limitations

    FGN establishes a state-of-the-art approach for ensemble forecasting that captures joint spatial structure despite training on marginal forecast distributions.

    Source: §6 · pp. 13–14

    FGN forecasts can exhibit subtle mesh-related artifacts, especially in higher-frequency variables and some spectral outputs.

    Source: §6 · p. 13

Abstract

from arXiv · show

Machine learning (ML)-based weather models have rapidly risen to prominence due to their greater accuracy and speed than traditional forecasts based on numerical weather prediction (NWP), recently outperforming traditional ensembles in global probabilistic weather forecasting. This paper presents FGN, a simple, scalable and flexible modeling approach which significantly outperforms the current state-of-the-art models. FGN generates ensembles via learned model-perturbations with an ensemble of appropriately constrained models. It is trained directly to minimize the continuous rank probability score (CRPS) of per-location forecasts. It produces state-of-the-art ensemble forecasts as measured by a range of deterministic and probabilistic metrics, makes skillful ensemble tropical cyclone track predictions, and captures joint spatial structure despite being trained only on marginals.

1. Introduction

Probabilistic weather forecasting is essential because complex, partially observed dynamics make precise forecasts impossible, while FGN introduces a flexible CRPS-trained approach that models aleatoric and epistemic uncertainty separately and outperforms GenCast and ENS across broad benchmarks.

Motivation

  • Probabilistic forecasts capture ranges of plausible weather conditions, enabling decisions that prepare for likely outcomes and dangerous but less probable storms.Complex nonlinear dynamics and partial observations leave unobserved factors that can substantially affect future conditions.Source: §1 · p. 1

Related work

  • ML-based probabilistic weather forecasting remains newer than deterministic ML forecasting, with recent generative and hybrid approaches only beginning to compete with operational systems.GenCast was the first model reported to clearly outperform ECMWF’s ENS, though it is slower than some ML models and less readily adaptable beyond dense grids.Source: §1 · p. 1

Contributions

  • FGN comprehensively outperforms GenCast and ENS across deterministic and probabilistic metrics, with similar-to-better calibration, at least as strong extreme prediction, improved joint structure, and significantly better tropical-cyclone tracks (p < 0.05).The method is trained on ERA5 reanalysis and operational HRES initial conditions to optimize CRPS.Source: §1 · p. 2

Method overview

  • FGN models epistemic uncertainty with independently trained model ensembles and aleatoric uncertainty by conditioning forecasts on independently sampled low-dimensional noise.These mechanisms generate ensemble diversity through model perturbations during forward passes and across multiple models.Source: §1 · p. 2, Figure 1 · p. 2

2. Methods

FGN models joint weather-forecast uncertainty through functional variability, combining independently trained models with learned parameter perturbations while optimizing marginal CRPS. Its constrained architecture is designed to produce globally coherent ensemble variability, with autoregressive rollout fine-tuning improving training but not being essential.

2.1. Problem formulation

  • Forecasting targets samples from the joint distribution of T-step trajectories conditional on assimilated prior weather states, factorized using a second-order Markov assumption.Each state is sampled autoregressively conditional on the previous two states, and repeated sampling generates ensemble trajectories.Source: §2.1 · p. 3

2.2. Functional generative networks: skillful joint distributions through functional variability

  • FGN models both epistemic and aleatoric forecast uncertainty through functional variability, learning joint output distributions from marginal-only losses in a suitably constrained setting.The approach is proposed as simple and scalable for skillful joint distributions.Source: §2.2 · p. 3

2.2.1. Modeling epistemic uncertainty with multi-model ensembling

  • Epistemic uncertainty is approximated with deep ensembles of J=4 independently initialized models, using each model consistently across all timesteps of a trajectory.The ensemble approximation informally marginalizes posterior uncertainty about model parameters given the training dataset.Source: §2.2.1 · pp. 3–4

2.2.2. Modeling aleatoric uncertainty with learned functional perturbations

  • Aleatoric uncertainty is represented by independently sampling learned parameter-space functions for each ensemble member and timestep, creating structured variability through parameter sharing.This parallels stochastic physics while enabling spatially structured perturbations rather than spatially independent model noise.Source: §2.2.2 · p. 4

2.2.3. Reparameterization trick and CRPS marginal training

  • The reparameterization θ=θ*+Δ·ε separates deterministic mean parameters from covariance control, while inference composes model-level and noise-level uncertainty.Training uses the fair CRPS estimator, averaged over locations, variables, and levels, with N=2 samples and specified loss weighting.Source: §2.2.3 · pp. 4–5

2.3. Model Architecture

  • FGN uses a GenCast-like graph-transformer architecture, but each model has ∼180m parameters, latent dimension 768, 24 processor layers, and 6-hour forecast timesteps.A global 32-dimensional Gaussian vector enters shared conditional-normalization layers, constraining marginally trained outputs toward globally coherent variability.Source: §2.3 · p. 5

2.4. Auto-regressive training

  • FGN begins with single-step training and is finetuned using autoregressive rollouts up to 8 steps; rollout training helps performance but is not essential for skillful joint distributions.The loss is averaged across rollout steps with gradients propagated through the rollout.Source: §2.4 · p. 6

3. Experiments

Experiments train and evaluate FGN using ERA5 pre-training and operational HRES-fc0 fine-tuning, compare it with GenCast in the operational setting, and assess computational efficiency. FGN produces 15-day forecasts substantially faster than GenCast despite a larger model and finer timestep.

3.1. Training and evaluation data

  • FGN is pre-trained on ERA5, fine-tuned on operational HRES-fc0, validated on 2022, and tested on 2023 after freezing its architecture and training protocol.ERA5 provides a large historical reanalysis corpus, while HRES-fc0 supplies real-time operational data.Source: §3.1 · p. 6

3.2. Baseline: GenCast

  • GenCast is the state-of-the-art probabilistic medium-range ML weather baseline, and this study compares FGN with its operational WeatherNext Gen forecasts.The operational forecasts are fine-tuned on HRES-fc0 and evaluated using 2022 for validation and 2023 for testing.Source: §3.2 · pp. 6–7

3.3. Training and inference efficiency

  • 8 times faster: FGN generates a 15-day forecast than GenCast despite being larger and producing twice as many frames, because it uses one network pass instead of iterative diffusion refinement.A single trajectory takes just under 1 minute on one TPU v5p, and ensemble members can be generated in parallel.Source: §3.3 · p. 7

4. Evaluation Methodology

FGN is evaluated against HRES-fc0 in operationally realistic settings using marginal, joint-distribution, and tropical-cyclone-track metrics. The methodology combines deterministic, probabilistic, calibration, economic-value, spatial-dependence, cross-variable, and physical-plausibility assessments.

4. Evaluation Methodology

  • Models are initialized and evaluated against HRES-fc0 at 00z, 06z, 12z, and 18z, with 56-member ensembles for both models.For FGN, each model seed generates 14 members; metrics and statistical significance testing follow Price et al. (2025).Source: §4 · p. 7

4.1. Evaluating marginal forecast distribution

  • Marginal forecasts are assessed using ensemble-mean RMSE, biased ensemble CRPS, spread-skill ratios, and Relative Economic Value across variables, levels, thresholds, and cost-loss ratios.Spread-skill ratios near 1 indicate calibration, while REV is normalized between climatological (0) and perfect (1) forecasts using 1993–2016 ERA5 climatology.Source: §4.1 · pp. 7–8

4.2. Evaluating correlation structure of the joint distribution

  • Joint-distribution quality is evaluated with spatially pooled CRPS, CRPS for derived 10m wind and 300–500hPa geopotential differences, and spherical-harmonic power spectra.Spatial pools range from 120 km to 3828 km; pooled CRPS tests spatial dependence, derived quantities test cross-variable dependence, and spectra diagnose physical plausibility and blurring.Source: §4.2 · p. 8

4.3. Evaluating tropical cyclone tracks

  • Tropical-cyclone tracks are evaluated by ensemble-mean position error and track-probability REV using IBTrACS truth, Tempest Extremes tracking, and 1° probability heatmaps.Track probabilities are computed as the fraction of ensemble members placing a cyclone center in each cell, while mean-track existence requires at least 50% of members to retain a track.Source: §4.3 · pp. 8–9

5. Results

FGN outperforms GenCast on marginal and joint-distribution forecast skill while maintaining good calibration, and it delivers more accurate and better-calibrated tropical-cyclone track predictions. These gains persist across spatial and inter-variable evaluations, though some fields show minor high-frequency artifacts.

5.1. Results on marginal predictive skill and calibration

  • FGN beats GenCast on marginal CRPS for 99.9% of targets, averaging 6.5% improvement, while achieving 100% win coverage and 5.8% average improvement in ensemble-mean RMSE.FGN maintains spread-skill scores close to 1 across lead times, matches GenCast on extreme 2m-temperature REV, and outperforms it for extreme 10m-wind REV.Source: §5.1 · p. 9, Figure 2 · p. 10, Figure 2b

5.2. Results on structure of the joint distribution

  • FGN improves average-pooled and max-pooled CRPS by 8.7% and 7.5%, winning 99.9% and 99% of evaluations, respectively, indicating captured spatial dependencies.It also improves derived 10m-wind and z300 −z500 CRPS by 4.8% and 7.8% on average, with short-lead gains reaching 10.4% and 15.6%; spectra generally match truth and avoid high-frequency power loss, although some smoother fields contain minor artifacts.Source: §5.2 · pp. 9, 12, Figure 3 · p. 11

5.3. Results on tropical cyclone tracks

  • FGN’s ensemble-mean cyclone track is approximately 24 hours more accurate than GenCast at 3–5.5-day leads, while its track-probability REV is better across relevant cost/loss ratios.A 12-hour-step FGN model still beats GenCast from 2-day leads onward, showing that most improvement reflects forecast accuracy rather than timestep; probability gains are strongest for smaller cost/loss ratios and longer leads.Source: Figure 4 · p. 12, §5.3 · p. 12

6. Discussion

FGN substantially improves ML-based probabilistic weather forecasting, outperforming GenCast in marginal and joint-distribution-dependent skills while efficiently modeling epistemic and aleatoric uncertainty. Its constrained stochastic architecture captures spatial dependence despite marginal-only CRPS training, though artifacts and unstable seeds remain deployment concerns.

Overall results

  • FGN outperforms GenCast in marginal forecast skill, including extreme weather, and in joint-distribution-dependent metrics such as tropical cyclone tracks.It also generates forecasts more efficiently than GenCast and supports autoregressive training for longer lead times.Source: §6 · p. 13

Extensions

  • An experimental cyclone-specific FGN extension reportedly improves cyclone-track, intensity, and size prediction beyond the tracker-based results presented above.The extension demonstrates that FGN can be broadened beyond dense, gridded analysis data.Source: §6 · p. 13

Joint spatial structure

  • Even a stripped-down FGN with one model seed, 512 latent dimensions, 16 layers, and marginal-only CRPS training outperforms the prior state-of-the-art almost across the board.This demonstrates that marginal optimization can still yield strong joint spatial forecasts, despite CRPS alone providing no guarantee of good joint distributions.Source: §6 · p. 13
  • FGN’s joint-structure capability likely arises because a shared 32-dimensional stochastic vector drives conditional normalization across the entire output, imposing distributional constraints that encourage modeling dependencies.The effective stochastic dimensionality is independent of output-grid resolution and can extend to dense or sparse predictions.Source: §6 · p. 13

Limitations

  • FGN can produce subtle mesh-shaped honeycomb artifacts and spectral kinks, possibly from over-constraining output covariance; higher-dimensional noise may mitigate them.Artifacts are most visible in some high-frequency variables, while spectral kinks in smoother variables may remain visually imperceptible.Source: §6 · pp. 13–14, Figure 5 · p. 14
  • Poor model seeds can cause unstable forecast rollouts, so trained models require careful validation before deployment.One unstable seed was detected through averaged validation-year spectra and resolved by retraining with a different seed.Source: §6 · p. 14

Appendix · A.1. Data

This appendix section describes the ERA5 and HRES-fc0 datasets, including preprocessing details added for FGN finetuning.

A.1. Data

A.1.1. ERA5

The FGN pre-training dataset uses a subset of ECMWF’s ERA5 reanalysis, covering global weather across selected atmospheric and surface variables from 1979 to 2018 at six-hour intervals.

ERA5

  • The ERA5 archive provides global weather reanalysis from 1959 onward at hourly resolution, using ECMWF’s HRES model and 4D-Var assimilation of observations and forecasts.The deterministic archive has native 0.281 25◦ latitude/longitude resolution.Source: §A.1.1 · p. 20
  • FGN’s dataset samples selected ERA5 variables on 13 WeatherBench pressure levels from 1979-01-01 to 2018-01-15 at 6-hour intervals.Most variables are subsampled, while total precipitation is accumulated over the preceding six hours.Source: §A.1.1 · p. 20
  • Sea-surface-temperature values over land are replaced with the minimum globally observed sea-surface temperature to remove ERA5 NaNs.This preprocessing is applied to SST training and evaluation data.Source: §A.1.1 · p. 20

A.1.2. HRES-fc0

HRES-fc0 supplies ECMWF HRES zero-step forecasts for finetuning and evaluation, using observations within ±3 hours and requiring preprocessing choices for consistency with ERA5. The setup excludes HRES-fc0 total precipitation as an input and retains ERA5 precipitation for training and evaluation.

Dataset

  • HRES-fc0 provides ECMWF HRES zero-step forecasts at 00z, 06z, 12z, and 18z, assimilated with the latest NWP model and observations from ±3 hours.The dataset resembles ERA5 but reflects the latest ECMWF NWP assimilation at each forecast time.Source: §A.1.2 · p. 20

Preprocessing

  • ERA5 sea-surface-temperature NaNs are matched in HRES-fc0 by imputing the corresponding locations with the minimum sea-surface temperature.This removes the datasets’ inconsistent land representation, because HRES-fc0 uses a placeholder rather than NaNs.Source: §A.1.2 · p. 20

Variables

  • The datasets use ECMWF variables partitioned into predicted outputs and input-only context, including static, surface, and atmospheric properties.Table A.1 documents variable names, short names, parameter IDs, types, and model roles.Source: Table A.1 · p. 21

Preprocessing

  • FGN predicts total precipitation without using it as input and retains ERA5 precipitation for finetuning and evaluation because HRES-fc0 precipitation is zero at initialization.HRES-fc0 would also require forward accumulation, whereas ERA5 accumulates precipitation over the preceding interval.Source: §A.1.2 · p. 22

A.2. Model Training

FGN is trained through four stages that progressively increase data resolution and introduce autoregressive HRES-fc0 training. The pipeline uses consistent batching, resolution-specific preprocessing, and an adjusted graph-transformer architecture for high-resolution inputs.

Training schedule

  • FGN training progresses from 1° ERA5 inputs at 12-hour resolution to 0.25° ERA5 and autoregressive HRES-fc0 inputs at 6-hour resolution.The schedule comprises 400000, 100000, 32000, and 18000 steps across the four stages, with the final stage including 8000 1AR, 4000 2AR, and 1000 each at 3–8AR.Source: §A.2 · p. 22

Data processing

  • The 6-hourly data are subsampled with stride two to generate 12-hourly data, while accumulated variables are accumulated over the longer interval; ERA5 is downscaled from 0.25° to 1° spatially.Source: §A.2 · p. 22

Architecture adaptation

  • The graph-transformer mesh is refined from five levels for 1° training to six for 0.25° training, with shared weights and encoder message normalization by 4.The architecture is updated to accept and produce higher-resolution data while preserving the incoming signal scale in Stage 3.Source: §A.2 · p. 22

A.3. Architecture

FGN largely follows GenCast’s architecture but uses a single-pass forecast conditioned only on the two prior weather states, alongside modified normalization, capacity, global-feature, and grid-to-mesh designs.

Conditioning on input physical states

  • FGN generates forecasts in one forward pass, conditioning only on the two prior weather states rather than noisy targets required by GenCast’s iterative diffusion refinement.This conditioning matches GraphCast’s use of prior states without noisy target inputs.Source: §A.3 · p. 22

Shared encoder for conditional layer-norm layers

  • FGN’s conditional layer normalization encodes a 32-element noise vector with one matrix multiplication, which was later found to aid training optimization.Unlike GenCast’s Fourier features and 2-layer MLP for noise-level encoding, FGN directly produces a 32-element conditioning vector.Source: §A.3 · pp. 22–23

Capacity

  • FGN increases core capacity to hidden and output sizes of 768, 6 attention heads, and 24 transformer layers, while reducing edge-feature embedding sizes to 32 without harming performance.The reduction applies to grid-to-mesh and mesh-to-grid edge-feature embeddings, saving compute and memory for graphs with O(10^5) edges.Source: §A.3 · p. 23

Global features

  • FGN broadcasts sine and cosine of year progress across the mesh as well as the grid, concatenating them with mesh input features.GenCast and GraphCast broadcast these spatially constant seasonal features across the grid and provide them to the encoder.Source: §A.3 · p. 23

Encoder from the grid to mesh

  • FGN removes receiver-mesh-node conditioning from the grid-to-mesh message function with no performance cost, saving compute and memory.The original bipartite GNN message function conditions on both sender grid and receiver mesh features for every edge.Source: §A.3 · p. 23

A.4. Ablations … A.6. Forecast visualizations

The supplementary results show that FGN’s gains persist under controlled ablations and operational comparisons, while autoregressive training, scaling, and ensembling further improve skill and rollout stability. Additional diagnostics cover extreme events, precipitation, spectra, and representative forecast visualizations.

A.4. Ablations

  • FGN beats GenCast even when controlling model capacity, ensemble size, and rollout-training steps, while preserving spatial and inter-variable correlations.The corresponding evaluations include pooled fields, derived quantities, and power spectra.Source: §A.4 · p. 24, Figure A.3 · p. 27
  • The single-step FGN model exhibits rare unstable, artifact-dominated late rollouts at roughly 0.1% prevalence at 15-day lead times, whereas autoregressively trained models show none.This observation suggests that autoregressive training improves rollout robustness and stability.Source: §A.4 · p. 24
  • 99.8% of targets favor autoregressively trained FGN over GenCast, increasing average CRPS improvement from 2.7% to 4%.The controlled model matches GenCast’s timestep, seed count, and capacity, while autoregressive training extends to eight rollout steps.Source: §A.4 · p. 24, Figure A.2 · p. 26
  • Cyclone-track REV improves successively with autoregressive training, larger models, and model ensembling, with the full configuration performing best.The single-step FGN variant matching GenCast’s size performs as well as or better than GenCast.Source: §A.4 · p. 24, Figure A.4 · p. 28

A.5.1. Comparisons with ENS

  • 99.3% of targets favor FGN over ENS, with average CRPS improvement of 10.8% and a maximum of 27.7% across 2023 operational initializations.Before and after the ENS resolution change, FGN outperforms ENS on more than 99% of targets using the fair estimator, with nearly indistinguishable results from the biased setting.Source: §A.5.1 · p. 28, Figure A.8 · p. 32

A.5.2. Extended Results

  • FGN outperforms GenCast on ensemble-mean RMSE in 100% of cases, with statistically significant improvements averaging 5.8% and reaching 18%.The extended evaluation also finds significantly higher REV for FGN on extreme wind, temperature, and pressure forecasts, with only a few longer-lead exceptions.Source: §A.5.2 · pp. 28, 36, Figure A.11 · p. 35
  • The supplementary evaluations extend FGN’s advantage to 12-hour and 24-hour precipitation using RMSE, CRPS, and SEEPS, and to spectra for 10 representative variables.SEEPS is computed from ensemble-mean categorical precipitation forecasts, while the spectral results supplement the main paper.Source: §A.5.2 · pp. 28, 36, Figure A.12 · p. 35, Figure A.13 · p. 36

A.6. Forecast visualizations

  • Forecast visualizations cover 10 representative variables, selecting the 2023 initialization with median CRPS for each variable–lead-time combination and plotting the first ensemble member.Each figure therefore shows a snapshot from a different forecast trajectory at each lead time.Source: §A.6 · p. 43

Related papers

Loading 2506.10772v1…