PaperlayerView exact revision on arXiv

Source-linked AI summary

Proper Scoring Rule-based Diffusion for Probabilistic Weather Forecasting

Joonhyeong Park, Giung Nam, Hyungi Lee, Kyunghyun Cho, Byoungwoo Park, Juho Lee

arXiv:2609.38632v1cs.LG

TL;DR

Context-only CRPS forecasting becomes harder as uncertainty grows with longer horizons. This paper adds auxiliary conditional denoising tasks to a stochastic predictor while retaining one-pass ensemble generation. Experiments report improved one-step forecasts, with larger gains at higher uncertainty and benefits extending to high-dimensional global forecasting.

  • Problem

    Context-only probabilistic forecasters can struggle as forecast uncertainty grows, motivating ways to learn predictive distributions more effectively.

    Source: §1 · p. 1

  • Method

    A stochastic predictor learns conditional distributions for auxiliary denoising tasks using context, a corrupted future, and its noise level, with CRPS training as the endpoint-only special case.

    Source: §1 · p. 2

  • Results

    The framework improves one-step forecasting over standard CRPS training across architectures and forecast horizons, with larger gains under high uncertainty.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The gains extend to high-dimensional global forecasting under training from scratch and fine-tuning, while evaluations under distribution shift and in tropical cyclone cases suggest potential generalization benefits.

    Source: §1 · p. 2, §5 · p. 10

Abstract

from arXiv · show

Recent probabilistic weather forecasters train stochastic predictors with the continuous ranked probability score (CRPS) to generate each ensemble member in a single forward pass. These models learn the predictive distribution from the forecast context alone, which becomes difficult at longer forecast horizons where uncertainty is high. To learn the predictive distribution more effectively, we introduce auxiliary conditional denoising tasks that predict the same future state from the context and its corrupted version, which provides partial future information that can reduce prediction ambiguity. Building on distributional diffusion models, we learn the conditional distributions of these tasks with a single stochastic predictor by minimizing a proper scoring rule across noise levels. At inference, the predictor can still generate each ensemble member in a single forward pass at the fully corrupted endpoint. Standard CRPS training is recovered as the endpoint-only special case of our formulation, so our framework extends existing CRPS-based forecasters with only additional conditioning inputs. Controlled experiments show that the auxiliary tasks improve one-step forecasting across architectures, with larger gains at longer forecast horizons. The gains extend to high-dimensional global weather forecasting under both training from scratch and fine-tuning, along with improved calibration and potential benefits for generalization under distribution shift.

1. Introduction

Probabilistic forecasting must represent uncertainty, but iterative diffusion sampling is costly and context-only CRPS training becomes harder as forecast uncertainty rises. The paper adds auxiliary conditional denoising tasks to one-pass stochastic forecasting and reports stronger performance, especially at longer horizons.

1. Introduction

  • Context-only CRPS training can become harder as forecast uncertainty grows, while iterative diffusion sampling requires repeated model evaluations for each ensemble member.The motivation is one-step forecasting several days ahead, where CRPS-trained models tend to overfit.Source: §1 · p. 1
  • The paper frames the approach as learning auxiliary tasks that predict the same future using partial future information alongside forecast context.The corrupted future is uninformative at the fully corrupted endpoint, and its noise level controls how much future information it provides.Source: §1 · p. 2
  • The framework trains a stochastic predictor on the same future conditioned on context, a corrupted future, and its noise level, with CRPS training as the endpoint-only case.At the fully corrupted endpoint it generates ensemble members in one forward pass; Figure 1 contrasts the added conditioning with diffusion and CRPS approaches.Source: §1 · p. 2, Figure 1 · p. 2
  • Auxiliary conditional denoising improves one-step forecasting across architectures and horizons, with larger gains under high uncertainty.The reported benefits extend to high-dimensional global forecasting under training from scratch and fine-tuning.Source: §1 · p. 2

2. Preliminaries

The preliminaries define probabilistic weather forecasting as modeling future states conditioned on forecast context and describe scoring-rule and diffusion-based approaches. They also contrast iterative diffusion sampling with one-pass CRPS ensemble generation.

2. Preliminaries

  • Probabilistic forecasting models the distribution of the next atmospheric state given context, and autoregressive forecasting factors trajectory distributions into one-step conditionals.The context includes the current state and may also include previous states, static variables, and forcing variables.Source: §2 · p. 3
  • A scoring rule assigns a scalar loss to a predictive distribution and outcome; it is proper when expected loss is minimized by the true distribution and strictly proper when that minimizer is unique.The paper adopts lower scores as better.Source: §2 · p. 3
  • CRPS is a common weather-forecasting score and can train predictive distributions whose ensemble members are generated in a single forward pass.With multiple ensemble members, the paper estimates CRPS using fair CRPS.Source: §2 · p. 3
  • Diffusion-based forecasters iteratively denoise noise into future states conditioned on context, with repeated denoiser evaluations making ensemble generation costly.The denoiser is conditioned on context, the corrupted state, and noise level; coarse time discretization can degrade sample quality.Source: §2 · p. 3
  • In the 3-day forecasting ablation, endpoint-only training closely matches the CRPS baseline, while lowering t_min broadens auxiliary tasks and mitigates performance degradation.Figure 4 reports validation nCRPS, spread–skill ratio, and test nCRPS change relative to the CRPS-trained baseline.Source: Figure 4 · p. 4

3. Proper Scoring Rule-based Diffusion Forecaster

The section motivates auxiliary denoising with context-only overfitting at longer horizons, then formalizes noise-indexed conditional prediction under a proper scoring rule. It relates standard CRPS to the endpoint case and describes training that adds corrupted-future conditioning while retaining one-step ensemble generation.

3.1. Motivating Observations: Challenges of Context-Only Forecasting

  • At 3 and 5 days, validation loss initially improves and then deteriorates while training loss keeps decreasing, indicating overfitting in context-only CRPS training.At 12 hours and 1 day, validation loss generally improves; similar patterns appear across diverse architectures.Source: §3.1 · p. 4
  • Jointly learning predictions of the same future with partial future information may reduce ambiguity and provide shared supervision, with possible regularization benefits.The diffusion task family varies the noise level to control supplied information, and its independent-noise endpoint recovers context-only forecasting.Source: §3.1 · pp. 4–5
  • Lowering the training noise range improves one-step 3-day forecasting and mitigates overfitting, while endpoint-only training closely follows the context-only baseline.Inference uses a single forward pass at t=1 (NFE = 1).Source: §3.1 · p. 5

3.2. Proper Scoring Rule-based Diffusion for Auxiliary Denoising Tasks

  • A shared stochastic predictor learns conditional distributions across noise levels by minimizing a proper scoring-rule objective, with the corrupted future and noise level as inputs.The objective uses a generalized energy score; the study uses uniform noise-level weighting.Source: §3.2 · p. 5
  • At the independent-noise endpoint, the learned predictive distribution matches the context-only target and ensemble members can be sampled in one forward pass.The text also states that multi-step sampling recovers the same target distribution.Source: §3.2 · p. 5

3.3. Revisiting CRPS Training from a Diffusion Perspective

  • Standard CRPS training is the endpoint-only special case; the proposed formulation generalizes it with auxiliary denoising tasks at t<1.The CRPS equivalence uses the component-wise score with λ=β=1 and training restricted to t=1.Source: §3.3 · p. 6
  • Training adds a sampled noise level and corrupted future as conditioning inputs relative to standard CRPS training.The algorithm retains stochastic ensemble generation and updates the predictor using a fair CRPS loss.Source: §3.3 · p. 6

4. Experiments

The experiments test auxiliary conditional denoising across forecast horizons, architectures, and training regimes. DDM improves direct and rollout forecasting, with broader gains in high-dimensional forecasting and promising calibration and generalization results.

4.1. Pilot Experiments: Analysis of Auxiliary Conditional Denoising Tasks

  • At NFE = 1, DDM consistently outperforms standard CRPS training in direct forecasts, with larger gains at 3d and 5d than at 12h and 1d.The authors suggest richer auxiliary supervision may help under high uncertainty; broadening the noise range also improves 3d forecasting.Source: §4.1 · pp. 7–8
  • For autoregressive rollouts, DDM gives modest gains at a 12h rollout interval and a clear advantage at 3d, while alleviating longer-interval underdispersion.SSRs stay closer to 1, suggesting better calibration; Figure 6 analyzes SSR.Source: Figure 6 · p. 7, §4.1 · p. 8

4.2. Scaling to High-Dimensional Global Weather Forecasting

  • Across representative variables, DDM improves CRPS and ensemble-mean RMSE for MOSAIC and improves or matches them for U-Cast at NFE = 1 per rollout step.Table 3 covers three variables at 1-, 3-, and 10-day lead times.Source: §4.2 · pp. 8–9, Table 3 · p. 8
  • Across all 84 MOSAIC forecast channels, DDM achieves lower CRPS in 787 of 840 channel–lead pairs (93.7%); for U-Cast, gains are modest at 82.0% of pairs, where objectives differ only during brief fine-tuning.Figure 7 visualizes the MOSAIC comparison across 10-day rollouts.Source: §4.2 · p. 9, Figure 7 · p. 9
  • Across all 84 channels and 1-, 3-, and 10-day lead times, DDM shifts SSR distributions toward 1 and generally brings spread–skill curves closer to the 1:1 line.Figure 8 presents the SSR distributions and binned spread–skill plots.Source: §4.2 · p. 9, Figure 8 · p. 9

4.3. Generalization under Distribution Shift and Extreme Weather

  • On HRES-fc0, DDM beats IFS-ENS in 86.8% of channel–lead pairs, versus 58.4% for CRPS-trained MOSAIC; both models deteriorate under the shift.For ERA5 evaluation, the corresponding shares are 100% for DDM and 99.5% for CRPS-trained MOSAIC.Source: §4.3 · p. 10, Table 4 · p. 10, Figure 9 · p. 10
  • In the Hurricane Ian case study, DDM produces track distributions closer to the IBTrACS best track for both architectures.Figure 9 shows the ensemble tracks and uncertainty cones enclosing 66% of the track distribution.Source: §4.3 · p. 10, Figure 9 · p. 10

5. Conclusion

The paper extends CRPS-based one-step forecasting with auxiliary conditional denoising, reporting improved forecasting and potential generalization benefits while retaining one-step ensemble generation. It positions this approach alongside scoring-rule and diffusion forecasters, and notes that theoretical guarantees remain unestablished.

5. Conclusion

  • The framework adds auxiliary conditional denoising to proper-scoring-rule training, recovers standard CRPS training as an endpoint-only case, and improves one-step forecasting, especially under high uncertainty.The conclusion says the approach uses only minor conditioning changes and retains one-step ensemble generation.Source: §5 · p. 10
  • The experiments report benefits across architectures, training regimes, and high-dimensional forecasting, with distribution-shift and tropical-cyclone evaluations suggesting potential generalization benefits.The conclusion does not establish theoretical guarantees for these benefits.Source: §5 · p. 10

A. Related Work

  • Unlike one-step methods that use separate CRPS fine-tuning after deterministic transport learning, this formulation applies a scoring rule across noise levels during training.The related-work discussion says it requires only additional conditioning inputs to existing CRPS-based forecasters.Source: A. Related Work · p. 14
  • Intermediate denoising tasks provide shared-parameter supervision even when inference uses only the fully corrupted endpoint, linking the approach to denoising autoencoders and multitask learning.The paper presents implicit regularization as a possible explanation, not an established guarantee.Source: A. Related Work · p. 14

B.1. Sampling Procedures

Sampling can use one step or a finite grid of steps, and under exact conditional-distribution recovery it recovers the target predictive distribution for any finite step count. At the fully corrupted endpoint, the formulation connects to standard CRPS training.

B.1. Sampling Procedures

  • The sampler updates its state from time t to s by mixing the clean prediction with the current state using the ratio s/t.The context stays fixed during sampling; the cited procedure uses one-step sampling primarily (NFE = 1).Source: §B.1 · pp. 14–15
  • Under exact joint conditional recovery, DDM sampling recovers the target predictive distribution for any finite number of steps, including one-step sampling.The result also holds for any corruption-noise prior independent of the context and target.Source: §B.1 · p. 15

B.2. CRPS at the Distributional Diffusion Endpoint

  • When predictive distributions are independent of endpoint corruption noise, endpoint-only DDM training reduces to standard CRPS training.If predictions depend on that noise, the objectives need not coincide, though their population-optimal forecast marginals still match.Source: §B.2 · pp. 15–16

C.1. Data and Preprocessing

Experiments use two ERA5 grid resolutions and fixed train, validation, and test periods. Inputs are standardized, while target normalization and training weights vary with prediction parameterization and resolution.

C.1. Data and Preprocessing

  • Experiments use 2.8125° grids at 64 × 128 for pilot studies and 1.5° grids at 121 × 240 for high-dimensional studies, with 6-hourly ERA5 data.Training covers 1979–2017, validation 2018–2019, and testing 2020.Source: §C.1 · p. 16
  • Dynamic input states are standardized channel-wise, while target normalization depends on whether a model predicts the future state or its residual from the current state.For residual prediction, the target uses the training-period standard deviation of the future-minus-current state for that channel and forecast lead.Source: §C.1 · p. 16
  • Spatially aggregated metrics use normalized cosine-latitude weights to account for unequal grid-cell areas.The spatially and channel-averaged training loss combines grid-point losses with these area weights and channel weights.Source: §C.1 · p. 16
  • The pilot setting uses nine uniformly weighted channels, whereas the 1.5° setting uses 84 dynamic channels with pressure-level and surface-field weights.The high-resolution setting assigns weight 1.0 to T2M and 0.1 to the other surface and near-surface fields.Source: §C.1 · p. 16, Table 5 · p. 17

C.2. Evaluation Metrics

The evaluation reports CRPS, ensemble-mean RMSE, and spread–skill ratio, with normalized aggregates and corrections for finite ensemble size. DDM adaptations add corrupted-target and noise-level conditioning to the forecasting backbones.

C.2. Evaluation Metrics

  • The evaluation includes fair CRPS, ensemble-mean RMSE, and spread–skill ratio, plus normalized CRPS and RMSE aggregates.Predictions are evaluated in state space after reconstructing states, and spatial aggregation uses normalized cosine-latitude weights.Source: §C.2 · pp. 17–18
  • The spread–skill ratio uses a finite-ensemble correction that accounts for sampling variability of the ensemble mean.When aggregated across channels, channel-wise ratios are averaged.Source: §C.2 · p. 18

C.3. Architectural Modifications

  • DDM adaptations condition forecasting backbones on the corrupted target and noise level while retaining their prediction parameterization and ensemble-generation mechanism.The architectural changes expand input layers and add time-conditioning modules.Source: §C.3 · p. 18

C.4. Experimental Details at 2.8125 degrees

The pilot compares four backbones under matched optimization settings across four forecast leads. DDM changes are limited to additional conditioning inputs and time-conditioning modules.

C.4. Experimental Details at 2.8125 degrees

  • The pilot study compares Swin, U-Net, FGN, and FCN3, training each backbone under both objectives with matched optimization settings.The four models include two general-purpose architectures and two CRPS-based probabilistic weather forecasters.Source: §C.4 · p. 18, Table 7 · p. 18
  • Switching from CRPS to DDM expands the input layers for the corrupted target and adds time-conditioning modules.The prediction targets and noise-entry points of the backbones are summarized in Table 7.Source: §C.4 · p. 18, Table 7 · p. 18
  • Each pilot run targets one of four forecast leads—12h, 1d, 3d, or 5d—and draws two ensemble members per training step.Widths and depths are held fixed across objectives; evaluation uses EMA weights.Source: §C.4 · p. 19, Table 8 · p. 19

C.5. Experimental Details at 1.5 degrees

The 1.5° experiments adapt MOSAIC and U-Cast while retaining their original prediction parameterizations and ensemble-generation mechanisms. In pilot training, validation nCRPS at longer horizons worsens even as training loss falls, with stronger overfitting at five days.

C.5. Experimental Details at 1.5 degrees

  • MOSAIC and U-Cast retain their original prediction parameterizations and ensemble-generation mechanisms, with DDM adding conditioning on the corrupted target and its noise level.This conditioning increases parameter counts by 3.23% for MOSAIC and 1.06% for U-Cast.Source: §C.5 · p. 19, Table 9 · p. 19
  • The CRPS and DDM variants use matched training settings; DDM uses spherical Gaussian corruption with a heat-kernel schedule at κ0 = 0.01.The choice follows pilot results, and the backbones follow their published recipes with the specified stage and ensembling adjustments.Source: §C.5 · p. 19

D.1. Training Dynamics of Context-Only CRPS Training

  • At 3- and 5-day horizons, validation nCRPS rises after an early minimum while training loss continues to fall, with larger overfitting at 5 days across four architectures.At 12 hours and 1 day, validation nCRPS instead decreases and levels off.Source: §D.1 · p. 19
  • Training and validation losses use different normalizations for FGN, Swin, and U-Net, so overfitting is assessed from their trends rather than their absolute scales.FCN3 uses state normalization for both losses.Source: §D.1 · p. 19

D.2. Proof-of-Concept Experiment in Section 3.1

The proof-of-concept compares CRPS training with auxiliary denoising tasks across two architectures, then extends the pilot analysis to additional backbones and rollouts. Auxiliary tasks improve one-step metrics and help preserve spread–skill calibration, while gains generally persist in autoregressive rollouts.

D.2. Proof-of-Concept Experiment in Section 3.1

  • The proof-of-concept evaluates direct 3-day forecasts at 2.8125° using FCN3 and FGN, with FGN extended to 100 training epochs.Checkpoints are selected by validation nCRPS using 240 initializations from 2018–2019 and 50 ensemble members.Source: §D.2 · p. 20
  • At t_min = 0, FCN3 nCRPS and nRMSE fall 8.10% and 6.55%, while FGN falls 9.96% and 8.55%, relative to their CRPS baselines.For both architectures, lowering t_min progressively improves both metrics; endpoint-only training closely matches CRPS training.Source: §D.2 · p. 20
  • Broader training-noise ranges mitigate post-improvement validation nCRPS deterioration and keep SSR closer to 1, although underdispersion remains, for both FCN3 and FGN.The pattern is consistent with a regularizing effect of jointly learned auxiliary tasks through shared parameters.Source: §D.2 · p. 20

D.3.1. Direct Forecasting and Autoregressive Rollout Performance

  • DDM generally improves nCRPS and nRMSE across four backbones in one-step forecasts, with larger gains at longer rollout intervals that partly persist during rollouts.The cited passage reports results in Table 13 and extended FGN and FCN3 rollout evaluations in Tables 14 and 15.Source: §D.3.1 · p. 21

D.3.2. Noise Prior and Schedule Ablation

The appendix examines noise choices, global forecast performance, robustness to an ERA5–HRES shift, and cyclone tracking. Results favor DDM over CRPS training across forecast metrics and settings, while the best noise prior and schedule vary by horizon and backbone.

D.3.2. Noise Prior and Schedule Ablation

  • Spherical Gaussian noise is evaluated alongside independent grid noise, and the heat-kernel schedule attenuates fine-scale target information more rapidly than linear interpolation.The schedule’s scale dependence strengthens with larger κ0.Source: §D.3.2 · pp. 21–22
  • Noise and schedule rankings depend on rollout interval and backbone: heat-kernel scheduling generally gives the lowest nCRPS at 12 hours and 1 day, while Grid or SpG leads at 3 and 5 days.All tested DDM configurations outperform the CRPS baseline at the two longer intervals; the selected configuration uses the lowest validation nCRPS.Source: §D.3.2 · pp. 22–23, Table 16 · p. 26

D.4.1. Forecast Performance

  • DDM achieves lower CRPS in 93.7% of MOSAIC and 82.0% of U-Cast channel–lead pairs; ensemble-mean RMSE improves in 92.0% and 81.0%, respectively.Improvements span surface variables and most tropospheric levels, while degradations concentrate at the lowest-pressure levels.Source: §D.4.1 · p. 23, Figure 13 · p. 27
  • For MOSAIC, NFE = 2 lowers CRPS in 803 of 840 channel–lead pairs, versus 787 pairs with NFE = 1, at the cost of one additional model evaluation per member and rollout step.The same DDM checkpoint is evaluated with both NFE settings.Source: §D.4.1 · p. 23, Figure 17 · p. 29

D.4.2. Robustness to Distribution Shift

  • Under the ERA5–HRES shift, DDM has lower CRPS and ensemble-mean RMSE across all 190 shared channel–lead pairs, with SSR generally closer to 1.Both models deteriorate relative to ERA5 evaluation, but DDM shows smaller average increases in nCRPS, nRMSE, and mean SSR deviation.Source: §D.4.2 · p. 24, Figure 18 · p. 29, Figure 19 · p. 30

D.4.3. Tropical Cyclone Tracking

  • DDM improves ensemble-mean track error and track CRPS over CRPS training for Hurricane Ian and Hurricane Michael.The tracking evaluation uses 50-member ensembles and IBTrACS best tracks as the reference.Source: §D.4.3 · p. 25, Table 17 · p. 29, Figure 21 · p. 31
  • For Hurricane Ian, DDM places more ensemble members near the reference center across the tested radii; at 200 km, the fractions rise from 62% to 94% for MOSAIC and 36% to 46% for U-Cast.The comparison uses 50-member ensembles at 48 h for MOSAIC and 120 h for U-Cast.Source: §D.4.3 · p. 25, Table 18 · p. 32, Figure 22 · p. 32

Related papers

Loading 2609.38632v1…