PaperlayerView exact revision on arXiv

Source-linked AI summary

SimCast-S2S: An Efficient Generative Model for Subseasonal Precipitation Forecasting via Transfer Learning from Climate Simulations

Hiep V. Dang, Antonios Mamalakis

arXiv:2608.26594v1cs.LG

TL;DR

S2S precipitation forecasting must represent uncertain outcomes despite weak signals, limited data, and costly ensemble generation. SimCast-S2S uses latent diffusion, VAEs, and LoRA transfer learning from climate simulations, outperforming deep-learning baselines and competing favorably with ECMWF-S2S. Its main limitation is that latent-space sampling produces statistical rather than explicitly physical realism.

  • Problem

    S2S precipitation forecasting requires probabilistic uncertainty representation despite weak predictive signals, data-hungry generative models, and high ensemble-generation costs.

  • Method

    SimCast-S2S uses diffusion-based stochastic sampling in VAE-compressed latent space, with LoRA transfer learning from large climate-simulation ensembles to limited reanalysis data.

  • Results

    SimCast-S2S outperforms deterministic AI baselines, shows stronger probabilistic skill across much of the globe, and generates 100-member ensembles in minutes on a small GPU node.

  • Takeaways & Limitations

    Latent generative modeling with simulation-to-reanalysis transfer learning offers an efficient and scalable approach to probabilistic S2S precipitation forecasting.

  • Takeaways & Limitations

    SimCast-S2S can generate statistically plausible fields without guaranteeing conservation laws or dynamical balances because sampling occurs in latent rather than physical space.

Abstract

from arXiv · show

Subseasonal-to-seasonal (S2S) precipitation forecasting has substantial financial and societal impact, yet remains challenging because of weak predictive signals, high associated uncertainty, and the computational cost of operational systems, which constrains simulation fidelity. We introduce SimCast-S2S, a generative latent-diffusion framework for probabilistic S2S precipitation forecasting that addresses three major bottlenecks in data-driven prediction. First, because S2S prediction requires uncertainty quantification rather than only deterministic point forecasts, SimCast-S2S is the first data-driven system that uses a diffusion-based generative pipeline for S2S prediction, enabling effective sampling from the underlying conditional distribution. Second, since generating large probabilistic ensembles is computationally costly in physical space, SimCast-S2S instead operates in a compact latent space learned by variational autoencoders, enabling efficient large-ensemble generation. Third, diffusion models typically require large training datasets; SimCast-S2S overcomes this via transfer learning with low-rank adaptation (LoRA), pretraining on large ensembles of climate simulations before fine-tuning on limited reanalysis data. On reanalysis data, SimCast-S2S outperforms deep learning baselines, including convolutional neural networks and U-Net architectures. Notably, despite using only a subset of atmospheric input variables and no post-processing, bias correction, or calibration, SimCast-S2S remains competitive with, and in many cases outperforms, state-of-the-art operational systems such as the ECMWF-S2S baseline. These results indicate that latent generative modeling combined with simulation-to-reanalysis transfer learning offers an efficient and scalable path toward data-driven probabilistic S2S precipitation forecasting.

1 Introduction

S2S forecasting occupies a difficult transition between weather and climate timescales, where atmospheric errors have grown while slower variability provides limited stable predictability. SimCast-S2S addresses the resulting uncertainty, data, and computational challenges with latent diffusion, VAEs, and simulation-to-reanalysis transfer learning.

  • Forecasting challenge: S2S forecasting, typically 3–6 weeks ahead, combines growing atmospheric-initial-condition errors with unstable slower sources of predictability.This transition zone requires models to capture both fine-scale atmospheric variability and slower climate influences.
  • Forecasting challenge: Ensemble prediction systems generate multiple perturbed forecasts because deterministic predictions are insufficient for chaotic atmospheric systems and expanding S2S uncertainty.This is especially important for precipitation, whose nonlinear processes make it difficult to predict.
  • Generative forecasting: Generative models approximate conditional distributions and can draw multiple plausible forecasts more efficiently than repeated numerical integrations.This makes them suitable for probabilistic S2S forecasting, particularly for precipitation.
  • Open limitations: Deep generative models require large datasets, while GANs can suffer from instability, mode collapse, and optimization sensitivity; existing encoder–decoder systems still require substantial data.FuXi-S2S outperformed ECMWF-S2S for selected variables but did not assess uncertainty capture.
  • SimCast-S2S approach: SimCast-S2S compresses physical fields into low-dimensional Gaussian latent representations with domain-specific VAEs and uses diffusion to generate probabilistic forecasts.Its design targets large ensembles, limited training data, and the computational cost of physical-space generation.
  • SimCast-S2S approach: Approximately 12 seconds are required to generate one ensemble member on an A100 GPU, with members parallelizable across GPUs for inexpensive large-ensemble forecasting.This efficiency also enables pretraining on 28 CESM2-LE ensemble members before adapting to limited reanalysis data.

2 Results

SimCast-S2S achieves strong deterministic and probabilistic precipitation-forecast skill while producing efficient, large ensembles. Diagnostics show improved spatial structure and interval scores, but both models remain underdispersive and forecast errors exceed ensemble spread.

  • Deterministic skill: SimCast-S2S achieves the lowest CESM2-test MAE, (1.25 ± 0.06) × 10−2 std, versus (1.31 ± 0.07) × 10−2 std for the best CNN and UNet baselines.Without ERA5 fine-tuning, it achieves 1.32 × 10−2 std on ERA5, indicating partial simulation-to-reanalysis transfer but a remaining gap.
  • Probabilistic skill: SimCast-S2S shows higher CRPSS than ECMWF-S2S across most of the globe, with stronger tropical patterns and broader extratropical advantages.The advantage extends from tercile-based RPSS to the continuous predictive distribution.
  • Probabilistic skill: SimCast-S2S maintains higher CRPSS than ECMWF-S2S across all seasons and regions, with the largest differences in the tropics.Spring and summer are especially difficult extratropical seasons; ECMWF-S2S has negative CRPSS there, while SimCast-S2S remains positive.
  • Uncertainty quantification: Both models produce underdispersive ensembles, with U-shaped PIT histograms and realized errors larger than ensemble spread despite high spread–error correlations.SimCast-S2S has more spatially uniform dispersion, while ECMWF-S2S has better extratropical calibration but greater tropical deviation.
  • Uncertainty quantification: SimCast-S2S achieves slightly lower mean interval scores than ECMWF-S2S globally, tropically, and extratropically, especially at high tropical coverage levels.Lower interval scores indicate better combined sharpness and coverage.
  • Spatial structure: SimCast-S2S better matches ERA5 meridional and diagonal spatial autocorrelation at small lags, while both models reproduce the basic autocorrelation decay.This indicates closer agreement with fine-scale precipitation structure in those directions.
  • Computational efficiency: A 100-member ensemble takes approximately 20.5 minutes on one A100 GPU, 7.3 minutes on three, and 5.7 minutes on four GPUs.Scaling is close to linear but declines with more GPUs because of loading, data-transfer, synchronization, and communication overheads.
  • Computational efficiency: ECMWF’s reported 20-hour product interval likely includes initialization, integration, post-processing, and dissemination rather than model runtime alone.The paper therefore treats the comparison with SimCast-S2S runtime cautiously.

3 Discussion

SimCast-S2S addresses ensemble size, data scarcity, computational cost, and uncertainty representation through diffusion-based generative modeling, latent-space sampling, and simulation-to-reanalysis transfer learning. Its forecasts show strong deterministic, probabilistic, spatial-dependence, and efficiency results, while remaining limited by incomplete physical consistency and interpretability.

  • Design contributions: SimCast-S2S addresses three S2S bottlenecks through diffusion-based distribution modeling, compact latent-space generation, and LoRA transfer learning from climate simulations to limited reanalysis data.The design targets uncertainty representation, ensemble-generation cost, and data requirements.
  • Forecast skill: SimCast-S2S exhibits higher probabilistic skill across much of the globe alongside lower deterministic errors, indicating predictive signal beyond member-specific stochastic variability.The authors connect probabilistic skill with stronger deterministic performance and ensemble-mean behavior.
  • Uncertainty quality: Both SimCast-S2S and ECMWF-S2S are underdispersive, but SimCast-S2S is more competitive in the tropics and lies closer to the spread–RMSE 1:1 reference without comparable calibration.ECMWF-S2S has stronger extratropical uncertainty behavior, whereas SimCast-S2S is evaluated from raw ensemble output.
  • Spatial structure: SimCast-S2S better captures precipitation-dependence decay, especially in meridional and diagonal directions, preserving spatial organization across distinct precipitation regimes.The discussion contrasts this with the overly smooth fields favored by point-wise-loss convolutional models.
  • Computational efficiency: SimCast-S2S generates ensemble members independently and in parallel across accelerators, changing the practical cost structure of subseasonal ensemble prediction.The method directly models the mapping from the current atmospheric state to target weeks rather than repeatedly integrating a full numerical model.
  • Limitations: SimCast-S2S realism is currently statistical rather than explicitly physical, and its latent-space sampling does not directly enforce conservation laws or dynamical balances.The authors propose occasional physical-space correction as a future hybrid approach.
  • Limitations: Extreme-event representation remains a key weakness shared by SimCast-S2S and ECMWF-S2S, motivating future incorporation of physical constraints.The limitation is stated as a scope boundary for both systems.
  • Limitations: Interpretability remains incomplete because the analysis does not identify which fields, regions, or dynamical patterns the model uses, complicating diagnosis of physically misleading successes.The authors propose perturbation, gradient, latent-sensitivity, and counterfactual methods to connect skill with mechanisms.

4.1 Data

SimCast-S2S transfers learning from CESM2 climate simulations to ERA5 reanalysis using physically corresponding atmospheric variables and standardized anomalies. The data pipeline aligns variables across datasets and removes dominant climate-background structure before training.

  • Simulation-to-reanalysis transfer: SimCast-S2S is pretrained on 28 CESM2 ensembles and then transferred to observation-constrained reanalysis data.CESM2 represents coupled atmosphere, ocean, land, sea-ice, and other climate components.
  • Cross-dataset alignment: Input variables are selected for availability or close physical correspondence across CESM2 and ERA5, spanning 850, 500, and 200 hPa.These levels represent the lower, middle, and upper troposphere.
  • Input variables: Wind, geopotential, temperature, and humidity jointly describe atmospheric dynamical and thermodynamical conditions at each pressure level.The variables represent transport, circulation structure, thermal stratification, and moisture supply.
  • Cross-dataset alignment: CESM2 and ERA5 fields with differing conventions or quantities require physically motivated basis conversions before transfer learning.Examples include reversing ERA5 top net longwave radiation sign and relating geopotential height to geopotential.
  • Preprocessing: All input and target variables are standardized anomalies, separating subseasonal departures from seasonal cycles, spatial climatology, and slow trends.Validation and test processing reuses training-period statistics to avoid information leakage.
  • Latent representation: The VAE maps each variable-group anomaly field to a Gaussian latent distribution, whose sampling introduces stochasticity before reconstruction.The encoder produces distribution parameters and the decoder reconstructs the sampled representation as a physical field.

4.2 Physical and Latent Space

SimCast-S2S represents grouped atmospheric fields in both high-dimensional physical space and compact probabilistic latent space. VAEs compress and reconstruct each group, while stochastic sampling, ELBO-based training, and selected compression levels balance uncertainty, fidelity, and efficiency.

  • Physical space: Physical-space samples are standardized anomaly fields on a latitude–longitude grid grouped by circulation, mass, thermodynamic, moisture, and precipitation processes.The physical representation remains meteorologically interpretable but is high-dimensional and redundant.
  • Latent space: Each variable group xg is mapped from physical dimension CgHW to latent dimension dg, with dg ≪ CgHW.The VAE provides a compact probabilistic representation for each group.
  • Encoding and decoding: The encoder predicts a Gaussian approximate posterior, samples zg through reparameterization, and the decoder maps that sample back to reconstructed anomaly field x̂g.Sampling uses zg = µg + σg ⊙ ϵ with ϵ drawn from a standard Gaussian.
  • Stochasticity: Latent sampling and information loss make reconstructions differ from inputs, providing a natural perturbation source for SimCast-S2S stochasticity.Repeated encoding and decoding of the same anomaly field can produce multiple possible reconstructions.
  • VAE training: The VAE loss combines reconstruction negative log-likelihood with KL divergence, encouraging accurate reconstruction while regularizing the posterior toward the prior.Under a Gaussian likelihood, the reconstruction term becomes a scaled MSE loss.
  • VAE architecture: The convolutional encoder and decoder progressively downsample and upsample fields, with block count controlling latent dimension and compression.The architecture uses ConvStack–DownScaling blocks in the encoder and mirrored upsampling in the decoder.
  • Variable grouping: The 21 variables are divided into five physically motivated groups—wind, mass, thermal, hydro, and precipitation—with each group retaining its own physical and latent dimensions.The groups share a 192 × 288 physical grid.
  • Compression choice: The study uses 16× compression for wind, mass, thermal, and hydro groups and 4× compression for precipitation to balance efficiency against reconstruction accuracy.The authors report that reconstruction error decreases approximately exponentially as latent dimension increases.

4.3 SimCast-S2S model

SimCast-S2S performs probabilistic precipitation forecasting by learning diffusion dynamics in a compact latent space conditioned on historical atmospheric representations. Its reverse process progressively denoises Gaussian latent states, with sampling variants that trade off diversity and stability.

  • Latent diffusion: SimCast-S2S models stochastic evolution in a low-dimensional latent space by adding Gaussian noise forward and progressively removing it conditionally during reverse diffusion.The conditioning representation is formed from concatenated latent representations of the initial physical state.
  • Forward process: The forward Markov chain gradually degrades the latent signal with √1−βk while injecting Gaussian noise of variance βk, approaching N(0, I) at step K.The closed-form transition q(zk | z0) enables direct sampling from a clean latent without iterating through every diffusion step.
  • Reverse process: The learned reverse transition is Gaussian, with its mean computed analytically from the network’s predicted noise and its variance set to the true posterior variance.The model uses pθ(zk−1 | zk)=N(zk−1; µθ(Zcond,t,zk,k), ΣkI) and sets Σk=˜βk.
  • Latent diffusion: Historical physical fields are separated into five groups, encoded by pretrained VAEs, and concatenated into the conditioning latent Zcond.The groups are wind, thermal, mass, hydro, and precipitation.
  • Sampling variants: η interpolates between deterministic DDIM sampling at η=0 and standard stochastic sampling at η=1, allowing a diversity–stability trade-off.Intermediate values 0<η<1 partially reduce the injected variance.
  • Velocity prediction: SimCast-S2S predicts a velocity variable v instead of noise ϵ, representing a rotated coordinate on the clean-latent/noise plane and yielding more stable training.The diffusion trajectory can be viewed geometrically as a rotation on the unit circle or hypersphere.

4.4 Transfer Learning: LoRA

SimCast-S2S transfers knowledge from large CESM2 simulation ensembles to ERA5 using LoRA while retaining the pretrained model weights. The approach adapts only compact low-rank corrections, with additional precipitation encoder and decoder fine-tuning for distribution shift.

  • Transfer strategy: SimCast-S2S is pretrained on 28 CESM2 atmospheric simulations and transferred to ERA5 using a computationally efficient latent-space framework.This addresses limited real-world atmospheric training data through simulation-to-reanalysis transfer.
  • Domain adaptation: The VAE encoders and decoders for wind, mass, thermal, and hydro are not adapted, whereas the precipitation encoder and decoder are fine-tuned because precipitation has stronger CESM2–ERA5 distributional shift.The denoising network is also adapted with LoRA.
  • LoRA adaptation: LoRA freezes each pretrained weight matrix W and learns only a low-rank update ∆W=BA, preserving CESM2 structure while adapting the model to ERA5.The update rank satisfies r≪min(din,dout).
  • LoRA adaptation: LoRA reduces trainable parameters from O(dindout) to O(r(din+dout)), making fine-tuning more memory efficient and less prone to overfitting.Only the factor matrices A and B are optimized during fine-tuning.
  • Transfer outcome: CESM2 pretraining followed by ERA5 LoRA fine-tuning performs substantially better than ERA5-only training and is identified as a main reason SimCast-S2S outperforms ECMWF-S2S.During fine-tuning, the task is to adjust for the domain shift from many simulated worlds to one real world.

4.5 Deep Learning Baselines: CNN, UNet

The study benchmarks SimCast-S2S against CNN and U-Net architectures designed for gridded spatial prediction. CNNs learn local patterns through shared kernels, while U-Nets add multiscale encoder–decoder processing with skip connections.

  • CNN baselines: CNN baselines use stacks of convolutional layers with fixed embedding dimensions, and larger models increase both layer count and embedding dimension.This configuration is summarized in Table 4.
  • Architecture rationale: CNNs provide a natural gridded-prediction baseline by learning local spatial patterns through shared convolutional kernels.U-Nets extend this design with skip connections that preserve multiscale spatial information.
  • U-Net baselines: U-Net baselines use symmetric encoder–decoder structures with four sampling blocks and embedding dimensions specified across blocks.The decoder reverses the encoder’s spatial-resolution changes.
  • Baseline design: SimCast-S2S is benchmarked against three CNN and three U-Net baselines with different model sizes.Tables 4–5 summarize the corresponding architectural configurations.

4.6 Physical Baseline: ECMWF-S2S

ECMWF-S2S serves as the strong process-based operational reference for evaluating SimCast-S2S. Unlike neural-network baselines, it is generated by integrating a physically based Earth-system model.

  • Operational reference: ECMWF-S2S is used as the operational physical baseline and represents a highly developed dynamical forecasting system.ECMWF is described as one of the world-leading numerical weather prediction centers.
  • Operational reference: ECMWF-S2S forecasts are produced by integrating a physically based Earth-system model that explicitly resolves atmospheric dynamics.This distinguishes the reference from neural-network baselines.

4.7 Evaluation Metrics

The evaluation uses deterministic metrics for anomaly magnitude and spatial pattern, probabilistic skill scores for categorical, continuous, and extreme-event forecasts, and diagnostics for ensemble uncertainty consistency, sharpness, and coverage.

  • Deterministic metrics: Deterministic skill is evaluated with ensemble-mean precipitation-anomaly forecasts across test samples.MAE measures average grid-level error relative to local climatology, while ACC measures agreement in spatial anomaly patterns.
  • Deterministic metrics: Lower MAE and higher ACC jointly indicate forecasts that match observed anomaly magnitude and spatial structure.MAE rewards smaller absolute errors; ACC rewards stronger agreement in wet and dry anomaly patterns.
  • Probabilistic skill metrics: Probabilistic forecasts are evaluated from empirical ensemble distributions that should assign probability to observations and improve over reference distributions.The ensemble members define an empirical predictive distribution at each grid point.
  • Probabilistic skill metrics: RPSS evaluates tercile probabilities, CRPSS evaluates the full continuous distribution, and BSS evaluates extreme precipitation probabilities above the 90th percentile.Positive RPSS, CRPSS, and BSS indicate improvement over climatological references.
  • Uncertainty quantification: PIT values should be uniformly distributed on [0, 1] when observations are statistically consistent with the predictive distribution.A U-shaped PIT histogram indicates underdispersion, whereas an inverted-U shape indicates overdispersion.
  • Uncertainty quantification: Spread–RMSE consistency, χ2-distance to flatness, and MIS assess ensemble dispersion, uncertainty calibration, and interval quality.Spread below RMSE indicates underdispersion; spread above RMSE indicates overdispersion, while lower MIS indicates better interval forecasts.
Loading 2608.26594v1…