PaperlayerView exact revision on arXiv

Source-linked AI summary

From Nowcasting to Forecasting: Adapting a Reanalysis-Trained

Mikko Partio, Leila Hieta, Ossi Laine

arXiv:2609.03763v1cs.LGphysics.ao-ph

TL;DR

Cloud-cover forecasts matter for temperature, radiation, and solar-power operations, but observation-based methods lose skill as clouds evolve and NWP may not match the observed cloud state at initialization. CloudCast v2 learns cloud evolution from reanalysis, adapts to satellite cloud fields with conditional flow matching, and uses NWP inputs; it maintains higher skill later in the forecast window than CloudCast v1.

  • Problem

    Short-range cloud forecasting must bridge observation-based initialization and atmospheric evolution, while extrapolation skill declines as clouds form, dissipate, or deform and NWP may not represent the observed initial cloud state.

    Source: §1 · p. 1, §4 · p. 16

  • Method

    CloudCast v2 pre-trains on reanalysis, adapts to satellite-derived cloud fields with clean-target conditional flow matching, and uses atmospheric NWP inputs across the forecast horizon.

    Source: §1 · p. 2, §2.2 · p. 4, §2.4 · p. 6

  • Results

    CloudCast v2 maintains higher skill later in the forecast window, greater neighborhood-scale spatial agreement, and better performance in cases with large observed cloud-cover changes than CloudCast v1.

    Source: §4 · p. 16

  • Takeaways & Limitations

    Observation-initialized machine-learning forecasts can extend cloud-cover prediction beyond short-range extrapolation when supplied with NWP atmospheric fields, complementing rather than replacing NWP.

    Source: §4 · p. 17

    Evaluation over additional years and geographic domains is needed to assess robustness and transferability.

    Source: §4 · p. 16

Abstract

from arXiv · show

Accurate cloud-cover forecasts are important for temperature prediction, radiation forecasting, and solar-power operations. Short-range forecasting methods can preserve observed cloud placement during the first forecast hours, but their skill decreases when cloud fields evolve through formation, dissipation and deformation. Longer lead times require accounting for atmospheric evolution, but operational numerical weather prediction (NWP) forecasts may not accurately represent the satellite-observed cloud state at initialization. We develop CloudCast v2, a machine-learning model for 12-hour cloud-cover forecasting from observation-based initial conditions. The model is first trained on the Copernicus European Regional Reanalysis (Ridal2024) to learn cloud-evolution dynamics, and is then adapted to satellite-derived cloud fields using conditional flow matching (Lipman2023), a generative method that transforms noise into cloud-cover forecasts conditioned on the observed initial cloud fields and NWP inputs. CloudCast v2 reduces mean absolute error by 10% relative to its predecessor, CloudCast v1 (Partio2025), over the 1-12 h range. It also overtakes CloudCast v1 in fractions skill score, a neighborhood-based measure of spatial agreement, after approximately 3-6 h, depending on the cloudiness category. These results show that observation-initialized machine-learning forecasts can extend beyond the usual 1-3-hour nowcasting range while retaining spatial detail from satellite cloud fields.

1 Introduction

CloudCast v2 aims to unify observation-based cloud nowcasting and NWP-informed forecasting in a 12-hour model. The introduction motivates this approach by the limits of extrapolation at longer leads and the underexplored adaptation of reanalysis-trained models to satellite cloud fields.

1 Introduction

  • Accurate cloud forecasts matter for weather prediction and solar-power operations because cloud errors can bias temperature and radiation forecasts and increase uncertainty in energy decisions.At local scales, such errors can reduce the reliability of operational decision-making.Source: §1 · p. 1
  • Extrapolation often works at one to two hours but loses skill from three hours onward as cloud formation and dissipation matter, while NWP represents atmospheric dynamics but remains sensitive to initial-state errors.Cloud evolution also depends on moisture, vertical motion, and parameterized subgrid processes.Source: §1 · p. 1
  • Operational users need one forecast across lead times, yet blending extrapolation and dynamical forecasts can create visible discontinuities when their predictions disagree.This transition remains a central challenge for operational cloud-cover prediction.Source: §1 · pp. 1–2
  • Machine-learning and generative forecasting advances have focused mainly on radar precipitation, leaving their applicability to satellite-derived fractional cloud cover uncertain.Cloud cover has bounded fractional values and complex partly or broken-cloud regimes.Source: §1 · p. 2
  • Reanalysis-trained weather models forecast atmospheric evolution, but adapting them to satellite-derived fractional cloud cover for 12-hour regional prediction remains underexplored.Prior cloud-focused models include CloudCast v1, which forecasts up to five hours with reported skillful lead time of about three hours; recent satellite generative approaches predict brightness temperature rather than fractional cloud cover.Source: §1 · p. 2
  • CloudCast v2 combines reanalysis pre-training, adaptation to observed cloud fields, and NWP inputs for unified 12-hour cloud-cover forecasting.The paper evaluates it against nowcasting and NWP baselines using pixel-wise error, spatial verification, and skill scores for large-change cases.Source: §1 · p. 2

2 Data and methods

CloudCast v2 is evaluated on a common 5 km Scandinavian grid, with CERRA providing reanalysis pretraining, NWCSAF supplying satellite-observed cloud fields, and MEPS providing atmospheric forcings. The methods combine a Swin-based forecasting model with staged observation-space adaptation and complementary verification protocols.

2.1 Study domain and datasets

  • The study aligns CERRA, NWCSAF, and MEPS fields on a common 5 km Scandinavian grid; CERRA supports pretraining, NWCSAF supplies satellite cloud fields, and MEPS supplies atmospheric forcings.CERRA fields are hourly through filling gaps between three-hour analyses; NWCSAF effective cloudiness is available every 15 minutes, while MEPS provides hourly forecast fields.Source: §2.1 · p. 3

2.2 Forecasting task

  • Each case uses cloud-cover fields at t −1 and t, static fields, and atmospheric forcings through t + 12 to predict cloud cover at leads 1–12.During pretraining, CERRA supplies cloud fields and forcings; during fine-tuning and inference, NWCSAF supplies cloud fields and MEPS supplies forcings.Source: §2.2 · p. 4

2.3 Model architecture

  • A U-shaped Swin Transformer encodes two-step cloud and forcing histories, then uses lead-time forcing to decode a cloud-cover tendency added to the current field.Windowed attention supports the regional grid, while convolution and decoder cross-attention add local mixing and encoded context.Source: §2.3 · pp. 4–5, Figure 2 · p. 5, Figure 3 · p. 5

2.4 Observation-space adaptation with conditional flow matching

  • Conditional flow matching adapts deterministic forecasts to observation-space cloud fields by learning to transform Gaussian noise into forecasts conditioned on analysis fields and future forcings.The clean-target formulation predicts the NWCSAF field directly at each denoising step, preserving the original cloud-cover prediction task and allowing initialization from deterministic weights.Source: §2.4 · p. 6

2.5 Training protocol

  • Four training stages progress from CERRA pretraining to NWCSAF adaptation: one-step and autoregressive pretraining, direct deterministic prediction, then direct flow-matching prediction.The observation-space stages use BCE and Soft-FSS; stage 4 starts from the deterministic stage-3 model and produces the final flow-matched model.Source: §2.5 · pp. 6–7, Figure 4 · p. 7

2.6 Evaluation protocol and metrics

  • Evaluation uses the held-out 2025 test year and compares CloudCast v2 with CloudCast v1, MEPS, a no-CFM ablation, and persistence using metrics appropriate to each cloud quantity.The common test set contains 2,855 initializations; CloudCast v1 is resampled to the common grid, causing a small nonzero error at lead zero.Source: §2.6 · p. 7
  • Verification combines direct errors, anomaly- and reference-based scores, and neighborhood FSS; category-based FSS excludes MEPS because it predicts cloud area fraction rather than NWCSAF effective cloudiness.For substantial cloud changes, the study evaluates the top 20% of cases by observed 12-hour change against an optical-flow-advected persistence baseline.Source: §2.6 · pp. 7, 9
  • Bias and MAE are computed only for systems predicting NWCSAF effective cloudiness directly, while ACC and MSESS permit comparison with MEPS despite differences in cloud quantities.MSESS compares forecast MSE with Eulerian persistence; MEPS scores remain affected by the mismatch between cloud area fraction and effective cloudiness.Source: §2.6 · pp. 8–9

3 Results

CloudCast v2 sustains greater skill than CloudCast v1 as lead time increases, including during substantial cloud-cover changes. Conditional flow matching improves neighborhood-scale spatial agreement, while the no-CFM variant scores better on conventional error-based metrics.

3.1 Pixel-wise verification

  • From about t + 3, both CloudCast v2 variants have lower MAE than CloudCast v1 and Eulerian persistence, with their advantage widening at longer lead times.CloudCast v1 has the lowest MAE during the first two forecast hours; the no-CFM variant is consistently slightly lower in MAE than the CFM model.Source: §3.1 · p. 10, Figure 5 · p. 10

3.2 Pattern and reference-based skill

  • Both CloudCast v2 variants maintain higher ACC than CloudCast v1, MEPS, and Eulerian persistence through most of the forecast window, and remain above MEPS through t + 12.The no-CFM variant has the highest ACC at all lead times; the full CFM model remains close to it and above CloudCast v1 after the first few hours.Source: §3.2 · pp. 10–11, Figure 6 · p. 10

3.3 Spatial skill across cloudiness regimes

  • The full CloudCast v2 overtakes CloudCast v1 in FSS at about t + 2 for mostly cloudy, t + 4 for clear-sky and overcast, and t + 6 for partly cloudy conditions.FSS is lower for partly and mostly cloudy conditions than for clear-sky and overcast conditions, consistent with greater spatial complexity in intermediate cloud fields.Source: §3.3 · p. 11, Figure 7 · p. 11

3.4 Skill during large cloud-cover changes

  • On the top-quintile dynamic cases, both CloudCast v2 variants add skill beyond Lagrangian persistence throughout the 12-hour window, and CloudCast v2 beats CloudCast v1 in 96.3% of cases at t + 5.At t + 12, CloudCast v2 outperforms CloudCast v1 in most high-change cases, with most cases above the diagonal in every season.Source: §3.4 · p. 12, Figure 8 · p. 12

3.5 Overall model performance

  • The no-CFM model has the lowest MAE and highest ACC and MSESS, while the full model has higher FSS across all cloudiness categories.Thus, CFM slightly reduces performance on conventional error-based metrics while improving neighborhood-scale spatial agreement.Source: §3.5 · p. 12
  • Both CloudCast v2 variants become more advantageous than CloudCast v1 as lead time increases and perform better in cases with large observed cloud-cover changes, sustaining skill across 12 hours.CloudCast v1 remains competitive earliest, but its skill deteriorates more rapidly.Source: §3.5 · pp. 12–13

3.6 Illustrative case studies

  • In the July case, both CloudCast v2 variants capture regional clearing and cloud increase better than CloudCast v1, while CFM preserves sharper cloud edges.In the winter case, CFM produces a cloud structure prematurely over the southern domain; it resembles the observed cloudier pattern more closely by t + 12, indicating a timing error in this case.Source: §3.6 · p. 13, Figure 9 · p. 14, Figure 10 · p. 15

4 Discussion and Conclusions

CloudCast v2 combines satellite-observed initial cloud fields, reanalysis-learned evolution, and NWP forcings to forecast up to 12 hours. It maintains higher skill later in the forecast window, while metric trade-offs, comparison caveats, and limited evaluation scope qualify the results.

4 Discussion and Conclusions

  • CloudCast v2 maintains higher skill later in the forecast window, greater neighborhood-scale spatial agreement, and better performance in cases with large observed cloud-cover changes.It complements rather than replaces NWP by combining atmospheric forecasts with the latest observed cloud field at small additional computational cost.Source: §4 · pp. 16–17
  • Satellite fields anchor the initial cloud distribution, reanalysis pre-training supplies learned evolution, and NWP forcings describe atmospheric development beyond persistence or advection.The benefit becomes most apparent after the first few hours, as persistence and CloudCast v1 skill declines.Source: §4 · p. 16
  • Conditional flow matching produces sharper, more spatially coherent structures with higher FSS, but displacement or timing errors can increase local errors.Sharper forecasts may better represent cloud morphology while also producing confident local false positives or false negatives.Source: §4 · p. 16
  • Positive skill against Lagrangian persistence indicates improvement over simply advecting the initial field, but does not prove explicit representation of cloud formation or dissipation.The evaluation targets cases where formation, dissipation, or deformation likely contribute to observed changes.Source: §4 · p. 16
  • CloudCast v2 maintains higher ACC than MEPS throughout the 12-hour window, while MEPS becomes more competitive as lead time increases.The comparison requires caution because MEPS forecasts cloud area fraction, whereas CloudCast v2 forecasts NWCSAF effective cloudiness; MSESS is also affected by this difference.Source: §4 · p. 16
  • Once MEPS fields are available, CloudCast v2 generates a complete 12-hour forecast in approximately 90 seconds on an NVIDIA H100 GPU, but cannot precede MEPS availability.It therefore adds little computational delay before dissemination but does not replace the NWP production cycle.Source: §4 · p. 16
  • Evaluation uses NWCSAF effective cloudiness on one regional domain and one held-out year, so direct transfer to other regions is not established.The reference field also does not capture all cloud-cover aspects relevant to radiation or solar-power forecasting, and the test year may not represent other weather regimes.Source: §4 · p. 16
  • Future work could develop stochastic CFM into probabilistic forecasts, test additional years and domains, and explore sub-hourly forecasts when suitable forcings are available.The forcing set was not extensively optimized, and broader or better-selected predictors may improve forecast quality.Source: §4 · pp. 16–17

Related papers

Loading 2609.03763v1…