Single-stage joint diffusion of sky images and clear-sky index for ultra-short-term irradiance forecasting: what the frame-to-irradiance stage discards

2.3. Sky-video prediction and the two-stage cascade

• The dominant cascade predicts future frames and separately regresses irradiance, so its forecast inherits both frame-generation and regression error.Separate training also means the regressor does not see the predicted frames it receives at test time; prior work sometimes uses real future images to estimate the cost of imperfect generation.Source: §1 · p. 3, §2.3 · p. 6

1. Introduction

• The model co-denoises future sky frames and clear-sky-index values, then predicts irradiance from a denoised estimate of the future sky without measured irradiance history.This removes the separately trained frame-to-irradiance regressor used by two-stage methods such as SkyGPT and Xiong et al..Source: §1 · p. 4
• The proposed model reaches CRPSS +0.34 against smart persistence and leads SUNSET in CRPSS by 0.26 to 0.30 in every sky-condition tertile, although SUNSET has lower pooled RMSE: 97.6 against 104.1 W m−2.SUNSET also reads the irradiance history and provides no predictive distribution.Source: §1 · p. 4

2.1. Ultra-short-term forecasting and its inputs

• At minutes-to-half-hour horizons, individual clouds drive irradiance variability, while numerical weather prediction updates too slowly and resolves too coarsely for that process.The study forecasts site-independent GHI rather than power tied to a particular array.Source: §2.1 · p. 5

2.2. Estimating irradiance from a sky image

• Sky-image irradiance estimation is supervised regression from observed images, but forecasting requires future imagery and is a distinct task.Varaschin and Silva report estimation errors far below forecasts at the same site.Source: §1 · p. 3, §2.2 · pp. 5–6

2.4. Probabilistic forecasting and its evaluation

• Probabilistic forecasts provide distributions relevant to reserve planning, but reporting of calibration and reference-forecast skill has been uneven.This study uses smart persistence as a common reference and reports CRPSS, interval coverage, and spread–skill alongside point error.Source: §2.4 · p. 6

2.5. Diffusion models and single-stage joint forecasting

• The proposed irradiance head reads a denoised future-sky estimate inside each reverse step, unlike PV-MM-Diffusion, which conditions on the noised image state, or Xiong et al., which uses separately generated frames.The model is assembled from standard diffusion, latent-image, and spatial-conditioning components rather than a task-specific architecture.Source: §2.5 · pp. 6–7

2.6. Position of this work

• The study evaluates joint diffusion for GHI without measured irradiance input on one year of SIRTA data, against twelve baseline configurations using a shared test subset and smart-persistence reference.It reports probabilistic and point scores, run-to-run variation, and ablations of the irradiance head’s inputs.Source: §2.6 · p. 7

3.1. Preliminaries: denoising diffusion

• Diffusion training corrupts clean samples over timesteps, trains a network to predict added noise, and repeatedly applies learned reverse steps from noise toward a sample.The denoised estimate is a recomputed clean-sample estimate, distinct from the noisy state advanced by the sampler.Source: §3.1 · pp. 7–8

3.2. Problem setup

• At each forecast origin, the model takes eight preceding RGB frames and four deterministic covariates, then generates eight future frames and eight CSI values over a 16-minute horizon.Image frames are diffused as VAE latents, while the CSI sequence is diffused directly; both streams share a timestep and the forecast uses no measured irradiance history.Source: §3.2 · pp. 8–9

3.3. Framework

• A Stable-Diffusion VAE, image tokenizer, CSI transformer, and image denoiser jointly process the two streams, with a ControlNet and temporal refiner in the image branch.The CSI denoiser attends across future positions and to image tokens; the temporal refiner corrects image-noise estimates across all eight frames.Source: §3.3 · p. 9

3.4. One denoising step

• Each denoising step runs the CSI denoiser to condition image denoising, forms a denoised future-sky estimate, then runs the CSI denoiser again to produce the loss-bearing forecast.The CSI-to-image link is undetached, but the image-to-CSI preview is built from detached image noise, so irradiance loss does not update the image denoiser.Source: §3.4 · pp. 10, 12

3.5. Training and sampling

• Training proceeds through five sequential stages, ending with joint diffusion phases; at inference, image latents and CSI are co-denoised over 50 reverse steps and mapped to frames and irradiance.The CSI forecast is read directly from its denoiser, with no frame-to-irradiance regression stage.Source: Figure 1 · p. 10, §3.5 · p. 13

4.1. Dataset

• SIRTA samples pair eight past and eight future sky frames at two-minute cadence, covering a 16-minute horizon; the 2023 test set contains 12,906 sequences from 43 days.Cross-method comparisons use 1,998 test sequences, with every method and smart-persistence reference scored on the same inputs.Source: §4.1 · p. 14

4.2. Experimental protocol

• The proposed model is evaluated on the same 2023 matched test sequences after retraining on 2021–2022, isolating the training-year shift.The cross-year test uses no training or validation data from the test year.Source: §4.1 · p. 14, §4.2 · p. 15
• Sky-condition regimes are origin-time clear-sky-index tertiles, while a ramp requires a GHI change of at least 200 W m−2 between consecutive forecast steps.The tertile labels describe the origin-time index, not measured variability.Source: §4.2 · p. 15
• The frozen U-Net regressor scores real and generated future frames, while the model’s irradiance head is scored on the same samples.The regressor is trained on real sky images and then frozen; video-prediction baselines are scored through it as well.Source: §4.2 · p. 15

4.3. Evaluation metrics

• CRPS skill compares probabilistic CRPS with smart persistence’s MAE, while point accuracy uses RMSE and forecast skill against smart persistence.Scores are pooled over sample–step pairs before skill-score ratios are computed.Source: §4.3 · p. 16

5. Performance comparison

• +0.341 CRPSS is the proposed model’s highest comparison score, while its 104.1 W m−2 pooled RMSE is second-lowest to SUNSET’s 97.6 W m−2.The proposed model’s CRPSS confidence interval lies entirely above zero; SUNSET is deterministic and provides no predictive distribution.Source: §5 · p. 18, Table 2 · p. 19
• DLinear, PatchTST, and TimeXer reach 109–113 W m−2 RMSE but CRPSS values of −0.08 to +0.03; adding clear-sky irradiance lowers their RMSE by 1.0–3.6 W m−2.Video predictors scored through the frozen regressor range from 112.3 W m−2 for PhyDNetGAN to 145.3 W m−2 for ConvLSTM, all with negative CRPSS.Source: §5 · p. 20
• The model of Xiong et al. has the highest baseline CRPSS at +0.148, while the proposed future-sky arm improves over its past-sky arm by 10.0 W m−2 RMSE and 0.106 CRPSS.The Xiong et al. model also reads irradiance history, unlike the proposed model.Source: §5 · pp. 18, 20

5.1. Cross-year performance

• Cross-year retraining leaves all six methods within 1.8 W m−2 in RMSE and 0.03 in CRPSS of in-period scores, with the ranking unchanged.The proposed model’s CRPSS changes from +0.341 to +0.343, less than the reported sampler variation of 0.006.Source: §5.1 · p. 20, Table 3 · p. 21

6.1. Per-horizon performance

• CRPSS exceeds every other method at every horizon, rising from +0.03 at i+1 to about +0.41 before settling near +0.39.Forecast skill is negative at the first step and approaches SUNSET’s without overtaking it; the pooled RMSE advantage is a mid-to-late-horizon effect.Source: §6.1 · p. 21, Figure 4 · p. 21

6.2. Performance across sky conditions

• The proposed model has a positive, nearly flat CRPSS margin over SUNSET in all three sky-condition tertiles, while SUNSET has lower RMSE in each.Each tertile contains 666 matched test samples.Source: Table 4 · p. 22, §6.2 · p. 22, Figure 5 · p. 22

6.3. Reliability of the predictive intervals

• No method reaches nominal 90% coverage off ramps; the proposed model is roughly calibrated off ramps and modestly under-dispersed on ramps.Its coverage falls by 0.026 from non-ramp to ramp events, versus 0.206 for Xiong; the proposed model’s reliability curve also shows under-dispersion across nominal levels.Source: §6.3 · pp. 22–23, Table 5 · p. 23

6.4. Ramp events

• On 334 ramp samples, the proposed model attains CRPSS +0.440, while SUNSET has the lowest RMSE at 216.4 W m−2.The proposed model’s ramp CRPSS exceeds its +0.342 on the past-sky arm and Xiong’s +0.310.Source: §6.4 · pp. 23–24, Table 7 · p. 25
• The proposed model’s 90% interval widens from a mean 87 W m−2 off ramps to 417 W m−2 on ramps and tracks ramp magnitude with r=+0.85.Its interval width scales at 0.88 W m−2 per W m−2 of ramp, versus Xiong’s 0.54; ramp coverage is 0.605 at nominal 0.90, so operators should use empirical rather than nominal coverage.Source: Figure 7 · p. 24, §6.4 · p. 25, Table 6 · p. 25

7. Effect of removing the regression stage

• The frozen U-Net regressor is trained on real sky images, then evaluated with identical preprocessing on ground-truth future frames and generated frames.The comparison tests whether the two-stage pipeline’s deficit reflects generated-frame quality or the separate regression stage.Source: §7 · p. 26
• 150.6 W m−2 is the frozen U-Net regressor’s RMSE on generated frames, compared with 104.1 W m−2 for the proposed model’s irradiance head reading the same frames.On true future frames, the U-Net reaches 63.5 W m−2; the 104.1-to-102.2 W m−2 oracle-arm improvement is about 2 W m−2.Source: §7 · p. 26
• In the illustrated ramp-up case, no method closely reproduces the ground-truth future frames, and frame quality decreases with lead time.For that case, the measured irradiance rises by about 700 W m−2 between i+4 and i+7; the ensemble band contains most of the rise while deterministic forecasts follow it only partly.Source: Figure 8 · p. 27, Figure 9 · p. 28

8.1. Input of the irradiance head

• Reading the denoised estimate of future frames instead of past frames lowers RMSE by 10.0 W m−2 and raises CRPSS from +0.235 to +0.341.The arms share architecture and parameter count; the difference has the same sign across three seeds and its paired bootstrap interval excludes zero.Source: §8.1 · p. 28, Table 9 · p. 29

8.2. Measured irradiance history

• The measured-history arm adds eight past irradiance values, encoded as clear-sky index and raw irradiance, to the irradiance head’s inputs.Each seed is compared with a control trained on the same modified code path.Source: §8.2 · p. 29
• The measured-history ablation’s paired comparison uses same-code-path controls because initialization differences otherwise put the controls above the headline arm.Aligning the seed-1 control’s initial weights brings its RMSE within 0.05 W m−2 of the headline arm, and the paired comparison is reported as unaffected by this issue.Source: §8.2 · p. 29
• Adding measured irradiance history lowers RMSE by 4.8–7.1 W m−2 and raises CRPSS by about 0.09 at both seeds.The corresponding CRPS reductions are 13.3% and 12.3%; this ablation quantifies the value of an input excluded from the study’s sky-only configuration.Source: §8.2 · p. 29, Table 10 · p. 29

9. Conclusion

• 104.1 W m−2 for the jointly trained head versus 150.6 W m−2 for a separate regressor on the same generated frames isolates a 46 W m−2 advantage in reading those frames.The authors attribute the difference to the joint head learning the generator’s error statistics, not to better frames.Source: §9 · p. 30
• The proposed model does not achieve the best pooled point forecast; SUNSET, which also reads measured irradiance, wins, while the proposed intervals fall short of nominal coverage.The cross-modal coupling result rests on one mechanism at one training budget, so it is an absence of observed effect rather than a demonstrated absence.Source: §9 · p. 30
• The point-accuracy gap to the strongest deterministic baseline is an input difference rather than an architectural ceiling: a perfect future sky is worth about 2 W m−2, and eight steps of measured irradiance 5–7.The paper suggests a joint single-stage model that also reads measured irradiance could close much of the remaining point-accuracy distance while retaining the predictive distribution.Source: §9 · p. 30

A. Baseline fidelity

• The comparison audited released implementations for convergence and training-budget parity, correcting three baselines, including cases where corrections made the proposed result look worse.SkyGPT, ConvLSTM, PhyDNet, PhyDNetGAN, and SUNSET came from the reference implementation released with Nie et al.; DLinear, PatchTST, and TimeXer came from the Time-Series-Library.Source: A. Baseline fidelity · p. 30
• 97.6 W m−2 is corrected SUNSET’s pooled RMSE, beating the proposed model and widening its point-accuracy concession.Both SUNSET variants had trained past their validation minimum because patience was applied on a coarser schedule than intended; retraining changed only validation cadence.Source: A. Baseline fidelity · pp. 30–31
• SkyGPT’s initial port split the eight-frame horizon into two four-frame chunks, creating a fourth-to-fifth-lead-time discontinuity that disappeared after correcting clip length and retraining.The authors identify the discontinuity as a port artifact rather than a weakness of the architecture; Table 2 uses the corrected version.Source: A. Baseline fidelity · p. 31
• ConvLSTM, PhyDNet, and PhyDNetGAN were verified to have converged and were scored by passing their generated frames to the same frozen U-Net regressor.This scoring is the two-stage arrangement whose effect Section 7 measures.Source: A. Baseline fidelity · p. 31
• The Xiong et al. [7] reimplementation’s trained SDE was near-deterministic; its observed spread came from the sampler and fitted observation-noise term instead.The authors report this as a diagnosis of their reimplementation’s behavior, not a criticism of the method, and say it does not change the quoted numbers.Source: A. Baseline fidelity · p. 31

B. The wider design space

• The design-space survey uses the full 2023 test set (N=12,906), unlike the matched subset in the main text, so its numbers are not comparable with Table 2.Table B.1’s caption also identifies its 20 reverse diffusion steps, compared with 50 in the main text, and excludes the oracle from best-value selection.Source: B. The wider design space · p. 31, Table B.1 · p. 32
• The noised-state arm is the weakest forecaster, while concat, which reads both the noised state and past frames, sits between the two bracketing arms.Image quality and forecast skill are largely independent: concat has the worst imagery but a mid-range forecast, while noised has the reverse.Source: B. The wider design space · p. 31
• 3.7 W m−2 is the oracle arm’s advantage over the deployed arm on the full test set; the oracle reads true future-frame tokens at evaluation only.It is not a separately trained model, and its trained head cannot fully exploit real frames because it learned to read its own denoised estimates.Source: B. The wider design space · pp. 31–32
• 104.1 W m−2 for the jointly trained head versus 150.6 W m−2 for a frozen regressor on the model’s generated frames shows the head’s advantage on those inputs.The oracle arm remains above the 1.05 SSR target at 1.11, while the deployed arm reaches 1.24; removing uncertainty about future sky does not remove under-dispersion.Source: B. The wider design space · p. 32
• The added path derives future CSI from the noise estimate of the pass reading clean tokens, then supplies it to the image denoiser through a route separate from cross-attention and ControlNet.The variants use either future-sky tokens or past-sky tokens as the input choice, and the detached estimate carries forward values without closing a gradient cycle.Source: B. The wider design space · pp. 32–33
• The added cross-modal path feeds a denoised estimate of future CSI back into the image branch as an additional conditioning signal, detached to prevent gradients flowing through it.Figure B.1 places that signal in the timestep embedding; the caption contrasts the added orange dashed path with the proposed model’s black path.Source: B. The wider design space · p. 33, Figure B.1 · p. 34
• Adding the cross-modal path moved forecast skill by less than the reported noise floors, but neither comparison isolates the path because the paired arms differ in training history.The authors therefore report an absence of observed effect, not evidence of no effect, and make no claim about image quality.Source: B. The wider design space · p. 33
• The two cross-modal comparisons add the path to otherwise identical arms, but each input choice has its own phase 1 while both variants share a different phase-1 checkpoint.Table B.2 identifies the additional differences between the compared arms.Source: B. The wider design space · p. 33, Table B.2 · p. 33

C. Protocol details

• Every arm starts from the same ControlNet checkpoint, shares a 50-epoch ceiling and validation criterion, and is scored at its own best-validation checkpoint.The validation criterion sums generated-frame LPIPS and forecast-CSI RMSE; the future-sky arm selected epoch 959 at all three seeds, while the past-sky arm stopped at epoch 954.Source: C. Protocol details · p. 34
• Table C.2 reports per-horizon values for frozen-regressor imagery scores, CRPSS, nominal-90% coverage, and cross-year minus in-period forecast skill.Table C.3 compares point estimates on the matched subset with all windows, contiguous windows, and QC-valid windows; no method changes rank on either score in either comparison.Source: Table C.2 · p. 35, Table C.3 · p. 35
• The sampler uses 50 reverse diffusion steps after a sweep found forecast metrics stop improving materially beyond that setting; probabilistic forecasts use 10 independent members and the fair CRPS estimator.The naive CRPS estimator was not used because it gives materially more optimistic skill at this ensemble size.Source: C. Protocol details · p. 35
• At seed 1, matching every shared module’s initial weights yielded 104.63 W m−2 for the modified arm, versus 104.58 for the headline arm and 108.51 for its control.The reported offset was attributed to initialization rather than the modified code path or ablation; that arm was not retrained at seed 2.Source: C. Protocol details · p. 36
• The proposed generator has 198,834,269 parameters and is not the fastest at inference; removing the regressor is a structural claim about error propagation, not a compute claim.The required cascade regressor is a negligible fraction of total cost.Source: C. Protocol details · p. 36

• Kanawut Suwandee handled software, validation, formal analysis, investigation, data curation, visualization, and the original draft.Source: CRediT authorship contribution statement · p. 36
• Suwichaya Suwanwimolkul led conceptualization, methodology, supervision, and project administration.Source: CRediT authorship contribution statement · p. 36
• Suwichaya Suwanwimolkul also handled funding acquisition and review and editing of the manuscript.Source: CRediT authorship contribution statement · p. 36

• The repository provides code, trained weights, stored forecasts for every compared method, and training scripts.Scoring the stored forecasts reproduces every Table 2 entry exactly.Source: Data availability · p. 37
• Regenerated proposed-model forecasts reproduce exactly, but SkyGPT and Xiong et al. ensembles vary between runs.Reproduced scores for those methods fall within Table 2 confidence intervals; released configurations omit the stage schedule needed to reproduce proposed-model training.Source: Data availability · p. 37
• The sky images and irradiance measurements are not redistributed; SIRTA grants access to its data on request.The repository includes retrieval and preprocessing scripts, plus sample identifiers and frame timestamps.Source: Data availability · p. 37
• Rebuilding the dataset from current SIRTA files can omit test samples or use shifted frames because of the observatory’s review of the 2023 record.The repository lists affected samples, identifies shifted frames from timestamps, and gives expected rebuild scores.Source: Data availability · p. 37