PaperlayerView exact revision on arXiv

Source-linked AI summary

Ensemble Complexity in Photovoltaic Forecasting

Sun Ze, Zhou Liguo, Xu Yuqing, Yu Lei, Jiang Mingming

arXiv:2609.15049v1cs.LGcs.AI

TL;DR

Photovoltaic ensembles may reduce forecast error, but their components can add fitting and inference costs without contributing useful predictive information. This study compares and ablates a fixed heterogeneous predictor bank under controlled evaluation. Static fusion has corrected positive evidence against matched boosting only on OPSD, while weather gating shows no consistent additional benefit.

  • Problem

    Ensemble components can add fitting and inference requirements without contributing useful predictive information, and comparisons can confound fusion with weather inputs or weight-fitting labels.

    Source: §1 · p. 1

  • Method

    The study evaluates a frozen heterogeneous predictor bank with matched controls, chronological separation of fitting and scoring, and component ablations.

    Source: §1 · p. 1, §2.3 · p. 2

  • Results

    Static fusion has corrected positive evidence against matched boosting only on OPSD; weather gating shows no consistent additional benefit, and removals indicate dependence on neural and classical groups without showing every member is necessary.

    Source: §4.14 · p. 11, §5 · p. 11

  • Takeaways & Limitations

    The findings support reporting incremental predictive effects and measured costs rather than using ensemble size or a transformed accuracy score as evidence of value.

    Source: §5 · p. 11

    Weather-assisted results have information-availability limits: private weather comes from a historical target-time cache, and GEFCom2014 forecast-issuance timestamps are unavailable in the processed mirror.

    Source: §2.1 · p. 2

Abstract

from arXiv · show

An ensemble can improve photovoltaic forecasts while adding components that contribute little or increase computation. We assess these effects through matched comparisons and ablations of a fixed heterogeneous predictor bank. Hourly experiments use GEFCom2014 and three additional public datasets, with chronological partitions and three seeds. Under retrospective ERA5 assistance, static fusion reduces scaled mean absolute error against matched boosting by 1.11%, 4.41%, and 1.63% on PVDAQ, OPSD, and Ausgrid; only OPSD remains supported after multiple-comparison correction. Weather gating offers no consistent incremental benefit. Exploratory member removals show group-level dependence alongside individual redundancy. A separate, previously inspected fifteen-minute case replaces one neural member with a tree predictor: normalized error falls by 1.72%, but measured inference is slower. These findings support component-wise evaluation with explicit limits on weather availability and test-set reuse.

1. Introduction

The paper asks whether the predictive gains from a heterogeneous photovoltaic ensemble justify the added components, fitting, and inference requirements. It uses matched comparisons and component analyses to separate the value of fusion, weather inputs, and fitting-label access.

1. Introduction

  • The central question is whether ensemble error reductions justify added fitting and inference requirements when some components may contribute little.The study examines learned fusion, weather gating, member sensitivity, and a separate member replacement.Source: §1 · p. 1
  • Matched information and fitting-label access are needed to distinguish gains from forecast combination, extra weather inputs, and weight-fitting labels.Equal epoch counts per neural member do not equalize total bank-training cost against training one model.Source: §1 · p. 1
  • The study evaluates learned fusion against equal averaging and matched regression, then tests site and horizon dependence, shrinkage, weather gating, member sensitivity, and replacement costs.The replacement case uses a separately evaluated four-slot combination.Source: §1 · p. 1
  • The public hourly experiments adapt a fourteen-member bank and use matched regression controls with retrospective ERA5 weather.The primary implementation's archived private score is audited separately; exploratory removals do not select a new model from test results.Source: §1 · p. 1

2. Related works

Prior work motivates evaluating photovoltaic forecasting methods on shared targets and distinguishing forecast weather from realized weather. This study situates its fixed-bank combination among weather-conditioned models, diverse predictors, and forecast-combination methods while noting limits on direct score comparisons.

2.1. PV forecasting and weather-conditioned models

  • PV forecasting reviews identify variation in datasets, horizons, and metrics as obstacles to comparing published models, motivating evaluation of variants on shared targets.Weather-classification approaches are established, so weather-conditioned aggregation itself is not presented as new.Source: §2.1 · pp. 1–2
  • The evaluated weather adjustment is bounded and family-level, applied to a fixed predictor bank rather than training separate model copies for each weather class.Private weather context comes from historical weather at target timestamps; GEFCom2014 predictors lack retained forecast-issuance timestamps.Source: §2.1 · p. 2

2.2. Neural and regression forecasting models

  • The cited neural architectures motivate diversity, while the public benchmark also includes ridge regression, gradient boosting, and persistence baselines.The PV benchmark uses repository implementations, including task-adapted models, and does not rank the official implementations of all named architectures.Source: §2.2 · p. 2

2.3. Forecast combinations and evaluation

  • Forecast combinations range from averages to conditional weighting; station- and horizon-specific weights can capture local and lead-time differences, with shrinkage reducing sensitivity to sparse local windows.V9 fits bounded adjustments to a previously trained bank rather than jointly training a hierarchical mixture.Source: §2.3 · p. 2
  • Evaluation separates predictor fitting, selection, ensemble fitting, and final scoring; GEFCom2014 competition scores are contextual rather than directly comparable to this deterministic rolling four-hour task.The study evaluates forecast variants on controlled partitions rather than treating published probabilistic scores as numerical baselines.Source: §2.3 · p. 2

3. Method

The method evaluates a frozen heterogeneous predictor bank on chronological partitions, fitting station–horizon weights with global shrinkage and a bounded weather gate. Confirmation safeguards gate selection, while fixed-weight removals diagnose dependence on the existing bank.

3.1. Task and information set

  • Public forecasts use 24 hourly observations to predict the next four hours, and eligible inputs must be available at forecast origin.The archived private result remains retrospective weather-assisted until availability of its weather feed at origin is established.Source: §3.1 · pp. 2–3

3.2. Frozen heterogeneous predictor bank

  • The public adaptation trains fourteen predictors per run, one from each of nine neural families plus five classical predictors, without transferring private checkpoints or weights.The adaptation validates the combination mechanism at this scale rather than reproducing the private 82-constituent system.Source: §3.2 · p. 3

3.3. Station–horizon weights and shrinkage

  • Nonnegative station–horizon weights are fitted by minimizing MAE, then local estimates are shrunk toward a shared global vector.The shrinkage preserves convex weights; its parameter is selected on a later chronological partition.Source: §3.3 · p. 3

3.4. Weather regimes

  • The private weather regimes use irradiance, cloud cover, precipitation, and irradiance change; the public adaptation instead uses a training-period radiation-envelope index.The public index is not a physical clear-sky index, and its denominator uses only the base-training period.Source: §3.4 · p. 3

3.5. Bounded family-level weather gate

  • A bounded family-level gate adjusts static weights by weather regime without retraining predictors, and a zero gate recovers the static combination.The renormalized form preserves nonnegative constituent weights; the public adaptation treats each constituent as its own family.Source: §3.5 · pp. 3–4

3.6. Selection and confirmation

  • Gate choices are made chronologically, and the gate is retained only when its MAE improvement persists across all three confirmation blocks.After choices are frozen, weights are refitted on fitting, selection, and confirmation periods; failed confirmation falls back to static weights.Source: §3.6 · p. 4

3.7. Fixed-weight member-removal diagnostic

  • Member-removal diagnostics renormalize frozen weights and predictions; positive Δ_R means removal worsens the current combination.All fourteen members and both neural and classical groups are covered, but the analysis is exploratory after test results were inspected and does not establish the best retrained smaller bank.Source: §3.7 · p. 4

4. Experiments

The experiments distinguish archived private evidence from chronological public evaluations, with controls and protocols varying by setting. The archived private comparison reports a small gain over its recomputed static control, while the public setup specifies datasets, partitions, and model baselines.

4.1. Evidence scope

  • The evidence spans an archived private five-minute case, hourly public evaluations, exploratory removals on inspected public tests, and a separate private fifteen-minute replacement case.Scores and member counts are not ranked across settings because targets, inputs, fitting histories, and normalization differ.Source: §4.1 · p. 4

4.2. Archived private case study

  • Approximately 0.0561% relative nMAE reduction separates archived V9 from its recomputed static control.The archived private results are not a new rerun, and the test period was inspected by several historical versions.Source: §4.2 · p. 5, Table 1 · p. 5

4.3. GEFCom2014 dataset and forecasting protocol

  • The GEFCom2014 test covers January–June 2014 across three sites, using shared targets for a within-site temporal—not held-out-site—evaluation.The protocol uses 24 past hours and four subsequent targets; forecast-vintage availability is not independently certified because NWP issue timestamps are absent.Source: §4.3 · p. 5

4.4. Baselines and implementation

  • The public bank combines nine neural implementations with five classical predictors, while equal per-member epochs do not match total bank costs to a single regression control.Classical predictions are deterministic and shared across the three neural seeds.Source: §4.4 · p. 5

4.5. Metrics and uncertainty

The section reports forecast-error metrics and uncertainty procedures, then compares the original GEFCom2014 ensemble with boosting and evaluates weather gating. Additional public evaluations use ERA5 reanalysis retrospectively, so they do not validate an operational forecast-weather pipeline.

4.5. Metrics and uncertainty

  • The reported score is 100(1 − nMAE), while uncertainty is summarized across three seeds and, for gate–static comparisons, by paired resampling of calendar-day blocks.The resulting interval is conditional on the test period and fitted models and does not fully model dependence across consecutive days or alternative temporal splits.Source: §4.5 · p. 6

4.6. Original GEFCom2014 comparison results

  • nMAE 0.062028 versus gradient boosting’s 0.066130 is a 6.20% relative reduction for the static ensemble in the original benchmark.The regression controls omitted historical weather channels and later ensemble-fitting labels, limiting architectural attribution; the supplied benchmark table identifies the common 5,973-window test set.Source: §4.6 · p. 6, Table 3 · p. 7
  • The target-weather gate’s 0.00000754 normalized-MAE improvement over static weights has a day-block interval including zero, and confirmation fails for all three seeds.The confirmation-selected rule therefore uses the static ensemble in every run; the origin-regime variant retains the same NWP-driven base predictions.Source: §4.6 · p. 6

4.7. Additional datasets and retrospective weather condition

  • The additional public experiments use Ausgrid, OPSD, and PVDAQ with ERA5 reanalysis, but target-period reanalysis is unavailable at a real forecast origin.These are retrospective oracle-weather evaluations, not validation of an operational NWP forecast pipeline; the source locations and weather alignment also have stated limits.Source: §4.7 · pp. 6–7

4.8. Additional protocol and matched controls

The protocol uses chronological partitions, matched controls, and shared test conditions to compare fusion while controlling information and fitting-label access. Evaluation also distinguishes retrospective ERA5 assistance from operational weather availability.

4.8. Additional protocol and matched controls

  • Complete hourly histories and targets are split chronologically into base training, selection, fitting, confirmation, and test partitions; intervals cannot cross partition boundaries.No missing history or target is interpolated, and only target midpoints in [06:00, 20:00) local time are included.Source: §4.8 · p. 8
  • Errors use each site’s maximum valid training-period power for sMAE and sRMSE; predictions are clipped to [0, 1.25].Weather standardization also uses no later partitions; physical-unit and per-site and per-horizon errors are archived.Source: §4.8 · p. 8
  • Matched Ridge and boosting controls receive the ensemble’s complete feature information and final fitting-label access, while DLinear is a historical-power-only reference.The bank’s ridge and boosting constituents are trained only on base training so their weight-fitting predictions are out of sample.Source: §4.8 · p. 8
  • The bounded gate retains its original confirmation rule, requiring improvement in each of three chronological folds before candidates are scored on the common test set.The empirical radiation envelope uses only training-period monthly and hourly values.Source: §4.8 · p. 8
  • Primary contrasts compare static fusion with matched boosting and gating with static fusion; seed-averaged losses use circular seven-day bootstrap intervals.The report includes descriptive 95% intervals and Bonferroni intervals for the six primary contrasts, conditional on the sites, splits, and fitted runs.Source: §4.8 · pp. 8–9
  • Table 6 reports test sMAE with training-maximum normalization, and future ERA5 is a retrospective oracle input rather than an operational forecast.Entries are means ± sample standard deviations across three training seeds; matched Ridge/GB share the ensemble’s information and final fitting-label access.Source: §4.8 · p. 9

4.9. Component ablations

Static fusion’s matched-boosting gain is supported after correction only on OPSD; ablations show no supported incremental benefit from gating and dependence on both member groups, but not every individual member.

4.9. Component ablations

  • The ablations vary gate, shrinkage, site-specific weighting, equal weighting, or gate context while sharing the frozen bank and fitting labels.Origin-context gating retains oracle-informed base predictions and is not a full causal-input ablation.Source: §4.9 · p. 9, §4.11 · p. 10
  • PVDAQ’s single-site design makes local and global objectives coincide, so it cannot establish cross-site benefits of shrinkage or site dependence.Those mechanisms must be assessed on the two multihousehold datasets.Source: §4.9 · p. 9

4.10. Additional comparison and ablation results

  • Static fusion reduces sMAE relative to matched gradient boosting by 1.11% on PVDAQ, 4.41% on OPSD, and 1.63% on Ausgrid; only the OPSD contrast excludes zero after correction.Gating provides no supported incremental improvement on any dataset.Source: §4.10 · p. 9
  • Learned static weights lower mean error than uniform averaging on all three datasets; local weights alone are worse than shrinkage or global sharing on OPSD and Ausgrid.The small shrinkage–global differences provide little reason to prefer the more site-specific estimate, and these ablation comparisons are exploratory.Source: §4.10 · p. 9
  • Origin-context gating changes only gate context and retains oracle-informed base forecasts, so it cannot establish performance with real-time-only information.Changing gate context produces smaller and less consistent effects.Source: §4.10 · p. 9, §4.11 · p. 10

4.11. Dependence on individual members and groups

  • Removing all neural members or all classical members worsens sMAE on every added dataset and seed, with mean increases ranging from 0.001417 to 0.004949.These fixed-weight group-removal contrasts are exploratory test diagnostics without refitting.Source: §4.11 · p. 9, Table 8 · p. 10
  • 0.003721, 0.001827, and 0.001787 are the largest mean single-removal deteriorations for gradient boosting on PVDAQ, OPSD, and Ausgrid; removing TiDE or persistence lowers sMAE by 0.000177 or 0.000171 on OPSD or Ausgrid.These are small descriptive changes, not separately tested discoveries.Source: §4.11 · p. 9
  • The reconstructed diagnostics verify the complementary-sum calculation, not the independence of already inspected test sets; no new member selection or training follows.The independent implementation agrees in sMAE within 2.8 × 10^-17, and source-file hashes remain unchanged.Source: §4.11 · pp. 9–10

4.12. Secondary case: a native member replacement

Across the hourly study, the matched boosting comparison remains supported after correction only on OPSD; the native fifteen-minute replacement reduces error but increases measured inference latency. These results are qualified by exploratory removals and previously inspected test periods.

4.12. Secondary case: a native member replacement

  • 1.72% lower nMAE than the matched four-neural pipeline follows replacing one member in the native fifteen-minute case.The first pair shares the complete fitting and calibration procedure; eighteen days and one seed do not establish long-term stability.Source: §4.12 · pp. 10–11, Table 9 · p. 10

4.13. Measured inference cost of the replacement

  • 4.71 and 7.96 are the median paired latency ratios for individual windows and batches of 256, respectively, with the replacement slower.Table 10 reports median latency in milliseconds per window, with batch-256 times amortized; timing excludes loading, data retrieval, network service, and training.Source: §4.13 · p. 11, Table 10 · p. 11

4.14. Discussion and limits of the evidence

  • Only on OPSD does static fusion’s advantage over matched boosting survive the six-comparison correction.The original GEFCom comparison is less controlled because its regression baseline has fewer inputs and fitting labels.Source: §4.14 · p. 11
  • Weather gating has no consistent incremental support, while group removals show dependence on neural and classical predictions without establishing that every member is necessary.Because removals keep predictions and relative weights fixed without refitting, they cannot show whether a smaller retrained bank recovers the same information.Source: §4.14 · p. 11
  • Public tests use one temporal holdout per dataset and three seeds; retrospective target-period weather and historical test reuse limit operational generalization.The study does not claim prospective validation on genuinely unexposed dates.Source: §4.14 · p. 11

5. Conclusion

The study finds qualified evidence for static fusion and no consistent added benefit from weather gating. A separate native comparison improves nMAE with slower inference, while weather-oracle use, limited holdouts, and historical test reuse constrain generalization.

5. Conclusion

  • Static fusion has corrected positive evidence against matched boosting on OPSD, while weather gating shows no consistent additional benefit.Exploratory removals show dependence on neural and classical groups but do not establish that all fourteen members are necessary.Source: §5 · p. 11
  • A separate native fifteen-minute comparison finds 1.72% relative nMAE reduction after one member replacement, alongside slower inference and larger saved checkpoints.The comparison uses a previously inspected period.Source: §5 · p. 11
  • The weather-oracle condition, limited temporal holdouts, and historical test reuse require further validation before operational generalization or optimal ensemble size can be claimed.Source: §5 · p. 12

Data and code availability

The paper describes public data sources, reproducibility materials, and limits on access to private artifacts.

Data and code availability

  • Public sources include the GEFCom2014 researcher mirror, Ausgrid, original OPSD household feeds, and NREL PVDAQ system 34, with additional ERA5 weather served through Open-Meteo.Source: Data and code availability · p. 12
  • Download scripts preserve attribution, request parameters, response hashes, and preprocessing rules; existing packages retain model snapshots, checkpoints, weights, predictions, ablations, and audits.Source: Data and code availability · p. 12
  • The revision supplement adds fixed-weight removals and summarized native evidence, but private raw data, predictions, and checkpoints are not redistributed.The native case has auditable local evidence but is not fully reproducible from the public supplement alone.Source: Data and code availability · p. 12

A. Reproduction and evidence boundaries

The reproduction appendix specifies the public runner and records validation choices, while distinguishing numerical checks from broader evidence boundaries.

A. Reproduction and evidence boundaries

B. Complete member-removal diagnostics

The removal diagnostics report exploratory, descriptive changes on previously inspected test periods; no member is removed from the retained system based on them. The accompanying figures distinguish descriptive comparisons from inferential claims and qualify the stability of gating gains.

B. Complete member-removal diagnostics

  • Table 12’s fixed-weight removal differences are descriptive three-seed means on already inspected test periods; no member is removed from the retained system on this basis.Table 12 reports mean changes in sMAE multiplied by 10^3; positive values indicate deterioration after removal, with remaining station–horizon weights renormalized without refitting.Source: B. Complete member-removal diagnostics · p. 12, Table 12 · p. 13
  • Removal entries are test sMAE, not capacity-normalized nMAE; stations use training-period maxima, and future ERA5 is a retrospective oracle input rather than an operational forecast.DLinear uses historical power only, unlike weather-assisted models; the neural rows are retrained model families, and matched controls additionally fit ensemble meta-stage labels.Source: B. Complete member-removal diagnostics · p. 13
  • Figure 3’s small aggregate gate improvement does not establish a stable benefit, as assessed using individual seed gains and a paired calendar-day bootstrap interval.Source: Figure 3 · p. 14
  • Figure 4 compares static fusion with matched boosting and gating with static fusion; its descriptive intervals are conditional on three seeds, while text uses a six-contrast correction.Table 11 is identified as the complete model comparison on the three additional datasets; the supplied caption gives no cell values.Source: Table 11 · p. 13, Figure 4 · p. 15

Related papers

Loading 2609.15049v1…