PaperlayerView exact revision on arXiv

Source-linked AI summary

An Interpretable Probabilistic Model for Short-Term Solar Power Forecasting Using Natural Gradient Boosting

Georgios Mitrentsis, Hendrik Lens

arXiv:2108.04058v2stat.APcs.LG

TL;DR

The paper addresses the limited transparency and trustworthiness of black-box probabilistic PV forecasting models. It combines NGBoost for probabilistic forecasts with SHAP explanations, finding better performance than GP and LUBE while exposing nonlinear feature relationships and enabling feature selection.

  • Problem

    Complex probabilistic PV forecasting models can require substantial training resources and provide insufficient transparency for safety-critical power-system decisions.

  • Method

    A two-stage framework uses NGBoost to produce probabilistic forecasts and SHAP values to explain point forecasts, prediction intervals, and feature interactions.

  • Results

    NGBoost outperformed GP and LUBE across deterministic and probabilistic metrics regardless of seasonal variation and trained more than ten times faster.

  • Takeaways & Limitations

    SHAP analyses exposed physically plausible nonlinear relationships, supported transparent interpretation, and helped identify features whose removal improved accuracy and sharpened probabilistic forecasts.

Abstract

from arXiv · show

PV power forecasting models are predominantly based on machine learning algorithms which do not provide any insight into or explanation about their predictions (black boxes). Therefore, their direct implementation in environments where transparency is required, and the trust associated with their predictions may be questioned. To this end, we propose a two stage probabilistic forecasting framework able to generate highly accurate, reliable, and sharp forecasts yet offering full transparency on both the point forecasts and the prediction intervals (PIs). In the first stage, we exploit natural gradient boosting (NGBoost) for yielding probabilistic forecasts, while in the second stage, we calculate the Shapley additive explanation (SHAP) values in order to fully comprehend why a prediction was made. To highlight the performance and the applicability of the proposed framework, real data from two PV parks located in Southern Germany are employed. Comparative results with two state-of-the-art algorithms, namely Gaussian process and lower upper bound estimation, manifest a significant increase in the point forecast accuracy and in the overall probabilistic performance. Most importantly, a detailed analysis of the model's complex nonlinear relationships and interaction effects between the various features is presented. This allows interpreting the model, identifying some learned physical properties, explaining individual predictions, reducing the computational requirements for the training without jeopardizing the model accuracy, detecting possible bugs, and gaining trust in the model. Finally, we conclude that the model was able to develop complex nonlinear relationships which follow known physical properties as well as human logic and intuition.

I. INTRODUCTION

PV forecasting must address volatile generation and the limited transparency of complex probabilistic machine-learning models. The paper proposes an interpretable NGBoost-SHAP framework to improve forecasting quality while explaining predictions and feature interactions.

  • Motivation: Volatile PV generation complicates reliable electricity supply and system operation, increasing the value of accurate probabilistic forecasts.Probabilistic forecasts provide uncertainty information that can support lower-risk decisions by system operators, traders, and other stakeholders.
  • Related work: Probabilistic PV forecasting remains less developed than deterministic forecasting, with existing approaches spanning quantile regression, bootstrapping, LUBE, gradient boosting, kernel density estimation, and analog ensembles.The literature also includes parametric and nonparametric methods with different distributional assumptions.
  • Problem: Many machine-learning forecasting approaches are complex black boxes with high training requirements, making their results difficult to understand and trust in safety-critical power-system decisions.The paper specifically highlights this concern for transmission and distribution system operators.
  • Proposed approach: The proposed two-stage framework uses NGBoost for probabilistic PV forecasts and SHAP values for transparent explanations of point forecasts and prediction intervals.NGBoost is presented as modular and usable without expert programming knowledge or time-consuming hyperparameter tuning.
  • Interpretability: The paper analyzes SHAP interaction values to identify nonlinear relationships between features and whether the model captures known physical relationships.The authors distinguish this work from earlier SHAP studies that interpreted deterministic models without considering interaction values.

II. POWER AND METEOROLOGICAL DATA

The study uses quarter-hourly measurements from two nearby PV parks in Southern Germany and combines meteorological, temporal, and lagged-power variables. Exploratory analysis examines feature relationships before comparing them with relationships learned by the forecasting model.

  • Study data: Data came from two PV parks in Baden-Württemberg with nominal powers of approximately 3.2 MVA and 1.8 MVA, recorded from February 2018 to October 2019 at 15-minute resolution.The parks are approximately 50 km apart and nighttime values from 22:00 to 06:00 were discarded.
  • Features: Features included temperature, relative humidity, precipitation, wind speed, ground-level solar radiation, month, time of day, and power values lagged by 15, 30, and 45 minutes.The previous-day lag t-24h was discarded because it did not improve prediction accuracy.
  • Feature processing: Month was represented with sine and cosine components so December and January appear as consecutive points on a cyclical scale.No analogous transformation was applied to time of day because nighttime observations were removed.
  • Forecast design: The forecasting task targets a 36-hour horizon at 15-minute resolution using recursive multi-step predictions and sliding-horizon validation.Each prediction uses predicted values from previous time steps.
  • Exploratory analysis: Boxplots summarize output-power distributions across times of day, with nested widths representing 50%, 25%, 12.5%, and progressively smaller data fractions.Single dots denote outliers, and the plot is used to show how time of day influences PV generation throughout the year.
  • Exploratory analysis: Pearson correlations were estimated before examining pairwise feature interactions, using combined scaled data for the heatmap while training separate models for each PV park.Pearson correlation reveals only linear relationships.

III. FORECAST MODELS AND METRICS

The study benchmarks persistence, Gaussian process, and LUBE against NGBoost for probabilistic PV forecasting. NGBoost estimates distribution parameters through natural-gradient boosting, using base learners, a probability distribution, and a scoring rule.

  • NGBoost is benchmarked against persistence, Gaussian process, and lower upper bound estimation using metrics for accuracy, bias, reliability, sharpness, and overall probabilistic performance.
  • NGBoost sequentially combines scaled base learners, each optimized against the current ensemble residual, to solve probabilistic regression problems.
  • Its three main modules are base learners, a parametric probability distribution, and a scoring rule such as CRPS.
  • Shallow decision trees and a normal distribution are selected, with each tree leaf receiving its own normal distribution.
  • Natural gradients provide parametrization-invariant optimization for distributional scoring-rule minimization.
  • At each stage, natural gradients are estimated using the Fisher Information, then combined with inputs to train base learners and update distribution parameters.

B. Gaussian Process (GP)

Gaussian process regression models outputs through a joint multivariate Gaussian distribution governed by a covariance kernel. Training minimizes the negative log marginal likelihood, and predictions for new inputs have Gaussian means and variances.

  • Gaussian process regression assumes finite random variables follow a joint multivariate Gaussian distribution.
  • The covariance matrix encodes pairwise relationships between observations, while the covariance function introduces prior knowledge about expected outputs.
  • RBF, rational quadratic, and periodic kernels are available, and kernel combinations can model more complex relationships.
  • With Gaussian observation noise, the covariance matrix of the full system is K = Σ+σ_nI.
  • For a new input, the output distribution is Gaussian with a mean and variance estimated from the kernel parameters and noise variance.
  • Model parameters are estimated by minimizing the negative log marginal likelihood using the Adam optimizer.

C. Lower Upper Bound Estimation (LUBE)

LUBE is a neural-network method that directly estimates prediction-interval bounds. Its objective balances coverage of observed values against interval width, with simulated annealing used to optimize the nonlinear, discontinuous criterion.

  • LUBE uses two output neurons to estimate the lower and upper bounds of prediction intervals associated with a point forecast.
  • Its objective combines the number of observations inside the intervals with their total width.
  • PICP measures the percentage of outputs covered, while PINAW normalizes interval width to prevent excessively wide intervals.
  • The coverage width-based criterion uses the confidence level µ and penalty factor η to penalize differences between coverage and the target level.
  • Simulated annealing estimates the neural-network weights and biases because the coverage width-based criterion is nonlinear and discontinuous.
  • LUBE produces prediction intervals only, so its evaluation uses probabilistic metrics rather than point-forecast metrics.

D. Validation metrics

The validation framework evaluates point-forecast accuracy and bias alongside probabilistic reliability and sharpness. CRPS provides an overall score combining the desired properties of probabilistic forecasts.

  • Accuracy and bias measure how far point forecasts deviate from observed power values, using MAE, RMSE, and MBE.
  • Reliability concerns whether observations could have been drawn from the predicted distribution, whereas sharpness concerns distribution concentration.
  • PIT evaluates reliability by applying the predicted cumulative distribution function to observations, with uniform histograms indicating desirable calibration.
  • U-shaped PIT histograms indicate overconfidence, while concave-shaped histograms indicate the opposite reliability pattern.
  • PICP measures reliability and PINAW measures sharpness for prediction intervals.
  • CRPS jointly evaluates probabilistic forecast properties and reports a negatively oriented score in forecast-quantity units.

IV. MODEL INTERPRETABILITY

The framework uses SHAP values to explain NGBoost predictions through feature attributions, including global and local influences. Because NGBoost is tree-based, these explanations can be calculated exactly in polynomial time.

  • SHAP values: The explanation model represents predictions as a base value plus feature attributions weighted by binary indicators of observed features.The base value is the expected model output without observed features, while the total number of features defines the explanation space.
  • SHAP values: SHAP values quantify how individual features influence model outputs, either globally across the model or locally for a single prediction.The method evaluates model behavior with and without features and interprets attribution as a cooperative game-theory problem.
  • SHAP values: Feature influence is estimated by mapping binary feature subsets back to the input space and evaluating the model conditional on the observed subset.The conditional model value is expressed as E[f(x)|x_S] for the selected feature values.
  • SHAP values: Exact SHAP values for tree-based models require polynomial rather than exponential time, motivating their combination with NGBoost for efficient interpretability.The paper contrasts this with approximation-based SHAP calculations for other black-box models.

B. SHAP interaction values

The study evaluates NGBoost against GP, LUBE, and persistence across seasonal point-forecast and probabilistic metrics. NGBoost provides strong accuracy, narrow intervals, low bias, and substantially shorter training time, while SHAP interaction values expose pairwise feature relationships.

  • SHAP interaction values: The total interaction between d1 and d2 is the sum of the two equal interaction components, Φd1,d2 and Φd2,d1.These interaction values can reveal physics-based relationships otherwise missed by individual feature attributions.
  • SHAP interaction values: SHAP interaction values extend feature attribution by revealing pairwise relationships that ordinary SHAP values do not capture.The interaction between two features is obtained by comparing a feature's SHAP value when the other feature is observed versus unobserved.
  • Comparative results: NGBoost reduces MAE by around 56% and RMSE by around 40% versus GP, while both metrics drop by around 70% versus persistence.NGBoost also shows almost zero bias across seasons, whereas GP underestimates spring generation and overestimates winter generation.
  • Comparative results: NGBoost achieves an average CRPS drop of around 50% relative to GP and LUBE and remains comparatively robust across seasons.The reported NGBoost configuration uses depth 3, learning rate 0.01, 500 boosting iterations, logarithmic score, and a Gaussian distribution.
  • Comparative results: NGBoost trains in less than 2 minutes, whereas GP and LUBE each require around 25 minutes.The paper links this shorter training time to practical model development involving repeated feature and hyperparameter testing.

B. Probabilistic forecasting

NGBoost generates accurate, sharp, and reliable day-ahead probabilistic PV forecasts across seasonal generation patterns. Its PIT behavior is close to perfect reliability, and the reported performance supports short-term operational and market applications.

  • Probabilistic forecasting: NGBoost produces accurate and sharp day-ahead probabilistic forecasts across substantially different seasonal generation patterns.For the Gaussian distribution, the plotted intervals correspond approximately to 68%, 95%, and 99% coverage at one, two, and three standard deviations from the mean.
  • Probabilistic forecasting: The combined PIT histogram shows almost perfectly reliable forecasts for PIT values above 0.5, with a small tendency to underestimate lower-half density.The reported PIT values remain close to perfect reliability overall.
  • Probabilistic forecasting: The proposed model yields more accurate and reliable predictions than two state-of-the-art approaches.The paper identifies PV park operators, system operators, and power traders as potential beneficiaries of the forecasting accuracy.
  • Probabilistic forecasting: Higher forecasting accuracy could support short-term energy-market bidding, reduce activated balancing energy, and reduce imbalance costs.The paper describes these as potential benefits for system operators, PV park operators, and traders.

C. Interpretation of point forecasts

SHAP analyses show how lagged power, radiation, humidity, and time-related features influence point forecasts through nonlinear relationships consistent with PV-generation physics. Global, summary, interaction, and force plots provide complementary views from overall importance to individual predictions.

  • Global and local explanations: SHAP summary plots reveal both the magnitude and direction of each feature’s impact on point-forecast outputs across observations.Feature values are encoded by color, horizontal position shows impact, and vertical dispersion indicates how many observations share similar effects.
  • Global and local explanations: High t-15 values tend to increase predicted power, whereas near-zero t-15 values tend to decrease it.Higher t-15 values generally have larger impacts on the model output.
  • Feature effects: High radiation and t-45 values have larger impacts than lower values, while t-15 can suffice when radiation and lagged power are low.At higher radiation and lagged-power values, the model uses more features to produce accurate predictions.
  • Feature effects: Humidity, hour, and lagged-power interactions produce physically plausible effects, including lower predictions at higher humidity and positive mid-day impacts.These relationships align with declining PV-cell efficiency at high humidity and the bell-shaped daily radiation pattern.
  • Nonlinear interactions: SHAP interaction values are strongest among highly important feature pairs, while low-ranked features show no remarkable interactions.The model’s gradient-boosting point forecasts therefore capture nonlinear relationships rather than simple linear correlations.
  • Nonlinear interactions: For t-15 values between 0.5 MW and 1.5 MW, radiation can reverse the direction of the point-forecast influence, while its interaction weakens above 1.5 MW.High radiation raises predictions and low radiation lowers them; radiation becomes redundant at high lagged power.
  • Individual predictions: A force plot explains one prediction by showing each feature’s direction and magnitude, but it does not support general inference about model behavior.In the example, morning hours, high radiation, and moderate t-15 increase the output, whereas low t-30 and t-45 reduce it with lower magnitude.

D. Interpretation of model uncertainty

SHAP analyses of prediction intervals show that uncertainty depends on past power, radiation, humidity, and time of day through nonlinear feature interactions. The same framework explains both global uncertainty patterns and individual interval estimates.

  • Global uncertainty explanations: The uncertainty model’s global feature-importance ordering mostly follows the point-forecast model’s pattern.Bar plots and SHAP summary plots provide complementary global views of feature effects on uncertainty.
  • Feature effects on uncertainty: Small t-15 values substantially reduce uncertainty, whereas higher radiation usually lowers confidence in the prediction.When the PV park is not generating power, continued zero generation over the next 15 minutes is considered very likely.
  • Nonlinear uncertainty interactions: Radiation and hour form a nonlinear interaction affecting interval estimates, as shown in the corresponding interaction plots.The figure set examines humidity, lagged power, radiation, and hour as interacting contributors to uncertainty.
  • Feature effects on uncertainty: Precipitation has almost zero overall influence on uncertainty, while wind speed has no remarkable impact and some high-precipitation cases reduce confidence.The contribution of these weather variables is therefore limited or potentially negative for uncertainty estimation.
  • Nonlinear uncertainty interactions: Low humidity increases confidence, while similar and larger past-power values also increase confidence in the prediction intervals.These effects are shown through SHAP interaction plots for humidity and lagged power.
  • Individual uncertainty explanations: For one sample, past power, radiation, and humidity increase estimated standard deviation, whereas time of day reduces uncertainty.The uncertainty force plot uses the same sample as the point-forecast force plot.

E. Model performance with a subset of features

Feature analysis identified precipitation, temperature, and wind speed as low-use features for point forecasting and negatively contributing to uncertainty estimation. Removing them improved average test-set performance.

  • Performance after feature selection: Removing precipitation, temperature, and wind speed increased accuracy by around 6% in RMSE and around 10% in CRPS.The comparison used average forecasting metrics from test sets trained with all features versus a reduced feature set.
  • Performance after feature selection: The authors attribute the improvement possibly to a local optimum during training caused by the higher dimensionality of the full feature set.The excluded features may still help particular examples even though their overall impact can be negative.

VI. DISCUSSION

The discussion presents a transparent two-stage probabilistic framework that combines NGBoost forecasting with SHAP explanations for point forecasts and prediction intervals. Comparisons and feature analyses indicate improved performance, faster training, and physically interpretable nonlinear relationships, while the scope remains tied to the studied PV forecasting setting.

  • Problem setting: The discussion frames volatile, intermittent PV generation and limited interpretability of existing probabilistic models as barriers to safe power-system implementation.These constraints motivate transparent probabilistic forecasting for safety-critical power-system applications.
  • Proposed framework: The proposed framework targets accurate, reliable, transparent PV forecasts while minimizing expert knowledge and training requirements.It uses NGBoost for probabilistic forecasts and SHAP values for point forecasts and prediction intervals.
  • Forecasting performance: NGBoost performed better than GP and LUBE across deterministic and probabilistic metrics despite seasonal weather and power variations.The comparison covered both forecasting metric types and seasonal variation.
  • Forecasting performance: NGBoost trained more than ten times faster than the other two algorithms.The authors identify faster training as an advantage when feature and hyperparameter choices require repeated trials.
  • Interpretability: SHAP attributions exposed nonlinear feature relationships that followed known physical properties and human intuition for both point forecasts and prediction intervals.The authors present this interpretability as relevant to addressing limited trust in machine-learning models.
  • Feature selection: Discarding low-contribution features improved RMSE by around 6% and CRPS by around 10%.The discussion links feature analysis with reduced dimensionality and improved model performance, while noting that some features can remain useful for specific examples.
  • Practical relevance: The approach is intended for stakeholders such as system operators and traders whose decisions require transparency and involve financial risks.The authors position transparency as a way to use complex machine-learning models in settings where black-box reluctance matters.
Loading 2108.04058v2…