PaperlayerView exact revision on arXiv

Source-linked AI summary

Using SHAP Values and Machine Learning to Understand Trends in the Transient Stability Limit

Robert I. Hamilton, Panagiotis N. Papadopoulos

arXiv:2302.06274v1eess.SYcs.SY

TL;DR

Transient-stability assessment faces growing computational demands as renewables connect, while accurate ML models can be difficult to interpret. The paper uses SHAP to explain location-specific CCT predictions and analyze system-wide effects; its IEEE 39-bus case study shows feature effects beyond feature importance.

  • Problem

    Renewable integration increases the computational demands of transient-stability assessment, while accurate black-box ML models can be difficult to interpret.

    Source: §I · p. 1, §VI · p. 11

  • Method

    The paper applies SHAP to ML models predicting CCT at each busbar and uses covariance between a variable of interest and its location-specific SHAP values to examine stability-boundary trends.

    Source: §I · p. 2, §VI · p. 11

  • Results

    In the modified IEEE 39-bus case study, SHAP provided more detailed insights than permutation feature importance by capturing feature effects as well as feature importance.

    Source: §VI · p. 11

  • Takeaways & Limitations

    The locational insights could inform rules for system planners and operators about the system-wide impact of changes to a variable of interest.

    Source: §VI · p. 11

Abstract

from arXiv · show

Machine learning (ML) for transient stability assessment has gained traction due to the significant increase in computational requirements as renewables connect to power systems. To achieve a high degree of accuracy; black-box ML models are often required - inhibiting interpretation of predictions and consequently reducing confidence in the use of such methods. This paper proposes the use of SHapley Additive exPlanations (SHAP) - a unifying interpretability framework based on Shapley values from cooperative game theory - to provide insights into ML models that are trained to predict critical clearing time (CCT). We use SHAP to obtain explanations of location-specific ML models trained to predict CCT at each busbar on the network. This can provide unique insights into power system variables influencing the entire stability boundary under increasing system complexity and uncertainty. Subsequently, the covariance between a variable of interest and the corresponding SHAP values from each location-specific ML model - can reveal how a change in that variable impacts the stability boundary throughout the network. Such insights can inform planning and/or operational decisions. The case study provided demonstrates the method using a highly accurate opaque ML algorithm in the IEEE 39-bus test network with Type IV wind generation.

I. INTRODUCTION

Renewables complicate transient-stability behavior, while conventional assessment can be slow and difficult to interpret. The paper motivates SHAP as a way to explain location-specific CCT predictions and identify system-wide stability trends.

I. INTRODUCTION

  • Renewable integration can produce location-specific changes in the transient-stability boundary that depend on several parameters.The impact can vary, improving or deteriorating stability.Source: §I · p. 1
  • Time-domain simulations are computationally slow for CCT assessment, and their results can be difficult to interpret; sensitivity analysis does not reduce TSA computational burden.The burden grows when many operating conditions are considered.Source: §I · p. 1
  • TSA machine learning faces an accuracy–interpretability trade-off: simpler models may be less accurate, while higher-accuracy black-box models are difficult to interpret.Interpretability can build trust and understanding of prediction mechanisms.Source: §I · p. 1
  • Existing interpretation approaches have limits: LIME requires a difficult-to-define neighborhood, while PFI measures importance through model-performance decrease rather than feature effects.Feature effects are presented as useful for designing transient-stability enhancement measures.Source: §I · p. 2
  • SHAP provides local explanations that can be extended to consistent global explanations, feature effects in prediction units, and analysis of feature interactions.Its approximate Shapley values represent a feature’s average marginal contribution across coalitions.Source: §I · p. 2
  • The paper applies SHAP to CCT models at each busbar and uses covariance between a variable of interest and its location-specific SHAP values to identify stability-boundary trends.The authors distinguish this focus from prior power-systems SHAP applications in frequency stability, small-signal stability, and classification-based security assessment.Source: §I · p. 2

II. METHODOLOGY

The methodology builds operational scenarios, calculates location-specific CCTs and CCTmin, and trains corresponding ML models. SHAP explanations and covariance analysis are then used to examine feature effects and locational stability trends.

II. METHODOLOGY

  • The workflow calculates CCT for faults at each busbar and CCTmin, builds n+1 datasets, and trains n+1 ML models to predict those targets.The method can select an ML algorithm according to desired accuracy because SHAP is model-agnostic.Source: §II · pp. 2–3, Figure 1 · p. 3
  • SHAP values measure feature effects relative to a model’s expected baseline CCT, while covariance between a variable of interest and location-specific SHAP values identifies locational stability trends.A separate model trained for CCTmin also captures the variable’s impact on critical fault duration.Source: §II · p. 3

A. Generator Dispatch via AC Optimal Power Flow

  • AC OPF establishes generator dispatch for each operational scenario by minimizing generation cost subject to operating constraints and meeting demand.The resulting dispatch initializes dynamic studies and incorporates the modeled market dispatch into the scenarios.Source: A. Generator Dispatch via AC Optimal… · p. 3

B. Transient Stability and the Critical Clearing Time

CCT measures proximity to a fault-specific stability boundary, while the minimum across busbars captures the worst-case system boundary. The databases retain common features while varying the target, and feature selection determines which insights are available.

B. Transient Stability and the Critical Clearing Time

  • CCT measures proximity to the stability boundary for a given fault; CCTmin is the shortest busbar CCT and represents the worst-case system boundary.CCTmin also represents the minimum duration in which protection schemes must operate to maintain stability.Source: B. Transient Stability and the… · p. 3

C. Transient Stability Database Creation

  • Separate databases use unchanged features but different targets—the CCT at each busbar and CCTmin—yielding n+1 models.Each database row represents an operational scenario, with power-system variables as features and CCT as the target.Source: C. Transient Stability Database Creation · pp. 3–4

D. Feature Selection

  • Feature selection affects prediction accuracy and interpretability; this study uses pre-fault features, while post-fault data could change application suitability and activation windows.The selected pre-fault inputs cover synchronous generation, renewable resources, and system variables.Source: D. Feature Selection · p. 4

E. Machine Learning Algorithm Selection

The algorithm-selection process increases model complexity until a predefined accuracy threshold is met. Performance is assessed with average error metrics and maximum over- and under-estimation measures.

E. Machine Learning Algorithm Selection

  • Model complexity is increased until a predefined accuracy threshold is reached, rather than using unnecessary complexity.The selection is guided by the desired CCT estimation accuracy.Source: E. Machine Learning Algorithm Selection · p. 4

F. Performance Metrics

  • Performance comparisons average RSQ, MSE, and RMSE across all n+1 location-specific models for each algorithm.These metrics assess model performance against observed CCT values.Source: F. Performance Metrics · p. 4
  • MOE or MUE <0.02 sec is the accuracy threshold; critical-fault errors are also assessed when actual CCT <0.30 sec.The maximum over- and under-estimation errors distinguish risky overestimates from conservative underestimates.Source: F. Performance Metrics · p. 4

G. SHapley Additive exPlanations (SHAP)

SHAP explains predictions from complex models through feature attributions, supporting local and global interpretation across location-specific CCT models. Its explanations reflect the underlying model and do not establish causal effects.

G. SHapley Additive exPlanations (SHAP)

  • SHAP attributes feature effects to location-specific models predicting the transient stability limit, helping characterize the entire stability boundary.The framework uses an additive explanation model to approximate predictions from a complex, often black-box model.Source: G. SHapley Additive exPlanations (SHAP) · pp. 4–5, Figure 4 · p. 5
  • Shapley values represent average marginal feature contributions across coalitions and satisfy local accuracy, missingness, and consistency properties.SHAP values move the prediction from the training database's average CCT toward the current model output.Source: G. SHapley Additive exPlanations (SHAP) · p. 5
  • The paper uses DeepExplainer for neural networks, with approximate SHAP values whose computation time scales linearly with the number of background samples.Explainer choice involves trade-offs between model accuracy and speed, and SHAP implementation accuracy and speed.Source: G. SHapley Additive exPlanations (SHAP) · p. 5
  • SHAP explanations reflect any inaccuracy in the underlying model, and learned associations do not guarantee causal relations.Causal effects require further validation using domain knowledge or causal-inference methods.Source: G. SHapley Additive exPlanations (SHAP) · p. 5
  • Local SHAP values explain a prediction; aggregating scenarios produces a feature-by-scenario matrix, with a third dimension for the n+1 location-specific models.This matrix supports stability-boundary trend identification.Source: G. SHapley Additive exPlanations (SHAP) · pp. 5–6, Figure 5 · p. 6

H. Identification in Locational Trends in SHAP Values via Covariance

The method assesses covariance between a variable of interest and its location-specific SHAP values to characterize how their relationship varies across the network. Covariance signs indicate whether the variables move in the same or opposite directions.

H. Identification in Locational Trends in SHAP Values via Covariance

  • For each location, covariance between the actual variable of interest and its SHAP value reveals how that variable relates to the location-specific stability limit.The calculation yields n+1 covariance values per variable of interest, identifying locations with similar or dissimilar behavior.Source: H. Identification in Locational Trends… · p. 6, Figure 6 · p. 6
  • Positive covariance means the variables move in the same direction, while negative covariance means they move in opposite directions.The covariance is calculated across paired observations of the variable of interest and its SHAP value.Source: H. Identification in Locational Trends… · p. 6
  • Unlike PFI's ranked importance scores, SHAP provides feature effects; PFI importance is tied to model-performance decrease and does not indicate the effect on the target.PFI importance is calculated by shuffling a feature and measuring the resulting performance change.Source: H. Identification in Locational Trends… · p. 6

J. Implementation Details in a Realistic Setting

The study evaluates the method on an adapted IEEE 39-bus network, using scenarios that vary synchronous generation, renewable capacity, and demand. AC optimal power flow and RMS time-domain simulations produce CCT data for 25 fault locations.

A. Network Overview and Operational Scenarios

B. Generation of Operational Scenarios and AC OPF

  • Scenarios vary synchronous generation displacement, renewable capacity, and demand to represent operating conditions as renewables replace conventional generation.Demand ranges from 0.6 to 1.025 p.u.; renewable scaling includes high and very-high penetration cases.Source: B. Generation of Operational Scenarios… · p. 7
  • Of 3906 designed scenarios, 3762 (96.3%) converged and were passed to RMS time-domain simulations for CCT calculation.The scenarios comprise 3888 displacement cases and 18 cases without displacement.Source: B. Generation of Operational Scenarios… · p. 7
  • The case study evaluates three-phase-to-ground busbar faults at 25 locations, with 223 power-system variables in each location-specific dataset.Fault locations exclude synchronous-generator low-voltage terminals, renewable points of common coupling, and buses 1 and 9.Source: B. Generation of Operational Scenarios… · p. 7

IV. RESULTS

The study compares model accuracy and computational requirements for location-specific CCT estimation. The two-hidden-layer ANN is selected for its accuracy and reduced outliers; online inference is substantially faster than RMS time-domain simulation.

A. Accuracy

  • The two-hidden-layer ANN outperforms DT, RF, and XGBoost in accuracy and reduces outliers, so the authors use it for subsequent analyses.Its predictions closely replicate RMS time-domain simulation results; reducing large overestimates matters because they could contribute to loss of synchronism, cascading, and blackout.Source: A. Accuracy · p. 7

B. Computational Speed

  • 0.2 sec per inference is a 200-fold time reduction compared with RMS time-domain simulation.Training one ANN takes approximately 60 seconds, and SHAP explanations take under 1 second on the reported computer.Source: B. Computational Speed · pp. 7–8
  • Scaling to larger networks primarily adds offline RMS time-domain simulations for each busbar; DeepExplainer time scales linearly with observations.Reducing background samples can lower SHAP computation time but may reduce SHAP-value accuracy; this study uses the full training set as background.Source: B. Computational Speed · p. 8

C. Local SHAP Explanation: CCTmin Model

Local SHAP explanations decompose individual CCTmin estimates into feature effects, while global interpretations compare feature importance and effects across operating scenarios. The examples identify generator loading and renewable capacity as informative variables for CCTmin estimation.

C. Local SHAP Explanation: CCTmin Model

  • In the illustrated scenario, SHAP contributions move the 0.225 sec base value to a CCTmin estimate of 0.145 sec.The contributions sum to the difference between the expected value and the model estimate.Source: C. Local SHAP Explanation: CCTmin Model · p. 8, Figure 8 · p. 8
  • At 100% G09 loading, the highest-ranked feature reduces the prediction by 0.03 sec, while 20.7% G08 loading increases it by 0.01 sec.The lowest-ranked 214 features together add 0.01 sec from the base value.Source: C. Local SHAP Explanation: CCTmin Model · p. 8
  • Local explanations for short-CCT cases can help operators consider actions such as reducing G09 active power loading.The authors describe these explanations as a potential aid to operational decisions, not as a prescribed control action.Source: C. Local SHAP Explanation: CCTmin Model · p. 8

D. Global Interpretation via PFI: CCTmin Model

  • PFI and SHAP share 12 of their top 20 features, but SHAP captures feature interactions and reports feature effects that PFI does not.The importance-ranked PFI list and SHAP summary plot are shown in Figs. 9 and 10.Source: Figure 9 · p. 8, D. Global Interpretation via PFI:… · p. 9, Figure 10 · p. 9
  • G07 active power loading is the most important feature in the example: low values raise estimated CCTmin by up to 0.02 sec, while high values lower it by up to 0.015 sec.The summary plot's shared color scale does not give precise feature values; a separate dependence plot can examine G07 loading.Source: D. Global Interpretation via PFI:… · p. 9
  • The example associates low synchronous-generation capacity with reduced CCTmin estimates and high RES2 capacity with increased estimates.The authors note that these insights depend on the variables included in the training dataset.Source: D. Global Interpretation via PFI:… · p. 9

1) Detailed feature dependence insights:

The dependence analysis relates G07 loading and total renewable capacity to CCTmin predictions. It indicates a loading threshold for the direction of the G07 effect and an interaction with renewable capacity.

1) Detailed feature dependence insights:

  • Below 55% G07 loading, the SHAP effect on CCTmin is positive; above 55%, it is negative, with the effect also varying by total RES capacity.The dependence plot places G07 loading on the x-axis, its SHAP values on the y-axis, and total RES capacity on the color axis.Source: 1) Detailed feature dependence insights · p. 9, Figure 11 · p. 9
  • High RES capacity gives the largest CCTmin prediction improvement below 55% G07 loading, but the trend reverses above that threshold.The authors suggest maintaining G07 loading below 55%, particularly when RES capacity is high.Source: 1) Detailed feature dependence insights · pp. 9–10

F. Locational Trend Identification using SHAP Values

Covariance between a variable of interest and location-specific SHAP values reveals how variable changes align with stability-boundary trends across buses. The examples show both system-wide gains and local trade-offs, followed by a test of a SHAP-derived operational rule.

F. Locational Trend Identification using SHAP Values

V. EVALUATION OF INTERVENTIONS BASED ON SHAP INTERPRETATION RULES

  • B25 CCT lower extreme rose 0.19 to 0.25 sec (+0.06 sec or 32%), median 0.39 to 0.42 sec (+0.03 sec or 8%), and upper extreme 0.62 to 0.68 sec (0.06 sec or 10%).The rule was to avoid G08 disconnection; 423 scenarios were rerun without disconnecting it.Source: §V · p. 10, Figure 18 · p. 11

VI. CONCLUSIONS

On a modified IEEE 39-bus network with renewables, SHAP provided feature-effect insights beyond feature importance. The conclusion presents location-specific CCT models and covariance analysis as a way to reveal stability-boundary trends relevant to planning and operations.

VI. CONCLUSIONS

  • Transient stability is complex, high-dimensional, and location-dependent; TDS has computational limitations, while ML should be both accurate and interpretable for wider TSA adoption.Source: §VI · p. 11
  • The method trains CCT predictors for each busbar and an additional model for each scenario's minimum CCT, then applies SHAP to all n+1 models.Source: §VI · p. 11
  • On a modified IEEE 39-bus network with renewables, ANN reached acceptable accuracy, and SHAP provided more detailed insights than permutation feature importance by capturing feature effects as well as importance.Source: §VI · p. 11
  • Covariance between a power-system variable and locational SHAP values reveals system-wide stability-limit trends that could inform rules for planners and operators.Source: §VI · p. 11
Loading 2302.06274v1…