PaperlayerView exact revision on arXiv

Source-linked AI summary

LLaTSA: Large Language Model-Aligned General-Purpose Transient Stability Analysis

Chao Shen, Hongwei Zhen, Junyan Shao, Zhenghao Yang, Yifan Zhang, Mingyang Sun

arXiv:2609.14374v1eess.SYcs.AIcs.SY

TL;DR

Existing trajectory predictors often require substantial adaptation across changing system configurations, while Uni-TSA has limitations involving short observations, coupling, representation mismatch, and dense-backbone computation. LLaTSA combines scenario-aware conditioning, vocabulary-guided alignment, a sparse MoE backbone, coupling, and rollout-based training. Component ablations show reduced trajectory accuracy and stability discrimination when designs are removed.

  • Problem

    General-purpose trajectory prediction remains limited by Uni-TSA’s short-observation ambiguity, omitted state-variable coupling, numerical–textual mismatch, and dense-backbone computation.

    Source: A. Limitations of Uni-TSA for… · pp. 2–3

  • Method

    LLaTSA combines scenario-aware textual conditioning, vocabulary-guided trajectory alignment, a pretrained sparse MoE backbone, state-variable coupling, and rollout-based training.

    Source: §I · p. 2, E. State Variable Coupling for Dynamic… · p. 7, A. Curriculum Training · p. 8

  • Results

    Removing components reduced trajectory accuracy and stability discrimination; removing vocabulary alignment raised NRMSE from 0.31 to 0.43 in-range and from 0.48 to 0.68 under OOD conditions.

    Source: E. Ablation Study and Backbone… · p. 11

  • Takeaways & Limitations

    LLaTSA is presented as a framework for trajectory-based TSA across diverse operating conditions, contingencies, and power-system configurations.

    Source: §VI · p. 12

    Uni-TSA remains constrained in heterogeneous systems by short-observation discrimination, state coupling, representation mismatch, and dense-backbone computation.

    Source: A. Limitations of Uni-TSA for… · pp. 2–3

Abstract

from arXiv · show

Dynamic trajectory prediction has become an important paradigm for data-driven transient stability analysis (TSA), yet most existing predictors remain system-specific and require substantial retraining when network configurations, generation mixes, or state-variable sets change. Uni-TSA introduced a general-purpose TSA framework that combines channel-independent modeling with a pretrained large language model (LLM) predictor. Nevertheless, its application to heterogeneous systems is limited by ambiguity in short observations, a mismatch between numerical trajectories and LLM embeddings, neglected coupling among state variables, and the high inference cost of dense backbones. This paper proposes LLaTSA, an LLM-aligned framework for general-purpose trajectory-based TSA. LLaTSA first incorporates operating conditions, disturbance attributes, and state-variable identity through a structured textual prefix. It then aligns normalized temporal patches with a TSA-related vocabulary before processing them with a pretrained sparse decoder-only mixture-of-experts (MoE) backbone. A state-variable coupling module captures coordinated post-fault evolution, while teacher forcing and rollout-based training support iterative long-horizon prediction. Case studies on multiple test systems demonstrate accurate trajectory prediction, reliable stability discrimination, and effective adaptation across unseen scenarios.

I. INTRODUCTION

TSA needs accurate, efficient post-disturbance assessment, while data-driven trajectory prediction preserves temporal information but often assumes fixed system configurations. LLaTSA responds to the limitations of general-purpose prediction with scenario-aware conditioning, trajectory alignment, sparse MoE modeling, state-variable coupling, and rollout-based training.

A. Power System Transients and Data-driven TSA Paradigms

  • Time-domain simulation computes trajectories accurately but is costly for large-scale screening, while direct methods face practical limits from model dependence, conservatism, and implementation complexity.These constraints have motivated data-driven TSA for faster security assessment.Source: §I · p. 1
  • Trajectory prediction preserves time-resolved post-fault dynamics for stability and control analysis, unlike classification outputs that compress responses into labels or scalar indices.Predicted channels may include rotor angles, rotor speeds, and bus voltages.Source: §I · p. 1, A. Power System Transients and… · p. 2

I. INTRODUCTION

  • Most trajectory-prediction methods assume fixed system configurations, often requiring data-intensive redesign and retraining as system size, topology, or component mix changes.This motivates general-purpose prediction across evolving networks and generation portfolios.Source: §I · p. 1
  • LLaTSA combines scenario-aware textual conditioning, vocabulary-guided trajectory alignment, sparse MoE modeling, state-variable coupling, and rollout-based training for general-purpose trajectory-based TSA.Its stated motivation includes short-observation ambiguity, numerical–textual mismatch, omitted inter-variable dependence, and dense-backbone inference cost.Source: §I · p. 2

B. General-Purpose Trajectory Prediction for TSA

General-purpose TSA must accommodate changing system dimensions and heterogeneous dynamics; Uni-TSA’s channel-independent approach helps, but leaves several limitations. LLaTSA proposes conditioning, alignment, sparse modeling, and coupling to address them.

B. General-Purpose Trajectory Prediction for TSA

  • General-purpose TSA must accommodate changes in state-variable dimensions, heterogeneous system dynamics, and errors that can accumulate during short-window rollout.Uni-TSA uses channel-independent sequences and a shared pretrained LLM predictor, with teacher-forcing and scheduled-sampling fine-tuning; reported results include zero-shot generalization and few-shot adaptation using 5% target-system data.Source: B. General-Purpose Trajectory… · p. 2

A. Limitations of Uni-TSA for General-Purpose TSA

1) Insufficient Discrimination Under Short Observations:

  • Similar short channel prefixes can precede divergent futures across operating-disturbance cases, making long-horizon prediction from the univariate prefix ambiguous.The future trajectories may diverge because of different operating conditions or disturbance attributes.Source: 1) Insufficient Discrimination Under… · p. 3

2) Weak Representation of Coupled State Evolution:

  • Uni-TSA’s separate channel predictions omit correlations among state variables, although stable-case synchronous-generator rotor-angle trajectories have pairwise correlations above 0.9.Independent predictions may therefore be inconsistent with coordinated generator dynamics.Source: 2) Weak Representation of Coupled… · p. 3

3) Modality Gap Between Numerical Trajectories and Textual Representations:

  • Normalized trajectory and text embeddings show a center gap of 1.085 and MMD of 0.5236, indicating a modality gap in the shared embedding space.The passage identifies this as a potential limit on directly reusing a text-pretrained backbone for trajectory modeling.Source: 3) Modality Gap Between Numerical… · p. 3

4) Capacity-Efficiency Tradeoff in Dense Backbones:

  • Larger dense Uni-TSA backbones reduce trajectory error, but the gains are not proportional to model size and complete-trajectory inference time rises rapidly.LLaTSA’s proposed design uses sparse MoE routing and state-variable coupling to address dense scaling’s capacity–efficiency tradeoff.Source: 4) Capacity-Efficiency Tradeoff in… · p. 3

B. Dynamic Trajectory Encoding and Representation Alignment

LLaTSA prepares post-disturbance trajectories for a pretrained LLM by encoding numerical sequences and aligning them with textual representations. The procedure forms input–target windows, processes each state variable independently, and encodes ordered temporal patches.

B. Dynamic Trajectory Encoding and Representation Alignment

  • Numerical trajectory embeddings remain separated from textual embeddings, motivating sequence encoding and vocabulary-guided alignment before LLM prediction.LLaTSA decomposes, normalizes, and segments trajectories into temporal patches for alignment with pretrained token representations.Source: B. Dynamic Trajectory Encoding and… · p. 4

1) Dynamic Response Input-Output Pair Construction:

2) State-Variable Decomposition and Normalization:

  • State-variable decomposition lets the shared encoder handle different state counts, while independent normalization reduces scale differences among heterogeneous variables.The normalization statistics are computed from each input trajectory segment.Source: 2) State-Variable Decomposition and… · p. 4

3) Temporal Segmentation and Linear Encoding:

  • Overlapping normalized trajectory segments are mapped to the LLM dimension and positionally encoded to preserve temporal order.The resulting representations encode local post-disturbance variations across state variables.Source: 3) Temporal Segmentation and Linear… · pp. 4–5

4) State-Trajectory Vocabulary Alignment:

LLaTSA aligns trajectory patches with a compact set of pretrained vocabulary embeddings related to transient stability analysis. Cross-attention links each temporal segment to selected vocabulary entries and incorporates their textual information.

4) State-Trajectory Vocabulary Alignment:

  • A compact TSA-related vocabulary subset avoids irrelevant tokens from the full vocabulary and supplies the basis for trajectory alignment.The subset is retrieved from the frozen LLM vocabulary by selecting top-m tokens nearest to seed-token embeddings.Source: 4) State-Trajectory Vocabulary Alignment · p. 5
  • Cross-attention uses trajectory representations as queries and selected vocabulary embeddings as keys and values to associate patches with textual entries.The attention weights characterize the correspondence between each temporal segment and selected vocabulary entries.Source: 4) State-Trajectory Vocabulary Alignment · p. 5
  • Weighted vocabulary information is incorporated into the trajectory representation, reducing the modality gap before subsequent LLM-based prediction.The selected entries come from the pretrained vocabulary and are related to TSA.Source: 4) State-Trajectory Vocabulary Alignment · p. 5

C. Scenario-Aware Textual Prefix Encoding

LLaTSA prepends a textual prefix to the aligned trajectory representation to provide context alongside short post-fault observations. The prefix encodes task, operating-condition and disturbance, and state-variable descriptions.

C. Scenario-Aware Textual Prefix Encoding

  • Short post-fault trajectories may not distinguish cases with similar initial responses but different subsequent behavior, motivating additional scenario context.The prefix is prepended to the aligned trajectory representation.Source: C. Scenario-Aware Textual Prefix… · p. 5
  • The prefix combines task, operating-condition and disturbance, and state-variable descriptions for each sample and variable.Its records are tokenized with the pretrained LLM vocabulary and mapped to the backbone dimension.Source: C. Scenario-Aware Textual Prefix… · p. 5
  • Prepending the prefix yields a scenario-conditioned trajectory representation for the subsequent LLM backbone.Source: C. Scenario-Aware Textual Prefix… · p. 5

D. MoE-Based Pretrained LLM Backbone for Dynamic Trajectory Modeling

The backbone processes augmented trajectory representations through causal-attention and sparse-expert blocks. Causal masking lets trajectory segments use scenario information and preceding segments while blocking future segments.

D. MoE-Based Pretrained LLM Backbone for Dynamic Trajectory Modeling

  • Each shared backbone block applies causal self-attention followed by a sparsely activated MoE feed-forward sublayer.The backbone maps the augmented representation toward temporal representations for trajectory prediction.Source: D. MoE-Based Pretrained LLM Backbone… · p. 5
  • The framework’s backbone and coupling stages process augmented tokens before trajectory prediction.The figure identifies causal attention and sparse expert routing in the backbone.Source: Figure 5 · p. 6

1) Causal Self-Attention Mechanism:

2) MoE Feed-Forward Network:

The MoE feed-forward layer routes representations to selected experts, while a separate coupling module models relationships among state variables at each temporal patch.

2) MoE Feed-Forward Network:

  • The router computes routed-expert probabilities and activates only the top-K routed experts at each position.Source: 2) MoE Feed-Forward Network · p. 6
  • When shared experts are used, they remain active for all representations alongside the selected routed experts.The formulation also accommodates checkpoints without shared experts.Source: 2) MoE Feed-Forward Network · pp. 6–7
  • Sparse routing supports capacity for diverse post-fault responses while controlling activated computation relative to dense scaling.Source: 2) MoE Feed-Forward Network · p. 7

E. State Variable Coupling for Dynamic Trajectory Prediction

F. Post-Fault Dynamic Response Prediction

The prediction head transforms each state variable’s encoded representation into a response over the prediction horizon, then stacks the responses into a post-fault trajectory.

F. Post-Fault Dynamic Response Prediction

  • For each state variable, the encoded representation is flattened across patches and features, then truncated before response prediction.Source: F. Post-Fault Dynamic Response… · p. 7
  • Shared output parameters map each retained representation to its state-variable response over the prediction horizon.Stacking the predicted responses forms the post-fault trajectory.Source: F. Post-Fault Dynamic Response… · p. 7
  • Because the output parameters are not tied to the number of state variables, the same output form applies across systems with different state-variable counts.Source: F. Post-Fault Dynamic Response… · p. 7

IV. MODEL TRAINING AND ONLINE APPLICATION

LLaTSA predicts post-fault responses iteratively by appending each predicted segment to the trajectory record. Training uses teacher forcing followed by rollout-based training to align the model with this rolling application.

IV. MODEL TRAINING AND ONLINE APPLICATION

  • The two-stage curriculum combines teacher forcing with rollout-based training for iterative post-fault response prediction.Source: §IV · p. 7

A. Curriculum Training

  • Teacher forcing conditions the model on ground-truth trajectory samples and initializes it for rollout-based training.Source: A. Curriculum Training · p. 8
  • Rollout-based training appends predicted segments to the available record and penalizes accumulated prediction error along the rolling trajectory.The rollout loss aligns training with online TSA application.Source: A. Curriculum Training · p. 8

B. Online Application for TSA

  • Online, the model repeatedly uses the latest input window, appends each predicted segment, and updates the record for the next rollout step.The resulting trajectory can support stability judgment, stability-margin estimation, or emergency-control screening.Source: B. Online Application for TSA · p. 8

V. CASE STUDIES

The case studies span conventional and IBR-modified systems, varying renewable penetration, operating conditions, and contingencies. The dataset holds out selected systems and scenario factors for generalization tests.

1) Simulation Settings and Model Configuration:

  • Tests cover four conventional systems, an IBR-modified IEEE 39-bus system, and renewable penetration levels from 0% to 40%.The IBR-modified system uses VSG-controlled photovoltaic generation at 20% and 40% penetration.Source: 1) Simulation Settings and Model… · p. 8
  • Fault trajectories come from time-domain simulations, with DFIG cases spanning N-1, N-2, and N-3 contingencies and the IBR case using a short electromagnetic-transient simulation.The IBR simulation uses a 10^-5-s time step; its trajectories are resampled to 10^-3 s for model input.Source: 1) Simulation Settings and Model… · p. 8

2) Dataset Organization:

  • Scenarios are organized by system configuration, renewable penetration, and contingency order, with each instance identifying a system, penetration level, and contingency.The configurations include S39, S68, S118, S189, and SIBR.Source: 2) Dataset Organization · p. 8
  • Training excludes the IEEE 118-bus and IBR-modified systems, 40% renewable penetration, and N-3 contingencies, reserving them for out-of-distribution evaluation.Validation uses the training scenario range but disjoint operating conditions and fault locations.Source: 2) Dataset Organization · p. 9
  • Trajectory accuracy uses MAE and NRMSE, while stability discrimination uses unstable-class recall and balanced accuracy.Balanced accuracy weights stable- and unstable-class recall equally under class imbalance.Source: 2) Dataset Organization · p. 9

B. General-Purpose Dynamic Prediction Performance

On unseen operating conditions and fault locations, LLaTSA improves trajectory and stability metrics over Uni-TSA. It also generalizes to held-out scenario factors and adapts to unseen systems with limited target data.

B. General-Purpose Dynamic Prediction Performance

  • Average MAE falls from 0.83 to 0.20 and average NRMSE from 0.71 to 0.31 versus Uni-TSA; unstable-class recall rises from 82.4% to 96.2% and balanced accuracy from 84.9% to 96.8%.LLaTSA is best across all four metrics on unseen operating conditions and fault locations within the training scenario range.Source: Table I · p. 9, B. General-Purpose Dynamic Prediction… · p. 9
  • On held-out N-3 contingencies and 40% renewable penetration, LLaTSA remains best across all four metrics, with unstable-class recall above 94% and balanced accuracy above 95%.The results assess extrapolation to the two held-out scenario factors in the evaluated benchmark systems.Source: Table II · p. 9, B. General-Purpose Dynamic Prediction… · pp. 9–10

C. Cross-System and Cross-Dynamics Prediction

  • With 20% target-system data, LLaTSA beats a full-data target-specific LSTM on every reported metric for both unseen systems.On IEEE 118-bus, LLaTSA’s MAE is 0.58 versus 0.85 for the LSTM, and balanced accuracy is 75.1% versus 70.1%.Source: Table III · p. 10, C. Cross-System and Cross-Dynamics… · pp. 10–11
  • On the IBR-modified system, 20%-data LLaTSA lowers MAE from 0.78 to 0.37 and NRMSE from 0.62 to 0.40 versus the full-data LSTM.The IBR case includes VSG-controlled photovoltaic dynamics and resampled electromagnetic-transient trajectories.Source: C. Cross-System and Cross-Dynamics… · p. 11
  • At full adaptation on the IBR-modified system, LLaTSA improves MAE, NRMSE, unstable-class recall, and balanced accuracy over Uni-TSA.The respective values are 0.19 versus 0.76, 0.20 versus 0.45, 95.2% versus 91.3%, and 96.0% versus 94.1%.Source: C. Cross-System and Cross-Dynamics… · p. 11

D. Computational Cost for Online TSA

Under the stated protocol, LLaTSA combines lower NRMSE with substantially lower inference time than Uni-TSA at the largest configurations. Ablations and interpretability analyses examine its representation components, pretrained MoE backbone, and vocabulary-guided alignment.

D. Computational Cost for Online TSA

  • At the largest configurations, LLaTSA reaches an NRMSE of 0.31 versus 0.71 for Uni-TSA while retaining substantially lower end-to-end time.The comparison is a measured latency comparison under the stated protocol, not a parameter-count-normalized comparison, because the MoE activates only a subset of routed experts per token.Source: Figure 8 · p. 11, D. Computational Cost for Online TSA · p. 11

E. Ablation Study and Backbone Comparison

  • Removing vocabulary-guided alignment raises NRMSE from 0.31 to 0.43 in-range and from 0.48 to 0.68 under OOD conditions; removing both input-side components worsens target-system results.The combined removal raises IEEE 118-bus NRMSE from 0.72 to 1.02 and reduces IBR balanced accuracy from 90.1% to 84.3%; the three components make complementary contributions in evaluated settings.Source: E. Ablation Study and Backbone… · p. 11, Table IV · p. 12
  • Replacing OLMoE-1B-7B with Qwen1.5-MoE-A2.7B further improves performance across evaluated settings, particularly under distribution shift and limited-data transfer.The reported gains indicate that the proposed components remain effective across pretrained MoE backbones, while a stronger backbone provides additional capacity for heterogeneous post-fault dynamics.Source: E. Ablation Study and Backbone… · pp. 11–12

F. Interpretability Analysis

  • During training, patch-to-vocabulary attention becomes concentrated at channel- and patch-dependent locations rather than uniformly averaging candidate embeddings.This is consistent with selective encoding of early response features relevant to subsequent trajectory evolution.Source: F. Interpretability Analysis · p. 12, Figure 9 · p. 12
  • Projected numerical and textual representations have smaller within-backbone separation for LLaTSA than for Uni-TSA.Because the methods use different native embedding spaces and vocabularies, only this within-backbone comparison is meaningful.Source: F. Interpretability Analysis · p. 12
  • High-attention vocabulary entries describe rising, falling, oscillatory, damped, settling, and recovering trajectory behavior, without assigning tokens unique physical labels.The analysis is consistent with numerical response patches being placed in a representation space better matched to the pretrained backbone.Source: F. Interpretability Analysis · p. 12

VI. CONCLUSION

  • Across the reported comparison, LLaTSA reduces average NRMSE from 0.71 to 0.31 and increases balanced accuracy from 84.9% to 96.8%.On both held-out target systems, adaptation with 20% target-system data surpasses a target-specific LSTM trained on the complete target dataset.Source: §VI · p. 12

Related papers

Loading 2609.14374v1…