PaperlayerView exact revision on arXiv

Source-linked AI summary

A Unified Mamba--MoE Surrogate for Closed-Loop Simulation and Measurement-Window Forecasting of Inverter Transients

Haoguang Wang, Huy Hoang Le, Akhila Kandivalasa, Christian Moya, Marcos Netto, Guang Lin

arXiv:2608.15051v1cs.LGeess.SY

TL;DR

Modeling inverter transients is difficult, and it is unclear whether one learned surrogate can support both closed-loop simulation and measurement-window forecasting. This paper develops a task-conditioned Mamba–MoE surrogate with calibrated prediction intervals, retaining low errors across both tasks with 13% fewer parameters than specialist models.

  • Problem

    It is unclear whether one learned surrogate can support both closed-loop simulation and measurement-window forecasting of inverter transients.

  • Method

    A task-conditioned Mamba backbone with mixture-of-experts routing, task-matched objectives, and adaptive conformal prediction serves both transient-prediction tasks.

  • Results

    The unified surrogate keeps both-task errors near a Mamba specialist pair while reducing parameters by about 13%; intervals achieve about 94–96% mean marginal coverage.

  • Takeaways & Limitations

    Expert routing lowers errors across all outputs in both tasks under the evaluated larger operating-point shift, while limited-data shared-head adaptation reduces held-out forecasting error.

Abstract

from arXiv · show

This paper proposes a Mamba surrogate model with mixture-of-experts (MoE) routing to represent the transient dynamics of inverter-based resources. A Mamba surrogate model is a predictive machine learning model built on the Mamba architecture. MoE routing uses a router network to assign data-dependent weights to specialized subnetworks (experts). The resulting Mamba--MoE surrogate can perform two tasks: (i) closed-loop simulation and (ii) measurement-window forecasting of inverter transients. A single Mamba backbone with task conditioning and expert routing serves both tasks, replacing two separate specialists. Task-matched objectives fit each prediction form, and an adaptive conformal layer provides prediction intervals for both tasks. For the considered grid-following inverter, the unified surrogate model remains in the same low-error regime as a Mamba specialist pair while using 13% fewer parameters. The prediction intervals achieve 94--96% empirical mean marginal coverage across the two tasks. For transient dynamics---that is, beyond the vicinity of an equilibrium point---our surrogate model with MoE routing yields lower errors across all outputs in both tasks compared to a shared Mamba backbone without expert routing. A controller hardware-in-the-loop simulation validates our results and shows that adapting only the shared output head with limited measured data reduces held-out forecasting error.

I. INTRODUCTION … B. Deep Learning Sequence Architectures for Time Series

The paper motivates data-driven inverter transient surrogates because vendor-specific nonlinearities make first-principles and generic models difficult, then unifies closed-loop simulation and measurement-window forecasting in a task-conditioned Mamba–MoE model. The related work spans measurement-based, recurrent, convolutional, operator-learning, Gaussian-process, and state-space approaches.

  • I. INTRODUCTION: IBR penetration can exceed 60% of generation in ERCOT, increasing the need for accurate inverter dynamics models.The paper identifies rapidly shifting power-system dynamics as the motivation for improved IBR modeling.
  • I. INTRODUCTION: Vendor-specific hardware, firmware, protection logic, limits, and nonlinear controls make first-principles and generic IBR models unreliable or difficult to identify.Unknown or confidential parameters and control schemes further limit model accuracy and portability.
  • I. INTRODUCTION: Terminal measurement data can capture aggregation, control interactions, operating-point dependence, delays, and nonlinear limits without requiring OEM controller disclosure.Such surrogates can support contingency screening, controller testing, real-time simulation, and operational decision support.
  • I. INTRODUCTION: Closed-loop simulation autoregressively rolls predictions forward, whereas measurement-window forecasting uses recent fixed-size data windows to predict future outputs.Simulation errors can accumulate over the horizon, while forecasting directly uses a recent measurement window without auxiliary inputs.
  • I. INTRODUCTION: A shared Mamba backbone with task conditioning, task-matched objectives, MoE routing, and adaptive conformal prediction serves both transient prediction tasks.The unified network replaces two separate specialists while providing prediction intervals for each task.
  • A. Data-Driven Surrogates for IBR Dynamics: Data-driven IBR surrogates include recurrent, convolutional, operator-learning, and Gaussian-process models, alongside measurement-based learning from synchro-waveform data.Interface-aware LSTM models enforce terminal voltage-current consistency but require inverter equations that may be unavailable.
  • B. Deep Learning Sequence Architectures for Time Series: Structured SSMs use fixed input-independent matrices, while selective SSMs such as Mamba dynamically depend on inputs to filter information like an attention mechanism.PowerMamba applies Mamba to quasi-steady-state forecasting of load, electricity prices, ancillary-service prices, and renewable generation.

C. Task Conditioning and Expert Routing … B. Unified Task Formulation

The paper unifies closed-loop simulation and measurement-window forecasting with a compact, task-conditioned Mamba surrogate using dense soft expert routing. It formulates both tasks over the same observed outputs while distinguishing autoregressive rollout from direct multi-step prediction.

  • C. Task Conditioning and Expert Routing: A compact dense soft-routing layer follows the shared Mamba backbone and conditions expert mixing on task identity and trajectory features.This design targets task-conditioned transient behavior in a power-system surrogate rather than primarily increasing capacity or training efficiency.
  • A. Inverter–Infinite-Bus System: The system model is an averaged standard grid-following inverter connected to an infinite bus, with PLL, outer power, inner current, and dc-link controllers.The branch current is modeled algebraically rather than as an independent differential state.
  • A. Inverter–Infinite-Bus System: The inverter model uses per-unit quantities on a 2.75-MVA, 60-Hz inverter base, while the 100-MVA system base maps 0.075-pu grid reactance to 0.00206 pu.The grid resistance is zero and the infinite-bus voltage magnitude is fixed at 1.00001 pu.
  • B. Unified Task Formulation: The observed output vector is not a complete Markov state, so task mappings use encoded observation history and recurrent state.The output representation includes two coordinate forms of the same branch current, while the PLL angle is inferred rather than separately output.
  • B. Unified Task Formulation: Both tasks share the same output vector but differ in inputs and prediction forms: simulation uses step-by-step rollout, whereas forecasting predicts H future outputs in one pass.Simulation includes current outputs, active power, and the time-varying active-power reference; forecasting instead uses a recent measurement window.
  • B. Unified Task Formulation: Simulation predicts one-step residuals from sampled data, initializes a latent state from history, and feeds each predicted output back as the next input.This autoregressive feedback can accumulate error with the rollout horizon.
  • B. Unified Task Formulation: Forecasting fills the unavailable simulation auxiliary channels with zero markers and predicts H future residuals directly from the latest W current-output samples.The zero values denote missingness rather than physical commands, and the forecasting mode is identified by a task embedding; evaluation uses fixed reference-step timing.

IV. PROPOSED MAMBA–MOE SURROGATE FRAMEWORK · A. Architecture Overview · B. Mamba Backbone and Task Conditioning

The unified Mamba–MoE framework conditions one Mamba backbone on simulation or forecasting tasks, combines task information with routing for expert mixing, and produces shared outputs. Its selective state-space recurrence and FiLM conditioning provide input-dependent dynamics while preserving standard layer normalization at initialization.

  • A. Architecture Overview: The unified forward path projects inputs into dmodel-dimensional representations, embeds the binary task label, routes representations through experts, and passes mixed outputs to a shared output Mamba.The task label is m=0 for simulation and m=1 for forecasting; its embedding conditions Mamba blocks through FiLM and contributes to routing.
  • A. Architecture Overview: A single Mamba backbone produces time-step hidden representations ht for both simulation and forecasting within the unified architecture.The backbone is shared across tasks rather than duplicated into separate task-specific backbones.
  • B. Mamba Backbone and Task Conditioning: The Mamba backbone models trajectory sequences with an input-modulated selective state-space recurrence over hidden representations ht and internal states st.The recurrence is input-dependent, allowing the state-space dynamics to respond to trajectory inputs.
  • B. Mamba Backbone and Task Conditioning: The implementation introduces input dependence through ∆t, while B and C remain static learned projections shared across the recurrence.The state matrix is parameterized as A = −exp(Alog), ensuring A < 0 and components of ¯At lie in (0, 1).
  • B. Mamba Backbone and Task Conditioning: The discretized recurrence uses the exact transition ¯At = exp(∆t ⊙A) and first-order Euler-type input discretization ∆t⊙(htB).This specifies the transition and input terms used by the implemented selective scan.
  • B. Mamba Backbone and Task Conditioning: FiLM maps task embeddings through an MLP to generate scale and shift terms that modulate layer normalization in every Mamba block.The modulation is γ = 1 + gFiLM and LNFiLM(u) = γ ⊙LN(u) + β; initialization sets gFiLM = 0 and β = 0.
  • B. Mamba Backbone and Task Conditioning: At initialization, FiLM reduces to standard layer normalization because its scale adjustment and shift are both zero.Specifically, gFiLM = 0 and β = 0 initially.

C. MoE Routing and Expert Mixing · D. Unified Output Head and Task-Specific Readout

The unified surrogate adapts a shared Mamba representation through task- and condition-dependent dense MoE routing, then uses a common residual output space with task-specific readouts. Simulation and forecasting differ through their hidden-state usage, output-block application, residual interpretation, and training objectives.

  • C. MoE Routing and Expert Mixing: MoE routing conditions expert selection on the hidden representation and the simulation or forecasting task embedding.The router operates on the concatenation of h_r and c_m; h_r is taken from the current rollout step for simulation and the measurement window’s final representation for forecasting.
  • C. MoE Routing and Expert Mixing: The router produces K expert logits from the concatenated hidden representation and task embedding, with expert weights obtained by softmax normalization.The routing network maps [h_r; c_m] to ℓ ∈ R^K before softmax weighting.
  • C. MoE Routing and Expert Mixing: Dense soft routing includes all experts in every forward pass, adding small overhead while avoiding load imbalance associated with sparse top-k routing.Expert mixing is jointly determined by current dynamic features and task identity.
  • D. Unified Output Head and Task-Specific Readout: The expert-mixed representation feeds a shared output Mamba block and unified linear head that generate an H-output residual matrix.The shared block produces h_o ∈ R^dmodel, and the head outputs residual rows r_1, ..., r_H.
  • D. Unified Output Head and Task-Specific Readout: The output block carries its SSM state across simulation rollouts but is applied once to the forecasting window’s final representation.In both tasks, the linear head—not the output Mamba block—produces the H residuals.
  • D. Unified Output Head and Task-Specific Readout: Because the residual output space matches the current-output vector dimension, one output head serves both simulation and forecasting.Forecasting uses all H residuals as horizon-specific corrections to the measurement window’s last observed output.
  • D. Unified Output Head and Task-Specific Readout: The unified head establishes a common output-residual space, while task differences are handled by input structure, FiLM conditioning, MoE routing, and task-specific objectives.This separates shared output representation from task-specific processing and training.

V. TASK-MATCHED TRAINING OBJECTIVES … C. Joint Objective and Algorithm

The unified Mamba–MoE surrogate is trained in two stages: task-agnostic backbone pretraining followed by joint optimization with simulation- and forecasting-matched losses. Joint training also uses shared-router regularization and an algorithm that enables task conditioning, FiLM, MoE routing, and the shared output head.

  • V. TASK-MATCHED TRAINING OBJECTIVES: Two-stage training first pretrains the Mamba backbone with one-step output residual prediction, then jointly trains simulation and forecasting according to their respective prediction forms.Pretraining learns a shared current-response representation before complete-model joint training.
  • A. Backbone Pretraining: During backbone pretraining, a temporary residual head predicts δx_t = x_t+1 − x_t from the final hidden representation of an output window.The task-conditioning vector is zero, producing a task-agnostic transition representation; H_PT is discarded afterward.
  • B. Task-Matched Losses: The simulation loss matches autoregressive rollout by evaluating predictions over H_roll rollout steps while feeding each predicted output back as the next input.Both tasks predict the same current outputs, but simulation uses its rollout prediction form.
  • B. Task-Matched Losses: The forecasting loss directly maps a recent measurement window to H future output residuals and supervises the full forecasting horizon.This objective differs from simulation because it does not use autoregressive rollout.
  • C. Joint Objective and Algorithm: Joint training optimizes both task losses together with load-balancing and expert-diversity regularizers computed from routing decisions pooled across simulation and forecasting.The router is shared by both tasks; load balancing uses average soft routing weights and top-1 assignment frequencies.
  • C. Joint Objective and Algorithm: Algorithm 1 initializes task conditioning at zero, performs N_PT pretraining updates on D_PT, then enables c_m, FiLM, MoE routing, and the shared output head for N_joint updates.Joint updates sample simulation and forecasting batches, compute both losses, record routing weights, and form the regularizers.
  • C. Joint Objective and Algorithm: Fig. 4 presents the adaptive conformal calibration workflow associated with the unified training and prediction framework.The supplied passage provides the workflow caption but no additional calibration procedure.
  • C. Joint Objective and Algorithm: The diversity term penalizes positive cosine similarity between different expert outputs, while for K < 2 diversity is zero and load balancing is omitted.Thus, the K = 1 ablation is trained without MoE regularization.

VI. ADAPTIVE PREDICTION-INTERVAL CALIBRATION · A. Calibration Challenges for Simulation and Forecasting · B. Task-Specific Conformal Base Widths

The paper calibrates prediction intervals with task-specific conformal widths and an online per-output scale factor that tracks target coverage. Calibration accounts for distinct simulation and forecasting residual structures, using horizon-wise simulation widths and closest-window-merged forecasting residuals.

  • VI. ADAPTIVE PREDICTION-INTERVAL CALIBRATION: Adaptive conformal inference computes task-specific widths on the calibration split and adapts a per-output scale factor online to track target coverage.This workflow is summarized in Fig. 4.
  • A. Calibration Challenges for Simulation and Forecasting: Standard conformal prediction offers distribution-free coverage under exchangeable residuals, but simulation and forecasting have different residual-dependence structures.Simulation rollout reuses predictions, while forecasting residuals are more homogeneous after closest-window merging; operating-point shifts can still change residual levels.
  • A. Calibration Challenges for Simulation and Forecasting: Coverage reported in §VII is empirical rather than a finite-sample distribution-free guarantee because calibration residuals come from overlapping temporal windows.
  • B. Task-Specific Conformal Base Widths: Simulation uses horizon-wise conformal widths because rollout uncertainty grows with horizon.The widths are defined for target miscoverage α, horizon h, and output dimension j using sorted absolute calibration residuals and calibration counts.
  • B. Task-Specific Conformal Base Widths: Forecasting calibration follows closest-window merging, selecting the most recent eligible forecast origin and its associated horizon for each target time.Eligible origins satisfy 1 ≤ t′ − t ≤ H and are drawn from each trajectory’s input-window observation times.
  • B. Task-Specific Conformal Base Widths: Merged forecasting predictions are used to compute residuals for each output dimension before sorting them to form forecasting base widths.
  • B. Task-Specific Conformal Base Widths: All base widths use only the calibration split.

C. Online ACI Scaling

Online ACI adapts conformal interval widths to operating-point mismatch by using sequential coverage feedback. Intervals are constructed prequentially, with clipping and stream-based resets preventing future test labels from entering interval construction.

  • Online ACI Scaling: ACI scales conformal base widths using sequential coverage feedback to address residual shifts from calibration–test operating-point mismatch.The method applies separate base widths for simulation and forecasting intervals.
  • Online ACI Scaling: Each output interval is formed as [x̂_u,j ± κ_u(j)q_u,j], with κ_u(j) = exp(z_u(j)).The scaling factor is applied separately for each output dimension and evaluated interval.
  • Online ACI Scaling: c = 5 bounds the log-scale update, while validation processing starts from z_1(j) = 0 and resets each test trajectory to the validation stream’s final state.These settings define the update range and initialization/reset procedure.
  • Online ACI Scaling: Intervals are constructed strictly prequentially before labels are revealed; labels only score coverage and update z_u+1(j), and merged targets update when available.Thus, future test labels do not enter interval construction.

VII. CASE STUDIES AND RESULTS · A. Experimental Setup and Baselines

The study evaluates the unified surrogate through accuracy, efficiency, ablations, robustness, cross-topology and cross-system adaptation, and interval reliability. Experiments use controlled transient trajectories, task-specific baselines, and calibrated prediction intervals under a 95% target coverage.

  • VII. CASE STUDIES AND RESULTS: Six evaluation perspectives cover clean-condition accuracy and model size, ablations, robustness, cross-topology forecasting, cross-system adaptation, and interval reliability.These perspectives organize the reported case studies and results.
  • A. Experimental Setup and Baselines: 150 offline transient trajectories are generated with MATLAB ode15s, recorded every 5 ms, and split into 80 training, 20 validation, and 50 test trajectories.All output channels receive z-score normalization, and each trajectory applies a 0.5 pu-to-pm active-power-reference step at t = 3 s.
  • A. Experimental Setup and Baselines: 0.1% G-NRMSE corresponds to current RMSE on the order of 10^-3 pu, indicating errors below one percent of the normalized per-unit current range.The comparable output dynamic ranges prevent unequal denominators from explaining cross-channel G-NRMSE differences.
  • A. Experimental Setup and Baselines: 95% is the target coverage for marginal coverage and mean prediction interval width evaluation across simulation rollouts and merged forecasting target-time predictions.The evaluation sets distinguish rollout points for simulation from closest-window-merged target-time predictions for forecasting.
  • A. Experimental Setup and Baselines: K = 4 experts, dmodel = 128, four Mamba blocks, dc = 32, ds = 16, W = 10, H = 24, and Hroll = 20 define the unified model configuration.Static CP fixes κu(j) = 1, whereas ACI updates conformal scale factors online.
  • A. Experimental Setup and Baselines: 5-ms-ahead forecasting errors and intervals are reported after unit-stride origins and closest-window merging select h = 1, although training supervises all H outputs.The setup uses λsim = λfcast = 1 and λLB = λdiv = 5 × 10^-4.
  • A. Experimental Setup and Baselines: DeepONet, Transformer, and Mamba are trained as separate simulation and forecasting specialists with task-matched objectives.The passage also specifies AdamW pretraining and joint-training learning rates, weight decay, and ACI hyperparameters.

B. Clean-Condition Comparison with Specialists · C. Ablation Analysis · D. Robustness Under Distribution Shift and Measurement Corruption

Under clean conditions, Mamba–MoE remains in the same low-error regime as a Mamba specialist pair while using about 13% fewer parameters. Ablations and robustness tests show that shared-head design, expert routing, both task losses, and resistance to measurement corruption affect performance in task- and shift-dependent ways.

  • B. Clean-Condition Comparison with Specialists: In simulation, Mamba–MoE reduces d/r output errors from about 10% for zero-delta to at most 0.21%.For q/i outputs, errors decrease from 2.72% to about 1.39%.
  • B. Clean-Condition Comparison with Specialists: The Mamba specialist pair provides the strongest clean-condition specialist reference, especially for forecasting icv_r, while DeepONet and Transformer do not consistently outperform Mamba–MoE across both tasks.Mamba–MoE does not exceed every specialist metric under clean conditions.
  • B. Clean-Condition Comparison with Specialists: Mamba–MoE remains in the same low-error regime as the Mamba specialist pair while using about 13% fewer parameters.The unified model has 800,260 parameters versus 921,316 for the specialist pair.
  • C. Ablation Analysis: Two task-specific heads reduce forecasting d/r G-NRMSE from 0.10%/0.12% to 0.03%/0.03%, but neither head configuration dominates all tasks and outputs.Simulation errors change from 0.14%/0.21% to 0.15%/0.16%, motivating retention of the shared head.
  • C. Ablation Analysis: K = 4 gives lower errors for more outputs than K = 1, although K = 1 is better for simulation ifilt_r.The routing advantage increases as operating points move farther from the training range.
  • C. Ablation Analysis: Removing FiLM lowers simulation d/r G-NRMSE from 0.14%/0.21% to 0.13%/0.15% but raises forecasting errors from 0.10%/0.12% to 0.13%/0.17%.Its effect is task-dependent, while task conditioning through the router remains.
  • C. Ablation Analysis: Removing either supervised loss sharply increases error, with forecasting d/r G-NRMSE reaching 18.70%/22.74% when λfcast = 0 and simulation errors reaching 65.65%/121.70% when λsim = 0.Both losses are required under the current unified-surrogate training objective.

E. Cross-Topology Forecasting and Adaptation: 9-Bus System · F. Cross-System Adaptation to CHIL Measurements

On the 9-bus system, fine-tuning improves five-ms-ahead forecasting across distribution-shifted operating regimes, while CHIL adaptation sharply reduces cross-system forecasting error. The adaptation updates only the shared output head using limited measured data.

  • E. Cross-Topology Forecasting and Adaptation: 9-Bus System: The 9-bus evaluation uses 200 ID, 50 OOD-1, and 50 OOD-2 trajectories spanning pm ranges of [0.3, 0.7], [0.7, 0.9], and [0.9, 1.1] pu.Fine-tuning uses 160 ID trajectories, with the remaining 40 reported as ID validation data; both OOD sets are reserved for evaluation.
  • E. Cross-Topology Forecasting and Adaptation: 9-Bus System: Fine-tuning lowers zero-shot G-NRMSE in all 12 evaluated set–channel comparisons on the 9-bus system.G-NRMSE is normalized by each target set’s per-channel range, so comparisons are made within each set.
  • F. Cross-System Adaptation to CHIL Measurements: 13.09% to 3.07%: mean normalized RMSE decreases after adapting the shared output head with the first 10 s (5.1%) of CHIL data.Adaptation uses only the first-horizon loss; evaluation is performed five ms ahead after normalization by each channel’s test-block range.
  • F. Cross-System Adaptation to CHIL Measurements: The CHIL values are not converted to the offline benchmark’s pu base during cross-system evaluation.The channel mapping pairs measured currents in the PLL-aligned dq frame with the offline output representation.

G. Interval Reliability: Static CP and ACI · VIII. CONCLUSION

The section evaluates static conformal prediction (CP) and adaptive conformal inference (ACI) for simulation and forecasting, then concludes that a unified Mamba–MoE surrogate preserves specialist-level accuracy with fewer parameters. ACI provides near-target coverage, while expert routing and limited output-head adaptation improve transient and held-out forecasting performance.

  • G. Interval Reliability: Static CP and ACI: ACI raises simulation MCOV from 91.70% to 94.47%, below but closer to the 95% target.It also reduces mean MPIW from 0.005648 to 0.004823.
  • G. Interval Reliability: Static CP and ACI: Forecasting static CP reaches 94.55% MCOV with mean MPIW 0.002072.ACI shifts forecasting coverage to 96.04%, according to the supplied passage.
  • G. Interval Reliability: Static CP and ACI: Simulation residuals are more nonstationary than forecasting residuals because rollout errors evolve over time.Closest-window merging produces more homogeneous forecasting residuals.
  • VIII. CONCLUSION: Mamba–MoE uses one temporal representation for shared transient dynamics, with task conditioning and expert routing adapting task-dependent residuals.The unified surrogate supports autoregressive simulation and single-pass multi-step forecasting from a measurement window.
  • VIII. CONCLUSION: 13% fewer parameters keeps both-task errors close to those of a Mamba specialist pair.This result is reported on the inverter benchmark.
  • VIII. CONCLUSION: 94–96% mean marginal coverage brings ACI intervals close to the 95% target.At the larger operating-point shift, expert routing lowers errors across all outputs in both tasks.
  • VIII. CONCLUSION: Shared-output-head adaptation with limited measured data reduces mean held-out forecasting error in the cross-system CHIL study.The supplied passage identifies this as a cross-system controller hardware-in-the-loop evaluation.
Loading 2608.15051v1…