PaperlayerView exact revision on arXiv

Source-linked AI summary

Physics-Aware Distillation of a Grid Foundation Model for Real-Time Edge Inference

Vlad Alexandru Sănduleac, Shahariya Rasheed Rayamarakkarveettil Puthanpurayil, Noman Khan, Remus Teodorescu

arXiv:2610.21833v1cs.LG

TL;DR

Edge inference for low-inertia grids calls for models that fit the device and meet synchrophasor reporting frames, while existing GridFMs are sized for cloud or HPC. This paper distils a pretrained GridFM into a smaller student with physics-aware and representation-transfer losses; the student improves on its no-teacher counterpart, though the aggregated residual-transfer term degrades accuracy.

  • Problem

    GridFMs are too large for edge inference, and existing distillation does not transfer physical consistency to typed power-grid graphs.

    Source: §1 · p. 2

  • Method

    The study distils a frozen pretrained GridFM into a reduced student, combining the masked power-flow objective with output, physics-residual, latent-state and attention transfer.

    Source: §1 · p. 2, §3 · p. 3

  • Results

    Distillation lowers error by up to 48% against the same architecture trained without a teacher, while the aggregated physics-residual term degrades accuracy on all four quantities.

    Source: §5 · pp. 8–9

  • Takeaways & Limitations

    The resulting 32.5× smaller, half-depth student makes GridFM distillation relevant to edge inference.

    Source: §1 · p. 2, §5 · p. 8

    All configurations use a single seed, and the study reports no physics-residual metric, limiting conclusions about run-to-run variation and nodal power balance.

    Source: §4.1 · p. 8

Abstract

from arXiv · show

Grid foundation models (GridFMs) are graph transformers pretrained on a masked power-flow objective, and evaluate power flow orders of magnitude faster than an iterative solver. Their size, however, targets data-centre inference, whereas low-inertia operation increasingly asks for inference at the edge, next to the measurement, at batch size one and within a single synchrophasor (PMU) reporting frame. We distil a 20.1M-parameter, 12-layer GridFM teacher into a 0.62M-parameter, 6-layer student, a $32.5\times$ parameter reduction at half the message-passing depth, using an objective that combines the masked power-flow loss with output, physics-residual, latent-state and attention transfer. The physics-residual term, which matches the teacher's power-residual profile, has not previously been applied to typed power-grid graphs. We evaluate on a renewable variant of the IEEE 118-bus benchmark grid in which 26 of the conventional (synchronous) generators are replaced by inverter-based renewables with battery flexibility and standardised reactive-power limits (IEEE 1547), giving $97\,529$ physically valid (AC-OPF-feasible) operating scenarios, and we ablate all eight combinations of the transfer terms, reporting accuracy against ground truth and fidelity to the teacher separately. Distillation lowers student error on all four predicted quantities, by up to $48\%$ on active-power dispatch, and every distilled student is more accurate than its teacher on reactive-power dispatch. The gain is largely carried by the output matching term. The physics-residual term, in the layer-aggregated form used here, is consistently harmful, which we attribute to the spatial averaging it applies to a bus-level residual field. The resulting student is $32.5\times$ smaller in parameter count and has half the message-passing depth, making it a candidate for subsequent edge-device benchmarking.

1 Introduction

Low-inertia grid operation motivates local, frame-bounded inference, while GridFMs offer fast power-flow surrogates but are sized for cloud or HPC. The paper proposes physics-aware distillation to shrink a GridFM and evaluates it on a renewable IEEE 118-bus benchmark.

1 Introduction

  • Synchrophasors report every 20–100 ms, motivating local inference; GridFMs can solve power flow three to four orders of magnitude faster than iterative numerical solution.The 12-layer, 20-million-parameter teacher was designed for cloud or HPC, while edge inference requires batch-size-one execution within latency and memory budgets.Source: §1 · pp. 1–2
  • The proposed objective distils a pretrained GridFM into a 32.5× smaller, half-depth student using output, physics-residual, latent-state and attention transfer.The physics-residual term matches the teacher’s active/reactive residual profile rather than only penalising the student’s own residual.Source: §1 · p. 2
  • The renewable IEEE 118-bus benchmark replaces 26 synchronous units with converter-interfaced generators, battery flexibility and IEEE 1547 reactive bands, across 100,000 AC-OPF-feasible scenarios.Source: §1 · p. 2
  • The study ablates every combination of the four losses and reports ground-truth accuracy separately from teacher fidelity.The distinction matters because the two measures can diverge.Source: §1 · p. 2
  • The work includes edge deployment of a 0.6M-parameter student on an NVIDIA Jetson Orin, with measured latency and memory.Source: §1 · p. 2

2 State of the Art

Prior work develops grid GNNs, GridFMs, distillation, and edge-compression methods, but the paper identifies a gap at their intersection: physics-aware distillation and edge evaluation for typed grid models.

2.1 Related Work

  • Grid-model research spans physics-guided solvers, scalable GNN OPF, and GridFM pretraining, while topology generalisation remains an open problem; this work instead distils a pretrained GridFM.The closest named comparator is MxGPS, a 1.6M-parameter multiplex graph transformer trained jointly on state estimation and power flow [Papaioannou et al., 2026].Source: §2.1 · p. 2
  • Graph and transformer distillation transfers structure, hidden states, attention or predictions [Yang et al., 2020; Jiao et al., 2020], while physics-aware distillation targets PDE solvers [Bounja et al., 2025] and PDE foundation models [Musekamp et al., 2026].The reviewed approaches do not match teacher–student physical-residual distributions for power-grid models.Source: §2.1 · pp. 2–3
  • Edge compression methods include quantization, pruning and once-for-all supernets, but reported Jetson-class GNN measurements use generic graphs rather than grid models and omit physics metrics.The listed compression methods are orthogonal to distillation and can be combined with it.Source: §2.1 · p. 3

2.2 Research Gap

  • The paper addresses the uncombined gap by distilling a pretrained GridFM with output, latent-state, attention and residual-profile transfer, then ablating the losses on renewable IEEE 118 and reporting Jetson Orin latency and memory.Source: §2.2 · p. 3

3 Methodology

The methodology trains a reduced-parameter student on the masked power-flow task while transferring predictions and internal information from a frozen GridFM teacher. It evaluates the approach on a renewable IEEE 118-node system using AC-OPF-feasible operating scenarios.

3.1 Distillation

  • The student combines the shared masked power-flow objective with output, physics-residual, latent-state, and attention distillation from a frozen teacher.The comparison evaluates accuracy against ground truth, teacher fidelity, and physics residuals; architectural footprint is discussed separately.Source: §3 · p. 3, §3.1 · p. 4
  • Physics-residual distillation matches the teacher’s active- and reactive-power residual profile, summarized as a mean bus-level residual norm for each matched layer.This differs from penalizing only the student’s own residuals.Source: §3.1 · p. 4
  • Output transfer matches teacher predictions, latent transfer matches intermediate states, and attention transfer compares averaged maps when the student has fewer heads.The student’s layers are assigned to teacher layers for latent matching; the teacher’s weights remain frozen during training.Source: §3.1 · pp. 4–5

3.2 Domain

  • The IEEE 118-node benchmark is adapted by replacing 26 synchronous generators with flexible renewable generators using batteries and converter reactive-power control.The reactive-power band uses the wider of each replaced unit’s native range and the IEEE 1547 inverter requirement.Source: §3.2 · p. 5
  • Each load scenario is solved as a single-period AC OPF subject to power-flow equations, voltage and thermal limits, and unit bounds; non-converged cases are discarded.The resulting learning target maps load to a feasible state.Source: §3.2 · pp. 5–6
  • Of 100,000 generated load scenarios, 97,529 converge and are split by scenario into 78,025 training, 9,752 validation, and 9,752 test cases.No load profile appears in more than one split.Source: §3.2 · p. 6

4 Experiments

Distillation improves the student’s accuracy on all four predicted quantities, with output matching carrying most of the gain and the layer-aggregated physics-residual term consistently harmful. The experiments distinguish accuracy from teacher fidelity, while the discussion notes single-seed and deployment-measurement limits.

4 Experiments

  • 47.9% of baseline active-power dispatch error, 30.4% of voltage-angle error, 29.4% of voltage-magnitude error, and 28.6% of reactive-power dispatch error are removed by the best configurations.Every distilled configuration improves on the baseline student for every predicted quantity.Source: §4 · p. 6, Table 2 · p. 6
  • Output plus latent distillation is best for Vm, Va, and Pg, while Qg’s best configuration also includes attention distillation.The Qg errors span only 33.746–35.240 across distilled configurations, so their ordering should not carry an architectural conclusion on a single seed.Source: §4 · p. 7
  • All eight distilled configurations have lower Qg error than the teacher’s 35.692, whereas the undistilled baseline error is 47.236.The authors attribute this to distillation rather than the reduced architecture, while presenting the proposed explanation as plausible rather than isolated.Source: §4 · p. 7
  • Accuracy and teacher fidelity select different objectives for dispatch: output-only best matches the teacher on Pg, and output plus attention does so on Qg.The best configurations reduce teacher-referenced error by 57.4% on Vm, 56.2% on Pg, and 80.8% on Qg relative to the baseline.Source: §4 · p. 7
  • Output distillation dominates the marginal gains, while physics-residual distillation is harmful on all four quantities and attention effects are mostly unresolved against their spread.Latent distillation has a small reactive-power benefit; the output-KD estimate uses a single baseline-to-Lout contrast, unlike the other terms’ four contrasts.Source: §4 · p. 7, Table 3 · p. 7

4.1 Discussion

  • The aggregated residual loss discards where bus-level errors occur and can push the student toward worse physical consistency when the teacher’s residual is larger.The authors suggest testing per-bus, tail-weighted residual matching and a one-sided penalty as future work, not established results.Source: §4.1 · p. 8

4.2 Edge-Deployment Considerations

  • Single-seed runs and the absence of a physics-residual metric limit conclusions about smaller effects and possible power-balance gains.The 32.5× parameter reduction and 12-to-6-layer depth reduction do not establish edge-device latency or peak runtime memory; device benchmarking is deferred.Source: §4.1 · p. 8, §4.2 · p. 8

5 Conclusions

Distillation compresses the GridFM while recovering accuracy, but the transfer terms differ in value and teacher fidelity does not always track accuracy. The conclusions are bounded by a single seed and the lack of a physics-residual metric; the authors also disclose AI assistance.

5 Conclusions

  • Up to 48% lower error against a same-architecture student trained without a teacher, with every distilled student more accurate than its teacher on reactive-power dispatch.The 12-layer, 20.1M-parameter teacher is distilled into a 6-layer, 0.62M-parameter student on a renewable IEEE 118-bus variant.Source: §5 · p. 8
  • Output matching accounts for almost all transfer, latent matching adds a small reactive-power improvement, attention matching adds no separable gain, and physics-residual distillation degrades accuracy on all four quantities.The authors attribute the residual result to layer aggregation that collapses a bus-level field to one scalar and can incentivize reproducing the teacher’s physical error.Source: §5 · pp. 8–9
  • A single seed leaves smaller effects needing replication, and the absence of a physics-residual metric prevents judging whether the physics term trades accuracy for physical consistency.The 32.5× parameter reduction and halved depth are architectural properties, not latency claims; batch-one GNN latency must be established on-device.Source: §5 · p. 9

AI use statement

  • Generative AI assisted with literature search, drafting and editing, table formatting, arithmetic checks, and proofreading; the authors state they did not use it to generate ideas, data, experiments, or results.The authors say AI-suggested references were verified and AI-assisted text reviewed and revised.Source: AI use statement · p. 9

A Notation

The notation defines the teacher and student models, graph node and relation sets, matched layer pairs, and the quantities used in latent, residual, attention, and loss terms.

A Notation

  • fT and fS denote the frozen teacher and distilled student; Vb and Vg are batch-graph bus and generator nodes, Vτ denotes nodes of type τ, and T is the matched node-type set.T is typically {bus, gen}.Source: A Notation · p. 13
  • R and Er denote attention-matched relation types and their edge sets, while M contains matched student-teacher layer pairs with indices ℓS and ℓT.P_(ℓS,ℓT)_τ is a trainable projection from student latent dimension to teacher latent dimension.Source: A Notation · p. 13
  • The notation names bus and generator predictions, layerwise hidden states, active and reactive bus residuals, their mean norm, and head-averaged relation-specific attention weights.λout, λphys, λlat, and λatt weight the four distillation terms; LGraphKit is the standard GraphKit training loss.Source: A Notation · p. 13
  • The λ terms weight output, physics-residual, latent-state, and attention distillation, respectively, while LGraphKit denotes the standard GraphKit training loss.The residual notation includes layerwise active/reactive residuals and their mean bus-level norm; attention notation gives a head-averaged weight for each relation edge and layer.Source: A Notation · p. 13

Related papers

Loading 2610.21833v1…