PaperlayerView exact revision on arXiv

Source-linked AI summary

PowerZooJax: A JAX-based Power System Benchmark for Reinforcement Learning

Zhanhua Pan, Xiao Liu, Zhilong Cao, Jianhong Wang, Dawei Qiu

arXiv:2609.36052v1cs.AIcs.LGcs.SYeess.SY

TL;DR

Power-system RL benchmarks are often narrow in scope and computationally limited by CPU-based simulation. PowerZooJax provides five JAX-based CMDP tasks; across these settings it standardizes reporting of returns, safety costs, and stress-test performance, with transmission results illustrating an economic-safety trade-off.

  • Problem

    Power-system RL environments often use task-specific physical models and CPU-based simulation, limiting efficient large-scale evaluation.

    Source: §1 · p. 2

  • Method

    PowerZooJax provides a JAX-based benchmark with five operational tasks under a consistent CMDP interface and GPU-based compiled, vectorized rollouts.

    Source: §1 · p. 2

  • Results

    Across five tasks, PowerZooJax supports standardized reporting of returns, safety costs, and stress-test performance; in transmission, PPO-Lagrangian eliminates reserve shortfalls and reduces thermal overloads to 6% at £3.50M/day.

    Source: §6.3 · p. 8, §7 · p. 9

  • Takeaways & Limitations

    The benchmark supports evaluation of RL algorithms across diverse power-system settings under a common CMDP interface, including operational returns, safety costs, and stress-test performance.

    Source: §7 · p. 9

    The suite has limited grid scale, uncertainty modeling, operational constraints, and task diversity, and its public or synthetic models cannot fully capture real-time power-system operation.

    Source: §7 · p. 9

Abstract

from arXiv · show

Power system operation is a safety-critical sequential decision-making problem, making it a natural testbed for reinforcement learning (RL). However, existing RL environments for power systems are often narrow in scope and computationally limited by CPU-based simulation workflows, making large-scale evaluation difficult. We introduce PowerZooJax, a JAX-based benchmark suite for RL in power system operation. It provides five constrained Markov decision process tasks spanning generation, transmission, distribution, distributed energy resources, and data center microgrid. By rewriting power flow, economic dispatch, market clearing, and device dynamics as JAX computation graphs, PowerZooJax keeps the entire training and evaluation loop on the GPU. Experiments show substantial speedups over CPU-based simulations and demonstrate standardized evaluation of policy returns, safety violations, and out-of-distribution stress conditions. Our open-source benchmark is available at: https://github.com/powerzoojax/PowerZooJax.

1 Introduction

Power system operation is safety-critical and constrained, but existing RL environments face modeling differences and CPU-based simulation bottlenecks. PowerZooJax introduces a JAX benchmark with five tasks and a shared CMDP interface for scalable, standardized evaluation.

1 Introduction

  • Power system operation requires sequential decisions under uncertainty while satisfying strict physical, operational, and security constraints.Operators must balance supply and demand, manage constrained flows, coordinate distributed resources, and maintain reliable operation.Source: §1 · p. 2
  • Different power-system models and CPU-based simulation workflows limit efficient large-scale RL training and evaluation.CPU simulation and repeated CPU-GPU communication constrain rollouts and slow evaluation across many conditions, seeds, and trajectories.Source: §1 · p. 2
  • PowerZooJax implements a power-system RL benchmark in JAX, enabling compiled, vectorized GPU rollouts that reduce CPU-GPU communication overhead.The design is intended to make power-system RL evaluation more efficient.Source: §1 · p. 2
  • The benchmark covers five operational tasks under a consistent CMDP interface and provides baseline algorithms and evaluation metrics for scalable, reproducible assessment.The shared interface supports comparison across tasks with different physical models, constraints, and objectives.Source: §1 · p. 2

2 Related Work

Prior power-system RL benchmarks cover distinct grid layers, while many remain task-specific and computationally costly. PowerZooJax adds a broader JAX-based benchmark for power operation to the accelerator-compatible RL benchmark landscape.

2 Related Work

  • Prior power-system RL benchmarks span grid operation, end-use DER management, and microgrid scheduling, but address different operational areas.Examples include voltage control, transmission topology and redispatch, flexible consumption, and local generation and storage.Source: §2 · p. 2
  • Many existing power-system benchmarks focus on a specific task and lack standardized protocols, violation accounting, or evaluation records.CPU-based solvers can make large-scale rollouts costly; MARL2Grid-TR reports roughly 120,000 CPU hours for training and baseline evaluation.Source: §2 · p. 2
  • Earlier JAX benchmarks cover physics simulation, classic control, optimization, games, multi-agent tasks, and long-horizon exploration; PowerZooJax targets power-system operation.It exposes challenges including partial observability, mixed action spaces, heterogeneous agents, and multi-objective control.Source: §2 · pp. 2–3

3 Background and Problem Setting

The five tasks represent distinct power-system operating settings but share a sequential decision structure under uncertainty and safety constraints, formulated as CMDPs. They range from generation-market bidding to long-horizon data-center microgrid scheduling.

3 Background and Problem Setting

  • Across the five tasks, states, actions, and rewards reflect physical dynamics, device constraints, uncertainty, operational objectives, and safety costs.The tasks share a CMDP structure despite differing physical scales and modeling details.Source: §3 · p. 3
  • Five generation companies submit market bids over a 24-hour horizon, with realized profit as reward and thermal line overloads as safety costs.Market clearing uses security-constrained economic dispatch; agents observe their own generation status and public price signals.Source: §3 · pp. 3–4
  • The transmission task commits and dispatches 54 generators on a 118-bus network, enforcing ramping, minimum up/down-time, and DC OPF constraints.Negative operating cost is the reward; thermal overloads and reserve shortfalls are separate safety costs.Source: §3 · p. 4
  • The distribution task coordinates six flexible loads on a 33-bus network, using negative network loss as reward and voltage violations as safety costs.The agent selects curtailment and load-shifting fractions based on system state, including voltages, flows, demand, and deferred-load buffers.Source: §3 · p. 4
  • The DER task coordinates 12 heterogeneous assets on a 141-bus network using local observations and shared negative network-loss reward.Voltage, thermal, and device-feasibility violations are tracked as separate safety costs.Source: §3 · p. 4
  • The data-center microgrid task jointly schedules computing workloads, cooling, storage, diesel generation, and grid interaction over 288 five-minute steps.Its reward combines energy use, operating cost, and emissions; power imbalance, overheating, and incomplete workloads are safety costs.Source: §3 · p. 4

4 Power System Problem, RL Challenge and Evaluation Scope

PowerZooJax is intended to provide a unified testbed that reflects diverse RL challenges arising in practical power-system settings. Its task framework distinguishes centralized operator decisions from decentralized decisions by users, assets, or market participants.

4 Power System Problem, RL Challenge and Evaluation Scope

  • The five tasks are designed to highlight distinct RL challenges, including partial observability, mixed action spaces, heterogeneous agents, long horizons, and multi-objective control.They include both single-agent and multi-agent settings grounded in operational contexts.Source: §4 · p. 4
  • Table 1 relates power-system problems to RL challenges and distinguishes centralized operator decisions from decentralized local decisions.The authors state that this mapping helps researchers identify which algorithm features the task settings can meaningfully assess.Source: §4 · pp. 4–5, Table 1 · p. 5

5 How Does PowerZooJax Work?

PowerZooJax rewrites power-system computations as fixed-shape JAX graphs, keeping rollouts and training on the GPU. Power-flow and optimal-power-flow solvers are restructured for compilation and batching.

5 How Does PowerZooJax Work?

  • PowerZooJax rewrites per-step power-flow, optimal-power-flow, and device computations as fixed-shape JAX graphs, compiling full rollouts and training on the GPU.The compiled loop covers resets, transitions, rewards, safety costs, action sampling, and gradient updates.Source: §5 · p. 5

5.1 Rewriting Power Flow as a JAX Computation Graph

  • Precomputed topology matrices turn radial feeder sweeps into matrix-vector products, replacing tree traversal with fixed-shape operations.The downstream-aggregation and path matrices are reused throughout training.Source: §5.1 · p. 6
  • A device-side lax.while_loop, vmap, and scan batch PF solves across scenarios and time; one DSO or DERs episode’s 48 evaluations form one compiled rollout.The iterative body is traced into a static XLA computation graph, and the rollout combines power flow with device updates, rewards, and safety costs.Source: §5.1 · p. 6

5.2 Optimal Power Flow as a Computation Graph

  • TSO DC OPF uses ADMM and GenCos market clearing uses PDIPM; a precomputed grid-dependent KKT inverse and fixed-length loops keep solves JAX-compiled.The ADMM iteration uses fixed-shape matrix operations and elementwise projections rather than repeated external CPU-solver calls.Source: §5.2 · pp. 6–7

6 Evaluation

PowerZooJax is evaluated across five power-system tasks for computational speed and representative task-level outcomes, including safety and out-of-distribution stress performance. Solver-heavy tasks show especially large throughput gains, while task results illustrate economic, safety, and scheduling behaviors.

6 Evaluation

  • Five tasks were evaluated with task-specific budgets and five seeds, using PPO for single-agent tasks and IPPO for multi-agent tasks; DCMG also used SAC.The evaluation includes held-out data and task-specific OOD stress splits when applicable.Source: §6 · p. 7

6.1 Execution Speed and Scaling Performance

  • Solver-heavy GenCos and DERs achieve throughput speedups above 2,000×; DCMG gains least because its transitions are mostly arithmetic device updates.Table 2 compares backend speed per task; TSO and DSO also benefit from moving power-system computation onto the GPU.Source: Table 2 · p. 8, §6.1 · p. 8

6.2 GenCos: Bidding under Partial Observability

  • IPPO earns £395k/day in-distribution, exceeding Uniform-mid’s £303k/day but remaining below Max-markup’s £598k/day.The ordering holds under high-demand and low-renewable stress splits; above 800 MW, IPPO’s LMP is around £45/MWh versus Max-markup’s near £75/MWh.Source: §6.2 · p. 8, Figure 4 · p. 8

6.3 TSO: Economic–Safety Cost Trade-off

  • PPO-Lagrangian reduces thermal overloads to 6% and reserve shortfalls to zero, versus PPO’s 36% and 7%, at £3.50M/day.Under line-tightening and load-stress, its overload rates are 16% and 9%, versus PPO’s 49% and 41%; PPO costs £1.58M/day, illustrating the safety-cost trade-off.Source: §6.3 · p. 8, Figure 5 · p. 9

6.4 DCMG: Long-Horizon Scheduling for a Data Center

  • Over a representative 24-hour episode, SAC shifts battery charging and discharging around price peaks while serving the IT workload at every step.It charges during the first two low-price hours, discharges through the morning peak above £100/MWh, recharges when midday PV is available, and cycles again in the evening.Source: Figure 6 · p. 9, §6.4 · p. 9

7 Conclusion

PowerZooJax combines GPU-accelerated rollouts with operational metrics across five power-system tasks under a shared CMDP interface. The conclusion frames it as a benchmark for representative challenges, not a deployment-ready simulator or controller.

7 Conclusion

  • PowerZooJax evaluates five diverse power-system tasks through a shared CMDP interface, reporting returns, safety costs, and stress-test performance.Its GPU-accelerated pipeline is designed to support fast rollouts while retaining operational metrics relevant to grid decision making.Source: §7 · p. 9
  • Strong benchmark performance is not a guarantee of safe real-world operation because the suite has limited scale, uncertainty modeling, constraints, task diversity, and real-time operational data.The current environments use public or synthetic grid models and data.Source: §7 · p. 9

Appendix

  • The appendix documents task models and protocols, external data and stress-test splits, quickstart materials, algorithm configurations, supplementary results, and execution scaling.It is organized across Sections A–H.Source: Appendix · p. 14

A Constrained Markov Decision Process Formulation

A.1 The CMDP tuple

  • Each task is an episodic CMDP with state and action spaces, transitions, reward, cost channels, a discount factor, thresholds, an initial-state distribution, and a horizon.Reward and cost are separate outputs; GenCos and DER agents observe only state projections, making those tasks constrained partially observable problems.Source: §A.1 · p. 14

A.2 Constraint forms used in the suite

  • The suite uses discounted expected-cost budgets for specified DSO, DER, DCMG, and TSO Lagrangian channels, while TSO reserve-shortfall and thermal-overload channels require zero violations at every step.GenCos thermal overload is reported as a market-side diagnostic only.Source: §A.2 · p. 14

A.3 Lagrangian formulation

  • PPO-Lagrangian and IPPO-Lagrangian update multipliers against cost budgets while updating the policy on the Lagrangian objective; Sauté PPO instead adds remaining channel budgets to the observation.Sauté PPO then trains plain PPO on the augmented state.Source: §A.3 · p. 15

A.4 Per-task instantiation

  • Each task is instantiated through a TaskSpec/ConstraintSpec pair, with task equations, safety thresholds, and split definitions documented in separate sections or configuration files.Observation, action, reward, and cost equations appear in Section C; splits appear in Section D.Source: §A.4 · p. 15

B Physical Models

This section presents the physical kernels reused across PowerZooJax tasks and their fixed-shape JAX computation-graph representation. It also describes the GenCos market task and the TSO task’s generator commitment and dispatch on the IEEE 118-bus grid, including their respective decision and safety-cost characteristics.

B.1 Radial AC power flow as a JAX graph

  • Radial power flow uses precomputed topology matrices and fixed-shape JAX operations, with compiled loops and batching keeping all 48 episode solves accelerator-resident.The same kernel powers DSO and DERs.Source: §B.1 · pp. 16–17

B.2 DC optimal power flow with ADMM

  • The TSO task solves a DC OPF linear program at each step using a fixed-length ADMM loop compiled as one XLA program.The solver precomputes the case-dependent augmented KKT matrix; TSO uses 200 iterations and tolerance 10−4.Source: §B.2 · p. 17

B.3 Bid-based market clearing with PDIPM

  • GenCos market clearing solves security-constrained economic dispatch over piecewise-linear bids with a primal-dual interior-point method.The fixed-length JAX loop stops early when µ(k) < 10−5; the balance dual is recovered analytically and line-constraint duals determine LMPs.Source: §B.3 · p. 17

B.4 Resource device dynamics

  • Resource dynamics model batteries, PV inverters, flexible loads, and diesel generators for DERs and DCMG, with flexible loads also used by DSO.Battery actions split charging and discharging and incur an episode-end SoC penalty; PV setpoints are projected onto the capability circle; shifted load must be repaid within its horizon; DCMG diesel dispatch is a same-step residual-slack actuator.Source: §B.4 · p. 18

B.5 DCMG coupling

  • DCMG couples device dynamics to an IT thermal model, cooling control, a local power balance, workload SLA tracking, and energy, cost, and carbon rewards.Cooling COP decreases with outdoor temperature, and over-temperature and SLA-deficit costs are included in the cost vector.Source: §B.5 · p. 18

C.1 Generation-company bidding task (GenCos)

  • GenCos is a five-agent wholesale market task with rolling SCED clearings, where partially observable agents compete through joint actions that affect other bidders.The agents observe their own physical status and limited public market signals; shared market rewards and policy changes make the task non-stationary.Source: §C.1 · pp. 19, 21

C.2 Transmission dispatch task (TSO)

  • A single TSO controls 54 generators on the 118-bus, 186-line IEEE grid for 24 hours at 30-minute resolution, using GB net-load profiles and DC optimal power flow.Each episode has 48 decision steps; ramp limits and minimum up/down times constrain generator operation.Source: §C.2 · p. 21, Figure 8 · p. 22
  • The 108-dimensional action combines on/off commitment requests with dispatch preferences, which are applied subject to minimum up/down limits, ramp-adjusted ranges, and DC-OPF feasibility.Dispatch preferences bias marginal-cost intercepts so cleared output follows the preference when operational limits permit.Source: §C.2 · pp. 21–22
  • The task rewards negative operating cost and reports line-overload and reserve-margin shortfalls as separate violation channels under a zero-violation constraint.Reserve shortfall measures committed-headroom deficit relative to the configured 5% margin; line overload measures cumulative MW above line capacity.Source: §C.2 · p. 22
  • The benchmark includes All-On and Merit Order heuristics alongside unconstrained PPO and PPO-Lagrangian, which updates primal-dual parameters on the two violation costs.Merit Order commits the cheapest units until capacity covers demand plus a fixed 5% reserve margin.Source: §C.2 · p. 22
  • The challenge is coordinating 54 binary commitments and 54 continuous setpoints while maintaining OPF feasibility across ramp, minimum up/down, thermal, and reserve constraints.Sparse, bursty reserve violations make safe-RL credit assignment difficult.Source: §C.2 · p. 22

C.3 Distribution demand response task (DSO)

The DSO, DERs, and DCMG tasks model distinct power-system control settings with different agents, observations, and objectives. Each also poses safety or service constraints that interact with its operational goals.

C.3 Distribution demand response task (DSO)

  • DSO coordinates six flexible loads on a 33-bus feeder to reduce active power loss while keeping every bus voltage within [0.94, 1.06] p.u.A fully observable centralized agent acts every 30 minutes; loss reduction can push voltages toward band edges, while voltage couples controllers through shared power flow.Source: §C.3 · pp. 23–24, Figure 9 · p. 23

C.4 Distributed energy resource coordination task (DERs)

  • DERs assigns 12 heterogeneous battery, PV, and flexible-load agents to cooperate on network loss while respecting voltage and line-thermal limits.Agents use a local K=4 BFS-neighbor window plus a coarse global voltage summary, making credit assignment difficult under the shared reward and network-wide constraint.Source: §C.4 · pp. 24, 26, Figure 10 · p. 25

C.5 Data Center Microgrid task (DCMG)

  • DCMG schedules data-center workloads, cooling, storage, diesel, and grid imports over 288 five-minute steps while trading off energy, cost, and carbon.SLA, temperature, and power-deficit constraints interact with those objectives; outages force islanded operation, where unmet demand is a hard violation.Source: §C.5 · pp. 26–28

D.1 Data and splits

The benchmark documents its external traces, task splits, and evaluation budgets, then provides installation and rollout guidance. Its split taxonomy identifies the physical changes represented by stress tests.

D.1 Data and splits

  • Table 6 summarizes each task’s external time-series traces, network case, splits, and seed and episode budget; processed traces are included in the code release.Bold identifies the primary split for the headline metric; DCMG appendix-only splits follow the semicolon.Source: Table 6 · p. 28, §D.1 · p. 28

D.2 Split and stress-test taxonomy

  • The split taxonomy groups evaluations by the physical mechanism changed, including data-center cooling or workload stress and reduced backup-generation capacity.These splits test long-horizon scheduling under thermal, workload, or service stress, and sensitivity to reduced backup capacity.Source: §D.2 · pp. 28–29, Table 7 · p. 28, E Quickstart and API · p. 29

E Quickstart and API

  • The quickstart walks through installation, rollout, task and training configs, and launching benchmark runs; five task classes share the TaskSpec Protocol.The protocol exposes task names, default splits, environment creation, episode parameters, rollout, and constraint specifications.Source: E Quickstart and API · p. 29, §E.2 · p. 30

E.1 Installation

  • The base installation ships CPU-only JAX, while CUDA 12 support and RL, multi-agent, and benchmark dependencies are opt-in extras.The package supports Python 3.10–3.12.Source: §E.1 · p. 29

E.2 Minimal rollout

  • A minimal transmission rollout runs 48 fixed-action steps with jax.lax.scan, keeping every transition inside JIT.The shared task-level environment contract also applies across the other physical models.Source: §E.2 · p. 29

E.3 Reading a task config

The paper shows how to configure task and training runs and invoke the benchmark workflow. Its algorithm choices distinguish single-agent constrained tasks from competitive and cooperative multi-agent settings.

E.3 Reading a task config

  • The TSO task config specifies case118, 54 units, 48 half-hour steps, demand data, evaluation splits, baselines, seeds, and episode budget.Its primary split is iid, with load-stress and line-tightening evaluations also configured.Source: §E.3 · p. 30

E.4 Reading a training config

  • The TSO PPO training config sets 20 million steps, 256 parallel environments, 48 steps per update, and an in-training evaluation cadence.The config also records evaluation wall time for cross-backend learning curves.Source: §E.4 · p. 31

E.5 CLI commands

  • The benchmark CLI provides separate commands to run baselines, train, evaluate on selected splits, rebuild summaries, and regenerate plots.A lighter YAML- and preset-driven command-line interface is also available.Source: §E.5 · p. 31

F Training Setup and Hyperparameters

  • Training and evaluation details document algorithm rationale, per-task hyperparameters, architectures, and evaluation protocol for reported results.The protocol is described as supporting the reported numbers throughout the paper.Source: F Training Setup and Hyperparameters · p. 31

F.1 Algorithm choices

  • PPO is the unconstrained single-agent baseline, with PPO-Lagrangian used for safe-RL comparisons on TSO and DSO; IPPO variants cover DERs and GenCos.DERs uses type-specific parameter sharing, while GenCos shares one actor-critic across its five generation companies.Source: §F.1 · pp. 31–32

F.2 Per-task per-algorithm hyperparameters

The supplementary results report task-level outcomes alongside training and evaluation details. GenCos IPPO falls between the intermediate and maximum-markup strategies, while TSO results show a cost–safety trade-off with no method meeting both zero-violation channels.

F.2 Per-task per-algorithm hyperparameters

  • Table 8 summarizes the key per-task training hyperparameters, with safe-RL specifics including dual learning rates, a DSO augmentation budget, and a DERs voltage penalty.DERs IPPO-rs uses voltage penalty 8.0, twice IPPO’s 4.0.Source: §F.2 · p. 32, Table 8 · p. 32

F.3 Network architectures

  • The setup uses independent MLP actors and critics, with policy heads and activations varying by algorithm; DERs shares actors by agent type and GenCos across agents.The continuous policy distributions and action bounds differ across PPO, SAC, and IPPO variants.Source: §F.3 · pp. 32–33

F.4 Seeds and evaluation protocol

  • Each task uses five seeds, reports episode metrics with 95% bootstrap confidence intervals, and tests primary-metric comparisons with paired sign-flip permutation tests.Per-step safety rates are reported across seeds, and wall-clock time is recorded at evaluation points.Source: §F.4 · p. 33, G Additional Empirical Results · p. 33

G.1 GenCos

  • GenCos IPPO earns GBP 395,030 in mean in-distribution profit, between Uniform-mid (302,972) and Max-markup (597,864), and above Truthful (6,934).The Truthful < Uniform-mid < IPPO < Max-markup ordering persists on demand-shift and renewable-shock splits.Source: §G.1 · p. 33, Table 9 · p. 34, Table 10 · p. 34
  • GenCos IPPO has lower HHI than the three heuristics (0.5384 versus 0.5664, 0.5663, and 0.5662) and a lower ramp-binding rate (0.35 versus 0.58).Figure 11 plots prices, cumulative profit, HHI, and ramp-binding indicators over a representative in-distribution episode.Source: §G.1 · p. 33, Figure 11 · p. 34

G.2 TSO

  • No TSO method satisfies both zero-violation channels: PPO-Lagrangian and All-On have zero reserve shortfall, but every method has thermal overloads.Reported operating-cost differences are significant in the paired sign-flip tests (n = 250 pairs, p < 10^-4 for reported comparisons).Source: §G.2 · p. 34, Table 11 · p. 35
  • PPO has the lowest TSO operating cost (£1.58M per episode) but mean thermal cost 247.67; PPO-Lagrangian costs £3.50M with mean thermal cost 1.76.PPO-Lagrangian’s mean thermal cost is roughly two orders of magnitude below PPO’s; the figure shows per-episode dispersion.Source: §G.2 · p. 35, Figure 13 · p. 36
  • Across TSO training, PPO-Lagrangian holds reserve-shortfall incidence at zero and lowers final thermal-overload incidence well below PPO, but costs more than PPO and merit order.PPO drives cost toward its held-out value while both safety incidences rise over training.Source: §G.2 · p. 35, Figure 14 · p. 36

G.3 DSO

Across DSO and DER tasks, learned policies differ in how they trade network loss, voltage safety, and dispatch behavior. The execution-scaling analysis also shows that environment-side acceleration varies substantially by task.

G.3 DSO

  • PPO and Sauté PPO drive DSO voltage violations near zero within the first half of training; PPO-Lagrangian converges to a conservative region with low loss reduction and a heavier violation tail than PPO.No control has roughly 0.21 voltage violations per step; as violations become rare, the dual update's constraint signal becomes sparse.Source: Figure 16 · p. 37, §G.3 · p. 37
  • PPO and Sauté PPO use roughly 17 MWh per day of curtailment and 11 MWh per day of shifting, with voltage violations near zero on most episodes, while tail steps remain above zero.SAC reaches lower total loss than PPO-Lagrangian on every paired episode but has the highest per-step voltage-violation rate among the four learned methods.Source: §G.3 · pp. 36–37, Figure 17 · p. 38

G.4 DERs

  • IPPO and IPPO-rs achieved 0.1974 MW active-power loss versus 0.2052 MW for no control, a 3.8% reduction; IPPO-Lagrangian's 0.2051 MW was statistically indistinguishable from no control.IPPO and IPPO-rs beat both no control and voltage droop at p < 10−4; IPPO-Lagrangian was statistically worse than voltage droop.Source: §G.4 · p. 38, Table 13 · p. 39
  • Under voltage tightening, IPPO and IPPO-rs cut violation steps by 44% (4.91 vs 8.73 per episode), while IPPO-Lagrangian cut them by 9% (7.94).Figure 18 traces the policies' device schedules and active-power loss; the text attributes IPPO and IPPO-rs loss reduction to coordinated reactive-power dispatch rather than active-power curtailment.Source: §G.4 · p. 38, Table 14 · p. 39, Figure 18 · p. 39

G.5 DCMG

  • PPO records zero violations across the DCMG's three cost channels; SAC and the three heuristic methods have SLA rates of 10−7 to 10−6 and zero on the other two channels.The reported channels are SLA, over-temperature, and power deficit, each evaluated at a zero threshold.Source: §G.5 · pp. 39–40, Table 15 · p. 40

H Execution Scaling and Backend Comparison

H.1 Backend decomposition at Nenv = 212

  • At Nenv = 212, environment-side gains over SBX were 1,145× on GenCos and 2,040× on DERs, versus 39× on DSO, 33× on TSO, and 22× on DCMG.The largest gains occur where CPU-hosted OPF or power-flow solvers are fused into the JAX graph; DCMG's device updates require no external solver call.Source: §H.1 · p. 40

H.2 Per-task throughput scaling

The throughput sweeps show increasing PowerZooJax throughput with parallelism across DCMG, DSO, and DERs. The section also reports first-compilation cost and compares wall-clock learning for DSO and TSO.

H.2 Per-task throughput scaling

  • JAX/GPU throughput rose from 32.1k to 307.7k env steps/s on DCMG, 25.0k to 388.4k on DSO, and 21.6k to 283.8k on DERs.Matched-range speedups over SB3/CUDA reached 7.4× on DCMG at nenv = 64 and 39× on DSO at nenv = 128.Source: Figure 20 · p. 41, §H.2 · p. 41

H.3 First-compile cost

  • First JAX compilation costs are single-digit seconds for the three tasks in the scaling sweeps; subsequent updates run from the cached compiled program.Source: Table 17 · p. 41, §H.3 · p. 41

H.4 Wall-clock learning curves for DSO and TSO

  • PowerZooJax reaches the same training budget in a fraction of the time required by either CPU-hosted backend for DSO and TSO.All three backends ran to the same training budget and recorded evaluation-point wall-clock timestamps for these tasks.Source: §H.4 · pp. 41–42, Figure 21 · p. 42

Related papers

Loading 2609.36052v1…