PaperlayerView exact revision on arXiv

Source-linked AI summary

Data-Driven Static Output-Feedback Control for Multi-Agent Systems

Paul Serna-Torre, Adam Sedlak, Patricia Hidalgo-Gonzalez

arXiv:2610.02623v1math.OCcs.SYeess.SY

TL;DR

Data-driven output-feedback synthesis for multi-agent systems remains challenging because existing approaches can require full state information, online data, or iterative learning, while realistic MAS also have nonlinear dynamics. This paper combines offline-data Nash-equilibrium algorithms with an SDP for static output-feedback design. In nonlinear power-system models, the framework stabilizes the grid and provides regulation services, with up to 78% less overshoot and 30% faster settling time than proportional-integral control.

  • Problem

    Data-driven MAS control methods may require full state information, while direct output-feedback synthesis without online data or iterative learning remains an open question; realistic MAS also have nonlinear dynamics that small numerical examples may not capture.

    Source: §I · pp. 1–2

  • Method

    The framework uses offline state and input trajectories to compute a Nash equilibrium with either of two algorithms, then applies an SDP to synthesize static output-feedback controllers.

    Source: §II · p. 2, §VII · pp. 8–9

  • Results

    Up to 78% less overshoot and 30% faster settling time than traditional strategies such as proportional-integral control were reported alongside successful grid stabilization and inverter regulation services.

    Source: §VII · p. 9

  • Takeaways & Limitations

    High-fidelity nonlinear power-system models demonstrate the framework’s performance in realistic multi-agent settings beyond small-scale numerical examples.

    Source: §I · p. 2

    With corrupted data samples, the proposed algorithms yield zero-valued P matrices and output-feedback synthesis fails; the case studies instead use system identification to clean data.

    Source: §VI · p. 8

Abstract

from arXiv · show

This work proposes a data-driven framework that synthesizes output-feedback control for linear time-invariant multi-agent systems (MAS). We first develop two algorithms that use offline data samples of state and input trajectories to determine a Nash equilibrium (NE) for MAS. Next, we formulate a semidefinite program that, given the NE solution, computes the static output-feedback controllers for the agents in MAS. Compared to prior work, virtues of our framework include weaker controllability assumptions, the absence of online information and data-driven estimation of states. Next, we use the proposed framework to design control schemes for voltage and frequency regulation for nonlinear multi-nodal electrical networks with grid-forming and grid-following inverters. Thus, compared to prior work that implements their data-driven frameworks in small-scale numerical examples, we validate the performance of our data-driven framework in high-fidelity nonlinear models of MAS. This implementation in realistic MAS provides insights that small-scale numerical examples may fail to capture. Results show successful grid stabilization and provision of regulation services from the inverters in the grid using our proposed framework. It can attain up to 78% less overshoot and 30% faster settling time than traditional strategies such as proportional-integral control.

I. INTRODUCTION

Data-driven control for multi-agent systems must account for agents’ separate objectives and limited access to full state information. The paper proposes a two-step framework for Nash-equilibrium computation and static output-feedback synthesis, then applies it to realistic power-system models.

I. INTRODUCTION

  • Prior data-driven MAS methods either require full state information or rely on observers, online data, and iterative learning, leaving a need for direct output-feedback synthesis from offline data.Earlier frameworks also often assume a system-wide objective, whereas MAS agents may have distinct objectives and self-interested strategies.Source: §I · pp. 1–2

II. CONTRIBUTIONS

  • The framework computes a Nash equilibrium using offline state and input data, then uses an SDP to synthesize static output-feedback controllers without state estimators or online data.The authors also provide an open-source toolbox and assess scalability in realistic inverter-dominated power grids using EMT models.Source: §II · p. 2

A. Problem 1: Nash equilibrium solution

  • The first problem is to determine state-feedback gains K that attain a Nash equilibrium when the system matrices A and B_i are unknown.Each agent minimizes its own cost, which depends on the strategies of all agents through the shared state.Source: A. Problem 1: Nash equilibrium solution · p. 2

B. Problem 2: Stabilizing output feedback strategies

  • The second problem is to find static output-feedback gains F that stabilize the closed-loop system and make it dissipative, while limiting each agent to its output measurements.The output-feedback cost is also compared with the cost under the Nash state-feedback strategies.Source: B. Problem 2: Stabilizing output… · pp. 2–3

IV. CONTROL FRAMEWORK

The preliminaries review data-driven LQR synthesis for systems with unknown matrices. They use trajectory data to construct an SDP and recover the Riccati solution through a Sylvester equation.

A. Preliminaries

  • The preliminaries frame LQR as a single-agent optimization problem and state that its gain depends on the solution P of the ARE.When A and B are unknown, the ARE cannot be solved directly.Source: A. Preliminaries · p. 3
  • For an LQR system with unknown A and B, persistently exciting input and state trajectories provide data for synthesizing the gain without directly solving the ARE from known system matrices.The sampled state data are used to construct a finite-difference approximation, and an SDP yields K = UZW^-1.Source: A. Preliminaries · p. 3
  • The paper proposes recovering the ARE solution P by solving a Sylvester equation using matrices derived from the data-driven SDP solution.The construction sets M = eXZW^-1 and N = -Q - K⊤RK; uniqueness is tied to the stated eigenvalue condition.Source: A. Preliminaries · p. 3

B. Nash equilibrium

The section presents two data-driven algorithms for finding Nash equilibria in LTI multi-agent systems using offline trajectory data. Their resulting solutions support controller synthesis, while Algorithm 2 uses weaker controllability/stabilizability assumptions than Algorithm 1.

B. Nash equilibrium

  • Algorithm 1 uses data to determine the Nash-equilibrium matrices K and P when the system matrices A and B_i are unknown.Its inputs include each agent’s Q_i and R_i, a sampling time, a CARE tolerance, and a maximum iteration count.Source: B. Nash equilibrium · p. 3
  • When the CARE tolerance is small, such as 10^-3, the final iteration’s P and K are treated as the CARE solution.The section states that the resulting P satisfies the CAREs.Source: B. Nash equilibrium · p. 4
  • Algorithm 1 adds an equation that enables output-feedback determination and a parallel loop that improves its convergence speed.The paper contrasts these features with the similar scheme in [14, Algo. 2].Source: B. Nash equilibrium · p. 4
  • Algorithm 1 assumes stabilizability for every i ∈ C, which can be difficult to meet in some multi-agent systems; Algorithm 2 does not require this assumption.Source: B. Nash equilibrium · p. 4
  • Algorithm 2 uses an SDP at each iteration to drive the CARE residuals toward zero as part of finding a Nash equilibrium.The SDP aims to make every entry of the residual matrix T^(r) equal to zero.Source: B. Nash equilibrium · pp. 4–5

C. Stabilizing output-feedback strategies

The section uses the Nash-equilibrium matrices from either algorithm as the basis for an SDP that determines static output-feedback strategies. The proposed formulation does not require knowledge of A or B_i.

C. Stabilizing output-feedback strategies

  • Lemma 1 uses the matrices K and P from either algorithm as the basis for determining output-feedback strategies u_i = F_i y_i.The resulting strategy is stated to satisfy the objectives of Problem 2 when the SDP’s stated condition holds.Source: C. Stabilizing output-feedback… · p. 5
  • The lemma’s stated feasibility condition is that the optimal SDP solution returns α_i > 0 for at least one agent.Under that condition, the resulting F matrices form the output-feedback control strategy.Source: C. Stabilizing output-feedback… · p. 5
  • The SDP reformulates a model-based synthesis approach to use the computed K and P instead of requiring A and B_i.The paper states that Lemma 1 does not require knowledge of those system matrices.Source: C. Stabilizing output-feedback… · p. 5

A. Computational implementation

The work implements its data-driven control framework in CMAS, an open-source toolbox for multi-agent system control synthesis.

A. Computational implementation

B. Numerical cases

The numerical cases compare the proposed framework with model-based methods and examine Algorithm 1’s convergence speed. The proposed controllers have less than 6% error across K and P, and the parallel implementation is 19% faster on average.

B. Numerical cases

  • The illustrative case uses an unstable fourth-order LTI system and random square input signals, with resulting state trajectories used for the data-driven algorithms.The model-based controllers are computed using and explicitly require A and B as input data.Source: B. Numerical cases · p. 6
  • Less than 6% error across K and P shows that the framework computes controllers almost identical to model-based methods.Table I compares controllers from the proposed framework with those from model-based methods [28].Source: Table I · p. 6, B. Numerical cases · p. 6
  • Table II is identified as a comparison of performance between the proposed control framework and other methods.Source: Table II · p. 6
  • 19% faster on average is the convergence speed of parallel Algorithm 1 versus the synchronous traditional for-loop implementation in [14, Algo. 2].The comparison uses ten randomly generated LTI systems.Source: B. Numerical cases · p. 6

C. Frequency regulation

The frequency-regulation case study applies the framework to a modified IEEE 230-kV 9-bus system, using a reduced frequency-dynamics model for three grid-forming inverters. The synthesized controllers outperform integral-action strategies in the reported time-domain comparisons.

C. Frequency regulation

  • The case study models a modified IEEE 230-kV 9-bus grid with three grid-forming inverters in a 94th-order EMT simulation.For synthesis, the unknown model represents frequency dynamics with 6 states and 3 input channels.Source: C. Frequency regulation · pp. 6–7
  • System identification generates noise-free state trajectories for the Nash-seeking iterative LQR and subsequent SDP controller synthesis.The original EMT trajectories contain numerical noise that can make the SDP infeasible or produce zero state-feedback matrices.Source: C. Frequency regulation · p. 7
  • Up to 78% less overshoot and up to 16% less settling time were achieved than with integral controllers.The comparison uses integral controllers that receive frequency deviation and act on each inverter’s power reference.Source: C. Frequency regulation · p. 7, Figure 2 · p. 7

D. Voltage regulation

The voltage-regulation case study applies the framework to a 20-kV two-bus system with a grid-following and a grid-forming inverter. Its synthesized control scheme improves the reported voltage-regulation performance relative to no control and PI control.

D. Voltage regulation

  • The 20-kV two-bus EMT scenario includes a grid-following inverter, a grid-forming inverter, a long transmission line, and loads.The grid-forming inverter supplies a fixed 60-Hz frequency signal to the grid-following inverter.Source: D. Voltage regulation · p. 7
  • The unknown dq-frame small-signal model has 13 states and 3 input channels, with the two inverters as agents.Controller synthesis follows the frequency-regulation procedure but uses Algorithm 2 instead of Algorithm 1.Source: D. Voltage regulation · pp. 7–8
  • Up to 74% less overshoot and up to 30% less settling time were achieved than with no control or PI control.The PI controller was configured through trial and error for close-to-zero steady-state error and no instability.Source: D. Voltage regulation · p. 8, Figure 4 · p. 8

VI. CHALLENGES FOR DATA-DRIVEN FRAMEWORK IMPLEMENTATION

The implementation discussion identifies noisy trajectory data and model complexity as practical challenges for applying the framework to realistic multi-agent systems. It describes system identification and reduced-order models as ways used or considered to address these challenges.

VI. CHALLENGES FOR DATA-DRIVEN FRAMEWORK IMPLEMENTATION

  • Noisy or numerically corrupted state data can yield zero-valued P matrices in Algorithms 1 or 2, causing output-feedback synthesis to fail.The paper uses system identification in its case studies to generate data from an identified model; other clean-data approaches are possible.Source: §VI · p. 8
  • Algorithm 2 can be faster and less computationally expensive than Algorithm 1 because it requires data samples only once.Algorithm 1 requires exciting the system and obtaining data samples in each iteration.Source: §VI · p. 8
  • A reduced-order model may be better suited when modeling all states and input channels of a complex MAS causes the framework to fail.For frequency regulation, the paper reduces a 94-state, 9-input power system to a 6-state, 3-input frequency-dynamics model and validates control in the high-fidelity nonlinear model.Source: §VI · p. 8

VII. CONCLUSIONS

The framework uses offline trajectory data to determine a Nash equilibrium, then uses an SDP to synthesize output-feedback strategies; it is tested in a high-fidelity nonlinear power-system MAS.

  • The framework determines a Nash equilibrium with two alternative algorithms using offline state and input trajectory data, then feeds that solution to an SDP for output-feedback synthesis.Source: §VII · pp. 8–9
  • Algorithm 1 avoids identifying matrices A and Bi for all i in C, whereas Algorithm 2 requires identifying Bi for all i in C.Source: §VII · p. 9
  • Up to 78% less overshoot and 30% faster settling time than traditional strategies such as proportional-integral control were achieved, alongside successful grid stabilization and inverter regulation services.Source: §VII · p. 9
  • The paper reports implementation challenges in realistic settings, which small-scale numerical tests with known state-space models may overlook.Source: §VII · p. 9
Loading 2610.02623v1…