Source-linked AI summary
Online Optimization as a Feedback Controller: Stability and Tracking
Marcello Colombino, Emiliano Dall'Anese, Andrey Bernstein
TL;DR
The paper asks how to regulate an LTI dynamical system toward the solution of a time-varying convex optimization problem without knowing disturbances. It designs a feedback primal-dual controller using a proximal augmented Lagrangian and analyzes it with IQC-based LMIs. The results establish stability and bounded tracking under suitable conditions, including time-scale separation, with a power-system application and an approximate method for less restrictive assumptions.
Problem
Regulating dynamical-system outputs to time-varying optimization solutions is difficult when disturbances are unknown and conventional methods require disturbance models or forecasts.
Method
The paper modifies continuous-time primal-dual dynamics for a proximal augmented Lagrangian and embeds them in feedback with an LTI dynamical system.
Results
IQC-based LMI conditions guarantee global exponential stability and bounded tracking error, and become satisfiable under sufficient time-scale separation between the system and optimizer.
Takeaways & Limitations
The approach enables disturbance-independent online tracking and compresses secondary-control and tertiary-optimization time scales in a DC-OPF transmission-system example.
Takeaways & Limitations
A rank condition can restrict hard constraints to at most as many outputs as inputs, motivating an approximate algorithm under less restrictive assumptions.
Abstract
from arXiv · showhide
This paper develops and analyzes feedback-based online optimization methods to regulate the output of a linear time-invariant (LTI) dynamical system to the optimal solution of a time-varying convex optimization problem. The design of the algorithm is based on continuous-time primal-dual dynamics, properly modified to incorporate feedback from the LTI dynamical system, applied to a proximal augmented Lagrangian function. The resultant closed-loop algorithm tracks the solution of the time-varying optimization problem without requiring knowledge of (time-varying) disturbances in the dynamical system. The analysis leverages integral quadratic constraints to provide linear matrix inequality (LMI) conditions that guarantee global exponential stability and bounded tracking error. Analytical results show that, under a sufficient time-scale separation between the dynamics of the LTI dynamical system and the algorithm, the LMI conditions can be always satisfied. The paper further proposes a modified algorithm that can track an approximate solution trajectory of the constrained optimization problem under less restrictive assumptions. As an illustrative example, the proposed algorithms are showcased for power transmission systems, to compress the time scales between secondary and tertiary control, and allow to simultaneously power re-balancing and tracking of DC optimal power flow points.
I. INTRODUCTION
The paper addresses regulation of dynamical-system outputs toward time-varying optimization solutions without relying on disturbance knowledge. It proposes a feedback-based primal-dual approach that incorporates system dynamics through a proximal augmented Lagrangian.
- Motivation: Traditional engineering architectures separate optimization and regulation across time scales, using slow reference signals for system control.These references are based on operational costs, engineering constraints, and algebraic input-output models.
- Motivation: Model-based optimization and predictive-control approaches rely on system models and accurate forecasts of disturbances.This dependence constrains their use when disturbances are uncertain.
- Motivation: Feedback-based optimization uses system measurements to steer dynamics toward optimization solutions with limited input-output-map knowledge and no disturbance information.Prior work also considered tracking time-varying problems.
- Contribution: The paper proposes continuous-time primal-dual gradient methods applied to a proximal augmented Lagrangian as feedback controllers for LTI dynamical systems.Stability and tracking are analyzed using integral quadratic constraints.
- Problem formulation: The formulation optimizes steady-state control and output costs subject to time-varying convex constraints over an LTI system's steady state.The objective includes control effort, output penalties or soft constraints, and a possibly nonsmooth constraint or regularization term.
- Algorithmic basis: The proximal augmented Lagrangian preserves optimal primal-dual saddle points while making the nonsmooth term continuously differentiable through its Moreau envelope.The associated saddle-flow algorithm globally converges under strong convexity, with exponential convergence under further technical assumptions.
C. Online implementation
The online implementation closes the primal-dual algorithm around measurements from the LTI system, so disturbance effects are observed rather than modeled. Its stationary point corresponds to the time-varying primal-dual optimum, while stability determines tracking behavior.
- Online feedback implementation: The online primal-dual saddle flow is implemented in feedback with the dynamical system to track its time-varying optimal trajectory.The implementation requires neither disturbance measurements nor knowledge of how disturbances affect the output.
- Stationary points: A time-varying stationary point is defined as a state where the time-dependent closed-loop vector field equals zero for the current disturbance.This definition characterizes instantaneous equilibria rather than the evolving state trajectory.
- Stationary points: For every feasible time, the stationary-point set is a singleton whose state equals -A^-1Bu⋆(t), with control and dual components u⋆(t) and λ⋆(t).Thus the stationary point corresponds to the unique primal-dual optimum at that time.
- Stationary points: The closed-loop stationary conditions are equivalent to the first-order optimality conditions of the time-varying constrained problem.This establishes the optimization meaning of the feedback equilibrium.
- Assumptions: The analysis allows time-varying problem data and disturbances to vary discontinuously while retaining Carathéodory solutions under the stated measurability and continuity assumptions.Feasibility and measurability of the optimal primal-dual trajectory are also assumed.
- Feedback representation: The feedback interconnection combines an LTI system G(s), containing the physical system and part of the optimizer, with a time-varying nonlinearity containing objective gradients and constants.This representation supports the subsequent robust-control analysis.
III. STABILITY AND PERFORMANCE ANALYSIS
The paper provides a tractable stability test for the online optimization scheme and its ability to track the optimizer of a time-varying problem.
- Stability and tracking: The main analysis supplies LMI conditions, based on an IQC characterization, that guarantee closed-loop stability and output tracking of the time-varying optimizer.These conditions provide a tractable test for the proposed feedback control law.
A. Tracking optimal trajectories for time-varying disturbances
The feedback interconnection is analyzed through an LMI condition that guarantees exponential stability and tracking of time-varying optimal trajectories. The tracking error is driven by the rate of change of the optimizer.
- Feedback formulation: The online primal-dual implementation is represented as an LTI system interconnected with a time-varying nonlinearity.The analysis uses the system state, control input, multiplier, and measured output within this feedback representation.
- Optimal trajectory: The optimal trajectory is time-varying because it depends on the time-varying disturbance and optimization solution.The trajectory is formed from a primal-dual optimal pair and the corresponding steady-state system state.
- Stability certificate: An LMI condition based on integral quadratic constraints certifies exponential tracking of the optimal trajectory.The condition is stated using Aρ, B, C, D, a positive-definite matrix P, and IQC multipliers.
- Tracking guarantee: The LMI result can be interpreted as input-to-state stability with the optimizer’s rate of change as the input.This connects optimizer variation directly to the tracking analysis.
- Special cases: For constant disturbances, the closed-loop system exponentially converges to the unique constant optimal solution.When the optimizer changes at a bounded rate instead, the result provides an asymptotic tracking-error bound.
B. Time-scale separation and feasibility of the LMI condition
The paper shows that sufficient time-scale separation makes the LMI stability condition feasible under a rank assumption, while clarifying that separation is sufficient rather than necessary. A faster system can preserve the optimal trajectory while enabling the feasibility result.
- System scaling: The scaled faster system has the same steady-state map and therefore the same optimal trajectory as the original system.The proof studies the limit of the scaled transfer function as ǫ approaches zero.
- Proof mechanism: The proof establishes strict positivity at ρ = 0 and extends it to positive ρ and sufficiently small ǫ by continuity.The argument uses a Schur complement and the projection-matrix property of the relevant input-output term.
- Design implication: Theorem 2 provides a constructive stabilizing design by artificially slowing down the optimization algorithm.This is one way to create the required time-scale separation.
- Scope and limitation: Time-scale separation guarantees LMI feasibility but is not necessary, since the condition may hold without apparent separation.The rank condition still limits hard constraints imposed through the indicator function to at most as many outputs as inputs.
IV. APPROXIMATE ONLINE OPTIMIZATION
The approximate online optimizer relaxes the rank requirement by regularizing selected output constraints. It enforces some constraints strictly while allowing the remaining constraints to be satisfied approximately, with the approximation controlled by γ.
- Motivation: The regularized algorithm is proposed as an alternative when Π2uΠ⊤2u is not positive definite.It can replace soft constraints in settings where the original control algorithm cannot be used.
- Rank-deficient case: When the number of constrained outputs exceeds inputs, the original algorithm is unavailable because the rank condition fails.The approximate formulation addresses this case by choosing a matrix Q that selects which constraints are regularized.
- Constraint handling: For a separable time-varying constraint set, the proximal operator becomes projection onto that set.The resulting multiplier stationarity condition links the projected quantity to the constrained output.
- Approximate enforcement: The choice Q = diag(0p, Im) enforces the first p constraints strictly and the remaining m−p constraints approximately.The approximation level is adjustable through the regularization parameter γ.
A. Stability analysis of the approximate algorithm
The approximate algorithm is analyzed with IQC and LMI tools analogous to the original method. Under its stated assumptions, sufficient time-scale separation yields stability despite relaxing the rank requirement.
- Tracking target: The approximate optimal trajectory is introduced as the reference for analyzing the regularized algorithm’s tracking properties.The section begins by defining the trajectory that the approximate dynamics are intended to follow.
- LMI analysis: The stability analysis uses an LMI membership condition involving Aγρ, B, C, D, P, and IQC multipliers.The KYP Lemma provides the equivalence between the frequency-domain condition and the LMI test.
- Stability result: Under the rank-free approximate formulation’s assumptions, sufficiently separated time scales produce parameters for which the LMI condition holds.The result requires µ ≥ max{ˆLf, Lh} and establishes positive ǫ and ρ together with suitable multipliers and P.
- Frequency-domain characterization: The frequency response Γ0(ω) characterizes the regularized approximate algorithm in the stability analysis.It is defined using µ, γ, and Q through a rational matrix expression in ω.
- Proof conclusion: Continuity arguments extend strict feasibility from the unshifted case to positive stability margins and sufficiently small scaling parameters.This parallels the feasibility argument used for the original algorithm.
V. POWER SYSTEMS EXAMPLE
The section introduces illustrative numerical results based on a power-systems case study.
- The proposed algorithms are evaluated through a power-systems case study.The section presents numerical results for the case study described next.
A. Online OPF for constraint-aware grid rebalancing
The paper formulates a time-varying DC-OPF problem over swing dynamics and develops soft-constraint and approximate hard-constraint online optimization variants for power-system regulation.
- The example uses linearized swing dynamics to model generator and inverter behavior.The system state contains phase angles and frequencies, with grid admittance, inertia, and damping matrices defining the dynamics.
- A coordinate transformation removes the marginal average-angle mode, yielding Hurwitz-stable dynamics while preserving line-power and average-frequency outputs.The transformed state represents rotor angles relative to the average angle and unaltered rotor frequencies.
- The time-varying optimization problem corresponds to DC-OPF with uncontrollable loads and renewable injections represented by the disturbance.The usual rank condition limits the number of outputs imposed through hard constraints, motivating soft line constraints.
- Soft constraints for the transmission lines: Soft constraints are introduced for transmission-line powers while frequency constraints remain hard in the online formulation.The resulting power-system algorithm uses a soft-constraint reformulation of the optimization problem.
- Approximate online optimization: The approximate algorithm retains hard constraints and regularizes the dual update for line constraints with γ > 0.The frequency deviation remains hard-constrained through a matrix choice, while γ controls the regularization.
- Decentralized integral frequency control can be destabilized by an infinitesimal bias, motivating a formulation that avoids the corresponding rank-condition violation.An equality constraint on frequency would produce a decentralized integral controller and violate the rank condition.
- The proposed algorithms are tested on a simple power-systems test case.The section transitions from the formulation to the numerical test case.
- Without control, the IEEE9 system fails to react to Line 1 tightening or regulate frequency under load variation.The uncontrolled system cannot keep frequency and line powers within limits as the load changes.
B. IEEE9 Test Case
The IEEE9 study compares uncontrolled, soft-constraint, and approximate hard-constraint online optimization under changing loads and a tightened Line 1 constraint.
- The modified lossless IEEE9 case has three controllable generators, three uncontrollable loads, and swing dynamics with specified inertia and damping parameters.Controllable generator setpoints are updated by the online optimization algorithms.
- Without control, the system cannot react to Line 1 constraint tightening or keep frequency and line powers within limits during load changes.The figures compare average frequency and line powers against the line constraints.
- Soft constraints: The soft-constraint optimizer regulates frequency and reacts to tightening but does not achieve steady-state constraint satisfaction.The residual violation is attributed to the use of soft constraints.
- Approximate online optimization: The approximate hard-constraint algorithm outperforms the soft-constrained method in frequency regulation and constraint enforcement.It uses γ = 10^-2, µ = 4, and ϵ = 10^-2, with hard constraints and regularized line-constraint dual updates.
- Both algorithms receive stability certificates from appropriate LMIs with ρ = 10^-3.The framework also includes specialized frequency-control algorithms and extends them to more general constraints.
- Distributed implementation of the proposed algorithms remains an open topic for future research.
VI. CONCLUSION AND OUTLOOK
The paper develops feedback primal-dual optimization for regulating LTI-system outputs toward time-varying convex optimizers and establishes stability and tracking guarantees.
- The method interconnects an augmented-Lagrangian primal-dual saddle-flow algorithm with an LTI dynamical system.The resulting feedback interconnection is designed for time-varying convex optimization problems.
- IQC-based LMI conditions guarantee stability and tracking of the time-varying optimizer.The analysis provides global stability and output tracking guarantees for the interconnection.
- With sufficient time-scale separation between the fast system and slow optimizer, the interconnection is always stable under mild conditions.
- The stability tests are not very conservative, but the guaranteed convergence-rate estimate is extremely conservative.The authors propose Popov or Zames-Falbes IQCs as possible ways to obtain tighter rates.