PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimization Algorithms as Robust Feedback Controllers

Adrian Hauswirth, Zhiyu He, Saverio Bolognani, Gabriela Hug, Florian Dörfler

arXiv:2103.11329v2math.OCeess.SY

TL;DR

The paper addresses the gap between numerical, offline optimization and robust optimization performed directly in closed loop with physical systems. It surveys feedback-based optimization as dynamical-system control, emphasizing stability, constraints, data-driven operation, and applications. The survey reports theoretical guarantees and real-world deployment, while identifying limitations in constraint enforcement and time-varying non-convex optimization.

  • Problem

    Offline optimization separates set-point computation from physical operation, leaving solutions vulnerable to model discrepancies and motivating closed-loop methods that use real-time measurements.

  • Method

    The survey interprets optimization algorithms as dynamical systems and reviews their interconnection with physical plants, including primal-dual methods, data-driven and model-free operation, and constraint-enforcement designs.

  • Results

    The survey covers closed-loop stability and constraint guarantees, robustness to small inaccuracies, and deployment of a feedback controller for voltage-constrained reactive-power optimization in a real distribution grid.

  • Takeaways & Limitations

    Feedback-based optimization supports autonomous, measurement-driven optimization of physical systems and has progressed from theory to industrial operation.

  • Takeaways & Limitations

    Penalty and barrier terms enforce inequality constraints only approximately, while time-varying non-convex problems may prevent continuous tracking of a single global minimizer.

Abstract

from arXiv · show

Mathematical optimization is one of the cornerstones of modern engineering research and practice. Yet, throughout all application domains, mathematical optimization is, for the most part, considered to be a numerical discipline. Optimization problems are formulated to be solved numerically with specific algorithms running on microprocessors. An emerging alternative is to view optimization algorithms as dynamical systems. Besides being insightful in itself, this perspective liberates optimization methods from specific numerical and algorithmic aspects and opens up new possibilities to endow complex real-world systems with sophisticated self-optimizing behavior. Towards this goal, it is necessary to understand how numerical optimization algorithms can be converted into feedback controllers to enable robust "closed-loop optimization". In this article, we focus on recent control designs under the name of "feedback-based optimization" which implement optimization algorithms directly in closed loop with physical systems. In addition to a brief overview of selected continuous-time dynamical systems for optimization, our particular emphasis in this survey lies on closed-loop stability as well as the robust enforcement of physical and operational constraints in closed-loop implementations. To bypass accessing partial model information of physical systems, we further elaborate on fully data-driven and model-free operations. We highlight an emerging application in autonomous reserve dispatch in power systems, where the theory has transitioned to practice by now. We also provide short expository reviews of pioneering applications in communication networks and electricity grids, as well as related research streams, including extremum seeking and pertinent methods from model predictive and process control, to facilitate high-level comparisons with the main topic of this survey.

1. Introduction

The article contrasts offline, feedforward optimization with feedback-based optimization, which embeds optimization dynamics in physical systems using real-time measurements. It emphasizes robustness to uncertainty, constraint handling, autonomous operation, and closed-loop stability.

  • Offline optimization computes decisions from known data before operation, so model–system discrepancies can leave the realized state suboptimal.
  • Feedback-based optimization casts optimization algorithms as dynamical systems and interconnects them with physical plants to drive the system toward an optimal steady state.
  • Real-time feedback addresses inaccurate and time-varying data by reacting to the disturbance actually realized rather than conservatively enumerating all possible scenarios.
  • Closed-loop optimization can exploit physical constraints and runtime measurements while optimizing economic objectives without external setpoints or reference signals.
  • A central technical challenge is certifying closed-loop stability, since plant–algorithm interconnections can diverge or converge to different minimizers depending on the control gain.
  • The survey focuses mainly on continuous-time systems, while discrete-time implementation issues often require domain-specific solutions outside its main scope.

2. Optimization Algorithms as Dynamical Systems

Optimization algorithms can be studied as dynamical systems, allowing their convergence, stability, and constraint-enforcement properties to be analyzed through continuous-time flows. The section reviews gradient, saddle-point, and projected dynamics, emphasizing assumptions needed for convergence and feasible operation.

  • Optimization dynamics: First-order optimization algorithms can be interpreted as dynamical systems, providing a bridge to sampled-data and closed-loop implementations.This dynamical-systems perspective is used to connect optimization flows with feedback-system implementations.
  • Gradient flows: Gradient flows move along descent directions toward critical points, whose stability is related to the Hessian and local optimality.Positive-definite Hessians yield locally exponentially stable minima, while a negative Hessian eigenvalue implies instability.
  • Gradient flows: Under compact sublevel sets, gradient trajectories remain bounded; analyticity or convexity ensures convergence to a single point, with exponential convergence under strong convexity or the Polyak-Lojasiewicz inequality.Without additional structure, trajectories may converge to an entire set of critical points rather than one point.
  • Primal-Dual Saddle-Point Flows: Primal-dual saddle-point flows use gradient descent in primal variables and ascent in dual variables to enforce equality constraints through Karush-Kuhn-Tucker conditions.For convex problems, equilibria satisfying the KKT conditions are optimal; convergence arguments rely on monotonicity and can fail beyond convex-concave settings.
  • Comparison of Constraint-Enforcement Mechanisms: Inequality constraints require mechanisms such as penalties, barriers, dualization, or projection because unconstrained gradient flows do not naturally enforce them.Penalties and barriers provide approximate outer or inner enforcement, while dualization permits transient violations and projection enforces feasibility directly but produces nonsmooth dynamics.
  • Comparison of Constraint-Enforcement Mechanisms: Constraint-enforcement mechanisms involve trade-offs: steeper penalties or barriers improve precision but can create stability issues, while saddle-point tuning may produce undesirable transients outside the feasible region.These limitations are especially relevant when optimization dynamics operate online or in closed loop.

3. Online Feedback-Based Optimization

This section reviews how optimization algorithms are implemented in closed loop with physical plants, emphasizing robust interconnection, closed-loop stability, constraint enforcement, and reduced model dependence.

  • Closed-Loop Optimization: Feedback-based optimization interconnects optimization dynamics with physical plants to drive systems toward optimal steady states under operational constraints.The design must remain robust and implementable despite limited model knowledge.
  • Core Challenges: The review organizes robust closed-loop design around stability guarantees, constraint enforcement using models and measurements, and data-driven or model-free operation.These themes structure the technical review and its tutorial exposition.
  • Closed-Loop Stability: Closed-loop stability generally requires sufficient timescale separation between fast plant dynamics and slow optimization dynamics.A simple gradient-based controller may be unstable unless its gain is sufficiently small.
  • Singular Perturbation Analysis: Singular perturbation methods provide broadly applicable stability conditions for nonlinear stable plants and convex or non-convex optimization dynamics, but may be conservative.The analysis decomposes the system into reduced and boundary-layer error dynamics.
  • LMI Stability Certificates: LMI-based certificates offer less conservative computational stability guarantees, but are limited to linear time-invariant plants and convex optimization dynamics.Passivity provides another possible certificate when the loop can be represented as an interconnection of passive systems.
  • Constraint Enforcement: Penalty and barrier functions can enforce inequality constraints only approximately, and steeper penalties reduce the allowable optimization gain under singular-perturbation bounds.Log-barrier functions without Lipschitz gradients require particular care because these stability bounds may not apply.

3.2. Constraint Enforcement in Closed Loop

Feedback-based optimization enforces constraints through plant saturation, anti-windup compensation, projections, and output-constraint mechanisms. These designs can converge to feasible KKT points, while trade-offs arise between global guarantees, distributed implementation, and computational effort.

  • Feedback-based optimization robustly enforces complicated constraints despite model uncertainty.
  • Input Saturation via Projection and Anti-Windup: Plant input saturation can act as a projection onto feasible inputs, outsourcing constraint handling to physical dynamics.This reduces computational requirements and the need to model time-varying or unknown constraints exactly.
  • Input Saturation via Projection and Anti-Windup: Continuous-time saturation requires anti-windup compensation because a cascaded integrator and saturation element can otherwise produce integrator windup.The anti-windup design smoothly approximates projected gradient flows as K →0+.
  • Input Saturation via Projection and Anti-Windup: The saturated control converges to KKT points, and to the global minimizer when the reduced problem is convex.At convergence, the plant is optimal even though the unsaturated control need not be.
  • Output Constraints via Dualization and Approximate Projections: Projected and augmented saddle-point feedback controllers handle general output constraints, with convex problems admitting global convergence and non-convex variants requiring careful augmentation choices.Dual augmentation can provide global convergence for non-convex problems but may shift equilibria away from the true KKT points.
  • Output Constraints via Dualization and Approximate Projections: Projected feedback dynamics can converge to KKT points under non-convexity, but some implementations require solving a quadratic program and do not naturally distribute.For the cited scheme, equilibrium points are feasible KKT points under weak technical assumptions and sufficiently small step size α.

3.3. Data-Driven and Model-Free Operation

Feedback-based optimization depends on steady-state input-output sensitivities, which may be unavailable from accurate models. The survey therefore reviews sensitivity learning and model-free alternatives, showing robustness to small estimation errors but degradation with larger errors and reduced sample efficiency without structural information.

  • Feedback-based optimization requires steady-state input-output sensitivities, but complex plants and missing parameters can make them difficult to obtain.
  • Small sensitivity-estimation inaccuracies preserve robust feedback-based optimization, whereas larger errors cause suboptimality and even instability.The experiment perturbs H as H + σM while using the same step size η across σ values.
  • Sensitivity learning uses offline or online data to estimate surrogates that are inserted into gradient-based controller updates.Reviewed approaches include informative input-output trajectories and recursive least-squares estimation.
  • Online Kalman-filter sensitivity learning produces an asymptotically unbiased estimate with bounded variance, while solution-accuracy bounds depend on that variance.
  • For nonlinear dynamic plants, insufficiently accurate or fast sensitivity learning causes suboptimality through cumulative gradient-step errors.
  • Model-free methods generally require more actuation steps to reach a given solution accuracy than schemes with perfect sensitivities.They discard useful structural information from prior knowledge or approximate models.

4. Application Example: Optimal Reserve Dispatch in Electricity Grids

The paper formulates real-time reserve dispatch as a nonconvex AC optimal power flow problem and evaluates feedback-based controllers for economically efficient, constraint-aware grid operation. The methods achieve near-optimal cost, minor temporary violations, and, with output projection, almost no transient line-flow violations.

  • 4.1. Formulation as a Real-Time Optimal Power Flow Problem: Real-time reserve dispatch is formulated as tracking a time-varying AC optimal power flow solution under volatile renewable generation.The ACOPF includes nonlinear power-flow equations, generation and voltage bounds, and transmission-line current limits.
  • 4.1. Formulation as a Real-Time Optimal Power Flow Problem: ACOPF is nonconvex because nonlinear power-flow equations create computationally demanding optimization problems.The nonlinear active-power, reactive-power, and squared-current expressions depend on transmission-line parameters.
  • 4.2. Real-Time Operation as Closed-Loop Optimization: Closing the loop enables economically efficient redispatch that responds to unscheduled events while respecting voltage and line-flow limits.The optimization is interpreted as a residual problem around a pre-planned generation schedule, with bounds representing dispatchable reserves.
  • 4.2. Real-Time Operation as Closed-Loop Optimization: Feedback-based optimization achieves almost perfect cost performance relative to an a-posteriori sequential ACOPF solution using omniscient information.The comparison is reported for the left panel of Figure 17.
  • 4.2. Real-Time Operation as Closed-Loop Optimization: Constraint violations remain very minor and temporary because the controller jointly manages active and reactive power, including renewable curtailment to prevent line overloads.Both solar and wind generation are curtailed when necessary to maintain line limits.
  • 4.2. Real-Time Operation as Closed-Loop Optimization: The linearized output projection method has similar performance to projected saddle-point flow while nearly eliminating transient line-flow violations.It predicts and avoids violations using first-order model information, but requires solving a quadratic program at each iteration.
  • 4.2. Real-Time Operation as Closed-Loop Optimization: The feedback controller has reached industrial deployment for reactive-power optimization in a real distribution grid subject to voltage constraints.The deployed system supports existing infrastructure and 24/7 uninterruptible operation.

5. Conclusions and Outlook

The survey frames optimization algorithms as dynamical systems interconnected with physical plants through feedback. It emphasizes robustness and constraint satisfaction, while identifying unresolved challenges for future research.

  • Conclusions: The survey interprets classical numerical optimization algorithms as dynamical systems and interconnects them with asymptotically stable physical plants.The interconnection uses well-defined input-output behavior to realize closed-loop optimization.
  • Conclusions: Online feedback optimization improves robustness to uncertain data while reducing required model information and computational effort through physical constraint enforcement.The approach exploits physical properties such as steady-state response and saturation effects.
  • Conclusions: The survey focuses on guarantees for closed-loop robustness and constraint satisfaction.These guarantees support controller designs that meaningfully exploit physical system properties.
  • Outlook: Many questions remain unanswered, motivating two proposed avenues for future inquiry.The paper presents feedback-based optimization as an active research area rather than a completed theory.

General Equilibrium Seeking in Closed Loop

Closed-loop optimization can target general equilibrium conditions beyond KKT optimality conditions. The same structure supports non-cooperative equilibrium seeking and robust tracking of time-varying Nash equilibria.

  • General Equilibrium Seeking in Closed Loop: Feedback controllers can seek states satisfying variational inequalities and other equilibrium conditions instead of only optimization KKT conditions.This extends the target beyond steady-state optimality specifications.
  • General Equilibrium Seeking in Closed Loop: In non-cooperative settings, feedback controllers can drive players toward a Nash equilibrium.Online measurements also support robust tracking when disturbances and problem parameters change.

Time-Varying Non-Convex Optimization

Time-varying optimization has mainly been studied for convex problems, where a unique global optimizer defines a trackable trajectory. Time-varying nonconvex problems are substantially harder because minimizers may be multiple, appear, or disappear.

  • Time-Varying Non-Convex Optimization: Time-varying optimization studies have focused primarily on convex problems, often with strongly convex objectives and unique global optimizers.The unique optimizer forms a well-defined trajectory that controllers and algorithms can track.
  • Time-Varying Non-Convex Optimization: Time-varying nonconvex optimization is more challenging because multiple local minimizers may exist and global minimizers can appear or disappear.These changes can make continuous tracking of one global minimizer impossible.

A.1. Network Congestion Control

Network congestion control forms a distributed feedback loop in which source controllers and link dynamics jointly track the solution of a utility maximization problem. The resulting projected saddle-point flow converges to a solution while using locally available information.

  • Link prices rise under overload and packet loss, decrease otherwise, and remain nonnegative.
  • Source rates adapt by minimizing local cost functions corrected by aggregate congestion prices, equivalently maximizing utility.
  • The controllers are fully distributed and require only locally available information.
  • The closed-loop system is a projected saddle-point flow whose trajectories converge to a solution of the network utility maximization problem.
  • Source controllers and link dynamics form a closed feedback loop that implicitly tracks the underlying utility maximization solution.

A.2. Optimal Frequency Control

Optimal frequency control reframes grid regulation as a closed-loop optimization problem for power balance and frequency regulation. The formulation exposes a saddle-point structure but also reveals sensitivity to controller-model matching.

  • Frequency control is framed as closed-loop optimization to improve upon classical frequency-control schemes.
  • The construction assumes a lossless AC transmission network modeled around an operating point with deviations from nominal values.
  • At steady state, frequency deviations are driven to zero and line-flow deviations from nominal values are compensated.
  • The grid dynamics realize a partial saddle-point flow whose optimization solution guarantees power balance and exact frequency regulation.
  • Choosing controller gains and β_j appropriately recovers the physical grid model, but the condition B_j = D_j requires accurate damping estimates.
  • Numerical experiments indicate stability even when B_j = D_j does not hold exactly.

B. Related Research Streams

The survey contrasts feedback-based optimization with extremum seeking, modifier adaptation, and real-time iteration schemes. These approaches differ in model requirements, computational burden, dimensionality, and constraint handling.

  • The survey highlights extremum seeking, modifier adaptation, and real-time iteration as related optimization-and-feedback approaches.
  • Extremum seeking is model-free and probes the system with perturbation signals, but is best suited to low-dimensional inputs without complicated constraints.
  • Modifier adaptation mitigates model bias in successive optimization problems but does not reduce computational requirements relative to feedforward optimization.
  • Real-time iteration schemes address constrained receding-horizon problems with limited resources, generally require a full plant model, and incur burden scaling with planning horizon.
  • Iterative feedback tuning and iterative learning control optimize controllers through repeated experiments or operation, making them only remotely related because of their episodic nature.
  • Reinforcement learning is excluded from the review because it is traditionally framed as optimal control over Markov decision processes.

B.1. Extremum Seeking

Extremum seeking optimizes an unknown plant objective by injecting a probing signal and using measured objective values to approximate gradient-based dynamics. Its model-free operation is attractive, but multidimensional inputs and engineering constraints limit its scope.

  • Extremum seeking steers a plant toward an extremum rather than tracking a setpoint.
  • A dither signal locally explores the objective and enables optimization without plant-model information or substantial computation.
  • For a scalar plant, sinusoidal input perturbation produces averaged dynamics that approximate gradient flow while requiring only measurements of Φ(y).
  • Extremum-seeking systems generally involve plant, probing-signal, and slow averaged-optimization timescales.
  • Projection maps and other formulations have extended extremum seeking to constrained model-free optimization.
  • Multidimensional inputs require carefully chosen noninterfering probing frequencies, contributing to extremum seeking’s confinement to relatively low-dimensional systems.

B.2. Modifier Adaptation

Modifier adaptation repeatedly updates an optimization problem using plant measurements to reduce discrepancies caused by model bias. It can handle constraints, but requires estimating true plant sensitivities and does not reduce computation or model-information requirements.

  • Modifier adaptation: Modifier adaptation repeatedly incorporates plant measurements into the optimal steady-state problem to reduce discrepancies between model-based solutions and the physical plant.The optimization is performed repeatedly, with each iteration modifying the problem using measurements from the previous iteration.
  • Modifier adaptation: MA corrects cost and constraint functions with adaptation terms rather than directly learning or identifying a better plant model.
  • Convergence: If the adaptation scheme converges to u⋆, the limit is a critical point of the modified objective ˜Φ(u).
  • Constrained adaptation: MA applies to constrained problems by introducing modifiers on constraints analogously to modifiers on the objective.
  • Sensitivity estimation: The method’s difficult step is estimating the true plant sensitivities, which can be achieved using finite differences.
  • Limitations: MA does not reduce computational burden or the amount of model information required.

B.3. Model Predictive Control with Incomplete Optimization and Real-Time Iterations

Model predictive control repeatedly solves finite-horizon optimal control problems while implementing only the first computed input, whereas real-time iterations approximate each solution with a single SQP step. These approximations trade optimization accuracy for computation time, with stability and convergence available under additional assumptions, but MPC remains computationally demanding and model-dependent.

  • Model predictive control: MPC stabilizes constrained plants by repeatedly solving finite-horizon optimal control problems and applying only the first input before updating the state measurement.
  • Computational limitations: High computational requirements have restricted MPC applications to relatively slow and low-dimensional process-engineering plants.
  • Explicit MPC: Standard linear MPC can avoid online receding-horizon optimization by using explicit MPC, which implements a precomputed solution as a lookup table.
  • Real-time iterations: Real-time iteration schemes approximate each receding-horizon solution by performing only a single SQP iteration.
  • Real-time iterations: RTIs trade approximation error for computation savings because more frequent solves make the receding-horizon problem change less between samples.
  • Stability and convergence: Under additional assumptions, RTI schemes admit proofs of stability and convergence.
  • Economic MPC: Economic MPC remains computationally expensive, lacks an a-priori guarantee that its optimum is a plant steady state, and requires a full plant-dynamics model for stability analysis.
Loading 2103.11329v2…