Source-linked AI summary
Extremum Seeking Control: Three Revolutions and the Road Ahead
Tiago Roux Oliveira
TL;DR
The paper examines why extremum seeking has remained relevant across changing technologies and mathematical settings. It connects that durability to the enduring engineering problem of learning to improve performance while a system operates under uncertainty, while identifying challenges in extending guarantees to more complex architectures.
Problem
Extremum seeking addresses how an autonomous system can discover its best operating point without complete knowledge before acting.
Method
The article reflects on extremum seeking's historical development and its expansion across delays, PDEs, games, and increasingly complex cyber-physical settings.
Results
Extremum seeking has remained relevant for more than a century by repeatedly adapting to new mathematical techniques, technologies, and application domains.
Takeaways & Limitations
Because it was conceived as a feedback algorithm grounded in stability, robustness, adaptation, and dynamical systems, extremum seeking remains positioned for safe real-time autonomous decision-making.
Takeaways & Limitations
Extending rigorous guarantees while improving convergence speed, preserving robustness, and handling high-dimensional, distributed, and sophisticated learning architectures remains challenging.
Abstract
from arXiv · showhide
The history of extremum seeking is not merely the history of an algorithm; it is the history of an idea. Few ideas in control engineering have demonstrated the remarkable longevity of extremum seeking control (ESC). Invented more than one hundred years ago, ESC has continually reinvented itself while remaining faithful to its original objective: enabling systems to optimize their performance without relying on accurate mathematical models. This perspective article proposes that the evolution of ESC is best understood through three scientific revolutions. The first Engineering Revolution (1922-1999) established the engineering principles of model-free optimization; the second Mathematical Revolution (2000-2010) provided the rigorous mathematical foundations that transformed ESC into a mature discipline of nonlinear control; and the third ongoing Infinite-Dimensional and Cyber-Physical Revolution (2010-present) continues to expand its scope toward delays, partial differential equations, distributed optimization, event-triggered implementations, and increasingly complex cyber-physical systems. Beyond recounting this historical evolution, we offer a personal perspective on why ESC has remained relevant across successive technological eras. We argue that its enduring influence arises because the fundamental engineering challenge has never changed: "how can a dynamical system learn to improve its own performance when the optimum is unknown?" As optimization, learning, and feedback control become increasingly intertwined, ESC appears uniquely positioned to contribute to a new generation of intelligent autonomous systems, pointing toward what may become the field's fourth scientific revolution.
1. Historical Perspectives: From Engineering Intuition to Learning Through Feedback
Extremum seeking control evolved from an engineering heuristic into a mathematically grounded framework for model-free optimization and autonomous learning. Its development spans practical experimentation, rigorous convergence analysis, and extensions to increasingly complex dynamical systems.
- Model-free optimization: ESC optimizes unknown or difficult-to-model systems by probing performance, observing changes, and steering operation toward improvement.Its model-free approach allows optimization and control to occur during system operation.
- Engineering intuition: Early extremum-seeking methods showed practical success but remained largely heuristic, with limited understanding of convergence conditions and performance guarantees.For decades, the mathematical foundations of these methods remained elusive.
- Mathematical foundations: Rigorous nonlinear analysis established exponential convergence to an arbitrarily small neighborhood of the unknown optimum under suitable perturbation choices.This foundation transformed extremum seeking into a mature branch of nonlinear adaptive control.
- Three scientific revolutions: The field’s evolution can be organized into three revolutions: engineering development from 1922–1999, mathematical formalization from 2000–2010, and ongoing expansion after 2010.The third stage addresses delays, partial differential equations, distributed optimization, and other complex environments.
- Infinite-dimensional and cyber-physical systems: Recent work extends extremum seeking to delays, PDEs, and multi-agent settings while preserving its model-free character and broad engineering applicability.These developments support a view of ESC as a general framework for autonomous learning in dynamical systems.
2. Pushing the Boundaries of Extremum Seeking
ESC has expanded from model-free optimization into delayed, game-theoretic, infinite-dimensional, resource-aware, constrained, resilient, and finite-time settings. Across these developments, the field preserves its core objective while adding mechanisms for stability, feasibility, resource allocation, and speed.
- Historical challenges: Historical ESC was framed as a model-free approach whose scalability was questioned for systems with many adjustable parameters and multiple time lags.The 1958 perspective anticipated that increasingly accurate mathematical models might eventually be required.
- Delays and games: Delay-compensated ESC uses predictors and perturbation-based Hessian estimates to address arbitrarily long input-output delays.The design assumes a known constant delay and uses estimated second-order information for prediction.
- Delays and games: For quadratic games, ESC extends to decentralized Nash equilibrium seeking, including unique equilibria under suitable matrix assumptions and delayed player actions.The formulation requires no knowledge of other players’ payoffs or actions, while prediction feedback supports collective stabilization under delays.
- Cyber-physical developments: Event-triggered ESC adds resource awareness by updating gradient feedback only when needed while retaining practical convergence and excluding Zeno behavior.The framework allows optimization performance to be traded against communication, computation, and actuation effort.
- Cyber-physical developments: Saturated, cyber-resilient, and sliding-mode ESC broaden the design objectives to include bounded updates, operation under attacks, and finite-time convergence.Together, these developments position ESC as a convergence-, constraint-, resilience-, and resource-aware optimization architecture.
3. Where the Field Is Going Next
Extremum seeking is expanding from real-time optimization toward feedback-based learning and autonomous operation in complex, changing systems. Its next challenges concern integrating these capabilities while preserving robustness, stability, and safety.
- Industry 4.0 and the Age of Autonomous Optimization: Modern cyber-physical systems motivate ESC because models can become outdated as sensors, actuators, communication networks, and operating conditions change.
- One Principle, Countless Applications: ESC is being applied across automobiles, wind turbines, industrial processes, telecommunications, robotics, renewable energy, and other domains while retaining the same underlying methodology.
- Beyond Optimization: A New View of Learning: ESC learns through feedback by exploring the environment, extracting information from measured responses, and incorporating it into future decisions.
- Extremum Seeking and Artificial Intelligence: Future architectures may combine machine-learning representations with ESC adaptation mechanisms to support predictive, mathematically grounded real-time optimization.
- What Challenges Still Remain?: Open challenges include faster convergence, robustness to noise and uncertainty, lower computational burden, safe constrained optimization, and guarantees for integrated learning architectures.
- Toward the Fourth Revolution: The field’s next revolution is framed as expanding classical control principles into increasingly autonomous and interconnected engineering systems.
4. Personal Reflections and Conclusions
The authors interpret ESC’s longevity as a consequence of addressing a persistent engineering problem: improving performance when the optimum is unknown. They view its future in the integration of optimization, learning, prediction, communication, and decision-making within feedback architectures.
- Why Has Extremum Seeking Endured for More Than a Century?: ESC has remained relevant because engineering systems have become more complex while the question of improving performance without knowing the optimum has persisted.
- Why Has Extremum Seeking Endured for More Than a Century?: Successive generations changed ESC’s engineering intuition, mathematical tools, and application domains while preserving the central challenge of learning during operation.
- Toward the Fourth Revolution: The proposed fourth revolution would integrate optimization, learning, prediction, communication, and decision-making into a unified feedback architecture.
- Toward the Fourth Revolution: Future autonomous systems are expected to perceive, predict, cooperate, adapt, and optimize multiple objectives, with learning intrinsic to the feedback loop.
- Toward the Fourth Revolution: ESC is positioned to contribute because its foundations emphasize stability, robustness, adaptation, and dynamical systems for safe real-time decisions.
- Personal Reflections and Conclusions: The authors portray ESC as a century-spanning principle that migrates across disciplines while preserving conceptual simplicity and mathematical foundations.
A.1. Fundamentals of Extremum Seeking
Basic ESC estimates the gradient of an unknown static map by perturbing the input and using the measured response to update an estimate of the optimizer. Averaging analysis establishes exponential convergence to a neighborhood whose size decreases with suitable perturbation choices.
- Basic gradient-based ESC: The common gradient-based ESC version uses perturbation signals to estimate the gradient of an unknown static map.
- Perturbation and adaptation: The actual input combines an optimizer estimate with a sinusoidal perturbation, while the estimate is updated using an integrator with adaptation gain k.
- Perturbation and adaptation: The perturbation signal can be replaced by other zero-mean signals, including square waves or stochastic noise.
- Averaged analysis: Averaging replaces sinusoidal signals by their means and yields an exponentially stable average system under suitable conditions.
- Convergence guarantee: The input converges exponentially to a neighborhood of the unknown optimizer with order O(1/ω + a), and the output approaches the vicinity of the optimal output.
A.2.1. Basic Idea of Predictor Feedback Design with Actuator Delay
Predictor feedback design addresses actuator delay by transforming a stabilizing undelayed control into an implementable delayed feedback law. The resulting law contains a distributed delay term, making it infinite-dimensional and requiring time-domain stability analysis.
- Problem setup: The section considers a controllable linear infinite-dimensional system whose input is delayed by D units of time.
- Stabilizing objective: Starting from a gain K that makes A + BK Hurwitz for the undelayed system, the design seeks delayed control with comparable stabilization behavior.
- Predictor feedback construction: The predictor-based control is rewritten using the variation-of-constants formula so it can be implemented from the current state rather than future state values.
- Implementation consequence: The implementable feedback law is infinite-dimensional because it includes a distributed delay term involving past controls.
- Initial interval: Before the delay elapses, t ∈ [0,D], the system state follows a separate initial-time evolution.
- Analysis framework: The law is connected to finite spectrum assignment and reduction approaches, while Lyapunov-Krasovskii analysis provides a time-domain route for stability analysis.
A.2.2. Backstepping Boundary Control of Transport PDEs
This section models actuator delay as a transport PDE and uses backstepping to transform the delayed system into a target system with a boundary control law. The resulting controller has an equivalent representation in terms of the input signal and matches the earlier controller.
- Transport-PDE model: Actuator delay D is modeled by a first-order hyperbolic transport PDE coupled to the original ODE system.The coupled equations form an ODE-PDE cascade driven by the boundary input U.
- Backstepping design: A stabilizing no-delay feedback with A + BK Hurwitz is used as the basis for the backstepping design.The transformation maps the delayed cascade into a target system.
- Boundary control law: The backstepping transformation yields gain functions and a boundary control law evaluated at the delay boundary x = D.The gains γ(x) and q(x,y) are substituted into the transformation before setting x = D.
- Controller representation: The controller expressed through the transport-delay state can also be written in terms of the input signal U(t).This input-based representation is obtained using the delay-state relation.
- Equivalence: The resulting controller is identical to controller (87) from Section A.2.1.
A.3. Basic Idea of Nash Equlibrium Seeking in a Two-Player Game
The duopoly example formulates firms’ price choices as a Nash equilibrium problem and applies model-free extremum seeking to recover the equilibrium using only payoff measurements. Compared with a standard parallel update scheme, the extremum-seeking approach reduces the modeling information required and has global convergence.
- Duopoly model: Each firm chooses a price to maximize profit, with sales and profit depending on both firms’ prices and market assumptions.The model assumes fixed total demand, a preference for P1, and conditions on the relative prices.
- Nash equilibrium: The firms’ profit functions are quadratic in the prices, allowing the Nash equilibrium to be determined analytically.The equilibrium is subject to a marginal-cost condition ensuring the modeled price constraints hold.
- Model-free seeking: Deterministic extremum seeking with sinusoidal perturbations lets both firms adjust prices without knowing consumer preference, total demand, or the other firm’s cost or price.The strategy uses time-varying perturbations μ_i(t) = a_i sin(ω_i t + φ_i).
- Information requirements: Standard parallel action updates require substantially more modeling information, including each firm’s marginal cost and the other firm’s previous price.P1 also needs the total demand and consumer preference parameter under that scheme.
- Convergence: The extremum-seeking algorithm requires only measurements of the firms’ own payoff functions, and its convergence is global.The supplied passage contrasts this with the convergence properties of the standard scheme.
CRediT Author Statement
The sole author handled the manuscript’s conceptual, investigative, technical, illustrative, writing, and editing responsibilities.
- Tiago Roux Oliveira was the sole author and performed the conceptualization, literature investigation, technical development, material preparation, drafting, and final review.