PaperlayerView exact revision on arXiv

Source-linked AI summary

A Historical Perspective of Adaptive Control and Learning

Anuradha M. Annaswamy, Alexander L. Fradkov

arXiv:2108.11336v2math.OCeess.SY

TL;DR

Adaptive control addresses real-time control of uncertain dynamic systems while connecting control with parameter learning. The paper surveys seven decades across deterministic and stochastic systems, organizing key problems and solutions around stability, robustness, and learning. Its central conclusion is that bounded closed-loop behavior and asymptotic performance can be guaranteed without guaranteed parameter learning, while robustness remains constrained by disturbances, unmodeled dynamics, and time-varying effects.

  • Problem

    Adaptive control seeks real-time control of uncertain dynamic systems while addressing whether boundedness and desired asymptotic behavior can be ensured amid parametric and non-parametric uncertainties.

  • Method

    The paper provides a historical synthesis across deterministic continuous-time and stochastic discrete-time adaptive control, organized around chronology, problem statements, solution highlights, and parameter-learning conditions.

  • Results

    The surveyed results establish bounded closed-loop behavior and asymptotic performance, with parameter learning requiring additional persistent-excitation conditions that cannot generally be guaranteed in closed loop.

  • Takeaways & Limitations

    Adaptive control prioritizes real-time boundedness and asymptotic performance, while complete learning and subsequent optimality require additional conditions.

  • Takeaways & Limitations

    The surveyed adaptive-control solutions face challenges from disturbances, unmodeled dynamics, time delays, and time-varying parameters, with disturbances potentially causing parameter drift.

Abstract

from arXiv · show

This article provides a historical perspective of the field of adaptive control over the past seven decades and its intersection with learning. A chronology of key events over this large time-span, problem statements that the field has focused on, and key solutions are presented. Fundamental results related to stability, robustness, and learning are sketched. A brief description of various applications of adaptive control reported over this period is included.

1 Introduction

This article surveys adaptive control across seven decades, emphasizing its intersection with parameter learning and spanning deterministic and stochastic systems. It organizes the field chronologically, by problem statements, and through solution highlights, while noting boundaries of coverage.

  • The article traces adaptive control advances across deterministic continuous-time and stochastic discrete-time systems over seventy years.
  • It focuses on advances with a significant intersection with parameter learning and chronicles developments from across the globe.
  • The survey presents the field through chronology, problem statements, and solution highlights, with references supporting deeper study.
  • It highlights applications of adaptive control and provides concluding takeaway messages.
  • The survey excludes several neighboring topics and defers comprehensive discussion of reinforcement learning and other adaptation-learning intersections.

2 A chronology

Adaptive control evolved from early adaptation concepts and aerospace applications into parallel deterministic and stochastic branches, with stability and robustness becoming central milestones. Across these developments, parameter adjustment supports control and learning, while robustness addresses disturbances and unmodeled dynamics.

  • Early adaptive control was motivated by aerospace autopilot design requiring online parameter adjustment across changing flight conditions.
  • Gradient-based adaptation and related early rules prompted stability analysis because phase lag could produce instability.
  • Adaptive control sought well-behaved closed-loop systems with asymptotic tracking or regulation under parameter uncertainty.
  • The field developed parallel deterministic and stochastic branches, using model-reference adaptive control and self-tuning regulators as representative approaches.
  • Parameter adjustment commonly uses a regressor and normalization component, while estimation methods include stochastic approximation and recursive least squares under persistent excitation.
  • The 1980s robustness framework extended adaptation to disturbances, time-varying parameters, and unmodeled dynamics after earlier algorithms proved inadequate.

3 Problem Statements

Adaptive control formulates real-time control of uncertain dynamic systems as a problem of achieving bounded, asymptotically desirable behavior while estimating unknown parameters. Its problem classes span deterministic continuous-time and stochastic discrete-time systems, with robustness extensions addressing disturbances, noise, time variation, and unmodeled dynamics.

  • 3.1 Problem formulation: Adaptive control designs real-time inputs for systems with potentially nonlinear dynamics, unknown parameters, disturbances, and stochastic noise.The system state and outputs may have different dimensions, with many physical systems satisfying n >> p > m.
  • 3.1.1 Boundedness and real-time decision making: The central control question is whether bounded closed-loop behavior and desired asymptotic tracking can be ensured despite parametric and non-parametric uncertainties.After these guarantees, adaptive control addresses learning through parameter convergence.
  • 3.1.1 Boundedness and real-time decision making: Adaptive controllers commonly combine a control law with an online estimator whose time-varying parameter estimate is updated from available data.The challenge is constructing these functions so the estimate learns the requisite unknown control parameter while supporting the control objective.
  • 3.1.4 Robust adaptive control: Robust adaptive control extends the idealized known-model setting to unmodeled dynamics, time-varying parameters, disturbances, and noise.The design objective is bounded signals with errors proportional to perturbation size, using persistent excitation or modified adaptive laws.
  • 3.2 Parameter estimation and learning: Parameter-estimation problems use recursive algorithms such as stochastic approximation and recursive least squares, with convergence tied to persistent excitation.A unified stochastic-approximation treatment establishes convergence under convexity, bounded growth, and Robbins–Monro conditions.
  • 3.4 Reinforcement Learning/Adaptive Dynamic Programming: Reinforcement-learning control under disturbances faces difficult stability verification because performance averages integrals over disturbance trajectories.The paper notes that few works establish stability under additional assumptions.

4 Solutions

The survey develops adaptive-control solutions from estimation and parameter adjustment to Lyapunov-based tracking, output-feedback, and passification methods. These solutions establish boundedness, asymptotic error convergence, and parameter convergence under progressively stronger structural assumptions.

  • 4.1 Adaptive Estimation and Control: Gradient descent updates θ using the measurable performance error e_y and regressor φ, while higher-order tuners additionally use Hessian information.
  • 4.1.2 Adaptive Controllers with State Feedback: Lyapunov analysis establishes e(t) → 0 for state-feedback adaptive control when a positive definite Q defines V with V̇ = −e^TQe.
  • 4.1.3 Adaptive Observers: Adaptive observers achieve asymptotic parameter convergence when the regressor is persistently exciting, with accelerated variants using matrix regressors, time-varying learning rates, or regressor mixing.
  • 4.1.4 Adaptive Controllers with Output Feedback - A special case: Output-feedback MRAC uses SPR reference models and the KYL to cancel cross terms, guaranteeing bounded closed-loop signals and e_y(t) → 0.
  • 4.1.5 Adaptive Controllers with Output Feedback - Passification approach: Passification provides convergence and Lyapunov guarantees when the relevant transfer function is hyperminimum-phase, a condition that is necessary and sufficient for the required Lyapunov function.
  • 4.1.6 Adaptive Controllers with Output Feedback - The General case: The general output-feedback case retains requirements on known order, relative order, high-frequency-gain sign, and stable plant zeros, while normalized adaptation ensures bounded parameters and asymptotic output-error convergence.

4.2 Learning and Persistent Excitation

Adaptive control distinguishes learning the unknown parameters from achieving safe real-time performance. Persistent excitation is required for parameter convergence, but satisfactory estimation and tracking can be guaranteed without it.

  • Persistent excitation is the necessary and sufficient condition discussed for convergence of the parameter error to zero.
  • Satisfactory output estimation and tracking do not require persistent excitation, allowing safe controlled behavior before parameter learning is complete.

4.3 Nonlinear Systems

The survey extends adaptive control to nonlinear and nonlinearly parameterized systems using backstepping, neural-network approximation, speed-gradient, and min-max methods. It also summarizes stochastic and discrete-time stability results, including adaptive minimum-variance control under structural assumptions.

  • 4.3 Nonlinear Systems: Backstepping became a major iterative design technology for adaptive nonlinear control, following efforts to relax matching conditions for nonlinearities.
  • 4.3 Nonlinear Systems: Neural networks approximate nonlinear dynamics for learning and control, while fixing inner-layer weights preserves tractability through a quadratic Lyapunov function.
  • 4.4 Nonlinearly Parameterized Systems: Speed-gradient methods address nonlinear parameterizations when a parametric stabilizing feedback law is known, with positive-definite gain matrices supporting Lyapunov analysis.
  • 4.4 Nonlinearly Parameterized Systems: Adaptive control applies to convex or concave nonlinearities, including neural networks with convex activations such as ReLU, but requires tools beyond gradient methods, including min-max methods.
  • 4.5 Stochastic and Discrete-Time Systems: Stochastic and discrete-time adaptive-control results establish stability for SA and RLS algorithms and self-tuning regulators under assumptions on model order, delay, gain, and zero locations.
  • 4.5.3 Adaptive LQG control: When unknown system parameters enter the LQG problem, the known-parameter control gain must be replaced by one depending on parameter estimates, making the problem significantly more complex.

4.6 Adaptive control of Continuous-time systems: Robustness

Adaptive control robustness depends on how disturbances, unmodeled dynamics, and excitation interact with parameter adaptation. The literature uses modified adaptive laws, persistent excitation, and bounded parameter updates to preserve boundedness, while insufficient excitation can produce imperfect learning and bursting.

  • 4.6.1 Modifications in the adaptive law: Disturbances can drive adaptive parameters to −∞ because the stabilizing adaptive controller behaves like a nonlinear integral controller with windup.The nominal Lyapunov guarantee applies when v(t) ≡ 0; anti-windup corrections seek negativity outside a compact error-parameter set.
  • 4.6.1 Modifications in the adaptive law: Robustness to unmodeled dynamics and delays is harder than robustness to bounded disturbances or time-varying parameters, but projection-based adaptation can maintain bounded parameters and solutions.The cited results constrain adaptation within a bounded set rather than guaranteeing unrestricted parameter convergence.
  • 4.6.2 Use of Persistent Excitation: Persistent excitation large relative to disturbance magnitude guarantees globally bounded adaptive-system solutions, whereas insufficient excitation can permit instability.For state-dependent disturbances, persistent excitation also supports local robustness through averaged-system exponential stability.
  • 4.7 Bursting Phenomenon and Imperfect Learning: Without sufficient persistent excitation, parameters need not converge to their true values, although closed-loop boundedness and performance can still hold.In a first-order example, estimates may converge to constants different from the true parameters, while the output approaches its target; persistent excitation restores true-parameter convergence.
  • 4.7 Bursting Phenomenon and Imperfect Learning: When perturbations occur without persistent excitation, parameter drift can move the closed loop toward unstable configurations and trigger bursting through oscillatory parameter readjustment.The phenomenon is associated with imperfect learning and can arise during simultaneous identification and control.

4.8 Adaptive Control in the Presence of Input and State Constraints

Adaptive control with input and state constraints represents saturation effects as disturbances and uses constrained Lyapunov constructions to preserve boundedness. These extensions retain real-time operation but may introduce additional assumptions, robustness trade-offs, or unresolved challenges for changing nonlinear plants.

  • 4.8 Adaptive Control in the Presence of Input and State Constraints: Saturation-aware adaptive controllers guarantee the actual plant input meets magnitude and rate limits while matching the unconstrained command when the command is small.Magnitude and rate saturation are represented through additive known disturbances that become nonzero only when their respective limits are exceeded.
  • 4.8 Adaptive Control in the Presence of Input and State Constraints: Input constraints require proving plant-state boundedness separately because bounded augmented error does not imply bounded original state error.The augmented error combines tracking error with saturation effects, so standard Lyapunov arguments alone are insufficient for the plant states.
  • 4.8 Adaptive Control in the Presence of Input and State Constraints: Barrier Lyapunov functions enforce state limits by growing large as error variables approach their boundaries.A logarithmic term can replace a quadratic term to prevent a state variable from exceeding its prescribed limit.
  • 4.9 Assumptions and Challenges: Most surveyed adaptive solutions are global and real-time, requiring no training, exploration, or simulation experiments.These properties are presented as distinctive advantages of adaptive control methods.
  • 4.9 Assumptions and Challenges: Extensions that relax structural assumptions can sacrifice robustness, require persistent excitation, or increase computational burden.The trade-offs arise in some extensions to high-frequency-gain and nonminimum-phase assumptions.
  • 4.9 Assumptions and Challenges: For neural adaptive controllers under unforeseen parameter and disturbance changes, guarantees of boundedness, convergence, and optimality remain an open problem.The paper links this setting to bursting when the system has not been satisfactorily trained under the changed conditions.

5 Applications

Adaptive control theory has been accompanied by applications across aerospace, industrial process control, and other engineered systems. Reported examples include aerial platforms, process-control products, and applications spanning ships, reactors, engines, furnaces, drives, and telescopes.

  • Aerospace applications: Applications span aerial platforms, including JDAM and X-36, reflecting continued transitions from conventional MRAC to adaptive OBLTR architectures.The cited discussion describes an ongoing technology transition in aerial-platform applications.
  • Industrial applications: Reported application areas include ship autopilots, cement mills, chemical reactors, diesel engines, glass furnaces, HVAC systems, motor drives, paper machines, optical telescopes, and titanium oxide kilns.The paper points readers to surveys and textbooks for these additional application accounts.
  • Industrial applications: Industrial products implementing MRAC, STR, and adaptive MPC include NOVATUNE, NOVAMAX, BrainWave, and Micro-Controller 2000X.The products were reported in connection with process-control applications and industrial implementations.

6 Summary and Concluding Remarks

The article surveys seven decades of adaptive control by organizing advances, problems, solutions, and applications across deterministic continuous-time and stochastic discrete-time systems. It emphasizes bounded closed-loop behavior, asymptotic properties, and the difficulty of optimizing over all future time under changing parametric uncertainty.

  • The survey attempts to condense seventy years of literature into highlights while avoiding deep theorem discussions and proofs.
  • The article covers adaptive-control advances in deterministic continuous-time and stochastic discrete-time systems through a chronological taxonomy, problem survey, solution highlights, and application review.
  • Adaptive controllers primarily target bounded closed-loop solutions and asymptotic output, and sometimes input, properties, with later results also addressing magnitude, rate, and state constraints.
  • Performance goals remain focused on real-time behavior because uncertainty may arise at any time, making cold-start optimization over all future instants difficult or impossible.

Stability framework

Adaptive control creates a nonlinear closed loop because estimation and control occur simultaneously, so stability is the first property requiring assurance. The framework uses Lyapunov functions to connect definiteness conditions on V and its derivative with stability guarantees.

  • Simultaneous estimation and control makes the control input depend on a parameter estimate that itself depends on the input, producing a nonlinear closed-loop system.
  • A Lyapunov function with positive definiteness, decrescence, negative-definite derivative, and radial unboundedness establishes uniform asymptotic stability in the large.
  • If the Lyapunov derivative is only negative semi-definite, uniform stability is ensured; a stronger bound yields exponential stability.
  • Adaptive-control stability analysis commonly represents tracking and parameter errors as system states and often uses quadratic Lyapunov functions.

Rational SPR functions and the KYL

The section introduces strictly positive real rational transfer-function matrices and states a basic characterization associated with the Kalman-Yakubovich-Lemma framework.

  • The section presents the SPR definition as preparation for a simple version of the KYL associated with adaptive-control analysis.
  • A rational transfer-function matrix Z(s) is defined as strictly positive real when its shifted-domain condition holds for some ϵ > 0.
  • The SPR definition applies to an n × n matrix whose elements are rational transfer functions.

The Kalman Yakubovich Lemma

The Kalman-Yakubovich Lemma characterizes strict positive realness for a transfer-function realization through symmetric positive definite matrices.

  • For a minimal realization with Z(∞) = 0 and poles in Re[s] < −µ, Z(s) is SPR if and only if symmetric positive definite matrices P and Q exist satisfying the stated conditions.
  • The characterization applies to a matrix of rational functions with a minimal realization {A, B, C}.
  • The pole-location requirement places all poles strictly in the left half-plane before applying the matrix characterization.

Bregman Divergence

Bregman divergences extend Lyapunov-function constructions for convergence proofs, enabling extended versions of speed-gradient algorithms.

  • Bregman Divergence: Bregman divergences provide a convenient component of extended Lyapunov functions for convergence proofs.The construction uses a twice differentiable function and its gradient-based divergence.
  • Bregman Divergence: The resulting constructions yield extended speed-gradient algorithms that generalize earlier algorithms.

Averaging

Averaging analyzes adaptive-system robustness by separating slow state variation from rapidly varying terms. Under bounded, almost-periodic, persistently exciting inputs and suitable conditions, the resulting system is exponentially stable, including many cases where the closed-loop transfer function is not SPR.

  • Averaging: The robustness analysis applies averaging when the state varies slowly, so rapidly varying terms do not affect its long-run variation.This method is used for adaptive systems with disturbances and unmodeled dynamics.
  • Averaging: The origin of the averaged error model is exponentially stable for sufficiently small µ when the stated excitation conditions hold.The theorem assumes that ω(t) is bounded, almost periodic, and persistently exciting.
  • Averaging: Stability depends critically on the excitation spectrum relative to the closed-loop transfer function.The excitation is expanded using an inverse Fourier transform, linking condition (152) to the spectrum of ω and W̄_m(s).
  • Averaging: The stability condition can hold for a large class of problems even when the closed-loop transfer function is not SPR.
Loading 2108.11336v2…