PaperlayerView exact revision on arXiv

Source-linked AI summary

Feedback Linearization based on Gaussian Processes with event-triggered Online Learning

Jonas Umlauft, Sandra Hirche

arXiv:1911.06565v1eess.SY

TL;DR

The paper addresses the gap between online adaptation of nonparametric models and formal stability guarantees for data-driven control. It proposes a Gaussian-process feedback-linearizing controller with uncertainty-triggered closed-loop updates and safe forgetting. The resulting tracking error is asymptotically stable for noiseless measurements and ultimately bounded for noisy measurements, with the stated probabilistic qualification.

  • Problem

    Existing methods lack nonparametric online learning control laws that guarantee safe tracking behavior while remaining computationally efficient for real-time applications.

  • Method

    The paper combines Gaussian-process feedback linearization, closed-loop identification, event-triggered model updates, and safe forgetting under a data-point budget.

  • Results

    The tracking error is globally asymptotically stable for noiseless data and globally uniformly ultimately bounded for noisy measurements, with probability 1−δ under the stated assumptions.

  • Takeaways & Limitations

    Event-triggered learning can provide data-efficient online adaptation while retaining convergence guarantees and excluding Zeno behavior through a positive inter-event-time lower bound.

  • Takeaways & Limitations

    The guarantees rely on a bounded RKHS-norm assumption that is difficult to verify, and general-kernel mutual information may lack an analytic computation.

Abstract

from arXiv · show

Combining control engineering with nonparametric modeling techniques from machine learning allows to control systems without analytic description using data-driven models. Most existing approaches separate learning, i.e. the system identification based on a fixed dataset, and control, i.e. the execution of the model-based control law. This separation makes the performance highly sensitive to the initial selection of training data and possibly requires very large datasets. This article proposes a learning feedback linearizing control law using online closed-loop identification. The employed Gaussian process model updates its training data only if the model uncertainty becomes too large. This event-triggered online learning ensures high data efficiency and thereby reduces the computational complexity, which is a major barrier for using Gaussian processes under real-time constraints. We propose safe forgetting strategies of data points to adhere to budget constraint and to further increase data-efficiency. We show asymptotic stability for the tracking error under the proposed event-triggering law and illustrate the effective identification and control in simulation.

I. INTRODUCTION

The paper addresses the lack of data-efficient online learning control laws that provide stability guarantees for initially unknown nonlinear systems. It combines Gaussian-process modeling with feedback linearization and event-triggered updates to control computational complexity while adapting during closed-loop operation.

  • Motivation: Gaussian processes provide model uncertainty information, enabling update decisions based on current model reliability rather than fixed time intervals.Their nonparametric flexibility supports complex dynamics, but computational cost grows with the training dataset.
  • Motivation: Existing approaches often separate fixed-dataset identification from control, while online adaptation of nonparametric models with formal stability guarantees remains unavailable.The paper identifies a need for safe online learning control that remains computationally efficient for real-time applications.
  • Contribution: The proposed controller performs closed-loop identification with Gaussian processes and updates the model event-triggeredly only when needed.The controller uses feedback linearization for an initially unknown system and adapts while control is active.
  • Contribution: For noiseless data, the approach provides global asymptotic tracking-error stability; for noisy output data, it provides global ultimate boundedness with high probability.A lower bound on inter-event times excludes Zeno behavior.
  • Contribution: The method also proposes a forgetting strategy that preserves convergence guarantees under a constrained data-point budget.This addresses the growing computational burden caused by accumulating Gaussian-process training data.

III. GAUSSIAN PROCESS LEARNING FOR CONTROL AFFINE SYSTEMS

This section introduces Gaussian-process regression as the basis for learning unknown system dynamics in closed loop. Because closed-loop measurements combine the effects of drift and control, the standard regression setup must be augmented with prior structural knowledge.

  • III-B. Closed-loop identification: The resulting Gaussian-process models provide approximations for the unknown functions f(·) and g(·) from closed-loop data.The subsequent construction uses structured kernels tailored to the control-affine system.
  • III-A. Gaussian process regression: Gaussian-process regression represents unknown functions through a mean and covariance function and provides posterior mean and variance predictions from observations.The kernel determines properties such as smoothness and signal variance, while the mean can encode prior knowledge.
  • III-A. Gaussian process regression: Hyperparameters are selected by maximizing the likelihood of the observed input/output data, although this optimization is generally non-convex.Conjugate-gradient methods are typically used for the optimization.
  • III-B. Closed-loop identification: Standard Gaussian-process regression cannot directly identify the separate drift and input-gain functions because closed-loop measurements do not observe them individually.The paper therefore augments the regression framework using the known control-affine structure and prior knowledge.

B. Closed-loop identification with prior knowledge

The closed-loop identification method embeds positivity and control-affine structure into Gaussian-process models. Compound kernels separate the contributions of the unknown drift and input-gain functions while retaining nonparametric flexibility.

  • Closed-loop identification with prior knowledge: A positive prior mean is used for the input-gain model so that the estimated gain remains positive and supports well-behaved feedback linearization.The paper treats positivity as prior knowledge from the system assumptions.
  • Closed-loop identification with prior knowledge: Compound kernels represent the observed dynamics as the sum of unknown drift dynamics and the control input multiplied by unknown input-gain dynamics.This structure enables identification from measurements of their combined effect rather than separate observations.
  • Closed-loop identification with prior knowledge: The structured kernel remains nonparametric: it encodes the additive control-affine decomposition without imposing a fixed finite-dimensional parameterization.Additional knowledge such as periodicity or state-variable independence can also be incorporated through kernel design.
  • Closed-loop identification with prior knowledge: The Gaussian-process estimates inherit boundedness and differentiability from the squared-exponential kernel, while the input-gain estimate can be made strictly positive.The paper notes that these conclusions extend to other bounded, differentiable kernels.

C. Discussion

The discussion emphasizes that Gaussian-process feedback linearization can adapt online while preserving analyzable tracking behavior, but computational cost and identification assumptions constrain deployment. Event-triggered updates and bounded data sets address these implementation challenges.

  • C. Discussion: Closed-loop identification is underdetermined because infinitely many pairs of differentiable functions can produce the same observed sum.The paper states that the formal guarantees hold independently of an optional temporary open-loop extension.
  • C. Discussion: The dominant Gaussian-process computation involves matrix inversion with O(N^3) complexity, motivating event-triggered updates and efficient inverse updates.A Sherman–Morrison rank-1 update reduces the per-update operation count to O(N^2), but the quadratic cost may remain substantial.
  • C. Discussion: Gaussian processes offer flexible nonlinear modeling and transfer prior assumptions consistently into the adaptive control law.Their uncertainty quantification is the basis for the paper’s model-error and stability analysis.
  • C. Discussion: Gaussian-process model updates create a switching control law because the dataset and corresponding estimates change only at adaptation times.The analysis tracks desired trajectories through the resulting time-varying closed loop.

A. Control law

The paper uses a feedback-linearizing controller with GP estimates updated over switching intervals, yielding stability guarantees under online model adaptation.

  • A. Control law: The controller uses GP mean functions to estimate f and g, while the deterministic control law is applied over switching intervals.The estimates are based on the current training dataset, and the GP posterior means—not stochastic samples—enter the control law.
  • A. Control law: The online adaptation forms a switching or hybrid closed-loop system whose stability is analyzed with a common Lyapunov function.The Lyapunov function is required to decrease outside an ultimate-bound ball for arbitrary switching sequences.
  • A. Control law: With unknown f and g, the tracking error is globally uniformly ultimately bounded when the controller gain exceeds a critical value.Theorem 1 establishes existence of a gain threshold k_c* under the stated assumptions.
  • A. Control law: When g is known, the tracking error is globally uniformly ultimately bounded for any positive controller gain k_c.This result applies when f is modeled by an online-adapted GP mean function under Assumptions 1–5.
  • A. Control law: The same any-gain boundedness result holds when the GP training dataset remains fixed after an update.Corollary 2 covers the corresponding non-adapting case.

C. Quantifying the ultimate bound

The paper converts GP uncertainty into a probabilistic tracking-error bound by restricting the modeled function to a bounded RKHS norm on a compact set.

  • C. Quantifying the ultimate bound: The GP error guarantee relies on a bounded RKHS norm for f with respect to a squared exponential kernel and known hyperparameters.Without prior knowledge or such a restriction, the paper does not expect a general deterministic model-error guarantee.
  • C. Quantifying the ultimate bound: The model-error guarantee holds only on a compact set, which is chosen from the compact region to which the tracking error converges.The analysis sets the compact GP-error domain to the ultimate-bound region established earlier.
  • C. Quantifying the ultimate bound: With probability 1 − δ, the tracking error is uniformly ultimately bounded for any k_c > 0 by a bound determined from GP uncertainty.The bound applies under Assumptions 1–6 and uses the GP model adapted at arbitrary switching times.
  • C. Quantifying the ultimate bound: The uncertainty term uses the GP variance over a compact set, with βκ depending on information gained from noisy samples.The maximum variance is taken over the compact analysis set, while γκ measures maximum mutual information from the samples.
  • C. Quantifying the ultimate bound: Theorem 1 supplies one bound valid across all intervals, whereas Theorem 2 provides a separate quantitative bound Bκ for each interval.The interval-specific bound need not be reached before that interval ends because convergence may take infinite time.

V. EVENT-TRIGGERED MODEL UPDATE

The proposed update rule replaces periodic adaptation with event-triggered measurements and model updates based on whether current uncertainty threatens the Lyapunov decrease condition.

  • V. EVENT-TRIGGERED MODEL UPDATE: Event-triggered updates add training data only when needed, reducing model updates and measurements used for online GP learning.The switching time is selected asynchronously rather than after a fixed interval.
  • V. EVENT-TRIGGERED MODEL UPDATE: Time-triggered updates cannot select a universally suitable fixed interval because required model precision varies across the state space.The necessary precision also changes as the tracking error becomes smaller.
  • V. EVENT-TRIGGERED MODEL UPDATE: Periodic updates can accumulate infinitely many measurements and make GP computation incompatible with real-time operation as the dataset grows.The model-update cost increases as O(N^3), or O(N^2) in the stated best case.
  • V. EVENT-TRIGGERED MODEL UPDATE: Time-triggered design exposes a precision–complexity trade-off: more data can reduce GP variance but can also undermine real-time capability.The paper motivates event-triggering as a way to limit unnecessary training points while retaining model precision.
  • V. EVENT-TRIGGERED MODEL UPDATE: The event is intended to occur before the Lyapunov derivative becomes non-negative, using an upper bound because the exact derivative cannot be computed.The derivation first considers noiseless highest-state-derivative measurements and then addresses measurement noise.

A. Asymptotic stability for noiseless measurements

Under noiseless measurements, uncertainty-triggered GP updates preserve asymptotic tracking stability while guaranteeing a positive minimum time between updates. The result relies on the GP variance as an error indicator and a Lyapunov argument.

  • A. Asymptotic stability for noiseless measurements: The trigger updates the model when βκσκ(x) reaches kc|r|, using GP variance to control the model-error contribution.The variance-based condition is supported by the GP error bound, while the event occurs at equality because variance and tracking-related terms are continuous.
  • A. Asymptotic stability for noiseless measurements: The tracking error is globally asymptotically stable with probability 1−δ under variance-triggered GP updates and a positive lower bound on inter-event times.The result holds under Assumptions 1–7 for any kc > 0.
  • A. Asymptotic stability for noiseless measurements: A common radially unbounded Lyapunov function shows convergence despite arbitrary switching between updated GP models.At each noiseless update, the GP mean is exact at the newest training point, and the trigger keeps the error term dominated between updates.
  • A. Asymptotic stability for noiseless measurements: With continuously available measurements, the same event-triggering design yields global asymptotic stability and a positive inter-event-time lower bound without the probabilistic qualifier.This idealized result requires continuous measurements and is stated mainly for completeness.

B. Ultimate boundedness for noisy measurements

With noisy measurements, event-triggered GP updates provide probabilistic ultimate boundedness rather than asymptotic convergence. The trigger avoids updates inside a noise-dependent error region and retains a positive inter-event-time lower bound.

  • B. Ultimate boundedness for noisy measurements: Compared with arbitrary GP updates, event-triggered updates shrink the ultimate bound to a size proportional to the measurement noise level.The result uses the event-triggering mechanism rather than merely allowing arbitrary update times.
  • B. Ultimate boundedness for noisy measurements: The noisy-measurement trigger is βκσκ(x) ≥ kc|r| together with e ∉ Bσon, so no events are generated inside the ultimate-bound set.This excludes Zeno behavior by restricting inter-event analysis to the region outside Bσon.
  • B. Ultimate boundedness for noisy measurements: Under noisy measurements, the tracking error is globally uniformly ultimately bounded to a noise-dependent set with probability 1−δ.The trigger is active only outside Bσon, where the Lyapunov derivative can be shown negative definite.
  • B. Ultimate boundedness for noisy measurements: The inter-event time remains lower bounded by a positive constant with probability 1−δ.Measurement noise makes the model error nonzero at update times, but the GP variance at training points can be bounded using the noise level.

C. Forgetting strategies

Safe forgetting strategies limit GP dataset growth while retaining the stability guarantees of event-triggered online learning. The approach can use a single recent point or a bounded reduced dataset, although retaining more points can reduce future updates.

  • C. Forgetting strategies: Event-triggered collection adds points only when necessary, while safe forgetting prevents computational growth when trajectories cover large regions or require high precision.The proposed forgetting rule stores only a single data point, and bounded-budget variants retain at most N̄ points.
  • C. Forgetting strategies: Storing only the newest data point also guarantees global asymptotic stability with probability 1−δ.This rule is maximally data-efficient but can trigger events more frequently than retaining multiple points.
  • C. Forgetting strategies: Keeping more than one point can avoid future measurements for periodic desired trajectories, whereas deleting all old points generally triggers events more frequently.The reduced-dataset search is combinatorial, and the retained set must include the most recent measurement.
  • C. Forgetting strategies: A bounded dataset containing at most N̄ points preserves global asymptotic stability with probability 1−δ for any kc > 0.The reduced dataset is selected at each event while retaining the newest measurement.
  • C. Forgetting strategies: Simulations illustrate tracking convergence and GP identification, with estimates becoming more precise near training data.The numerical example uses a modified pendulum system and compares actual and desired state trajectories.

A. Scenario 1: Time-triggered updates

Scenario 1 evaluates time-triggered model updates for feedback linearizing control under unknown dynamics and noisy measurements. The controller approaches desired states precisely, while continued periodic updates add unnecessary data after local accuracy is achieved.

  • A. Scenario 1: Time-triggered updates: Scenario 1 uses periodic updates every ∆t = 0.5, unknown f(x) and g(x), known positive g(x), noisy measurements, and N = 40 collected data points.The simulation runs for Tsim = 20 with hyperparameter optimization at each model update.
  • A. Scenario 1: Time-triggered updates: After 40 training points, the estimates of f(x) and g(x) are more precise near the collected training data.The identified kernel hyperparameters indicate that f(x) mainly depends on x1, while g(x) mainly depends on x2.
  • A. Scenario 1: Time-triggered updates: The controller approaches the desired stationary states (1, 0) and (0, 0) with high steady-state precision after collecting measurements nearby.A few measurements and subsequent model updates are required in each corresponding state-space region.
  • A. Scenario 1: Time-triggered updates: Time-triggered updates continue adding unnecessary data points once the model is already precise in the steady-state region.This motivates the event-triggered update mechanism illustrated in Scenario 2.

APPENDIX A EXPRESSING STRUCTURE IN KERNELS

Appendix A develops Gaussian-process kernel constructions that encode additive and multiplicative structure in unknown dynamics. These constructions allow separate function components to be inferred from observations of their sum and support the composite-kernel identification approach.

  • APPENDIX A EXPRESSING STRUCTURE IN KERNELS: The sum of two independent Gaussian-process functions is modeled by a Gaussian process with kernel ka(x, x′) + kb(x, x′).Conditioning on observations of the sum permits prediction of the individual functions.
  • APPENDIX A EXPRESSING STRUCTURE IN KERNELS: The composite-kernel formulation combines summation and product constructions to identify control-affine dynamics from closed-loop measurements.The appendix states that conditioning infers the component functions from the training data and test input.
  • APPENDIX A EXPRESSING STRUCTURE IN KERNELS: Multiplying an unknown function by a known function is represented with a Gaussian process using a scaled kernel.The construction preserves the product structure while accounting for noisy measurements.
  • APPENDIX A EXPRESSING STRUCTURE IN KERNELS: Using scaled observations as direct training data would undesirably scale observation noise, reduce numerical stability, and conflict with kernel summation.These constraints motivate scaling the kernel instead.

APPENDIX B IMPROVING IDENTIFICATION

Appendix B augments closed-loop data with safe open-loop measurements to improve identification. The authors expect practical identification gains but do not establish additional formal convergence guarantees for this extension.

  • APPENDIX B IMPROVING IDENTIFICATION: Open-loop operation with u = 0 for a finite interval records training points that measure f(x) directly while the bounded state remains safe.These observations can be merged with closed-loop training points in the Gaussian-process dataset.
  • APPENDIX B IMPROVING IDENTIFICATION: The extension is expected to significantly improve identification in practice, but it provides no additional formal guarantees regarding convergence.The authors therefore do not investigate it further.
Loading 1911.06565v1…