Source-linked AI summary
Safe Learning of Regions of Attraction for Uncertain, Nonlinear Systems with Gaussian Processes
Felix Berkenkamp, Riccardo Moriconi, Angela P. Schoellig, Andreas Krause
TL;DR
Model-based ROA estimates can include unsafe states because real nonlinear dynamics differ from approximate models, which is problematic for safety-critical control. The paper uses GP uncertainty, Lyapunov stability, and safe active experimentation to learn the real ROA while remaining inside it with high probability. Simulations show conservative estimates that improve toward the true Lyapunov level set as data increase and discretization becomes finer.
Problem
Model errors can make estimated ROAs differ drastically from the real system’s ROA, potentially including states outside the safe region required for safety-critical operation.
Method
The method combines an approximate model and Lyapunov function with GP uncertainty bounds and safe active learning to explore only within the estimated safe region.
Results
The algorithm provides a conservative ROA estimate that improves with more dynamics data and finer discretization, converging to the true level set in the limit.
Takeaways & Limitations
The framework provides a theoretical foundation for learning nonlinear-system ROAs through experiments without risking instability or leaving the true ROA with high probability.
Takeaways & Limitations
The analysis assumes a method can drive the system to every selected state without leaving the ROA; practical use must restrict selection by safe reachability.
Abstract
from arXiv · showhide
Control theory can provide useful insights into the properties of controlled, dynamic systems. One important property of nonlinear systems is the region of attraction (ROA), a safe subset of the state space in which a given controller renders an equilibrium point asymptotically stable. The ROA is typically estimated based on a model of the system. However, since models are only an approximation of the real world, the resulting estimated safe region can contain states outside the ROA of the real system. This is not acceptable in safety-critical applications. In this paper, we consider an approach that learns the ROA from experiments on a real system, without ever leaving the true ROA and, thus, without risking safety-critical failures. Based on regularity assumptions on the model errors in terms of a Gaussian process prior, we use an underlying Lyapunov function in order to determine a region in which an equilibrium point is asymptotically stable with high probability. Moreover, we provide an algorithm to actively and safely explore the state space in order to expand the ROA estimate. We demonstrate the effectiveness of this method in simulation.
I. INTRODUCTION
The paper targets safe ROA estimation for nonlinear systems whose model-based estimates may include unsafe real-system states. It combines Lyapunov analysis, Gaussian-process learning, and safe exploration to expand conservative ROA estimates without leaving the true ROA.
- Motivation: ROAs identify states where a controller renders an equilibrium asymptotically stable, defining a domain in which the system can be operated safely.This matters for safety-critical controllers that must guarantee safety over a specified operating domain before deployment.
- Motivation: Model errors can make purely model-based ROA estimates drastically different from the real system’s ROA.Such estimates may include states outside the true safe region.
- Contribution: The method learns the real ROA from experiments while remaining in it with high probability, avoiding failures relevant to safety-critical domains.The controller’s equilibrium remains attractive for every state visited during learning.
- Related work: Prior uncertain-system ROA methods use fixed uncertainties, whereas this paper reduces model uncertainty through learning.GPs provide uncertainty information that can support high-probability safety guarantees.
- Contribution: The approach combines GP learning, safe Bayesian optimization, and Lyapunov-function ROA computation for uncertain systems.It actively selects states that improve knowledge of the uncertain dynamics and reduce model uncertainty over time.
II. PROBLEM STATEMENT
The paper formulates safe ROA learning for a nonlinear system with known prior dynamics and unknown dynamics under a fixed policy. It assumes regularity, local Lyapunov stability, measurable noisy dynamics, and sufficient reachability for safe experimentation.
- System model: The closed-loop system combines known dynamics fπ(x) with unknown dynamics gπ(x) under a given control policy π(x).The policy is designed for the prior model, while gπ captures unknown dynamics and model uncertainties.
- Problem statement: The goal is to learn gπ from real-system experiments and estimate the policy’s ROA without leaving that ROA.Without assumptions on the model, this objective is impossible.
- Assumptions: The unknown dynamics are assumed to have bounded RKHS norm under a continuously differentiable, bounded kernel.This bounds their complexity relative to the chosen kernel.
- Assumptions: A fixed, twice continuously differentiable Lyapunov function is given, with a sublevel set S0 where its derivative is negative.This establishes local asymptotic stability around the equilibrium under the selected policy and Lyapunov function.
- Assumptions: The method requires state and derivative measurements, or a discrete-time approximation using state measurements alone, with bounded zero-mean noise.The known prior dynamics are subtracted to obtain measurements of the unknown model.
III. GAUSSIAN PROCESSES (GPS)
Gaussian processes model the unknown dynamics and provide posterior uncertainty from noisy observations. Under the RKHS and noise assumptions, these posteriors support high-probability bounds used by the safe ROA-learning method.
- GP model: A GP approximates the unknown function gπ over the state domain using the prior kernel shared with the RKHS assumption.The paper uses a zero-mean GP because prior dynamics knowledge is represented by fπ(x).
- GP model: GP regression treats function values as jointly Gaussian random variables and maps each state x to an estimate of gπ(x).The kernel encodes covariance between function values and assumptions about the unknown function.
- Posterior inference: Conditioning on n noisy measurements produces a posterior GP with mean µn(x), covariance kn(x, x′), and variance σn(x).Measurements are formed from observed dynamics, the known prior model, and Gaussian noise.
- Posterior inference: The kernel matrix Kn contains pairwise covariances among sampled states, while the identity matrix In represents the corresponding matrix dimension.The posterior uses covariances between a new state and the measurement states.
- Confidence bounds: With bounded RKHS norm and bounded noise, βn yields simultaneous high-probability bounds on gπ(x) over all n and x.The confidence scaling depends on the information capacity γn, which is sublinear in n for many common kernels.
IV. LYAPUNOV STABILITY
The paper uses GP confidence intervals and Lyapunov level sets to compute a high-probability ROA estimate on a continuous state domain from finite evaluations. The estimate is guaranteed to remain within the true ROA while becoming less conservative as model uncertainty decreases.
- Probabilistic stability bounds: GP predictions make the Lyapunov derivative probabilistic, so confidence intervals provide high-probability bounds for stability analysis.The derivative is affine in the GP-modeled unknown dynamics and is therefore itself normally distributed.
- Probabilistic stability bounds: If the upper confidence bound on the Lyapunov derivative is negative throughout a level set, that set is asymptotically stable with high probability.The condition is checked on discretized states and extended to the continuous domain using Lipschitz continuity.
- Continuous-domain guarantees: Lipschitz continuity allows finite-grid confidence intervals to provide guarantees over the continuous state domain.The discretization accuracy τ trades off ROA-estimate conservativeness against the computational cost of evaluating confidence intervals.
- ROA computation: The ROA estimate is obtained by maximizing the Lyapunov level-set parameter subject to the discretized stability condition.Theorem 1 solves this optimization efficiently using binary search.
- Safety guarantee: Sn = S0 ∪ V(cn) remains contained within the true ROA for all n with probability at least (1 −δ).The initial safe set is deterministic, while the learned level set is safe with high probability.
- ROA computation: More measurements reduce model uncertainty and allow the ROA estimate to expand beyond the initial deterministic safe set.A finer discretization reduces conservativeness, whereas additional experiments reduce uncertainty in the model estimate.
V. ACTIVE LEARNING
The active-learning procedure expands the safe ROA estimate by measuring unknown dynamics at informative states inside the current safe set. It maintains high-probability safety while reducing uncertainty and can approach the maximum safe level set after finitely many experiments.
- Safe exploration: New measurements are selected only at safe states within the current ROA estimate to actively expand the learned region.The procedure assumes access to a control method that reaches desired states without leaving the ROA.
- Uncertainty-directed sampling: The next evaluation state maximizes GP uncertainty over the current safe set, targeting dynamics measurements most relevant to ROA expansion.Reducing uncertainty about the unknown dynamics causes the safe region to expand over time.
- Safety and convergence: Every state selected by the active criterion lies inside the true ROA with high probability.A variant can find the maximum safe level set to a specified accuracy after a finite number of experiments.
VI. EXTENSION TO MULTIPLE DIMENSIONS
The analysis extends to multidimensional systems by modeling vector-valued unknown dynamics as a single GP over an augmented state space. This extension changes the confidence-parameter growth rate because the output dimension enlarges the effective domain.
- Multidimensional GP model: Vector-valued dynamics are modeled as one GP over X × I, where integer elements of I index the output dimension.This transfers the preceding one-dimensional analysis to multiple dimensions with more cumbersome notation.
- Confidence scaling: The multidimensional confidence parameter uses the combined function's information capacity over the augmented state space.The resulting βn′ grows at a faster rate because n′ = nq.
VII. DISCUSSION
The discussion identifies computational conservatism and reachability as practical boundaries, while noting that guarantees can be retained with more efficient sampling and state-only measurements.
- The discretization may be conservative, and computing GP predictions at all discretization points is computationally intensive.
- The analysis assumes a method can drive the system to every selected state without leaving the ROA, requiring further reachability restrictions in practice.The safely reachable set can be incorporated by selecting states from Sn ∩ R.
- Algorithm 1 can evaluate only states near the level-set boundary to improve data efficiency without losing guarantees.
- A discrete-time variant requires state measurements rather than derivative measurements, although nonlinear Lyapunov uncertainty propagation is not analytically tractable.
VIII. EXPERIMENTS
The simulated inverted-pendulum experiment tests learning with an inaccurate linear model and shows that Algorithm 1 conservatively approaches the true ROA as data and discretization improve.
- The experiment evaluates Algorithm 1 on a simulated inverted pendulum with bounded torque and recovery failure beyond |θ| > 30 deg.The state is x = (θ, ˙θ).
- After 100 data points, the prior model produces a safe set larger than the true ROA, whereas Algorithm 1 produces a conservative estimate.Algorithm 1 uses ˙V (x) ≤ −Lτ rather than ˙V (x) ≤ 0.
- The controller uses an LQR based on a linear approximation with neglected friction and a mass estimate 0.05 kg lighter than the real system.A corresponding quadratic Lyapunov function determines the ROA estimate.
- The GP models the dynamics error with a product of linear and Matérn kernels encoding high-probability regularity and growth properties.
- More data and smaller discretization values of τ improve the estimate, which converges to the true level set in the limit.
IX. CONCLUSION
The paper concludes that its Lyapunov- and GP-based algorithm learns the real ROA through safe experiments with high probability, while acknowledging restrictive practical assumptions.
- The algorithm learns the real ROA from experiments without leaving the true ROA with high probability.
- The results are presented as a theoretical foundation for learning algorithms that learn without risking instability, despite some restrictive assumptions.
A. Full Exploration Proof
The full-exploration proof defines an ideal ε-accurate safe-set expansion, then shows that a GP-confidence-based variant reaches it after finitely many experiments with high probability.
- The Lipschitz-based variant uses GP confidence intervals instead of perfect model knowledge and supports exploration guarantees.
- The ideal baseline expands a safe set using the Lipschitz constant and an error tolerance ε to identify states satisfying ˙V (x) < −Lτ.
- Iterating the baseline operator reaches the maximum ROA that can be determined with knowledge accurate to ε.
- The algorithm maintains intersected GP confidence intervals, initialized to certify S0 as safe, with ˙V (x) contained in them with probability at least (1 −δ).
- With probability at least (1 −δ), Sn remains contained in the ROA and satisfies Rl(Rε, ˙V (S0)) ⊆ Sn∗ ⊆ Rl(R0, ˙V (S0)).The result holds after the finite iteration count n∗ specified by the theorem.
- After n∗ evaluations, exploration reaches at least the baseline up to accuracy ε but no more than the perfectly informed safe set.