Source-linked AI summary
The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systems
Spencer M. Richards, Felix Berkenkamp, Andreas Krause
TL;DR
Safety-critical learning requires certificates that prevent exploration from entering unrecoverable states. The paper learns a structured neural-network Lyapunov function and adapts its level set to the largest safe region for general nonlinear closed-loop systems. On a simulated inverted pendulum, the learned level set moulds to the safe region and its covered fraction approaches 1, while the construction preserves provable safety.
Problem
Learning policies for safety-critical systems typically lack safety guarantees, so exploration can cause physical harm or enter states from which the system cannot recover.
Method
The method constructs a neural-network Lyapunov candidate with inherent positive-definiteness and Lipschitz continuity, then trains it by classifying states as safe or unsafe without assuming a specific dynamics structure.
Results
The inverted-pendulum experiment shows the learned level set moulding to the safe region, with its covered fraction approaching 1 as the safe level converges toward cS = 1.
Takeaways & Limitations
The approach can provide provable safe subsets while expanding the states available for exploration and can complement statistical models of dynamical systems.
Takeaways & Limitations
Training does not guarantee convergence to Sπ, monotonic volume growth, or convergence of the safe level to cS, although certified level sets remain safe subsets.
Abstract
from arXiv · showhide
Learning algorithms have shown considerable prowess in simulation by allowing robots to adapt to uncertain environments and improve their performance. However, such algorithms are rarely used in practice on safety-critical systems, since the learned policy typically does not yield any safety guarantees. That is, the required exploration may cause physical harm to the robot or its environment. In this paper, we present a method to learn accurate safety certificates for nonlinear, closed-loop dynamical systems. Specifically, we construct a neural network Lyapunov function and a training algorithm that adapts it to the shape of the largest safe region in the state space. The algorithm relies only on knowledge of inputs and outputs of the dynamics, rather than on any specific model structure. We demonstrate our method by learning the safe region of attraction for a simulated inverted pendulum. Furthermore, we discuss how our method can be used in safe learning algorithms together with statistical models of dynamical systems.
1 Introduction
The paper addresses the challenge of enabling learning in safety-critical dynamical systems by constructing safety certificates that support safe exploration. It introduces a neural-network Lyapunov candidate and training algorithm designed for general nonlinear systems without relying on a specific dynamics structure.
- Safety-critical autonomous systems need learning for unknown environments while avoiding dangerous states from which recovery is impossible.
- A region of attraction provides a safe set where trajectories remain bounded, converge to a fixed point, and support data collection during learning.
- Unlike approaches relying on polynomial dynamics or specific dynamics structure, the method targets general nonlinear dynamical systems.
- The proposed method constructs a neural-network Lyapunov candidate that inherently provides a provable safety certificate and adapts to trajectory shapes through safe-state classification.
- The approach is evaluated against other methods on a simulated inverted pendulum and discussed as a way to improve safe learning.
2 Problem Statement and Background
The paper formulates safe-set estimation as finding the largest invariant region of attraction without overestimating unsafe states. It uses Lyapunov level sets for certification while addressing limitations from candidate-function shape and computational assumptions.
- The closed-loop safe set Sπ is the largest region of attraction whose trajectories remain inside the set and asymptotically approach the equilibrium xO = 0.
- A reliable estimate of Sπ is needed before deploying a policy, and safety requires never classifying unsafe states as safe.
- Safety Certification with Lyapunov Functions: Lyapunov certification uses a positive-definite function whose value decreases under the closed-loop dynamics, making suitable level sets invariant and convergent.
- Safety Certification with Lyapunov Functions: Finite-point certification can use the tightened condition ∆v(x) < −L∆vτ, with Lipschitz and coverage terms accounting for continuous-state verification.
- Computing SOS Lyapunov Functions: SOS methods can optimize polynomial Lyapunov functions efficiently but require polynomial dynamics and a difficult choice of monomial features.
- Computing SOS Lyapunov Functions: When the safe region is not ellipsoidal, shape mismatch between a Lyapunov level set and Sπ limits the certifiable safe region.
3 Learning Lyapunov Candidates
The paper constructs a structurally valid neural-network Lyapunov candidate and trains it as a safe-region classifier. An iterative algorithm expands certified level sets using forward simulations, while retaining provable safety even though convergence to the largest safe region is not guaranteed.
- Neural Network Lyapunov Function: The candidate vθ(x) = φθ(x)⊤φθ(x) learns expressive features through a feed-forward neural network instead of manually engineered monomials.This construction is motivated by the SOS form and can closely match the true ROA with a level set.
- Neural Network Lyapunov Function: Layer-structure restrictions ensure vθ is positive-definite and Lipschitz continuous, satisfying the Lyapunov requirements for safety certification.The construction requires weight matrices and activation functions with trivial nullspaces; layer widths cannot decrease input dimension.
- Safe-Set Classification: The safe-set objective is reformulated as classification: states inside Vθ(cS) receive the safe label only when the Lyapunov decrease condition also holds.The decision boundary is vθ(x) = cS, and the decrease constraint ensures the estimated set satisfies the safety conditions.
- Safe-Set Classification: The method uses a perceptron loss rather than a maximum-margin hinge loss because states may lie arbitrarily close to the continuous safe-set boundary.The loss penalizes misclassifications more strongly when they occur farther from the decision boundary.
- Iterative Training Algorithm: Algorithm 1 iteratively expands a certified level set by sampling states in the gap between Vθ(ck) and Vθ(αck), forward-simulating them, and updating θ with batch SGD.It initializes the largest currently certified level set, then repeats sampling, simulation, parameter updates, and safe-level recomputation.
- Verification and Guarantees: Safety certification remains conservative and valid during training, but continuous-state verification and estimated labels are required because the loss alone does not enforce decrease everywhere.The paper checks a tightened decrease condition on a discretization and notes alternatives including adaptive discretization and formal neural-network verification.
4 Experiments and Discussion
The method trains a neural-network Lyapunov candidate to adapt its safe level set to the inverted pendulum’s largest safe region. The learned certificate approaches the prescribed safe region and can support broader safe exploration than standard candidates.
- Training and certification: Algorithm 1 trains vθ so its level set matches the largest safe region Sπ of the closed-loop dynamics.The training encourages the Lyapunov decrease condition while adapting the candidate to trajectory-based safe and unsafe classifications.
- Benchmark: The inverted pendulum uses a saturated torque input, making states outside Sπ unrecoverable as trajectories diverge from the upright equilibrium.The benchmark fixes the controller to the LQR solution for the discretized, linearized, unconstrained dynamics.
- Training and certification: A stricter decrease condition is checked on discrete points covering the state space to certify continuous level sets after parameter updates.The procedure does not guarantee that the estimated safe-set volume grows monotonically at every iteration.
- Training and certification: States near the origin receive greater weight because the training objective emphasizes samples with small vθ(x).
- Results: With three 64-unit tanh layers, the learned largest safe level set moulds to Sπ, while its level converges toward cS = 1 and coverage approaches 1.The true region is estimated by forward simulation over a discretized state space, and volume by counting discrete states.
- Safe learning implications: The neural-network candidate certifies more of Sπ than common LQR and SOS Lyapunov candidates, increasing the states available for safe exploration.The paper connects this broader certificate to learning under a wider range of operating conditions.
5 Conclusion
The paper demonstrates a method for learning safety certificates for general nonlinear dynamical systems. Its parameterized Lyapunov candidates and training algorithm adapt to the largest safe region, with potential application to safe exploration in uncertain environments.
- The paper develops a flexible parameterized Lyapunov candidate and training algorithm that adapt safety certificates to a closed-loop system’s largest safe region.
- The method is intended to improve safe exploration for real autonomous systems operating in uncertain environments.
A Proofs
The proof establishes that the neural-network Lyapunov candidate is positive-definite and locally Lipschitz under layerwise structural assumptions. It derives these properties from the network’s trivial nullspace and Lipschitz activations.
- Trivial nullspace: If every layer activation and weight matrix has a trivial nullspace, the network output φθ has a trivial nullspace.The proof establishes this inductively across successive layer transformations.
- Positive definiteness: Because vθ(x) = φθ(x)⊤φθ(x), the trivial nullspace of φθ makes vθ positive-definite with vθ(0) = 0.The squared inner product is zero only when the network output, and therefore x, is zero.
- Lipschitz continuity: Lipschitz-continuous activations make each layer, the composed network, and the Lyapunov candidate locally Lipschitz continuous.The argument uses Lipschitz continuity under function composition and the quadratic form in φθ.
- Weight construction: The weight-matrix structure is designed to preserve a trivial nullspace while reducing the number of free parameters.The construction uses block matrices involving Gℓ1, Gℓ2, the identity matrix, and ε > 0.