Source-linked AI summary
Neural Lyapunov Control
Ya-Chien Chang, Nima Roohi, Sicun Gao
TL;DR
Nonlinear control design needs stability guarantees beyond the small regions where linearization-based methods apply. The paper jointly learns control and neural Lyapunov functions and uses a falsifier with complete constraint solving to check Lyapunov conditions. Experiments show certified attraction regions much larger than those obtained by existing methods such as LQR and SOS/SDP.
Problem
The paper addresses the challenge of designing feedback controllers for highly nonlinear systems with provable stability, beyond the limited regions supported by linearization-based methods.
Method
A learner jointly optimizes a control function and neural Lyapunov function, while a falsifier searches for violating states and complete constraint solving certifies the Lyapunov conditions.
Results
Certified regions of attraction were much larger than those obtained by existing control methods across challenging nonlinear control problems.
Takeaways & Limitations
The framework simplifies nonlinear control design while providing an end-to-end provable correctness guarantee.
Takeaways & Limitations
The Lyapunov-function class uses smooth tanh feed-forward networks because analytical Lie derivatives are required, excluding non-smooth activations such as ReLU.
Abstract
from arXiv · showhide
We propose new methods for learning control policies and neural network Lyapunov functions for nonlinear control problems, with provable guarantee of stability. The framework consists of a learner that attempts to find the control and Lyapunov functions, and a falsifier that finds counterexamples to quickly guide the learner towards solutions. The procedure terminates when no counterexample is found by the falsifier, in which case the controlled nonlinear system is provably stable. The approach significantly simplifies the process of Lyapunov control design, provides end-to-end correctness guarantee, and can obtain much larger regions of attraction than existing methods such as LQR and SOS/SDP. We show experiments on how the new methods obtain high-quality solutions for challenging control problems.
1 Introduction
The paper asks whether learning can support classical nonlinear control design with provable stability guarantees. It combines neural representations, gradient-based learning, and complete constraint solving to produce certified controllers with larger verified attraction regions.
- The work targets feedback-controller design for nonlinear dynamical systems with provable stability guarantees, rather than relying only on learning through unknown-environment interactions.
- LQR provides stability only near equilibria where linearization is accurate, making it conservative for nonlinear systems outside those regions.
- The approach combines neural-network expressiveness, gradient descent, and complete nonlinear constraint solving to guarantee Lyapunov conditions.
- The framework jointly learns control and neural Lyapunov functions without local dynamics approximation, using a learner and a falsifier.
- Experiments on drone landing, vehicle path following, and planar robot balancing produced certified attraction regions significantly larger than previously established ones.
- Compared with related learning-based Lyapunov work, the approach jointly learns control and Lyapunov functions for continuous systems with generic feed-forward networks and provable guarantees.
2 Preliminaries
The preliminaries define controlled dynamical systems, stability, and Lie derivatives, then state the Lyapunov conditions that certify asymptotic stability. They also position LQR as a local method around a stationary point.
- A controlled dynamical system uses a Lipschitz-continuous vector field over an open state space and a continuous feedback control function.
- Stability requires trajectories starting sufficiently close to the origin to remain close, while asymptotic stability additionally requires convergence to the origin.
- The Lie derivative measures the rate of change of a scalar function along the system dynamics.
- A Lyapunov function is positive away from the origin, zero at the origin, and has a negative Lie derivative, which implies asymptotic stability.
- LQR is guaranteed within a small neighborhood around a stationary point where the dynamics can be approximated as linear.
3 Learning to Stabilize with Neural Lyapunov Functions
The framework jointly learns a control policy and neural Lyapunov function, then uses falsification to find violations and guide further learning. Complete constraint solving provides a stability certificate when no violating state remains.
- Candidate Lyapunov functions use multilayer tanh networks because analytical Lie derivatives are required for verifying Lyapunov conditions.
- The learner updates control and Lyapunov-network parameters by minimizing Lyapunov risk, which measures violations of positivity, negative Lie derivative, and zero value at the origin.
- The empirical Lyapunov risk is an unbiased estimator of the population Lyapunov risk, while true Lyapunov functions achieve global minimum risk equal to zero.
- The falsifier searches for state vectors violating Lyapunov conditions and adds discovered counterexamples to the learner's training set.
- Delta-complete solving guarantees that an unsatisfiable falsification formula has no solution, allowing the Lyapunov conditions to be certified over all state vectors.
- An ROA-tuning cost term adjusts learning toward larger regions of attraction, which are forward-invariant sets that can support safety guarantees.
4 Experiments
Experiments evaluate the learned Lyapunov approach on several nonlinear control problems, reporting certified regions of attraction larger than LQR and validating stability-related behavior.
- Experimental results: The region of attraction is enlarged by 300% to 600% compared to LQR across the reported examples.The final designs were re-verified with smaller ε, while the parameter choices do not affect the stability guarantees.
- Inverted pendulum: For the inverted pendulum, the learned Lyapunov function is valid within ∥x∥2 ≤6, compared with LQR diameters of 6 and 0.1 and SOS/SDP diameters of 1.75 and 1.2.The SOS Lyapunov function was the only SOS example to pass the complete constraint-solver check, enabling this comparison.
- Inverted pendulum: Figure 3 shows a negative Lie derivative over the valid inverted-pendulum region and trajectories near the learned region's boundary remaining bounded.Many trajectories initialized inside the SOS region escape, so that region fails the ROA properties.
- Caltech ducted fan: For the Caltech ducted fan, the learned design produces an ROA significantly larger than that obtained from LQR.The comparison concerns the aircraft's hover-mode nonlinear dynamics.
- Path following and robot balancing: The experiments also cover wheeled-vehicle path following and n-link planar robot balancing, including learned Lyapunov regions within ∥x∥2 ≤0.8 and ∥x∥2 ≤0.5, respectively.The robot experiments include a 2-link pendulum analysis and control on a 3-link model.
5 Conclusion
The paper presents methods for learning control policies and neural-network Lyapunov functions for highly nonlinear systems with provable stability guarantees. Experiments show simplified control design, end-to-end correctness guarantees, and larger regions of attraction than existing methods.
- Conclusion: The proposed methods learn control policies and neural-network Lyapunov functions for highly nonlinear systems with provable stability guarantees.The paper evaluates them on challenging nonlinear problems central to robot control.
- Conclusion: The approach simplifies nonlinear control design and provides an end-to-end provable correctness guarantee.The conclusion presents these as advantages over existing control methods.
- Conclusion: The learned designs can obtain much larger regions of attraction than existing control methods.The paper specifically highlights experiments on challenging nonlinear control problems.