PaperlayerView exact revision on arXiv

Source-linked AI summary

Safe Control with Learned Certificates: A Survey of Neural Lyapunov, Barrier, and Contraction methods

Charles Dawson, Sicun Gao, Chuchu Fan

arXiv:2202.11762v2cs.ROeess.SY

TL;DR

Learned controllers can lack transparent safety and stability guarantees, motivating methods that learn certificates alongside policies. This survey synthesizes neural Lyapunov, barrier, and contraction approaches, reviews practical and theoretical challenges, and presents case studies showing their application to nonlinear control. Its scope is promising but includes author-identified drawbacks and open research directions.

  • Problem

    Learned control systems can have reduced transparency and lack guarantees for safety or stability, while existing certificate-synthesis methods may be intractable, restricted to polynomial systems, or poorly scalable.

  • Method

    The paper surveys neural certificate methods that use neural networks to synthesize Lyapunov functions, barrier functions, and contraction metrics alongside safe controllers.

  • Results

    The survey reviews certificate-learning theory, synthesis methods, implementation challenges, two case studies, and future directions for nonlinear control.

  • Takeaways & Limitations

    Neural certificates provide verifiable correctness certificates alongside learned control policies and can support applications to complex nonlinear robotic systems.

  • Takeaways & Limitations

    The authors identify certificate-based learning for control as promising but having a number of drawbacks and open research directions.

Abstract

from arXiv · show

Learning-enabled control systems have demonstrated impressive empirical performance on challenging control problems in robotics, but this performance comes at the cost of reduced transparency and lack of guarantees on the safety or stability of the learned controllers. In recent years, new techniques have emerged to provide these guarantees by learning certificates alongside control policies -- these certificates provide concise, data-driven proofs that guarantee the safety and stability of the learned control system. These methods not only allow the user to verify the safety of a learned controller but also provide supervision during training, allowing safety and stability requirements to influence the training process itself. In this paper, we provide a comprehensive survey of this rapidly developing field of certificate learning. We hope that this paper will serve as an accessible introduction to the theory and practice of certificate learning, both to those who wish to apply these tools to practical robotics problems and to those who wish to dive more deeply into the theory of learning for control.

I. INTRODUCTION

Certificate functions provide proofs of desirable dynamical-system properties, while recent neural methods learn certificates alongside control policies to improve verification and training. This survey organizes theory, synthesis methods, implementation issues, case studies, and open problems across Lyapunov, barrier, and contraction certificates.

  • Lyapunov functions, barrier functions, and contraction metrics certify stability, safety, and differential stability, respectively.These are the three common certificate types covered by the survey.
  • Traditional certificate synthesis often required hand-designed functional forms and parameters, while some numerical approaches were computationally intractable.Examples include high-dimensional PDE methods and manually tuned polynomial certificates.
  • SoS methods formulate certificate search as convex semidefinite optimization for polynomial systems but scale poorly in higher dimensions.Their applicability is limited by both system class and computational scaling.
  • Certificate-based learning jointly searches for a control policy and a certificate proving the policy’s soundness, potentially using certificate-derived self-supervision.This differs from approaches that search only for a control policy.
  • The survey aims to provide an accessible, comprehensive treatment of neural certificate learning for practical robotics and deeper control-learning theory.It covers background, prior non-neural synthesis, neural synthesis, hardware implementation, case studies, and open problems.

B. Stability and Lyapunov certificates

Lyapunov functions certify stability by imposing positivity and decrease conditions on a scalar function. Control Lyapunov functions extend this idea to controller synthesis, often through quadratic programs that enforce stability or filter reference inputs.

  • A Lyapunov function certifies stability when it is positive away from the equilibrium and nonincreasing along closed-loop trajectories.Strict decrease yields asymptotic stability under the stated equilibrium conditions.
  • Sub-level sets of a Lyapunov function are forward invariant, and monotonic decrease toward a lower bound explains convergence toward the function’s minimum.The survey interprets the function as a generalized energy.
  • A Lyapunov function satisfying stronger decay bounds can establish exponential stability.The cited theorem connects bounds on V and its derivative to exponential convergence.
  • A control Lyapunov function certifies asymptotic stabilizability by requiring an admissible input that makes the Lyapunov decrease condition hold.The condition uses Lie derivatives along the drift and input dynamics.
  • The admissible CLF or ESCLF inputs form affine constraint sets, enabling quadratic programs that choose minimal control or remain closest to a reference policy.For convex input sets, the resulting controller is Lipschitz continuous and has a closed-form solution.

C. Safety and barrier certificates

Barrier certificates establish safety by proving forward invariance of a set containing the initial states and excluding unsafe states. Control barrier functions impose affine input constraints that can be combined with relaxed stability constraints in a single quadratic program.

  • Safety requires trajectories from the initial set to avoid the unsafe set for all future time.The certificate strategy is to prove forward invariance of a suitable safe set.
  • A barrier function defines a compact sublevel set whose forward invariance prevents trajectories from leaving the certified safe region.With initial states inside the set and unsafe states outside it, forward invariance implies safety.
  • The barrier condition makes h decrease on the set boundary, preventing exit and causing states that start outside the set to asymptotically converge toward it.A common choice is α(h) = γh for positive γ.
  • Control barrier functions parallel control Lyapunov functions and impose affine constraints in the control input, allowing quadratic-program implementation.Their constraints can be combined with a relaxed CLF condition.
  • In the combined CLF-CBF quadratic program, safety is enforced while stability may be temporarily relaxed, but the controller can deadlock when no safe direction approaches the goal.Switched controllers and unified Lyapunov-barrier certificates are cited as ways to alleviate this issue.

D. Contraction metric certificates

Contraction metrics certify that a closed-loop system can exponentially track any dynamically feasible reference trajectory, while bounded disturbances yield proportionally bounded tracking error. The metric also limits transient overshoot through its condition number.

  • D. Contraction metric certificates: Contraction theory extends stability analysis from fixed points to tracking general feasible reference trajectories.Fixed-point tracking is insufficient for general trajectories, motivating a richer contraction-based specification.
  • D. Contraction metric certificates: A contraction metric M(x) is a positive-definite matrix that measures distance between neighboring trajectories through virtual displacements.Contracting this metric means that displacement between neighboring trajectories shrinks exponentially over time.
  • D. Contraction metric certificates: The tracking controller has transient overshoot governed by the condition number of M, while worst-case steady-state tracking error is proportional to disturbance magnitude.The theorem identifies R as the square root of M's condition number and establishes disturbance-proportional tracking error.
  • D. Contraction metric certificates: A metric satisfying the contraction condition certifies exponential stabilization of every dynamically feasible nominal trajectory, rather than only one trajectory.Contraction is a property of the closed-loop system, not of a particular trajectory.

III. PRIOR WORK ON CERTIFICATE SYNTHESIS

Prior certificate-synthesis methods include numerical, polynomial-optimization, and simulation-guided techniques, but their scalability limitations—especially for nonlinear and high-dimensional dynamics—motivated neural approaches. Safe RL and reachability form related but distinct research lines.

  • III. PRIOR WORK ON CERTIFICATE SYNTHESIS: The survey organizes earlier certificate-synthesis work into numerical methods, polynomial optimization, and simulation-guided synthesis.These approaches form the historical background for later neural certificate techniques.
  • III. PRIOR WORK ON CERTIFICATE SYNTHESIS: Safe RL and Hamilton-Jacobi reachability are parallel approaches that address safety and stability but use different methods and vocabularies.The survey reviews their basic connections and differences without aiming for comprehensive coverage.
  • III. PRIOR WORK ON CERTIFICATE SYNTHESIS: Linear stable systems admit quadratic Lyapunov functions whose positive-definite matrix can be found by solving a continuous Lyapunov equation.This provides a numerically tractable special case for certificate synthesis.
  • III. PRIOR WORK ON CERTIFICATE SYNTHESIS: For polynomial dynamics, sum-of-squares constraints transform certificate synthesis into convex optimization using positive semidefinite matrices.The approach relies on the correspondence between fixed-degree sum-of-squares polynomials and positive semidefinite matrices.
  • III. PRIOR WORK ON CERTIFICATE SYNTHESIS: Simulation-guided synthesis samples the state space, simulates sampled points, and uses fixed basis functions to construct certificate candidates.A typical candidate has the form V(x) = z^T Pz, with sampled dynamics informing the certificate conditions.
  • III. PRIOR WORK ON CERTIFICATE SYNTHESIS: Poor scalability, particularly for nonlinear or high-dimensional dynamics, has limited widespread adoption of traditional synthesis methods.The survey notes that substantial practical scalability concerns remain even for sum-of-squares methods.

B. Parallel approaches: safe RL and HJ reachability

Certificate learning sits alongside safe RL and Hamilton-Jacobi reachability as an approach to guaranteed control. Neural methods optimize certificate conditions over sampled states, while certificates can also constrain RL policies or connect to reachability-based safety analysis.

  • B. Parallel approaches: safe RL and HJ reachability: Safe RL optimizes task performance while respecting behavioral constraints, and certificates can enforce safety or stability while RL pursues other objectives.Some safe RL methods use an a priori certificate, which requires a separate certificate-synthesis process.
  • B. Parallel approaches: safe RL and HJ reachability: Hamilton-Jacobi value functions and Lyapunov functions have related level-set interpretations for certifying forward invariance of safe regions.HJ methods seek optimally safe controllers and can therefore be conservative, although barrier-function connections have reduced that conservatism.
  • B. Parallel approaches: safe RL and HJ reachability: The generic certificate search requires a differentiable function satisfying certificate conditions throughout the state space.For a certificate V, the formulation seeks V such that each condition c_i(x,V) is nonpositive for every x in X.
  • B. Parallel approaches: safe RL and HJ reachability: Neural certificate synthesis searches over neural-network function spaces, whose depth and width are user-selected and can approximate continuous functions as size increases.This richer representation avoids restricting certificates to a manually chosen basis family.
  • B. Parallel approaches: safe RL and HJ reachability: Neural training replaces universal constraints with penalty terms evaluated on randomly sampled training points.The resulting empirical loss averages certificate-condition violations over the finite training set.
  • B. Parallel approaches: safe RL and HJ reachability: Zero empirical loss supplies statistical evidence rather than a guarantee that the learned certificate is valid.Neural-network verification strategies may supplement this evidence.

A. Remarks on certificate learning

Certificate learning can verify an existing controller or jointly learn certificates and policies in several settings. The framework covers differentiability requirements, certificate-quality objectives, behavior cloning, reward-based learning, and self-supervised policy synthesis.

  • A. Remarks on certificate learning: Continuous-time certificates should use continuously differentiable neural activations such as tanh, softplus, or ELU.This reflects the differentiability assumptions of much continuous-time certificate theory.
  • A. Remarks on certificate learning: Certificate search can optimize quality measures beyond feasibility, including the certified region of attraction for Lyapunov functions.Larger certified regions of attraction can be encouraged through additional objectives.
  • A. Remarks on certificate learning: Learning only a certificate is useful for verifying a known controller or deriving a controller from a CLF or CBF.An explicit control policy can instead be learned alongside the certificate when needed.
  • A. Remarks on certificate learning: Joint policy-and-certificate learning includes behavior cloning from expert controllers or observations of human actions.The resulting policy may compress an expensive controller or generate realistic pedestrian behavior predictions rather than directly control a system.
  • A. Remarks on certificate learning: Certificate-regularized reinforcement learning combines a reward with certificate information when no expert controller is available.One example uses the Lyapunov derivative alongside reward to update the policy.
  • A. Remarks on certificate learning: When no policy requirements exist beyond certification, the certificate loss can self-supervise both the certificate and control policy.This setting has been applied to Lyapunov, barrier, and contraction certificates, including partially known dynamics.
  • A. Remarks on certificate learning: The survey provides code examples covering learned Lyapunov, barrier, and contraction metric certificates.The implementation is linked through the neural-clbf repository.

C. Example: learning a Lyapunov function

The examples learn neural certificate functions for stabilization and station-keeping, training them with empirical losses that encode certificate conditions and controller feasibility. The learned CLF and CBF are then used with quadratic-program controllers in the inverted-pendulum and satellite tasks.

  • Learning a Lyapunov function: The inverted-pendulum CLF is a two-hidden-layer neural network trained on 10^4 uniformly sampled state-space points to stabilize x0 = [0, 0].The state is x = [θ, ˙θ], with torque input u = [τ].
  • Learning a Lyapunov function: The CLF loss includes terms approximating Lyapunov decrease conditions, while additional terms can facilitate learning.The first two terms are strictly sufficient for the stated conditions, whereas later terms provide useful training approximations.
  • Learned-controller behavior: The learned CLF-QP trajectories move toward the inverted-pendulum goal in decreasing V, while the CBF-QP trajectory respects station-keeping constraints.The CLF reaches V = 0 at the goal, and the CBF separates safe from unsafe sets.
  • Learning a control barrier function: The satellite CBF is learned from 10^5 labeled state-space samples using a two-hidden-layer network with 256 units per layer.The task enforces minimum and maximum distance constraints between chaser and target.
  • Learning a control barrier function: The CBF loss trains the zero-level set to separate safe and unsafe states and encourages feasibility of the corresponding CBF-QP.The learned barrier and a controlled chaser trajectory are shown in Figures 5 and 6.

E. Example: learning a contraction metric

The contraction-metric example learns a computationally inexpensive neural clone of an MPC trajectory-tracking controller together with a metric certifying its soundness. On a previously unseen reference, the learned controller tracks successfully while the metric value decreases exponentially.

  • Motivation and approach: The method jointly learns a neural control policy and contraction metric to replace an expensive onboard nonlinear MPC policy.The metric certifies the soundness of the computationally inexpensive cloned controller.
  • Training procedure: Training uses 100 random 10-second expert reference trajectories and 10^4 sampled tuples, with neural networks having two hidden layers of 32 neurons.The expert policy is generated by nonlinear MPC, and demonstrations receive a small amount of noise.
  • Training procedure: The metric is parameterized as M = A + A^T for symmetry, while the policy parameterization ensures u(x, x*, u*) = u* when x = x*.Both networks are optimized simultaneously using an empirical loss.
  • Results: The trained controller successfully tracks a previously unseen random reference trajectory, despite receiving only the current reference state and input.The original MPC policy instead receives a 1-second window of future reference states and inputs.
  • Results: The contraction metric value (x−x*)^T M(x−x*) decreases exponentially as the system converges to the reference trajectory.This behavior is reported in Figure 8 alongside the tracking example.

F. History of certificate learning

Certificate learning progressed from difficult or narrowly applicable synthesis methods toward neural approaches that jointly learn certificates and controllers. The survey also identifies hardware, measurement, disturbance, and model-uncertainty issues that constrain deployment and motivate further work.

  • Early development: Early neural Lyapunov proposals used neural networks as computational models or optimization representations, but lacked demonstrated learned certificates or scalable function approximation.Practical neural-certificate work re-emerged in robotics and control after the mid-2010s growth in neural-network interest and computation.
  • Fixed-controller certificate learning: Initial practical methods learned certificates for fixed controllers, including approaches that grew certified regions of attraction or used SMT solvers to generate counterexamples.Related work also applied this paradigm to barrier functions and contraction metrics.
  • Joint policy-certificate learning: Chang, Roohi, and Gao first proposed jointly optimizing a control policy and Lyapunov certificate, using alternating empirical optimization and SMT-based counterexample search.Their work reported improved performance in terms of a larger certified region of attraction.
  • Extensions: Subsequent work extended joint control-certificate learning to barrier functions, CLF-CBF combinations, contraction metrics, and partially known dynamics.These developments broadened simultaneous learning beyond Lyapunov certificates.
  • Deployment challenges: Hardware deployment is constrained by state-estimation error, control frequency and delay, disturbances, unmodeled dynamics, and actuator limits.These effects can degrade safety and stability unless mitigated.
  • Deployment challenges: Measurement-robust barrier functions have theoretical foundations but are more difficult to synthesize, and the survey reports no published work learning them automatically.The survey identifies learning measurement-robust barriers as an open direction.
  • Disturbance and uncertainty: For bounded additive disturbances, learned contraction metrics guarantee bounded worst-case tracking error proportional to disturbance magnitude.The survey distinguishes this case from more structured and unstructured model uncertainty.

D. Certificate adaptation

Certificate adaptation transfers certificates from nominal models toward true dynamics using learned model or Lie-derivative residues. Verification methods trade computational cost, guarantee strength, scalability, and training feedback.

  • Adaptation: Certificate adaptation asks whether nominal-model certificates can be transferred to true dynamics using limited real-system data.The motivation contrasts adapting certificates with robust methods that handle broad model uncertainty.
  • Adaptation: Neural networks or Gaussian-process regression can learn model or Lie-derivative residues that adjust certificate-based controllers for dynamics mismatch.The cited example adapts a simulated 14-state bipedal walker after changing link masses and inertias.
  • Verification frameworks: Generalization-error verification is computationally inexpensive and probabilistic, but its guarantees are relatively soft and often conservative.It samples states, checks certificate margins, and extrapolates validity with high probability.
  • Verification frameworks: Lipschitz verification provides deterministic continuous guarantees from gridded checks, but conservatism, dimensionality, and after-the-fact use limit it.The method requires constraint margins exceeding the Lipschitz variation between adjacent grid points.
  • Verification frameworks: Optimization-based learner-verifier architectures verify certificates during training and return counterexamples, while MILP- and SMT-based implementations remain computationally expensive.Their complexity typically grows exponentially with certificate-network neurons, limiting practical applicability to small networks.
  • Deployment: Nonverified certificates can still supervise policy training or support deployment with a real-time safety monitor checking certificate derivatives.These uses provide practical value without exhaustive verification.

F. Effects of actuator limits

Actuator limits can be incorporated directly into certificate-based quadratic-program controllers. This adds offline training cost but provides runtime assurance that learned policies respect admissible controls, while certificate controllers can also support faster online control than robust MPC.

  • Actuator-limit handling: Certificate-based quadratic-program controllers can incorporate actuator limits by restricting controls to the admissible set U.When U is a polytope, the constrained policy remains a quadratic program solvable efficiently online.
  • Actuator-limit handling: Training relaxes the constrained quadratic program and adds a loss term that minimizes the relaxation variable r.Differentiable convex programming enables back-propagation through the quadratic-program solution, with additional offline cost.
  • Runtime computation: At least an order of magnitude higher attainable control frequency than robust MPC is reported empirically for certificate-based controllers.Certificate synthesis shifts computation offline, while runtime evaluation uses local, single-step certificate properties instead of a long horizon.
  • Case studies: The survey illustrates certificate learning on nonlinear autonomous-car tracking and full-stack perception-feedback robotics problems.The accompanying code and pretrained models are presented as reference implementations for applying these techniques.

A. Learning a CLF for complex dynamics

The case studies show learned CLF, CBF, and hybrid certificate controllers applied to nonlinear vehicle tracking and perception-feedback navigation. They demonstrate robust tracking, safety-preserving exploration, and transfer to previously unseen environments, including hardware.

  • Learning a CLF for complex dynamics: The seven-state single-track car has nonlinear steering, friction, and load-transfer dynamics that challenge traditional synthesis methods.Sum-of-squares synthesis fails numerically even when these dynamics are polynomially approximated.
  • Learning a CLF for complex dynamics: A neural CLF parameterizes the optimization-based controller, is initialized from a linearized LQR Lyapunov function, and is then trained against CLF constraints.The learned function uses a squared neural-network output to encourage positive semidefiniteness.
  • Learning a CLF for complex dynamics: Robust CLF training targets reference-trajectory scenarios beyond those directly observed during training.The resulting neural CLF differs from the linearized Lyapunov function while supporting the tracking case study.
  • Learning a CLF for complex dynamics: A learned CLF successfully generalizes from circular training scenarios to more general reference paths during tracking.Figure 11 reports the tracking performance of the learned-CLF controller.
  • Safe visual-feedback control with barrier functions: For visual navigation, learned CBF and CLF networks use observations, correlating respectively with obstacle distance and goal distance.The CBF uses Lidar-related observations, while the CLF uses range and bearing to the goal.
  • Safe visual-feedback control with barrier functions: Because observation-space dynamics are not control-affine, the controller searches actions directly and switches between goal-seeking and exploratory modes when safety and progress conflict.Safety constraints remain enforced while Lyapunov conditions may be temporarily disabled to avoid local minima.
  • Safe visual-feedback control with barrier functions: Certificates trained in one random environment enabled safe navigation in previously unseen environments without certificate adaptation, including a hardware demonstration.The figure-based demonstration includes a newly introduced obstacle that the robot avoids.
  • Limitations and future work: The survey identifies drawbacks of certificate-based learning and pairs its practical case studies with discussion of future research directions.The supplied passages establish the existence of limitations but do not enumerate them here.

A. Limitations

The survey identifies unresolved theoretical and practical challenges in neural certificate learning, including data requirements, verification scalability, heterogeneous multi-agent systems, and distributed control. It also outlines directions for extending certificate learning to reinforcement learning and broader nonlinear robotics applications.

  • Data requirements & generalization: No firm theory yet predicts how much data neural certificate frameworks require, and existing consistency analysis assumes systems are known to be stable.Boffi et al.’s result concerns the shrinking invalid-volume region as data increases, but does not cover general nonlinear systems or jointly learned controllers.
  • Data requirements & generalization: Generalization bounds for learned certificates are often conservative, while the connection to almost-Lyapunov theory remains to be formalized.These approaches offer probabilistic guarantees, but their integration into certificate learning is incomplete.
  • Verification: Effective verification methods do not yet scale to networks involving more than 100 neurons, motivating probabilistic alternatives that remain underdeveloped.Generalization-error bounds and almost-Lyapunov theory could replace exhaustive verification, but neither has been fully integrated into the learning process.
  • Reinforcement learning: Connections between certificate learning and reinforcement learning remain insufficiently understood, including conditions under which a value function becomes a Lyapunov function.Several works learn certificate parameters through reinforcement learning, but the broader theoretical relationship is unresolved.
  • Heterogeneous multi-agent certificates: Existing multi-agent certificate-learning methods assume identical agent dynamics and constraints, leaving heterogeneous fleets as an open problem.Compositional verification is proposed as a possible route to combining certificates for subsets of agents with rigorous safety guarantees.
  • Distributed & network control: Distributed certificate learning must address communication delays, fault tolerance, scalability, and certificate construction for networked systems.The survey points to resilient control, decomposition methods, and graph neural networks as relevant directions for these challenges.
Loading 2202.11762v2…