Source-linked AI summary
Learning Control Barrier Functions from Expert Demonstrations
Alexander Robey, Haimin Hu, Lars Lindemann, Hanwen Zhang, Dimos V. Dimarogonas, Stephen Tu, Nikolai Matni
TL;DR
The paper addresses how to synthesize control barrier functions for known nonlinear systems from expert safe demonstrations, a problem prior analytic and sum-of-squares approaches address only with limited scope and scalability. It develops an optimization-based learning approach with provable safety guarantees under smoothness and sampling conditions, and evaluates learned CBFs on planar and realistic systems.
Problem
Prior work had not fully addressed synthesizing CBFs for general classes of systems, while analytic and SOS approaches were limited in scope and scalability.
Method
The paper learns CBFs from expert demonstrations by enforcing value and derivative constraints over sampled regions, using convex optimization for RKHS parameterizations and stochastic first-order methods for DNNs.
Results
The learned functions satisfy conditions yielding valid local CBFs and experimentally steer airplanes away from each other for all initial states.
Takeaways & Limitations
Under suitable smoothness assumptions and sufficiently fine sampling, the approach provides provable safety guarantees while supporting multiple CBF parameterizations.
Takeaways & Limitations
The approach depends on suitable smoothness assumptions, and exact computation of a DNN CBF's Lipschitz constant is NP-hard, with weight-norm products providing only a naive upper bound.
Abstract
from arXiv · showhide
Inspired by the success of imitation and inverse reinforcement learning in replicating expert behavior through optimal control, we propose a learning based approach to safe controller synthesis based on control barrier functions (CBFs). We consider the setting of a known nonlinear control affine dynamical system and assume that we have access to safe trajectories generated by an expert - a practical example of such a setting would be a kinematic model of a self-driving vehicle with safe trajectories (e.g., trajectories that avoid collisions with obstacles in the environment) generated by a human driver. We then propose and analyze an optimization-based approach to learning a CBF that enjoys provable safety guarantees under suitable Lipschitz smoothness assumptions on the underlying dynamical system. A strength of our approach is that it is agnostic to the parameterization used to represent the CBF, assuming only that the Lipschitz constant of such functions can be efficiently bounded. Furthermore, if the CBF parameterization is convex, then under mild assumptions, so is our learning process. We end with extensive numerical evaluations of our results on both planar and realistic examples, using both random feature and deep neural network parameterizations of the CBF. To the best of our knowledge, these are the first results that learn provably safe control barrier functions from data.
1 Introduction
The paper addresses safe controller synthesis for known nonlinear systems using safe expert demonstrations, targeting a gap in scalable CBF synthesis. It formulates an optimization-based method with provable safety guarantees across flexible function classes.
- Motivation: Known nonlinear systems with available safe demonstrations motivate learning control barrier functions for vehicles, UAVs, and manipulators.These scenarios share modeled nonlinear dynamics and accessible examples of safe behavior.
- Research gap: Prior analytic and sum-of-squares methods for synthesizing CBFs are limited in scope and scalability for general systems.
- Approach: The proposed method learns a zeroing CBF from expert trajectories for known control-affine nonlinear systems through optimization.The paper provides verifiable conditions involving trajectories, auxiliary data, and optimization hyperparameters.
- Optimization: The learning problem can be solved with convex optimization for RKHS function spaces or first-order stochastic methods for DNNs.The approach is presented as applicable across these function classes while retaining safety guarantees.
- Contribution: Unlike related imitation-learning work that empirically replicates CBF controllers, this approach targets provable correctness guarantees.
2 Preliminaries and problem formulation
The paper defines local CBF-based safety for nonlinear control systems and constructs a certification domain from expert state-action data. Safety follows when the learned function satisfies derivative and sampling conditions.
- 2.1 Valid control barrier functions: The system has state x(t), input u(t), locally Lipschitz dynamics, and a unique trajectory under a locally Lipschitz control law.
- 2.1 Valid control barrier functions: The trajectory’s maximum definition interval I may be bounded because forward completeness is not explicitly assumed.
- 2.1 Valid control barrier functions: A valid CBF h defines a set C intended to satisfy safety specifications and be made forward invariant through an admissible control action.The validity condition is imposed on an open domain D containing C and uses an extended class K function.
- 2.1 Valid control barrier functions: Under a valid CBF and locally Lipschitz compatible controller, initial states in C remain in C; compact C is also forward invariant and asymptotically stable.
- 2.1 Valid control barrier functions: Finding a CBF remains difficult because it must capture enough safe states while satisfying derivative constraints involving system dynamics, the function, and its gradient.
- 2.2 Problem formulation: The geometric safe set S is distinguished from the CBF-certified set C, with the construction requiring C ⊂D ⊆S.
- 2.2 Problem formulation: Expert data consist of state-action pairs demonstrating safe behavior, which need not come from sequentially sampled trajectories.
- 2.2 Problem formulation: D is built from ϵ-neighborhoods of expert states, must remain within S, and forms an ϵ-net supporting derivative-constraint enforcement.
3 An optimization based approach
The method learns a local CBF by constraining its values and derivative behavior on expert-safe and auxiliary samples. It supports RKHS and DNN parameterizations, with geometric sets organizing the certification domain.
- Optimization formulation: The optimization operates over twice continuously differentiable functions whose local Lipschitz bounds can be efficiently estimated.
- Geometric construction: Figure 1 organizes the construction as S containing expert trajectories, D as their ϵ-ball union, N as a surrounding layer, and C inside D.
- Function classes: RKHS and DNN function classes are highlighted as compatible parameterizations for the general learning framework.
- Data construction: Auxiliary samples XN are drawn from N to form an ϵ-net, and they require no associated expert inputs.They can be obtained through gridding or uniform sampling.
- Safety conditions: The learned CBF constraints and data-net conditions are designed to ensure validity on D before being used in an optimization problem.
3.1 Guaranteeing C ⊂D ⊆S
The learned CBF is constrained on safe and unsafe sample nets so its nonnegative set lies within the desired domain while containing a certified inner safe region. Lipschitz bounds and buffer choices make these guarantees verifiable, but overly demanding margins can make the optimization infeasible or shrink the certified set.
- The method requires h(x) to be nonnegative on safe samples and negative on unsafe samples, with γsafe > 0 ensuring C has non-empty interior.The unsafe-sample constraint is used to establish C ⊂ D ⊆ S, while the safe-sample margin supports a nontrivial certified region.
- If XN is an ε̄-net of N with ε̄ < γunsafe/Lh(xi), satisfying the unsafe constraints guarantees h(x) < 0 throughout N.The result follows by combining the sample margin h(xi) ≤ −γunsafe with the local Lipschitz bound.
- The constraints can be incompatible because large value changes between nearby safe and unsafe samples may exceed the allowed Lipschitz variation.Choosing γsafe and γunsafe too large can make the optimization infeasible and reduce the volume of C.
- If Xsafe is an ε-net of D with ε ≤ γsafe/Lh(xi), satisfying the safe constraint guarantees h(x) ≥ 0 throughout the certified inner region D̄.Thus D̄ is contained in C, providing a minimum-volume guarantee for the learned safe set.
3.2 Guaranteeing valid local control barrier functions
The derivative condition for a valid local CBF is extended from sampled states to the full domain using Lipschitz continuity of the derivative expression. Together with safe and unsafe level-set constraints, this yields a sufficient theorem for safety-certified local CBFs.
- For fixed control input ui, q(x)=⟨∇h(x), f(x)+g(x)ui⟩+α(h(x)) is Lipschitz because ∇h, f, and g are Lipschitz.This regularity permits derivative constraints checked on sampled states to extend across the domain.
- If Xsafe is an ε-net of D with ε ≤ γdyn/Lq(xi), sampled constraints q(xi) ≥ γdyn imply q(x) ≥ 0 for every x ∈ D.The margin γdyn offsets variation in q between each state and its nearby sample.
- Theorem 3.4 guarantees that the learned set satisfies C non-empty, D̄ ⊆ C ⊂ D ⊆ S, and h is a valid local CBF on D.The theorem requires the unsafe, safe, and derivative constraints together with the corresponding sample-net conditions.
3.3 Control barrier filters
The paper extends local CBFs with a control-barrier filter set F whose trajectories must pass through C before leaving S, allowing safe states outside C to be certified without artificial unsafe samples.
- For each ζ0 ∈ F, every trajectory that exits S must first pass through C, so C filters trajectories starting from F.This extension certifies F as safe even though F lies outside the learned nonnegative set C.
3.4 Computing a Control Barrier Function
The paper learns a CBF by optimization, with convexity available for linear parameterizations and first-order methods used for general nonlinear classes. Lipschitz bounds are central to verifying the sufficient safety conditions.
- The optimization problem learns a CBF from expert trajectories subject to safe, unsafe, and derivative constraints.
- Convexity holds when α is linear, Lipschitz constraints are verified by bootstrapping, and h is linearly parameterized over a convex parameter set.This includes possibly nonlinear feature transformations and can be solved with standard solvers.
- For DNNs or nonlinear α, the optimization is non-convex and can be relaxed into an unconstrained objective for gradient-based optimization.The relaxation uses weighted safe, unsafe, and derivative terms and can be optimized with Adam or SGD.
- Twice continuously differentiable functions on the compact working domain have uniformly Lipschitz h and ∇h, enabling computable bounds for selected function classes.
- Random Fourier features admit analytical Lipschitz bounds, whereas DNN bounds are harder to estimate and naive products of weight norms can be quite loose.Semidefinite-programming approaches can provide more accurate upper bounds for DNN h, while accurate gradient-Lipschitz bounds remain limited.
3.5 Data Collection
The safety guarantees require sufficiently fine sampling of relevant state and state-input regions. Expert demonstrations help focus sampling on desirable behavior, reducing the practical burden of covering the domain.
- The derivative constraint is checked using expert state-input pairs, with more transverse dynamics relative to CBF level sets supporting verification.
- An ϵ-net can be constructed by uniform random sampling or gridding when the relevant set is efficiently sampleable.
- The required sample count has undesirable exponential dependence on set diameter, but expert demonstrations can reduce diameters by restricting attention to desirable behavior.
- Figure 2 compares expert trajectories, dynamic samples, unsafe samples, and the learned CBF relative to the safe-set boundary.
4 Numerical Experiments
Experiments evaluate learned CBFs on a planar system and aircraft collision avoidance using random Fourier features and neural networks. The learned controllers reproduce safe behavior and avoid collisions in tested settings.
- 4.1 Planar Example: The planar experiment uses a feedback-linearizable system with safe set S = {x : x1 ≤1, x2 ≤1} and expert data generated by a CBF-QP controller.
- 4.1 Planar Example: The learned planar CBF has an annular safe set C with approximate radii .2333 and .4, while its validity set D has approximate radii .1333 and .4666.
- 4.1 Planar Example: Trajectories inside C replicate expert behavior, while tested trajectories outside C converge to the learned safe set.
- 4.2 Aircraft Collision Avoidance: Aircraft demonstrations come from both a CBF-MPC expert and a human-controlled simulator, with safe and unsafe samples added for training.
- 4.2 Aircraft Collision Avoidance: The neural-network planarization uses a two-hidden-layer tanh network trained with Adam on the relaxed loss and grid-searched hyperparameters.
- 4.2 Aircraft Collision Avoidance: Learned CBFs from both datasets steer the aircraft apart for all tested initial states, while the analytical CBF produces more aggressive separation at closer distances.
5 Conclusion
The paper establishes an optimization-based approach for learning CBFs from demonstrations in known nonlinear control-affine systems, with safety guaranteed under smoothness and sufficiently fine sampling. Numerical evaluations include CBFs learned from CBF-MPC and human demonstrations, while future work targets reducing the sample-complexity burden.
- The approach learns CBFs from expert demonstrations for known nonlinear control-affine dynamical systems.
- The evaluations include trajectories and learned CBFs generated from CBF-MPC and human demonstrations.
- Under suitable smoothness assumptions and sufficiently fine sampling, the learned CBF is provably valid and guarantees safety.
- Future work aims to reduce the method's sample-complexity burden by targeting safety for typical behaviors rather than uniform state-space coverage.