PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Control Barrier Functions from Expert Demonstrations

Alexander Robey, Haimin Hu, Lars Lindemann, Hanwen Zhang, Dimos V. Dimarogonas, Stephen Tu, Nikolai Matni

arXiv:2004.03315v3eess.SYcs.LGmath.OC

TL;DR

Synthesizing CBFs for general systems remains an open challenge, and analytic and SOS approaches are limited in scope and scalability. This paper learns CBFs from expert demonstrations and establishes local safety under stated conditions.

  • Problem

    Synthesizing CBFs for general systems remains incompletely addressed, while existing analytic and SOS approaches are limited in scope and scalability for safety-critical settings.

    Source: §1 · pp. 1–2

  • Method

    The paper uses optimization to learn CBFs from expert demonstrations for known nonlinear control-affine systems, with Lipschitz-based conditions supporting provable safety guarantees.

    Source: §1.1 · p. 3, §2.2 · p. 5

  • Results

    Under the stated sampling and constraint conditions, the learned function is a valid local CBF, and its set C is nonempty and contained in S.

    Source: §3.2 · p. 8

  • Takeaways & Limitations

    The safety guarantee is local: the certified set C lies within D and the system's safe set S under the theorem's sampling conditions.

    Source: §3.2 · p. 8

    For DNN parameterizations, exactly computing the CBF's Lipschitz constant is NP-hard.

    Source: §3.4.2 · p. 11

Abstract

from arXiv · show

Inspired by the success of imitation and inverse reinforcement learning in replicating expert behavior through optimal control, we propose a learning based approach to safe controller synthesis based on control barrier functions (CBFs). We consider the setting of a known nonlinear control affine dynamical system and assume that we have access to safe trajectories generated by an expert - a practical example of such a setting would be a kinematic model of a self-driving vehicle with safe trajectories (e.g., trajectories that avoid collisions with obstacles in the environment) generated by a human driver. We then propose and analyze an optimization-based approach to learning a CBF that enjoys provable safety guarantees under suitable Lipschitz smoothness assumptions on the underlying dynamical system. A strength of our approach is that it is agnostic to the parameterization used to represent the CBF, assuming only that the Lipschitz constant of such functions can be efficiently bounded. Furthermore, if the CBF parameterization is convex, then under mild assumptions, so is our learning process. We end with extensive numerical evaluations of our results on both planar and realistic examples, using both random feature and deep neural network parameterizations of the CBF. To the best of our knowledge, these are the first results that learn provably safe control barrier functions from data.

1 Introduction

The paper proposes learning control barrier functions from safe expert demonstrations for known nonlinear systems, addressing the challenge of synthesizing CBFs for general system classes. It gives conditions intended to guarantee safe execution and describes optimization approaches for different function classes.

1 Introduction

  • Self-driving cars, UAVs, and robotic manipulators motivate the approach because their dynamics are modeled accurately, are nonlinear, and have available safe expert demonstrations.The paper proposes using CBFs learned from those demonstrations to design safe controllers.Source: §1 · p. 1
  • Zeroing CBFs can provide minimally invasive corrections to nominal control while maintaining safety through a quadratic program.They do not require CBF sub-level sets to be invariant within the safe set.Source: §1 · p. 2
  • Prior analytic and SOS synthesis approaches for CBFs and CLFs are limited in scope and scalability, leaving synthesis for general system classes an open problem.Analytic and SOS-based methods are summarized in and include.Source: §1 · p. 2

1.1 Related work on learning and CBFs

  • Closest prior learning approaches [18] empirically validate learned CBFs but provide no correctness proofs; [18] also lacks an a priori forward-invariance guarantee.Other related work includes imitation learning of a CBF-based controller without theoretical correctness guarantees [15], and joint learning followed by post-hoc verification [16].Source: §1.1 · p. 2
  • The paper learns zeroing CBFs from expert trajectories for known control-affine nonlinear systems, with verifiable data and hyperparameter conditions ensuring safety and RKHS/DNN optimization.The authors describe these as the first results to learn CBFs from expert demonstrations with provable safety guarantees.Source: §1.1 · p. 3

2 Preliminaries and problem formulation

The section defines valid control barrier functions and their safety guarantees, then formulates learning one from safe state-action data over a domain contained in the geometric safe set. It motivates this formulation by the difficulty of finding functions that satisfy the required derivative condition.

2.1 Valid control barrier functions

  • The system is control-affine with locally Lipschitz dynamics and a locally Lipschitz feedback; its solution’s maximum definition interval need not be unbounded.The state and input lie in R^n and R^m, respectively.Source: §2.1 · p. 3
  • A valid CBF is twice continuously differentiable and defines C as its nonnegative superlevel set, with a derivative condition holding on an open domain D containing C.The condition uses an extended class K function and respects the allowed input set U.Source: §2.1 · pp. 3–4
  • A valid CBF and locally Lipschitz control that satisfies its derivative condition keep trajectories starting in C within C; if C is compact, it is forward invariant and asymptotically stable.The guarantee applies on the solution’s maximum definition interval; compactness additionally ensures that interval is [0, ∞).Source: §2.1 · p. 4
  • Finding a CBF is challenging because a geometrically intuitive safe set does not ensure that the function’s derivative satisfies the required condition across an open domain.This challenge motivates learning a CBF from demonstrations when the system dynamics are known.Source: §2.1 · p. 4

2.2 Problem formulation

  • The formulation distinguishes the geometric safe set S from the CBF-certified set C and uses safe state-action pairs that need not come from sequentially sampled trajectories.The pairs provide control inputs for enforcing the derivative condition during learning.Source: §2.2 · pp. 4–5
  • The data-derived domain D is an ϵ-net covered by sampled states and must lie within S; the choice of ϵ or omission of samples can ensure this containment.Conditions on ϵ are later specified to ensure validity of the learned CBF.Source: §2.2 · p. 5
  • Enforcing negative CBF values on the width-σ layer N around D places the zero level set within D, enabling a local CBF on N ∪ D with C ⊂ D ⊆ S.If the learned CBF satisfies the derivative condition throughout D, both C and D can be made forward invariant by an admissible CBF-consistent input.Source: §2.2 · p. 5

3 An optimization based approach

Section 3 develops an optimization-based method for synthesizing valid local control barrier functions from expert demonstrations. It derives sufficient conditions on the learned function and sampled data, then uses them to formulate an efficiently solvable optimization problem.

3 An optimization based approach

  • Figure 1 depicts the learned safe set C as a subset of D, which lies within S, and shows that artificial unsafe samples are not introduced in the center of the safe set.D is the union of ϵ-balls around expert trajectories; N is defined around D.Source: Figure 1 · p. 6
  • The general method applies to function spaces H including random Fourier feature RKHSs and deep neural networks.The approach assumes local Lipschitz bounds for functions in H can be efficiently estimated.Source: §3 · pp. 5–6
  • Expert-generated safe samples Xsafe are supplemented by an ϵ-net XN from N, which needs no expert inputs and can be obtained by gridding or uniform sampling.Conditions on the net radius are specified later in the section.Source: §3 · p. 6
  • The method constrains the learned CBF and data sets so that h(x) is a valid local CBF on D, then formulates an efficiently solvable optimization problem.The function space H contains twice continuously differentiable functions with efficiently estimable local Lipschitz bounds.Source: §3 · pp. 5–6

3.1 Guaranteeing C ⊂D ⊆S

The section gives Lipschitz-based guarantees that the learned CBF separates safe and unsafe regions, while identifying and mitigating possible infeasibility from the sampling and margin constraints.

3.1 Guaranteeing C ⊂D ⊆S

  • Requiring h to meet the safe-sample condition ensures that C={x:h(x)≥0} has nonempty interior.Source: §3.1 · p. 6
  • If X_N is an ϵ̄-net of N with ϵ̄ < γ_unsafe/L_h(x_i), then h(x)<0 throughout N, yielding C ⊂ D ⊆ S under constraint (3.1).The guarantee follows when the unsafe samples satisfy h(x_i) ≤ −γ_unsafe.Source: §3.1 · pp. 6–7
  • Large margins can make the constraints infeasible by requiring rapid h variation across nearby safe and unsafe samples, potentially making C undesirably small; a buffer relaxes (3.1).The buffer lets h vary from γ_safe to −γ_unsafe without an excessively large Lipschitz constant.Source: §3.1 · p. 7
  • For an ϵ-net of D with ϵ ≤ γ_safe/L_h(x_i), constraint (3.3) ensures h(x) ≥ 0 on D̄, the union of ϵ-balls around inner safe samples.Thus D̄ ⊆ C provides a minimum-volume guarantee on C.Source: §3.1 · p. 7

3.2 Guaranteeing valid local control barrier functions

The section gives sufficient conditions for a learned CBF to satisfy the derivative constraint throughout the safe domain, then combines them with level-set conditions to guarantee local validity.

3.2 Guaranteeing valid local control barrier functions

  • Because h is twice continuously differentiable on compact N ∪ D, ∇h is locally Lipschitz, and verifying validity requires one control input satisfying (2.3), for which the method uses expert demonstration inputs.The consequences of choosing those expert inputs are discussed in Section 3.5.Source: §3.2 · p. 8
  • If q(x_i) ≥ γ_dyn on an ϵ-net of D with ϵ ≤ γ_dyn/L_q(x_i), Lipschitz continuity guarantees q(x) ≥ 0 throughout D.For fixed u_i, q(x) = ⟨∇h(x), f(x) + g(x)u_i⟩ + α(h(x)); this establishes the derivative constraint (2.3) across D.Source: §3.2 · p. 8
  • Under the stated net conditions, constraints (3.2), (3.3), and (3.5) make h a valid local CBF on D, with ¯D ⊆ C ⊂ D ⊆ S and C non-empty.The theorem assumes h is twice continuously differentiable and the sample sets satisfy the conditions of Proposition 3.1, Corollary 3.2, and Proposition 3.3.Source: §3.2 · p. 8

3.3 Control barrier filters

The section extends local control barrier functions with a set F whose trajectories must pass through C to escape S. This makes F safe without artificial unsafe samples in clearly safe regions, reducing controller conservatism.

3.3 Control barrier filters

  • A set F is safe when every trajectory starting in F must pass through C before it can leave S, eliminating artificial unsafe samples and reducing controller conservatism.This extends the notion of a local CBF and follows the spirit of set invariance.Source: §3.3 · p. 9

3.4 Computing a Control Barrier Function

The section formulates CBF learning as an optimization over expert-trajectory data, with safety and Lipschitz constraints. It discusses when the optimization is convex and how Lipschitz bounds can be estimated for random Fourier features and neural networks.

3.4 Computing a Control Barrier Function

  • The learning optimization constrains q(x_i,u_i) = ⟨∇h(x_i), f(x_i,u_i)⟩ + α(h(x_i)) to be at least γ_dyn and bounds Lip(q, ϵ) by L_q over Z_dyn.Hyperparameters are chosen under Theorem 3.4's conditions; if the Lipschitz bounds are difficult to enforce, the method iteratively solves, checks the learned function's bounds, and adjusts hyperparameters.Source: §3.4 · p. 9

3.4.1 Convexity

  • The optimization is convex for linear α and a convex linear-in-parameters CBF class when the Lipschitz bounds are verified by bootstrapping; otherwise, it may be non-convex.For the non-convex case, the section proposes an unconstrained relaxation solved in practice with first-order stochastic gradient methods such as Adam or SGD.Source: §3.4.1 · pp. 9–10

3.4.2 Lipschitz continuity of H

  • Random Fourier feature parameterizations permit analytic upper bounds on the Lipschitz constants of h and ∇h, with the stated bounds holding with probability at least 1 − δ.The bounds use the feature-map Jacobian and the Hessian, respectively.Source: §3.4.2 · pp. 10–11
  • For DNNs, exactly computing h's Lipschitz constant is NP-hard; the product-of-weight-matrix-norms bound is loose, and accurate general bounds for Lip(∇h) remain limited.The section notes an SDP-based upper bound for Lip(h) and post-hoc sampling as the only general method it knows for bounding Lip(∇h).Source: §3.4.2 · p. 11

3.5 Data Collection

Data collection must provide sufficiently fine state-space coverage and demonstrations that show how to move away from unsafe states, supporting the constraints needed for a valid CBF.

What should the experts do?

  • Demonstrations should align the closed-loop vector field with the inward normal of the learned CBF, while greater actuation authority makes this alignment easier.This increases the derivative-constraint inner product without increasing the CBF’s Lipschitz constant.Source: What should the experts do? · p. 11

Constructing ϵ-nets

  • An ϵ-net can be constructed by repeated uniform sampling when efficient sampling from the set is available, or alternatively by gridding.Uniform sampling works with high probability; examples of efficiently sampleable sets include basic primitive sets and sets with efficient membership oracles.Source: Constructing ϵ-nets · p. 11
  • Although ϵ-net sampling has undesirable exponential dependence, expert demonstrations focus sampling on desirable behavior and reduce the diameters of the sets to be sampled.The passage notes that this can significantly reduce the sampling burden.Source: Constructing ϵ-nets · p. 11

4 Numerical Experiments

The numerical experiments evaluate learned control barrier functions in a planar system and an aircraft collision-avoidance task. The learned functions steer aircraft apart across tested initial states, while planar trajectories show convergence to the learned safe set, including from outside its certified domain.

4.1 Planar Example

  • The planar experiment used a feedback-linearizable system with safe set S = {x : x1 ≤ 1, x2 ≤ 1} and expert trajectories generated by tracking references with a CBF-QP.Circular and radial references were sampled to construct safe and unsafe points and a dense set of system samples.Source: §4.1 · p. 12

4.2 Aircraft Collision Avoidance

  • The aircraft task required two airplanes to reach target states while maintaining a minimum separation of Ds = 0.5 to avoid collisions.The setup used a six-dimensional joint state and four controls, with bounded translational and angular velocities.Source: §4.2 · p. 13

Generating training data

  • Aircraft demonstrations came from MPC with the analytical CBF and from human guidance in a web-based simulator, with no nominal controller used for the human trajectories.The MPC expert was run from 400 randomly generated initial conditions inside the safe set; both safe and unsafe samples were also collected.Source: Generating training data · p. 13

4.1 Planar Example

  • In the planar example, trajectories converged to the learned safe annulus, including trajectories initialized outside the domain where the CBF is provably valid.The learned safe set had approximate inner and outer radii of 0.2333 and 0.4, respectively; the certified domain had approximate radii of 0.1333 and 0.4666. Trajectories within the safe set also replicated expert behavior.Source: §4.1 · p. 13, Figure 3 · p. 14

Training procedure

  • The aircraft CBF candidate used a two-hidden-layer fully connected tanh network with 64 neurons per layer, trained for 105 epochs using JAX and Adam with cosine-decay learning rate.Loss weights were λs = 2.0, λu = 2.0, and λd = 15.0; safety, unsafe, and dynamics margins were γsafe = 3.0, γunsafe = 0.5, and γdyn = 0.05. Each hyperparameter was chosen via grid search.Source: Training procedure · p. 15

Closed-loop control with learned CBF

  • In aircraft collision avoidance, CBFs learned from both datasets steered the airplanes apart from all tested initial states, experimentally validating forward invariance of the safe set.Compared with the analytical CBF, the learned CBFs separated the airplanes at a greater distance; the analytical CBF produced more aggressive actions.Source: Closed-loop control with learned CBF · p. 15

5 Conclusion

The paper proposes an optimization-based method for learning control barrier functions (CBFs) from expert demonstrations in known nonlinear control-affine systems, with provable safety under suitable smoothness and sampling assumptions. It also identifies reducing sample complexity by focusing on typical behaviors rather than uniform state-space coverage as future work.

5 Conclusion

  • Under suitable smoothness assumptions and sufficiently fine sampling, the learned CBF is provably valid and guarantees safety.The paper notes that smoothness can be guaranteed for RKHS and DNN parameterizations using classic and recent results.Source: §5 · pp. 15–16
  • Figure 5 shows CBF-MPC and human-demonstration trajectories with safe and unsafe samples; Figure 6 shows learned CBFs, including a comparison with the closed-form CBF [20].The CBF-MPC trajectories and samples appear on the left of Figure 5, while human-demonstration trajectories and learned CBFs appear on the right of Figures 5 and 6, respectively.Source: Figure 5 · p. 16, Figure 6 · p. 16
  • Future work will use statistical learning theory to reduce sample complexity by guaranteeing safety for typical behaviors rather than uniformly covering the state space.This is the proposed alternative to the method’s uniform state-space coverage burden.Source: §5 · p. 16

Related papers

Loading 2004.03315v3…