PaperlayerView exact revision on arXiv

Source-linked AI summary

Safe Learning of Quadrotor Dynamics Using Barrier Certificates

Li Wang, Evangelos A. Theodorou, Magnus Egerstedt

arXiv:1710.05472v1cs.LGeess.SY

TL;DR

Unknown quadrotor dynamics make safe learning difficult because uncontrolled learning can destabilize the system. The paper combines GP modeling with barrier certificates, adaptive exploration, and recursive inference; simulations demonstrate safe region expansion and real-time performance.

  • Problem

    Accurate nonlinear models are often unavailable for complex dynamical systems, while model inaccuracies and unknown disturbances complicate safety-critical control.

  • Method

    The paper uses Gaussian processes to model unknown quadrotor dynamics and barrier certificates to regulate exploration within a forward-invariant safe region.

  • Results

    The quadrotor never leaves the barrier-certified safe region while it expands from µ = 6.3 to µ = 0.6, and recursive GP inference remains below 20ms.

  • Takeaways & Limitations

    The proposed approach demonstrates safe learning of 3D nonlinear quadrotor dynamics with reduced computation and real-time recursive GP inference.

Abstract

from arXiv · show

To effectively control complex dynamical systems, accurate nonlinear models are typically needed. However, these models are not always known. In this paper, we present a data-driven approach based on Gaussian processes that learns models of quadrotors operating in partially unknown environments. What makes this challenging is that if the learning process is not carefully controlled, the system will go unstable, i.e., the quadcopter will crash. To this end, barrier certificates are employed for safe learning. The barrier certificates establish a non-conservative forward invariant safe region, in which high probability safety guarantees are provided based on the statistics of the Gaussian Process. A learning controller is designed to efficiently explore those uncertain states and expand the barrier certified safe region based on an adaptive sampling scheme. In addition, a recursive Gaussian Process prediction method is developed to learn the complex quadrotor dynamics in real-time. Simulation results are provided to demonstrate the effectiveness of the proposed approach.

I. INTRODUCTION

The paper develops safe learning for partially unknown dynamical systems using barrier certificates, Gaussian-process uncertainty, adaptive sampling, and recursive online learning. Barrier certificates define broader forward-invariant safe regions than Lyapunov sublevel sets, enabling richer exploration while preserving safety.

  • Safe learning strategy: Barrier certificates establish forward-invariant safe regions and provide high-probability safety guarantees using Gaussian-process prediction statistics.The certified region is expanded as uncertainty decreases with additional data.
  • Barrier certificates: Barrier-certified safe regions permit richer control options than Lyapunov sublevel sets because they represent general invariant sets.The paper uses this less restrictive safety notion to support exploration of uncertain states.
  • Computational strategy: The adaptive sampling algorithm reduces computation by sampling certain state-space regions sparsely while retaining safety guarantees.More uncertain or small-margin regions receive denser sampling.
  • Contributions: The paper proposes three contributions: barrier-certificate safe learning, adaptive sampling to reduce computation, and recursive GP learning of 3D quadrotor dynamics online.These components target safe exploration and real-time modeling of nonlinear quadrotor dynamics.
  • Barrier certificates: The safe region estimated with barrier certificates is much larger than the region estimated with the optimal polynomial Lyapunov function.The comparison is illustrated in Fig. 1 for system (3).

B. Gaussian Processes

Gaussian processes model unknown dynamics while providing predictive uncertainty for safety analysis. The paper combines these predictions with confidence intervals and recursive computation for online use.

  • GP modeling: Gaussian processes provide nonparametric regression models that capture complex unknown functions and support high-probability statements.Each state-space point is associated with a normally distributed random variable.
  • Assumptions: The unknown dynamics is assumed Lipschitz continuous so that learned behavior can be generalized to previously unexplored states.The method is restricted to system classes addressable with existing computation tools.
  • GP modeling: The unknown dynamics d(x) is modeled dimension by dimension with zero-mean Gaussian processes and kernel-based covariance functions.Measurements are obtained by comparing noisy system-dynamics observations with inaccurate model predictions.
  • GP inference: Posterior GP predictions at query states are computed from collected measurements using the kernel matrix and query-kernel vector.Conditioning on past measurements yields the predictive mean and variance.
  • Uncertainty quantification: The confidence parameter k_δ determines the stated confidence level, with k_δ = 2 giving 95.5% confidence and k_δ = 3 giving 99.7% confidence.The parameter is used to construct a high-confidence interval for the unmodeled dynamics.

III. SAFE LEARNING WITH BARRIER CERTIFICATES

The safe-learning strategy begins with a conservative barrier-certified region and expands it through data-driven exploration. Its objective is to reduce GP uncertainty while maximizing the certified region under barrier constraints.

  • Incremental safe learning: The controller starts from a conservative certified region C0 and incrementally expands it as data about the dynamics are collected.This process is illustrated as growth from C0 toward Cn in Fig. 2.
  • Safety constraint: The barrier-certificate constraint incorporates a high-confidence interval for the unknown dynamics when determining admissible safe behavior.The optimization is reformulated using the GP uncertainty set D(x).
  • Learning objective: The exploration objective is to collect data that reduce GP uncertainty σ(x) and increase the volume of the barrier-certified region C.More data can make additional states satisfy the barrier-certificate constraint.
  • Optimization: The certified-region maximization problem is non-convex and infinite dimensional, so the method uses adaptive state-space sampling and parameterized safe-region shapes.These simplifications make the optimization tractable in practice.

A. Adaptive Sampling of the State Space

Adaptive sampling replaces exhaustive barrier validation with a finite, margin-aware procedure. Lipschitz continuity allows sampled constraints to certify neighboring states without losing the safety guarantee.

  • Finite validation: Lipschitz continuity allows safety to be evaluated on a discretized state space rather than at every continuous state.A finite sampled set X_τ is used to validate the barrier constraints.
  • Probabilistic guarantee: The discretization-based barrier condition provides a safety guarantee with probability (1−δ) over the state space.The probability statement follows from the GP high-confidence interval and the sampled constraints.
  • Local certification: If the barrier condition holds at a sampled point, Proposition 3.2 extends it to all states within a prescribed neighborhood.The neighborhood radius is determined by k_ττ.
  • Adaptive sampling: Sparse sampling is used where the safety margin is large, while dense sampling is reserved for locations with small safety margins.This adaptive allocation reduces the number of required samples without losing safety guarantees.

B. Parameterization of the Barrier Certificates

The barrier certificate is parameterized to simplify non-convex safe-region optimization. Adaptive sampling and QP-based control support efficient, safety-constrained exploration and region expansion.

  • Parameterizing hµ(x) with a positive semidefinite matrix µ simplifies maximum-volume barrier certificate optimization.The example hµ(x) = 1−Z(x)TµZ(x) converts volume maximization into trace minimization.
  • Shape parameterization and adaptive sampling produce a tractable sampled barrier certificate maximization problem.
  • The most uncertain state inside the current safe region is selected for exploration to improve learning efficiency.
  • A QP-based controller modifies the nominal GoTo controller so the system reaches the target while satisfying safety barriers.
  • Exploration ends when the certified safe region no longer grows, and the resulting barrier certificates can regulate other control tasks.

C. Overview of the Safe Learning Algorithm

The algorithm begins with a conservative certified region, safely explores uncertain states, updates the GP model, and expands the region. Quadrotor differential-flatness control combines feedforward inversion with feedback compensation.

  • C. Overview of the Safe Learning Algorithm: A conservative initial safe region C0 seeds the iterative barrier-certificate learning process.
  • C. Overview of the Safe Learning Algorithm: At each iteration, the controller drives the quadrotor toward the most uncertain state while remaining inside the current certified region.
  • C. Overview of the Safe Learning Algorithm: After sampling the target state, the GP model is updated and the safe-region volume is expanded using barrier optimization and adaptive sampling.
  • C. Overview of the Safe Learning Algorithm: Online GP learning is used because the quadrotor is nonlinear, unstable, and affected by uncertain unmodeled dynamics.
  • A. Differential Flatness of 3D Quadrotor Dynamics: The quadrotor model uses world, body, and intermediate coordinate frames with roll, pitch, and yaw Euler angles.
  • A. Differential Flatness of 3D Quadrotor Dynamics: Differential flatness represents the full state and control algebraically from the flat output and its derivatives, enabling trajectory-based feedforward control.
  • A. Differential Flatness of 3D Quadrotor Dynamics: The applied control combines differential-flatness feedforward input with feedback input that compensates for model and tracking errors.

B. Learning based Control Using Gaussian Process

Gaussian processes learn the quadrotor dynamics omitted by the nominal model, including damping, drag, and wind effects, and the learned dynamics modify feedforward control.

  • The nominal quadrotor model omits uncertain damping, drag force, and wind disturbances, motivating GP learning of unmodeled dynamics.
  • Six Gaussian processes model the unmodeled dynamics along the dimensions of the quadrotor state space.
  • The GP inputs are the full quadrotor state q, while observations contain translational accelerations and angular rates.
  • At a query state, each GP provides a mean and variance for the unknown dynamics.
  • The learned GP dynamics are incorporated into a differential-flatness feedforward controller through corrected desired accelerations.

C. Recursive Online GP Learning

Standard GP inference scales cubically with the number of data points, limiting online use. The proposed recursive method updates a compact active kernel matrix by adding relevant and deleting uninformative points.

  • Standard GP inference has time complexity O(N^3), with most computation spent inverting the kernel matrix.
  • The recursive online GP method computes exact inference for online quadrotor learning.
  • As the quadrotor moves, relevant data points are added to the kernel matrix at each time step.
  • Data points contributing least to inference are deleted during recursive updates.
  • The method saves the previous kernel-matrix inverse to support recursive computation at the next time step.

1) Adding Multiple New Data to the Kernel Matrix:

The method recursively adds new data to the kernel matrix and deletes selected old data while updating its inverse. These operations reduce the required inversion to an M × M matrix and enable efficient online GP inference.

  • M denotes the number of new data added, together with their new observations and kernel vector.
  • The recursive update requires inversion only of an M × M matrix rather than the full N × N kernel matrix.
  • For deletion, selected data points are permuted to the bottom of the kernel matrix before forming the reduced inverse.
  • Block matrix inversion recovers the updated inverse, with the only required deletion inverse being C_i^-1 ∈ R^M×M.
  • Together, recursive addition and deletion allow GP inference to be obtained efficiently online.

V. SIMULATION RESULTS

Simulations evaluate online GP learning for quadrotor dynamics and safe barrier-certificate learning under model error, unknown wind, limited thrust, and exploration of unknown states. Recursive inference improves tracking while remaining below 20ms per iteration, and barrier-regulated exploration expands the certified safe region without leaving it.

  • Simulation setup: The simulation uses a quadrotor with a 1.4-times-underestimated weight, unknown 0.1g wind, and thrust limited to fz ∈[−1.8mg,0].These conditions create model mismatch, disturbance, and actuator constraints during learning.
  • Online Learning of Quadrotor Dynamics: The quadrotor is intentionally pushed into unexplored regions to evaluate the scalability of online learning with an inaccurate model.The first example uses a differential-flatness controller with a PD stabilizing wrapper.
  • Online Learning of Quadrotor Dynamics: Tracking error is significantly smaller with GP inference than without it, using a recursive GP model fixed at 300 data points.At each timestep, the least relevant point is removed and the most relevant point is added according to the kernel function.
  • Online Learning of Quadrotor Dynamics: GP inference time remains below 20ms per iteration with the recursive learning strategy.The paper reports this timing as suitable for online learning of quadrotor dynamics.
  • Online Learning of Quadrotor Dynamics: Learning with q′ = [˙rT,θ,φ,ψ]T yields much better scalability than learning with q = [rT , ˙rT,θ,φ,ψ]T when the quadrotor is pushed into unexplored regions.The authors suggest position may be less important than the other features in this simulation.
  • Learning Safety Barrier Certificates: The barrier parameter µ regulates admissible vertical speed, with smaller µ allowing more aggressive flight; learning therefore minimizes µ.The quadrotor flies back and forth within an ellipsoid while learning under inaccurate modeling and limited thrust.
  • Learning Safety Barrier Certificates: Adaptive sampling concentrates samples near safe-region boundaries and at downward speeds, while sampling the center and upward speeds less densely.The reported asymmetry may reflect the lack of reverse thrust to counter unmodeled dynamics.
  • Learning Safety Barrier Certificates: The certified region expands from C0 with µ = 6.3 to Cn with µ = 0.6, and the quadrotor never leaves the barrier-certified safe region.The nominal exploration controller is regulated by barrier certificates through the QP-based controller in (15).

VI. CONCLUSIONS

The paper develops safe GP-based learning for nonlinear quadrotor dynamics using barrier certificates, adaptive sampling, and recursive inference. Simulations show safe expansion of the certified region and reduced computation time.

  • VI. CONCLUSIONS: Barrier certificates regulate the learning controller so the system remains outside unsafe states while GP models approximate unmodeled dynamics.The approach derives high-probability safety guarantees from GP uncertainty statistics.
  • VI. CONCLUSIONS: The barrier-certified safe region gradually expands as additional data reduces uncertainty in the learned dynamics.The method is applied to a quadrotor with 3D nonlinear dynamics.
  • VI. CONCLUSIONS: Adaptive sampling and recursive GP inference significantly reduce the computation time of the safe learning method.The paper reports simulation results demonstrating the effectiveness of the proposed method.
  • VI. CONCLUSIONS: Adaptive sampling covers the unexplored region between the current and optimized next safe regions using circular confident safe regions around sampled points.The figure distinguishes current and next regions with solid green and dashed red ellipses, respectively.
Loading 1710.05472v1…