PaperlayerView exact revision on arXiv

Source-linked AI summary

Cautious Model Predictive Control using Gaussian Process Regression

Lukas Hewing, Juraj Kabzan, Melanie N. Zeilinger

arXiv:1705.10702v4eess.SYmath.OC

TL;DR

Safe high-performance control requires models that capture nonlinear dynamics and residual uncertainty without making online computation infeasible. The paper combines nominal dynamics with an additive GP, approximates uncertainty propagation and chance constraints, and demonstrates improved performance and safety in simulation and autonomous-racing hardware. In the racing experiment, the GP-based controller improves average lap time by 0.71 s, or almost 7%, while maintaining safety.

  • Problem

    Accurate nonlinear models are difficult to identify, while control needs residual uncertainty to support safe and high-performance operation.

  • Method

    The approach combines a nominal system with additive GP dynamics, approximates state-distribution propagation, and formulates chance-constrained MPC using affine feedback and sparse approximations.

  • Results

    0.71 s, or almost 7%, average lap-time improvement was achieved by the GP-based controller in autonomous-racing experiments while improving performance and maintaining safety.

  • Takeaways & Limitations

    Cautious data-driven GP-MPC can improve both performance and safety for medium-sized systems with low sampling times.

Abstract

from arXiv · show

Gaussian process (GP) regression has been widely used in supervised machine learning due to its flexibility and inherent ability to describe uncertainty in function estimation. In the context of control, it is seeing increasing use for modeling of nonlinear dynamical systems from data, as it allows the direct assessment of residual model uncertainty. We present a model predictive control (MPC) approach that integrates a nominal system with an additive nonlinear part of the dynamics modeled as a GP. Approximation techniques for propagating the state distribution are reviewed and we describe a principled way of formulating the chance constrained MPC problem, which takes into account residual uncertainties provided by the GP model to enable cautious control. Using additional approximations for efficient computation, we finally demonstrate the approach in a simulation example, as well as in a hardware implementation for autonomous racing of remote controlled race cars, highlighting improvements with regard to both performance and safety over a nominal controller.

I. INTRODUCTION

The paper develops cautious GP-enhanced MPC by combining an approximate nominal model with data-learned nonlinear dynamics and residual uncertainty. It reviews uncertainty-propagation and constraint-handling techniques, then targets computationally feasible control and hardware validation.

  • Motivation: GP regression improves an approximate nominal system using available measurement data while retaining uncertainty estimates for cautious control.The nominal model can support operation, data collection, and pre-stabilizing ancillary control while the GP learns selected effects.
  • Motivation: The paper addresses the challenge of propagating state probability distributions when GP models represent system dynamics rather than disturbances.This propagation is required over the MPC prediction horizon.
  • Contributions: Sparse GPs and tailored inducing-point selection reduce computational burden for predictive control with millisecond-scale sampling times.The nominal model also permits learning in a reduced state-input subspace.
  • Contributions: The contributions include reviewing and extending approximation techniques for GP dynamics and uncertainty propagation with additive nominal dynamics.The resulting approximations support chance constraints on predicted state distributions.
  • Problem formulation: The system formulation combines known nominal dynamics with additive unknown dynamics learned from data and assumes process noise is independent and Gaussian.The learned dynamics lie in the subspace spanned by Bd, while state and input constraints are expressed as chance constraints.

C. Gaussian Process Regression

The GP models the unknown nonlinear deviation from nominal dynamics using state-input measurements, output-wise kernels, and predictive uncertainty. Standard GP evaluation can become costly as the dataset grows, motivating sparse approximations.

  • GP model: The GP infers the noisy vector-valued unknown dynamics from previously collected state-input pairs and deviations from the nominal model.The state and input form each GP input, while the output is the deviation from nominal dynamics.
  • GP model: Each output dimension is learned independently with its own kernel and prior mean function.The model assumes output components are conditionally independent given the input data.
  • GP model: The kernel choice and parameterization determine the inferred distribution of the unknown function.The paper uses a squared exponential kernel but allows other differentiable kernels.
  • Computational cost: GP predictive mean and variance scale with the number of data points, limiting large-data or fast real-time applications.The stated costs are O(ndnzM) for the mean and O(ndnzM^2) for the variance.

D. Sparse Gaussian Processes

The controller uses a GP approximation of additive unknown dynamics within stochastic MPC, propagating uncertainty and imposing chance constraints. Sparse GP techniques make predictive evaluation less dependent on the original dataset size.

  • Sparse approximation: FITC sparse GPs use inducing inputs to approximate the GP joint distribution and reduce predictive complexity.With M̃ inducing points, mean and variance costs become O(ndnzM̃) and O(ndnzM̃^2), respectively.
  • Sparse approximation: The paper proposes dynamically selected inducing points tailored to the predictive-control task.The selection uses transductive ideas to form a local approximation.
  • Stochastic MPC: The MPC prediction model combines nominal dynamics with GP-predicted additive dynamics and process noise.The GP produces a stochastic distribution at each step that is propagated forward through the horizon.
  • Stochastic MPC: Chance constraints impose prescribed maximum probabilities of state and input constraint violation.They are applied within a stochastic finite-time optimal-control formulation over input policies.
  • Efficient formulation: The exact optimization over general feedback policies is computationally intractable, motivating affine feedback and approximate mean-variance propagation.These approximations enable deterministic reformulations of chance constraints.

A. Ancillary Linear State Feedback Controller

The ancillary controller restricts policies to linear state feedback, allowing predicted inputs to remain stochastic and helping manage uncertainty. Closed-loop feedback reduces uncertainty growth relative to open-loop control, while gain selection remains difficult for nonlinear systems.

  • A. Ancillary Linear State Feedback Controller: Restricting general feedback policies to linear state feedback makes the MPC policy optimization tractable.The controller uses pre-selected gains and optimizes the mean applied input.
  • A. Ancillary Linear State Feedback Controller: Closed-loop state feedback limits uncertainty growth compared with open-loop input sequences in repeated double-integrator simulations.The figure compares trajectory means and 2-σ variances under the two control modes.
  • A. Ancillary Linear State Feedback Controller: Open-loop uncertainty growth can produce conservative control actions when chance constraints are enforced.The comparison uses a GP trained on a nonlinear friction term.
  • A. Ancillary Linear State Feedback Controller: Ancillary gain selection is generally difficult for highly nonlinear systems.The paper suggests linearization around an approximate prediction trajectory and finite-horizon LQR gains as a heuristic.
  • A. Ancillary Linear State Feedback Controller: State, input, and nonlinear disturbance are approximated as jointly Gaussian at each prediction step, with covariance terms retained for accurate uncertainty propagation.Linearization of nominal dynamics yields simple mean and variance updates.

2) Gaussian Process Prediction from Uncertain Inputs:

The paper reviews approximations for propagating Gaussian inputs through GP models, comparing their fidelity, conservatism, and computational cost. These methods become challenging in high-dimensional predictive control, motivating dimensionality and sparsity reductions.

  • Gaussian-input GP prediction methods include mean equivalent, Taylor, and exact moment-matching approximations.The methods differ in how they approximate the posterior mean, variance, and covariance.
  • Taylor approximation adds correction terms involving the posterior-mean gradient and input variance, with higher-order variants requiring more computation.Its one-step complexity is reported as O(ndn^2zM^2).
  • The posterior is generally non-Gaussian, so Gaussian approximations can incur prediction error even when the first two moments match exactly.Taylor approximation may locally fit the underlying distribution more closely than moment matching.
  • Mean Equivalent Approximation neglects covariance between the GP input and output, making it conservative and potentially overly confident depending on covariance sign.When combined with nominal dynamics, this neglected covariance can severely deteriorate prediction quality.
  • All approximations scale with input and output dimensions and data points, becoming expensive in high-dimensional spaces; restricting GP inputs to a subspace reduces the burden.Sparse GP approximations provide another computational reduction, often with little deterioration in prediction quality.

C. Dynamic Sparse GPs for MPC

For MPC with GP dynamics, the paper uses local sparse approximations and uncertainty propagation to make prediction and chance-constraint tightening computationally tractable. Inducing points follow an approximate trajectory, while probabilistic reachable sets tighten mean constraints.

  • C. Dynamic Sparse GPs for MPC: Dynamic sparse GP approximation selects inducing inputs locally along an approximate MPC trajectory to improve fidelity where predictions are evaluated.Previous solution trajectories provide practical test-point information for this transductive approximation.
  • C. Dynamic Sparse GPs for MPC: Full and sparse GP variances match closely along the predicted trajectory, while approximation quality deteriorates farther away.Because current and previous trajectories are similar, local information is sufficient for MPC computation in the illustrated example.
  • D. Chance Constraint Formulation: The Gaussian state and input distributions over the prediction horizon enable chance constraints to be approximated using probabilistic reachable sets.Constraint tightening is applied to the means using reachable sets for state and input errors.
  • D. Chance Constraint Formulation: Tightened mean constraints imply satisfaction of the original state and input constraints at the selected probability levels.The construction uses the Pontryagin set difference between the original constraint sets and probabilistic error sets.
  • D. Chance Constraint Formulation: Online computation and tightening can be computationally prohibitive for general convex constraint sets, so the paper focuses on cheaper tractable cases.State constraints are treated explicitly, with input constraints handled analogously.

1) Half-space constraints:

For half-space constraints, the paper projects Gaussian state-error uncertainty onto the constraint direction and uses a Gaussian quantile to construct a probabilistic reachable set and tighten the mean constraint.

  • 1) Half-space constraints:: Half-space constraints are handled by considering the marginal Gaussian error distribution in the direction of the constraint normal.This reduces the relevant uncertainty to the variance of the projected error.
  • 1) Half-space constraints:: The standard Gaussian quantile at probability level px defines an i-step probabilistic reachable set for the projected state error.The resulting set supports direct evaluation of the Pontryagin difference used for constraint tightening.
  • 1) Half-space constraints:: The same half-space procedure is extended to derive a tightened constraint directly on the state mean.The section then indicates an analogous derivation for the input constraint.
  • 1) Half-space constraints:: Polytopic constraints are represented as intersections of half-spaces, allowing the half-space treatment to serve as the basis for broader constraint sets.The passage introduces polytopic constraints through nj constituent half-spaces.

2) Polytopic Constraints:

The paper tightens polytopic chance constraints using probabilistic reachable sets, with alternative constructions based on polytope faces or marginal state distributions. Face-based tightening can become conservative as the number of faces grows.

  • Ellipsoidal probabilistic reachable sets yield a semidefinite-program formulation for tightening constraints, but solving it online is computationally demanding.
  • Polytope-face reachable sets bound violation probabilities for individual faces using Boole’s inequality.
  • Face-based tightening scales with the number of polytope faces and can be conservative when many similar faces are present.
  • Marginal-distribution reachable sets define a box-shaped uncertainty region whose scaling depends on the state dimension.
  • For a polytope and box, the Pontryagin difference tightens each inequality using the element-wise absolute value of the constraint matrix.
  • Constraining individual half-space violation probabilities can reduce the undesired conservatism of treating the entire polytope through a single bound.

E. Cost Function

The tractable formulation evaluates stochastic costs using approximate state and input distributions, primarily through expected-value objectives based on means and variances. The resulting GP-based MPC problem is non-convex and can be solved locally with nonlinear optimization methods.

  • Expected-value costs can be evaluated from approximate means and variances, including common stochastic objectives.
  • Quadratic costs use positive-semidefinite state weights Q and positive-definite input weights R.
  • Other supported objectives include saturating, risk-sensitive, and radial-basis-function-network costs.
  • Conditional value-at-risk and other stochastic cost measures can also be considered using the approximate state distributions.
  • The tractable MPC approximation combines the preceding approximations into an optimization problem with predicted means, covariances, costs, and constraints.
  • The GP-based MPC optimization is non-convex, but twice differentiability provides second-order derivatives for local SQP or nonlinear interior-point solutions.
  • The approach is demonstrated in simulation and hardware experiments, including an autonomous underwater vehicle and miniature autonomous race cars.

IV. ONLINE LEARNING FOR AUTONOMOUS UNDERWATER VEHICLE

The underwater-vehicle example uses an approximate linear model around a trim point and learns nonlinear friction effects online with a GP. Compared with linear MPC, GP-based MPC improves performance, particularly constraint satisfaction.

  • The AUV is simulated from nonlinear ground-truth dynamics while using an approximate linear model around horizontal trim for control.
  • Nonlinear friction effects are modeled as affecting only the continuous-time velocity states.
  • The linear model contains one integrating and two unstable modes, alongside one stable mode.
  • The controller tracks pitch references of 30° and 45° with a minimum-pitch safety constraint and ±20° rudder-deflection limits while learning online.
  • Online GP data is updated every 5 time steps while retaining 30 points, initialized with zero-input, zero-output data.
  • Figure 4 visualizes predicted pitch and rudder trajectories with mean predictions, 2-σ confidence regions, constraints, and the pitch reference.
  • The MPC uses horizon N = 35, a quadratic cost, and an ancillary infinite-horizon LQR controller to stabilize predictions and reduce uncertainty growth.
  • GP-based MPC improves performance over linear MPC, especially constraint satisfaction; linear MPC violates the minimum-pitch constraint during both reference changes.

V. AUTONOMOUS RACING

The autonomous-racing controller augments nonlinear bicycle-model dynamics with GP-modeled mismatch and uncertainty affecting velocity states. Constraint tightening based on predicted position uncertainty is approximated to meet a 20 ms sampling-time requirement.

  • The racing task seeks rapid track completion while avoiding collisions with track boundaries using model predictive contouring control.
  • The nominal vehicle model is a bicycle model with nonlinear Pacejka tire forces and states for position, orientation, velocities, and yaw rate.
  • The discretized model adds GP-modeled unmodeled dynamics and Gaussian white noise to account for mismatch and limited model fidelity.
  • Model uncertainty and process noise are assumed to affect only longitudinal velocity, lateral velocity, and yaw rate, not position or orientation kinematics.
  • Track safety is represented by input limits and a position constraint that keeps the car within a fixed-width track around a spline centerline.
  • Position uncertainty is propagated approximately into an ellipsoidal reachable set and then outer-approximated by a ball for track-constraint tightening.
  • Figure 6 illustrates how predicted position uncertainty adjusts the effective track radius during constraint tightening.
  • The controller tightens constraints only for the first 20 prediction steps, uses 10 inducing points, and applies further approximations for 20 ms real-time operation.

C. Results

The hardware racing results show that the GP-based controller improves lap performance and dynamics modeling while providing cautious control that maintains safety. Its computational cost remains compatible with small sampling times.

  • The nominal controller produced erratic driving and small collisions, whereas the GP-based controller made the racelines more consistent across 20 laps.The GP error model was trained using 325 data points collected during operation.
  • The GP prediction mean and residual uncertainty matched the encountered yaw-rate errors well during the first race lap.Figure 8 used a sparse GP with 10 inducing inputs and displayed a 2-σ confidence interval.
  • 0.71 s, or almost 7%, average lap time improvement was achieved by the GP-based controller over the nominal controller.The minimum lap time gains were less pronounced because the cautious controller prioritized avoiding collisions and large raceline problems.
  • Average solve times were similar for the nominal and GP-based controllers, while their percentages of solutions under 20 ms differed significantly.Frequent large re-planning actions occurred with the nominal controller.
  • The GP-based controller significantly improved performance while maintaining safety in hardware experiments on a complex system with small sampling times.The approach combined improved modeling with cautious control in autonomous racing.
Loading 1705.10702v4…