Source-linked AI summary
Bayesian Optimization with Safety Constraints: Safe and Automatic Parameter Tuning in Robotics
Felix Berkenkamp, Andreas Krause, Angela P. Schoellig
TL;DR
Robotic parameter tuning can require costly manual experimentation and may evaluate unsafe settings. This paper develops SAFEOPT-MC, which optimizes performance under multiple separate safety constraints using Gaussian-process-based safe exploration, with theoretical analysis and quadrotor experiments supporting efficient automatic optimization.
Problem
Robotic parameter tuning is time-consuming and potentially safety-critical because poorly modeled systems require careful manual choices or large amounts of representative data.
Method
SAFEOPT-MC extends SAFEOPT to optimize performance under multiple arbitrary safety constraints decoupled from the objective, using Gaussian-process-based safe exploration from an initial safe set.
Results
The algorithm enabled efficient and automatic parameter optimization without violating safety constraints in quadrotor control experiments.
Takeaways & Limitations
Context variables can support safe transfer of knowledge to new situations and tasks, including different quadrotor flying speeds.
Takeaways & Limitations
The algorithm is mostly applicable to low-dimensional problems because of computational and statistical difficulties in high dimensions.
Abstract
from arXiv · showhide
Robotic algorithms typically depend on various parameters, the choice of which significantly affects the robot's performance. While an initial guess for the parameters may be obtained from dynamic models of the robot, parameters are usually tuned manually on the real system to achieve the best performance. Optimization algorithms, such as Bayesian optimization, have been used to automate this process. However, these methods may evaluate unsafe parameters during the optimization process that lead to safety-critical system failures. Recently, a safe Bayesian optimization algorithm, called SafeOpt, has been developed, which guarantees that the performance of the system never falls below a critical value; that is, safety is defined based on the performance function. However, coupling performance and safety is often not desirable in robotics. For example, high-gain controllers might achieve low average tracking error (performance), but can overshoot and violate input constraints. In this paper, we present a generalized algorithm that allows for multiple safety constraints separate from the objective. Given an initial set of safe parameters, the algorithm maximizes performance but only evaluates parameters that satisfy safety for all constraints with high probability. To this end, it carefully explores the parameter space by exploiting regularity assumptions in terms of a Gaussian process prior. Moreover, we show how context variables can be used to safely transfer knowledge to new situations and tasks. We provide a theoretical analysis and demonstrate that the proposed algorithm enables fast, automatic, and safe optimization of tuning parameters in experiments on a quadrotor vehicle.
1 Introduction
Robotics parameter tuning is difficult because real-system experiments are costly and unsafe, while existing optimization methods may ignore safety. The paper extends safe Bayesian optimization to multiple constraints separate from performance and evaluates it on a quadrotor.
- Motivation: Manual tuning and massive data collection are time-consuming and potentially safety-critical for poorly modeled robotic algorithms.Engineers must either select safe parameters carefully or gather enough representative data to induce safe behavior.
- Related work: Many robotics optimization methods do not account for safety and may require difficult-to-obtain gradients or impractical numbers of experiments.Policy-search approaches may repeatedly evaluate similar parameters to estimate performance gradients.
- Background: Bayesian optimization improves sample efficiency by using regularity assumptions and Gaussian-process models to select informative evaluations.It can seek global optima with relatively few parameter evaluations, which is useful when experiments consume time and wear the robot.
- Contribution: SAFEOPT-MC generalizes SAFEOPT to multiple arbitrary safety constraints decoupled from the performance objective while retaining high-probability safety and theoretical guarantees.The practical algorithm relaxes proof assumptions while preserving safety guarantees, according to the paper.
- Evaluation: The method is experimentally evaluated by safely optimizing linear and nonlinear control laws on a quadrotor vehicle.The experiments also examine context-based optimization for environment-dependent control laws.
2 Problem Statement
The paper formulates robotics tuning as maximizing an unknown performance function subject to multiple unknown safety constraints. Safe optimization starts from known safe parameters and uses regularity assumptions to explore only parameters likely to remain safe.
- Problem formulation: The tuning parameters a belong to a domain A, and the objective is to maximize a scalar performance measure observed through finite-horizon robot experiments.The performance may represent quantities such as negative tracking error or walking speed.
- Safety constraints: Safety is represented by q unknown constraints gi(a) ≥0, whose values may depend on robot states, inputs, or environment variables.Known constraints could be excluded directly, but the paper addresses constraints estimated through noisy experiments.
- Safety constraints: Only parameters satisfying every safety constraint may be evaluated on the real system.The optimization therefore selects experiments inside the feasible region rather than evaluating arbitrary parameter settings.
- Safe exploration: Because the functions are unknown, the algorithm assumes a nonempty initial safe set S0 from which safe-region exploration begins.Additional regularity assumptions support Gaussian-process modeling, reliable confidence intervals, and safe inference for unevaluated parameters.
- Objective boundary: Safety requirements generally limit the attainable solution to the safely reachable portion of the feasible region rather than the global optimum.The target is the optimum within the safe region reachable from S0.
- Assumptions: Safety constraints must hold with high probability across the entire sequence of experiments, despite noisy performance and constraint measurements.The theoretical noise model assumes zero-mean sub-Gaussian noise and does not analyze correlated noise variables.
3 Background
The paper uses Gaussian processes and Bayesian optimization to model unknown performance and constraint functions, select informative experiments, and extend safe optimization across multiple outputs and contexts.
- Gaussian processes: Gaussian processes provide nonparametric models for unknown functions and produce posterior mean and variance predictions from noisy observations.These predictions quantify uncertainty at arbitrary parameter locations.
- Multiple outputs: The paper represents performance and multiple safety constraints with a surrogate function over the extended space A × I.The additional index i selects the performance function or an individual safety constraint.
- Multiple outputs: A kernel combining separate output kernels with cross-covariance models similarities between performance and constraint functions.The surrogate enables standard GP prediction and supports lifting SAFEOPT theory to multiple constraints.
- Bayesian optimization: Bayesian optimization uses the GP model to choose informative sample locations when real-system evaluations are expensive.Repeated evaluations improve function estimates and reduce uncertainty near candidate optima.
- Contextual optimization: Contextual Bayesian optimization models performance and constraints as functions of externally fixed variables such as battery level, weather, or task.A product kernel correlates function values when both parameters and contexts are similar.
- Contextual optimization: After observing a particular context sufficiently often, contextual Bayesian optimization queries parameters that are close to optimal for that context.The cited framework provides convergence guarantees similar to ordinary Bayesian optimization.
- Safe Bayesian optimization: SAFEOPT restricts optimization to parameters estimated safe with high probability while balancing exploitation of the safe set and expansion of that set.The paper extends this framework from a performance-defined safe set to multiple separate constraints.
4 SAFEOPT-MC (Multiple Constraints)
SAFEOPT-MC safely optimizes performance under multiple unknown constraints by expanding the safely reachable region while searching for strong parameters. Under regularity, noise, and initial-safety assumptions, it provides high-probability safety and near-optimality guarantees, including knowledge transfer across safe contexts.
- Algorithm: SAFEOPT-MC evaluates parameters only while expanding the known safe region and optimizing within the current feasible set.Its two goals are to enlarge the safe region without violating constraints and find optimal parameters within that region.
- Assumptions: The analysis assumes Lipschitz-continuous performance and constraint functions, noisy observations, finite parameter sets, and a nonempty initial safe set.The finite-set focus makes the safe-set analysis tractable, while the accuracy parameter ϵ reflects learning constraints under noisy measurements.
- Algorithm: The method restricts evaluations to potential maximizers or expanders rather than learning the entire parameter function uniformly.Potential maximizers could improve the performance estimate, while expanders could enlarge the safe set.
- Theoretical Results: With probability at least 1 −δ, confidence intervals support safe exploration and the algorithm reaches an ϵ-close estimate of the optimum over the safely reachable region after at most n∗ samples.The guarantee follows when uncertainty in the potential-maximizer and expander sets is sufficiently reduced or the safe set expands.
- Contexts: The contextual variant retains the noncontextual safety guarantees and can identify context-specific optima after sufficient information is acquired.Its sample bound is conservative because correlations across contexts can transfer knowledge and reduce evaluations in practice.
- Practical Limitations: Choosing an unsuitable Lipschitz constant can make the method overly conservative or allow unsafe parameters to be evaluated.This practical issue arises because smoothness is also encoded through the kernel choice.
5 Quadrotor Experiments
Quadrotor experiments show SAFEOPT-MC tuning controller parameters while adapting exploration to safety and informativeness. Across step, circle, and context-dependent tasks, it improves tracking while evaluating only parameters that satisfy the specified safety constraints.
- Experimental Setup: SAFEOPT-MC experiments used a Parrot AR.Drone 2.0, with computations taking significantly less than one second per iteration.The setup enabled continuous experiments without interruptions or human interventions.
- Experimental Setup: The quadrotor model uses position, velocity, attitude, and angular-velocity states, while control inputs specify desired roll, pitch, vertical velocity, and yaw rate.The experiments focus on horizontal position control while fixing the z-position and yaw control laws.
- Linear Control: The algorithm first samples near the initial controller, then expands the safe region coarsely, and finally refines promising high-performance parameters.This adaptive strategy concentrates samples near the initial safe region and potential maximizers while leaving other areas more coarsely sampled.
- Nonlinear Control: With the angular-velocity constraint, SAFEOPT-MC stops before violations and selects smaller time constants with higher damping to improve performance safely.The optimized parameters are near the top-left of the safe set, combining aggressive tracking with moderate angular velocity.
- Circle Trajectory and Context-Dependent Optimization: For circle tracking, optimized parameters substantially improve the trajectory while keeping all evaluations within RMSE and angular-velocity constraints.Using speed as a context transfers prior data to higher velocities, enabling safe optimization up to 1.8 m/s; beyond that, the performance constraint is violated.
6 Conclusion and Future Work
The paper concludes that SAFEOPT-MC enables efficient optimization without violating safety constraints and extends this framework to context-dependent control laws. It also identifies low-dimensional applicability as a current limitation.
- SAFEOPT-MC enabled efficient parameter optimization without violating safety constraints in quadrotor control.
- Speed context enabled knowledge transfer across speeds and produced speed-dependent optimal control parameters.
- At 1.8 m/s, actuator saturation limited performance, while safe optimization identified the maximum speed satisfying the deviation constraint.
- The algorithm is currently mostly applicable to low-dimensional problems because optimizing the acquisition objective and defining suitable high-dimensional GP priors are difficult.
7 Proofs
The proofs establish high-probability confidence bounds, characterize information gain and safe-set expansion, and derive safety and performance guarantees under GP regularity assumptions.
- Information gain: Because SAFEOPT-MC samples multiple functions at each iteration, its information capacity grows faster than in the single-function case.
- Information gain: Information capacity measures the maximum mutual information obtainable from n measurements and is sublinear for many commonly used kernels.
- Confidence bounds: Confidence intervals contain the true functions jointly with probability at least 1 −δ, supporting the algorithm’s safety reasoning.
- Safe-set expansion: The safe set expands after finitely many evaluations whenever safely reachable parameters remain outside the current safe set, unless further expansion is impossible.
- Performance guarantee: When the safe set stops expanding, the algorithm achieves performance within ϵ of the best safely reachable value with probability at least 1 −δ.
- Assumptions: The analysis assumes bounded RKHS norm, sub-Gaussian measurement noise, a nonempty initial safe set, and initially safe constraints.