PaperlayerView exact revision on arXiv

Source-linked AI summary

Data-efficient Auto-tuning with Bayesian Optimization: An Industrial Control Study

Matthias Neumann-Brosig, Alonso Marco, Dieter Schwarzmann, Sebastian Trimpe

arXiv:1812.06325v2eess.SYcs.LGcs.ROcs.SY

TL;DR

Controller tuning can be expensive, while manual calibration may leave controllers at low performance or unused. The paper uses Bayesian optimization to tune controllers from experiments, and reports performance on throttle-valve control that is on par with or better than hand tuning with around ten evaluations.

  • Problem

    Controller parameters are costly to tune experimentally, and manual tuning can consume substantial operator time while leaving controllers at low performance or unused.

    Source: §II · p. 3

  • Method

    The paper proposes direct Bayesian optimization for controller tuning, combining a Gaussian-process model of the objective with experiments and a framework that incorporates ADRC.

    Source: §I · p. 1, Related work · p. 2, Contributions · p. 2

  • Results

    BO achieves on par or better performance than hand tuning with only around ten experimental evaluations on throttle-valve control.

    Source: §I · p. 2

  • Takeaways & Limitations

    The tuning approach can handle widely different control functionals, and measurements from before a changed objective can inform the GP model afterward.

    Source: Different functionals · p. 10

    The article does not aim to systematically compare different BO algorithms.

    Source: Contributions · p. 2

Abstract

from arXiv · show

Bayesian optimization is proposed for automatic learning of optimal controller parameters from experimental data. A probabilistic description (a Gaussian process) is used to model the unknown function from controller parameters to a user-defined cost. The probabilistic model is updated with data, which is obtained by testing a set of parameters on the physical system and evaluating the cost. In order to learn fast, the Bayesian optimization algorithm selects the next parameters to evaluate in a systematic way, for example, by maximizing information gain about the optimum. The algorithm thus iteratively finds the globally optimal parameters with only few experiments. Taking throttle valve control as a representative industrial control example, the proposed auto-tuning method is shown to outperform manual calibration: it consistently achieves better performance with a low number of experiments. The proposed auto-tuning framework is flexible and can handle different control structures and objectives.

I. INTRODUCTION

Controller tuning is often expensive and manual tuning can leave controllers performing poorly or unused. The paper proposes Bayesian optimization (BO) for data-efficient controller tuning and reports performance on throttle-valve control that matches or exceeds hand tuning with around ten evaluations.

I. INTRODUCTION

  • Controller parameters are costly to tune, and manual tuning can consume substantial operator or engineer time while leaving controllers at low performance or unused.The paper describes tuning as a widespread industrial need because parameter choices are critical to performance.Source: §I · p. 1
  • The proposed BO framework seeks globally optimal controller parameters from few experiments without requiring a process model.BO models the unknown objective probabilistically and selects subsequent experiments systematically, for example to maximize information gain.Source: §I · p. 1
  • BO achieves on par or better performance than hand tuning with only around ten experimental evaluations.The study applies BO to throttle-valve control using active disturbance rejection control (ADRC).Source: §I · pp. 1–2
  • The framework combines BO with ADRC, balancing prior structural knowledge, especially system order, with learning from data.It is demonstrated under different control objectives, including response speed and robustness.Source: Contributions · p. 2
  • The method is direct: it adjusts controller parameters from closed-loop data, while incorporating poles of an approximate linear model to facilitate learning.Unlike indirect tuning, it does not first learn a dynamics model; a systematic comparison among BO algorithms is outside the article's scope.Source: Related work · p. 2, Contributions · p. 2

II. LEARNING CONTROL PROBLEM

The learning-control problem is to choose parameters for an output-feedback controller that optimize a designer-specified cost. Evaluating that cost requires experiments on an unknown system, which are costly, noisy, and difficult to optimize.

II. LEARNING CONTROL PROBLEM

  • The system dynamics and output map are unknown, although the throttle-valve example exploits structural knowledge while leaving the complete dynamics unknown.The system may also include general disturbances such as process disturbance or sensor noise.Source: §II · p. 2
  • The controller is output-feedback and tracks a reference signal using the system output.The controller may itself have internal states.Source: §II · p. 2
  • The controller is parameterized over a compact domain, and tuning seeks parameters that minimize the cost induced by its closed-loop response.The designer specifies the cost functional, which maps controller parameters to cost values.Source: §II · p. 3
  • Experiments sample the cost from recorded inputs, outputs, and references, but their expense permits only a few evaluations.Measurements may be uncertain, and the objective is generally nonconvex with no readily available gradients.Source: §II · p. 3

III. DATA-EFFICIENT BAYESIAN OPTIMIZATION

The paper presents Bayesian optimization (BO) as a way to optimize costly black-box objectives with few evaluations. Gaussian processes model the unknown cost and provide predictions used to guide subsequent experiments.

III. DATA-EFFICIENT BAYESIAN OPTIMIZATION

  • BO addresses expensive black-box optimization by combining a probabilistic prior over the objective with an acquisition function that selects the next parameter combination to test.The section introduces entropy search and expected improvement as its two specific BO algorithms.Source: §III · p. 3

A. Gaussian Processes (GPs)

  • Gaussian processes model the unknown cost function and predict cost values at unobserved controller parameters without assuming a parametric structure.The GP is a collection of random variables whose finite subsets have joint Gaussian distributions.Source: A. Gaussian Processes (GPs) · p. 3
  • A GP's prior mean and covariance encode prior information about expected cost and relationships between function values, including assumptions such as smoothness.The covariance function is also called the kernel.Source: A. Gaussian Processes (GPs) · p. 3
  • GP predictions provide a mean and variance at a candidate parameter setting conditioned on past observations; kernel and noise hyperparameters may be estimated from data.The paper also describes marginal-likelihood maximization and methods that marginalize the acquisition function over hyperparameters.Source: A. Gaussian Processes (GPs) · p. 3

B. Bayesian optimization (BO)

Bayesian optimization uses a Gaussian-process posterior to guide costly controller-parameter evaluations toward the global minimum. Expected improvement and entropy search differ in how their acquisition functions choose the next evaluation.

B. Bayesian optimization (BO)

  • The GP posterior conditions on observed evaluations, with uncertainty shrinking locally around data points.The figure illustrates five noisy evaluations.Source: Figure 3 · p. 4
  • The acquisition function uses the posterior to select the next parameter setting, whose physical-system evaluation updates the GP before the next iteration.The search stops after a suitable criterion; these experiments use a fixed number of evaluations.Source: B. Bayesian optimization (BO) · p. 4
  • Entropy search selects the location expected to provide the most information about the global minimum, while expected improvement seeks lower cost.The outcome, including the number of experiments required, may differ by BO method.Source: B. Bayesian optimization (BO) · p. 4
  • Stochastic optimizer computations and noisy observations can affect the repeatability of the final outcome.Entropy search uses internal numerical approximations based on sampling, and observation noise also affects the GP posterior and hyperparameter optimization.Source: B. Bayesian optimization (BO) · p. 4

C. Expected improvement (EI)

  • Expected improvement selects a setting expected to improve on the lowest cost observed so far.Its acquisition function is the expected value of max(0, η − J(θ)).Source: C. Expected improvement (EI) · p. 4

D. Entropy search (ES)

Entropy search (ES) represents uncertainty about the global minimum's location and chooses evaluations for their expected information gain. It then estimates the current minimum from the GP posterior mean.

1) Distribution about the location of the minimum:

  • ES represents knowledge of the global minimum's location with a probability distribution, approximated on an irregular grid with greater resolution in likely-minimum regions.In some cases the distribution cannot be strictly defined as a probability density, so ES approximates it as one for computation.Source: 1) Distribution about the location of… · p. 5

2) Acquisition function:

  • ES compares its approximate minimum-location distribution with a uniform distribution using Kullback–Leibler divergence to measure information about the minimum.A more concentrated distribution indicates greater information about the minimum's location.Source: 2) Acquisition function · p. 5
  • ES selects the next evaluation where the expected increase in information about the minimum is greatest.The increase is evaluated in expectation because the result of a prospective experiment is not yet known.Source: 2) Acquisition function · p. 5, D. Entropy search (ES) · p. 5

3) Global minimum computation:

  • After collecting a new data point, ES estimates the global minimum at the minimum of the GP posterior mean.This can be computed with local methods using sufficient random restarts, since GP gradients are analytically available.Source: 3) Global minimum computation · p. 5

IV. THROTTLE VALVE CONTROL

Throttle-plate control must be retuned for new plate models and balances response speed and minimal overshoot, alongside secondary objectives. The experiments use a physical throttle plate controlled through a MicroAutoBox-based setup.

A. System description

  • Throttle-plate tuning recurs with each new model, while switching spring behavior and nonlinear friction are key control challenges.The plate's two springs have different stiffnesses and rest positions, producing switching behavior.Source: §IV · p. 5, A. System description · p. 5
  • The plate's motor-driven opening angle is governed by a second-order nonlinear system with external disturbance and friction terms.The electrical and mechanical parts comprise a DC motor linked to the plate and a plate attached to two springs.Source: A. System description · p. 5

B. Control objectives

  • The primary objectives are fast response and minimal overshoot; secondary objectives include robustness, low noise amplification, and disturbance rejection.Speed is measured by T90-time, the average time for closed-loop absolute error to decrease by 90% after a step input.Source: B. Control objectives · p. 5

C. Experimental Setup

  • The experiments use a throttle plate, MicroAutoBox (DS1401), a DC-motor half-bridge, and supporting equipment; communication uses Matlab and the dSPACE ASAM XIL API at 1 kHz.The figure shows the throttle plate and MicroAutoBox.Source: C. Experimental Setup · p. 5, Figure 5 · p. 6

V. CONTROLLER STRUCTURE

The throttle-valve controller uses ADRC, which combines unknown system components into a total disturbance estimated by an observer. Pole placement sets the observer and nominal-system feedback structure, with controller parameters tuned for the desired cost.

V. CONTROLLER STRUCTURE

  • ADRC groups unknown system components into a total disturbance, and its observer estimates both the state and that disturbance.The extended model includes a linear approximation term; the authors found including it critical for performance despite adding optimization variables.Source: §V · p. 6
  • The nominal model need not accurately reproduce the full system if its errors help minimize the specified cost.The optimization chooses the linear approximation parameter to support the desired control goals.Source: §V · p. 6

A. Extended state-observer design

  • The physical experiments use a pole-placement Luenberger observer with one shared optimization variable for its triple pole; more general observer structures were not pursued after satisfactory test-system performance.A nonlinear observer might improve performance, but the study did not investigate that option.Source: A. Extended state-observer design · p. 6

B. State-feedback controller for nominal system

  • Pole placement designs the nominal-system feedback matrix, while the controller adds a fixed pre-amplification for unit DC gain and optimizes the closed-loop pole value.All desired feedback poles were set to the same value in the physical experiment.Source: B. State-feedback controller for… · p. 6

C. The full controller

The throttle-valve controller uses ADRC with a small parameter set, chosen for its tuning properties and suitability to this system. Experiments use ten evaluations per BO run.

C. The full controller

D. ADRC and throttle plate control

  • ADRC pole locations determine observer estimation speed and closed-loop speed, while pole placement leaves few parameters for BO to tune.An observer pole that is too fast can amplify sensor noise; one that is too slow can leave nonlinearities uncompensated at mid to high frequencies.Source: D. ADRC and throttle plate control · pp. 6–7
  • ADRC is considered suitable for systems whose dynamics are close to integrator chains, as the throttle plate is; its stability guarantee assumes ψ is constant.The authors state that stability cannot be guaranteed under sufficiently high uncertainty or arbitrary violations of that assumption.Source: D. ADRC and throttle plate control · p. 7

VI. DESIGN OF LEARNING EXPERIMENTS

  • Each BO-tuning experiment had a budget of ten evaluations, and measured data were filtered with a sixth-order, non-causal, zero-phase Butterworth filter.The controller has no internal states and uses a saturated observer input, so the authors report that windup causes no problems for this structure.Source: §VI · p. 7, D. ADRC and throttle plate control · p. 7

A. Choice of functionals

The experiments compare a heuristic objective that directly reflects speed and overshoot goals with a system-norm objective balancing robustness, speed, and overshoot. The latter uses frequency-domain estimates from a chirp experiment.

A. Choice of functionals

1) The heuristic functional:

  • The heuristic functional is constructed from the goal of speed with minimal overshoot, using measured overshoots and T90-times from a series of reference steps.Each step lasted 2 seconds, and each series lasted 2 minutes; T90 is retained because actual and nominal system speeds are not assumed equivalent.Source: 1) The heuristic functional · p. 7

2) The system norm functional:

  • The system-norm functional combines sensitivity H∞ and transfer-function H2 norms with a speed term to trade off robustness, speed, and overshoot.Sensitivity and closed-loop transfer estimates are obtained by FFT from throttle-plate position and reference signals, without fitting a system model.Source: 2) The system norm functional · pp. 7–8
  • The system-norm experiment uses a chirp from 0.1 to 30 Hz, while Fig. 6 reports estimated absolute transfer and sensitivity values over 0.5–28 Hz.The figure caption specifies the frequency interval used in the experiments.Source: 2) The system norm functional · p. 7, Figure 6 · p. 7

B. Hyperparameters

Entropy Search used hyperparameters selected from prior measurements, and safety bounds constrained controller parameters. The reported experiments were intended to compare BO tuning with manual tuning, not to rank BO variants.

B. Hyperparameters

  • Entropy Search hyperparameters were generated by maximum likelihood on ten prior measurements, with ardSE selected for the heuristic functional and ardRQ for the system-norm functional.The lengthscale for P1 in the system-norm functional was so large that P1 was not expected to significantly affect its GP approximation.Source: B. Hyperparameters · p. 8

C. Safety bounds

  • Safety bounds constrained Tset to [60 ms, 200 ms] and Tobs to [10 ms, 40 ms] to limit noise and set observer speed relative to nominal system speed.The bounds also restrict two stable, real-valued poles to ranges intended to provide one slow and one fast pole.Source: C. Safety bounds · p. 8

VII. EXPERIMENTAL RESULTS

  • Repeated tuning runs showed no repeatability problems from optimization stochasticity, but outcomes depended notably on choosing appropriate hyperparameters, including kernels.The reported results list one set per experiment; the authors say their goal was to contrast BO with manual tuning rather than systematically compare Entropy Search and Expected Improvement.Source: §VII · p. 8

A. Controllers tuned with the heuristic functional

For the heuristic objective, BO-tuned controllers outperform expert manual tuning on the primary control objectives, and the difference is statistically significant. System-norm tuning trades robustness for speed while keeping robustness above the stated threshold.

A. Controllers tuned with the heuristic functional

  • The unfiltered Fig. 7 measurement shows step responses for a controller tuned by Entropy Search with the heuristic functional.Its lower subfigure is an earlier step from the same measurement, zoomed in time and out in angle.Source: A. Controllers tuned with the… · p. 8, Figure 7 · p. 9
  • All controllers have sufficient robustness, comparable noise standard deviations, and acceptable disturbance rejection on the secondary objectives.Table III presents the secondary objectives for the heuristic controllers.Source: A. Controllers tuned with the… · p. 9
  • BO-tuned controllers outperform expert manual tuning on the heuristic functional's primary objectives, with a statistically significant difference between manual tuning and BO.Table II identifies the primary control objectives for the heuristic controllers; the authors report comparable results for the two BO controllers.Source: A. Controllers tuned with the… · pp. 8–9

B. Controllers tuned with the system norm functional

  • System-norm tuning trades robustness for speed, but the robustness measure remains above 0.6.The authors caution that the H2-norm's overshoot bound is generally not tight, so the system-norm and heuristic tuning experiments are not directly comparable.Source: B. Controllers tuned with the system… · p. 9
  • System-norm BO can incorporate control-theoretic characteristics such as robustness rather than necessarily producing a black-box tuning approach.Tables IV and V report the main and secondary objectives for the system-norm controllers.Source: B. Controllers tuned with the system… · p. 9, Table IV · p. 9, Table V · p. 9

C. Discussion

The discussion reports competitive Bayesian-optimization tuning alongside practical workflow advantages over manual calibration. The conclusion presents a flexible BO–ADRC framework and identifies assumptions and directions for further research.

C. Discussion

  • BO's documented, repeatable steps make choices retraceable, while explicitly encoded assumptions ease transfer between engineers.Source: Structured approach · p. 9, Transferability · p. 10
  • Less than one hour for Jheur and roughly one hour for Jnorm included validation; BO required no explicit system identification or selected excitation signals.Source: Tuning times · pp. 9–10
  • The authors expect BO tuning to generalize to similar throttle valves, potentially requiring no or minimal changes across applications.Source: Generalization · p. 10
  • BO empirically handled widely different cost functionals, allowing the same setup to address different control objectives.Source: Different functionals · p. 10

VIII. CONCLUDING REMARKS

  • Competitive throttle-valve controllers were obtained with about ten trials, and the framework applies across controller structures and objectives.Source: C. Discussion · p. 9, §VIII · p. 10
  • Without prior assumptions about the objective, global optimality cannot be guaranteed; the GP prior enables probabilistic reasoning about minima.Source: §VIII · p. 10
  • ADRC combines a crude system-order model with an observer for unmodeled dynamics and disturbances, while BO learns key parameters from experiments.Source: §VIII · p. 10
  • Future work targets sample efficiency, problem-specific kernels, and extensions to high-dimensional controller-tuning problems.Source: §VIII · p. 10

Related papers

Loading 1812.06325v2…