PaperlayerView exact revision on arXiv

Source-linked AI summary

Stochastic data-driven model predictive control using Gaussian processes

E. Bradford, L. Imsland, D. Zhang, E. A. del Rio-Chanona

arXiv:1908.01786v2math.OCcs.LGcs.SYeess.SY

TL;DR

GP uncertainty in nonlinear model predictive control can lead to worse performance and constraint violations, while existing GP-MPC uncertainty propagation can be costly or conservative. This paper uses offline closed-loop GP samples to determine constraint back-offs for online control. The method is assessed in a semi-batch bioprocess case study, and the authors report fast online evaluation and reduced conservativeness through online learning and state-dependent uncertainty.

  • Problem

    GP-based MPC needs an efficient way to account for model uncertainty, which can worsen control performance or cause constraint violations.

    Source: §1 · pp. 1–2

  • Method

    The method uses offline closed-loop Monte Carlo samples of GP models to choose explicit constraint back-offs for online GP-based NMPC.

    Source: §1 · p. 3, §7 · p. 23

  • Results

    The semi-batch bioprocess case study demonstrates the method's potential, with offline constraint tightening yielding relatively low online computational times.

    Source: §7 · p. 23

  • Takeaways & Limitations

    The approach can account for online learning and state-dependent uncertainty while reducing conservativeness in the studied control setting.

    Source: §7 · p. 23

    The formulation does not exactly optimize the original objective because obtaining the expectation of a nonlinear function is difficult.

    Source: §4.1 · p. 10

Abstract

from arXiv · show

Nonlinear model predictive control (NMPC) is one of the few control methods that can handle multivariable nonlinear controlsystems with constraints. Gaussian processes (GPs) present a powerful tool to identify the required plant model and quantifythe residual uncertainty of the plant-model mismatch. It is crucial to consider this uncertainty, since it may lead to worsecontrol performance and constraint violations. In this paper we propose a new method to design a GP-based NMPC algorithmfor finite horizon control problems. The method generates Monte Carlo samples of the GP offline for constraint tighteningusing back-offs. The tightened constraints then guarantee the satisfaction of chance constraints online. Advantages of our proposed approach over existing methods include fast online evaluation, consideration of closed-loop behaviour, and thepossibility to alleviate conservativeness by considering both online learning and state dependency of the uncertainty. The algorithm is verified on a challenging semi-batch bioprocess case study.

1. Introduction

NMPC can address nonlinear, constrained multivariable systems, but GP-based approaches need to account for model uncertainty without costly or overly conservative propagation. This paper proposes offline closed-loop GP sampling and constraint back-offs, with extensions for initial-state uncertainty, additive noise, state-dependent uncertainty, and improved back-off computation.

1. Introduction

  • GP model uncertainty can worsen performance and violate constraints, while existing uncertainty-propagation methods rely on approximations and can be computationally costly or overly conservative.Open-loop uncertainty growth is often prohibitively conservative.Source: §1 · p. 2
  • The extensions add initial-state uncertainty and additive disturbance noise, account for GP-noise state dependency, and use root-finding to compute back-offs.The root-finding approach is reported to converge better and improve satisfaction of the required probability bounds.Source: §1 · p. 2
  • The method uses offline closed-loop Monte Carlo GP samples to set explicit constraint back-offs, leaving online NMPC with cheap GP evaluations.The authors report considerably faster evaluation times with little effect on performance.Source: §1 · pp. 2–3
  • Back-offs are tuned with closed-loop simulations, avoiding the open-loop growth of uncertainty identified as a source of conservatism.Offline computations and independent samples are also cited as advantages of the approach.Source: §1 · pp. 2–3
  • A variance penalty makes the objective trade off exploiting the GP model against avoiding uncertain states, explicitly accounting for state-dependent uncertainty.The paper also identifies online learning and state dependency as ways to alleviate conservativeness.Source: §1 · p. 3
  • The data-driven method identifies a GP model from input/output pairs and targets finite-horizon control problems, including batch processes.Training data may come from high-fidelity-model simulations or experiments such as step tests.Source: §1 · p. 3

Notation

The notation section defines common set, distribution, and probability symbols used in the paper. It also specifies the beta inverse CDF and diagonal-matrix notation.

Notation

  • N and R denote the sets of natural numbers and real numbers, and diag(a0, a1, . . . , an) denotes a diagonal matrix.The diagonal entries are a0 through an, with zero off-diagonal entries.Source: Notation · p. 3
  • N(µ, Σ) denotes a Gaussian distribution with mean µ and covariance Σ, while φ ∼ N(µ, Σ) states that φ follows that distribution..Source: Notation · p. 3
  • betainv(P, A, B) returns a value C for a beta-distributed random variable with parameters A and B such that the probability of φ ≤ C is P.The notation section also defines expected value, an indicator function, and probability measure.Source: Notation · pp. 3–4

2. Problem definition

The problem is finite-horizon nonlinear control with noisy observations of an unknown system, input limits, and joint probabilistic state constraints. A GP trained on the available data represents uncertainty about the dynamics for constraint-aware control.

2. Problem definition

  • The system is a discrete-time nonlinear state equation with additive Gaussian disturbance noise, and its initial state is Gaussian distributed.The disturbance has zero mean and diagonal covariance; the initial state has mean µx0 and covariance Σx0.Source: §2 · p. 4
  • The control objective is to minimize a finite-horizon cost over control inputs subject to hard input constraints.The cost comprises stage and terminal terms.Source: §2 · p. 4
  • A joint chance constraint requires nonlinear state constraints to hold across the horizon with probability greater than 1 − ϵ.The constraint event covers all times t ∈ {0, . . . , T}.Source: §2 · p. 4
  • The dynamics are unknown; the available data are finite noisy input/output observations, and a GP is trained to approximate the dynamics.The uncertainty arises partly from the initial condition and disturbance, but mostly from not knowing the dynamics beyond noisy observations.Source: §2 · pp. 4–5

3. Gaussian processes

Section 3 builds GP regression from noisy data, extends it to a multi-output state-space model, and samples finite-horizon trajectories by recursively conditioning on sampled points.

3.1. Regression

  • GP regression estimates a latent function from noisy observations; its posterior mean predicts the function, while its variance measures uncertainty.The hyperparameters are inferred from data, for example by maximum likelihood estimation.Source: §3.1 · pp. 5–6
  • The study uses a zero-mean GP with a squared-exponential covariance, which assumes a smooth, stationary latent function.For highly non-stationary functions, non-stationary covariance functions may be necessary.Source: §3.1 · p. 5

3.2. Recursive update

  • For online updates, the method keeps hyperparameters fixed and recursively updates the GP mean and variance after adding each new input/output pair.Noiseless Monte Carlo samples are added without a measurement-noise term; efficient inverse-covariance updates reuse the previous inverse.Source: §3.2 · pp. 6–7

3.3. State space model

  • The state-space model uses a separate independent GP for each output dimension, with additive disturbance noise included in the posterior covariance.Each output GP can also be updated recursively when a new data point arrives.Source: §3.3 · p. 7

3.4. Monte Carlo sampling

  • Finite-horizon GP paths are sampled exactly by drawing an initial state, then conditioning each next-state draw on prior noiseless samples; feedback makes controls sample-specific.The sampled function is deterministic, and this process repeats until the horizon is reached.Source: §3.4 · pp. 7–8
  • Repeated trajectories yield independent Monte Carlo samples, but recursive updates may make large horizons costly and motivate approximate sampling.For moderate horizons, at most T new data points are added.Source: §3.4 · pp. 8–9

4. Solution approach

The approach uses a GP distribution over possible plant models to generate closed-loop trajectories offline, then tunes constraint back-offs for online GP-based NMPC. Its guarantees rely on a finite-sample confidence bound and, for the feasibility result, a correct GP description of system uncertainty.

4. Solution approach

  • Offline GP model samples generate closed-loop trajectories used to tune back-offs, producing tightened constraints for online NMPC.The online formulation predicts with the GP mean and covariance; the tightened constraints are designed to support closed-loop probabilistic constraint satisfaction.Source: §4 · pp. 9–10

4.1. Finite-horizon Gaussian process model predictive control formulation

  • The finite-horizon NMPC predicts states with the GP mean and optimizes against tightened constraints, with η factors available to penalize uncertainty.Scalar state-dependency factors require normalized data; the objective does not exactly optimize the expected nonlinear objective.Source: §4.1 · p. 10
  • At each sampling time, the controller applies only the first optimal action and resolves the NMPC problem after a new state measurement.The formulation assumes full state feedback, though an observer can be introduced to account for additional uncertainty.Source: §4.1 · pp. 10–11
  • The control policy can update the GP from new measurements or penalize state-dependent uncertainty; either choice adds computational cost or depends on normalized data.Setting all η factors to zero ignores possible state dependency, while retaining a fixed GP model is the non-learning option.Source: §4.1 · pp. 10–11

4.2. Probabilistic guarantees

  • The empirical cdf approximates the constraint-satisfaction probability, and its confidence bounds account for uncertainty from using a finite number of samples.More samples tighten the confidence bound; the stated guarantee is a conservative lower bound on the true feasibility probability.Source: §4.2 · pp. 11–12
  • If the GP correctly describes system uncertainty, a lower confidence bound of at least 1 − ϵ implies the original chance constraint holds with probability at least 1 − α.The bound is estimated from S independent samples; it accounts for finite-sample error.Source: §4.2 · p. 12

4.3. Determining back-off constraints

  • Back-offs are initialized from GP-sample constraint deviations, then scaled by bisection until the joint-constraint lower bound reaches 1 − ϵ.The initial individual chance constraints use a tuning parameter δ and are distinct from the joint chance constraint targeted by the root-finding step.Source: §4.3 · p. 13

4.4. Algorithm

  • The overall procedure selects learning and state-dependency options, computes back-offs through closed-loop simulations, then runs NMPC online with the tightened constraints.Updating back-offs online could reduce conservativeness but requires carrying out the costly offline calculations online.Source: §4.4 · pp. 14–15

5. Case study

The case study applies GP-based NMPC to phycocyanin production in a semi-batch bioprocess, with an economic objective and path and terminal constraints. It uses simulated data and two dataset designs to examine the role of state-dependent uncertainty.

5. Case study

  • The case study uses a simplified dynamic model to test GP NMPC on phycocyanin production, maximizing final-batch bioproduct concentration subject to path and terminal constraints.The process is photo-production by the cyanobacterium Arthrospira platensis, using a limited dataset.Source: §5 · p. 15

5.1. Semi-batch bioreactor model

  • The semi-batch bioreactor model has three ODE states—biomass, nitrate, and phycocyanin—and uses light intensity and nitrate inflow as control inputs.The model uses Monod kinetics for growth and the Aiba model for light effects on growth and bioproduction.Source: §5.1 · p. 15

5.2. Problem set-up

  • The batch horizon is 12 steps over 240 hours (20-hour sampling), with nitrate below 800 mg/L, a bioproduct-to-biomass ratio at most 11.0 mg/g, and terminal nitrate below 150 mg/L.The maximum probability of violating the joint chance constraint is ϵ = 0.1.Source: §5.2 · p. 16

5.3. Implementation details and initial dataset generation

  • Two Sobol-based dataset designs contrast broadly distributed input-state data with open-loop trajectory data to assess when state-dependent uncertainty matters.The first design is relatively evenly distributed, so avoiding sparse regions has essentially no effect; the second has defined trajectories that can be followed by accounting for state dependency.Source: §5.3 · p. 17

6. Results and discussions

On the real plant, several nominal variants violated nitrate constraint g1, while variants using back-offs remained feasible. Larger datasets and online learning improved objective outcomes, while state-dependent uncertainty produced a trade-off between back-off size and objective value.

6. Results and discussions

  • Converged variants had final satisfaction probabilities from 0.91 to 0.93 against the required 0.9; GP NMPC 50 did not converge, and single-evaluation times ranged from 135ms to 48ms.The authors attribute GP NMPC 50's failure to converge to insufficient data in crucial areas; the satisfaction probabilities were higher than required because of the conservative probabilistic lower bound.Source: §6 · pp. 17–18, Table 3 · p. 22
  • With back-offs, the tested approaches remained feasible on the real plant, while several nominal variants violated nitrate constraint g1 by up to 50 mg/L.The nominal GP NMPC SD 50 variant did not violate g1.Source: §6 · p. 18, Figure 8 · p. 21, Figure 9 · p. 21
  • GP NMPC NSD 50 had a larger trajectory spread and mean back-offs over 50% larger for g1/g3 and over 100% larger for g2 than GP NMPC SD 50, whose average objective was marginally lower and worse.The authors attribute this trade-off to accounting for state dependency reducing conservativeness while potentially producing a suboptimal solution due to conflicting objectives.Source: §6 · p. 18, Figure 5 · p. 20, Figure 10 · p. 21
  • GP NMPC objective values increased from 50 to 60 to 100 data points, and mean g2 back-off fell from 0.022 mg/L for GP NMPC 50 to 0.003 for GP NMPC 100.The mean g1/g3 back-off decreased from 250 mg/L for GP NMPC 50 to 34.2 mg/L for GP NMPC 60, then increased to 38.8 mg/L for GP NMPC 100.Source: §6 · pp. 17–18, Figure 10 · p. 21, Table 2 · p. 22
  • GP NMPC 50 learning achieved an average objective above GP NMPC 60, while GP NMPC 50 averaged over 30% lower objective than the remaining scenarios.The authors report that learning's benefit was nearly negligible at larger dataset sizes.Source: §6 · p. 18, Figure 10 · p. 21

7. Conclusions

The paper proposes GP state-space NMPC for finite-horizon control, using sampled possible plant models to set explicit constraint back-offs. It derives probabilistic guarantees from simulated constraint violations and demonstrates the method in a semi-batch bioprocess case study.

7. Conclusions

  • The method samples deterministic functions of possible GP plant models and uses explicit back-offs to tighten constraints for high-probability closed-loop feasibility.Probabilistic guarantees are derived from the number of constraint violations in simulations.Source: §7 · pp. 22–23
  • Online learning and state-dependent uncertainty can be incorporated explicitly, leading to overall less conservativeness.Source: §7 · p. 23
  • Constraint tightening is performed offline, and the paper reports relatively low computational times.Source: §7 · p. 23
  • The semi-batch bioprocess case study demonstrates the method’s efficiency and potential for optimizing complex stochastic systems, including biological processes.Source: §7 · p. 23

Appendix A. Recursive inverse matrix update

This appendix introduces a recursive formula for obtaining an inverse more cheaply by exploiting that Y is known a priori. The resulting inverted matrix is then given.

Appendix A. Recursive inverse matrix update

  • Because Y is known a priori, the appendix introduces a recursive formula to obtain the inverse more cheaply.Source: Appendix A · p. 26
  • The update is adapted from Strassen (1969) to this case.Source: Appendix A · p. 26
  • The appendix presents the inverted matrix after specifying the quantities needed to obtain it.Source: Appendix A · p. 26
Loading 1908.01786v2…