Source-linked AI summary
Performance-oriented model learning for data-driven MPC design
Dario Piga, Marco Forgione, Simone Formentin, Alberto Bemporad
TL;DR
MPC must manage uncertainty without sacrificing closed-loop performance, but conventional robust approaches can be conservative. This paper learns hierarchical MPC-oriented models and controller parameters from closed-loop experiments using Bayesian optimization, achieving a lower benchmark cost than an LQG controller while stabilizing the pendulum under constraints.
Problem
Uncertain prediction models can require robust MPC approaches whose increased robustness usually leads to conservative performance.
Method
The paper optimizes an inner controller, an outer MPC prediction model, and potentially the prediction horizon using closed-loop data and Bayesian optimization.
Results
The proposed approach achieved cost ˜J = −3.66 versus −2.41 for LQG, while stabilizing the pendulum and respecting cart-position constraints.
Takeaways & Limitations
Closed-loop experiments can yield satisfactory hierarchical MPC controllers without knowledge of the system dynamics or constraints on modeling accuracy.
Abstract
from arXiv · showhide
Model Predictive Control (MPC) is an enabling technology in applications requiring controlling physical processes in an optimized way under constraints on inputs and outputs. However, in MPC closed-loop performance is pushed to the limits only if the plant under control is accurately modeled; otherwise, robust architectures need to be employed, at the price of reduced performance due to worst-case conservative assumptions. In this paper, instead of adapting the controller to handle uncertainty, we adapt the learning procedure so that the prediction model is selected to provide the best closed-loop performance. More specifically, we apply for the first time the above "identification for control" rationale to hierarchical MPC using data-driven methods and Bayesian optimization.
I. INTRODUCTION
MPC performance depends on the prediction model, yet conventional uncertainty handling can become conservative. The paper applies identification for control to hierarchical MPC, selecting models and controller parameters through closed-loop performance optimization.
- Robust and stochastic MPC address uncertainty, but increasing robustness usually leads to conservative performance.
- Identification for Control selects models according to closed-loop control performance rather than solely output prediction accuracy.
- The paper proposes learning an MPC-oriented prediction model from experimental data while directly considering the control objective.
- The proposed hierarchical architecture combines a fast inner controller with an outer MPC reference governor that improves performance and handles signal constraints.
- The model of the plant is treated as a design variable and optimized together with the controller using closed-loop data and Bayesian optimization.
III. CONTROL ARCHITECTURE
The hierarchical multi-rate architecture uses a fast inner controller for system dynamics and an outer MPC for performance enhancement and constraint enforcement. The outer MPC model is selected directly for closed-loop performance measured on the actual plant.
- III. CONTROL ARCHITECTURE: The inner controller operates at Ts to handle fast dynamics and may stabilize an unstable plant, although inner-loop stability is not required.
- III. CONTROL ARCHITECTURE: The outer MPC treats its model of the inner loop as a design parameter and seeks the model yielding the best measured closed-loop performance.
- III. CONTROL ARCHITECTURE: The outer MPC operates at TMPC = NTs, enhances inner-loop performance, and enforces output constraints.
- III. CONTROL ARCHITECTURE: Closed-loop performance is measured directly from experiments on the actual plant rather than evaluated using a plant model.
A. Inner controller parameterization
The inner controller is represented by a parameter vector, with a discrete-time PID controller provided as an example. Its derivative-filter parameter limits high-frequency gain and is generally excluded from optimization.
- A. Inner controller parameterization: The inner controller K is parameterized by θ, with a discrete-time PID controller serving as an example.
- A. Inner controller parameterization: The PID design vector is θ = [θP θI θD]′, while Nd ≫1 limits high-frequency gain and is generally not included in θ.
B. Outer MPC parameterization
The outer MPC predicts outputs and inputs from the command g using a parameterized state-space model. Its prediction horizon may be optimized, while other weights and constraint-softening parameters specify the desired closed-loop performance.
- B. Outer MPC parameterization: The outer MPC model predicts y and u from command g and is represented in state-space form with state ξ and parameter vector μ.
- B. Outer MPC parameterization: At each MPC sampling instant, the controller solves an optimization problem over predicted trajectories generated by the model M.
- B. Outer MPC parameterization: The MPC enforces softened output, input, and input-increment constraints using slack penalties and bounds over the prediction horizon.
- B. Outer MPC parameterization: When Nu < Np, the control command remains constant from Nu through Np, while Nu influences computational complexity.
- B. Outer MPC parameterization: The prediction horizon Np can be optimized, whereas Nu and the remaining MPC weights and softening parameters are treated as performance specifications or fixed by online throughput.
IV. PERFORMANCE-DRIVEN PARAMETER TUNING
The hierarchical control-design problem is reformulated so closed-loop performance depends directly on inner-controller parameters θ and outer-MPC parameters ν.
- The hierarchical architecture makes the measured closed-loop cost a function of θ and ν, converting controller design into a parameter-optimization problem.
A. Bayesian optimization for parameter selection
Bayesian optimization iteratively models the closed-loop cost with a Gaussian process and selects controller parameters through an acquisition function. It balances exploration and exploitation while limiting costly closed-loop experiments.
- A. Bayesian optimization for parameter selection: Bayesian optimization initializes experiments, fits a Gaussian process to observed costs, selects new parameters, and augments the dataset until termination.The algorithm then returns the best parameters found.
- A. Bayesian optimization for parameter selection: The acquisition function balances exploration of uncertain parameter regions with exploitation of regions predicted to have favorable closed-loop cost.
- A. Bayesian optimization for parameter selection: Expected Improvement selects parameters by estimating improvement over the best objective value observed at the current iteration.Under the Gaussian-process model, it can be evaluated analytically.
- A. Bayesian optimization for parameter selection: Bayesian optimization is derivative-free and reduces the number of closed-loop evaluations needed when the performance function lacks a closed-form expression and experiments are costly.
B. Restricting the parameter space
The search over controller and prediction-model parameters can be narrowed using stability, control-design, structural, and prior-knowledge constraints.
- B. Restricting the parameter space: Restricting the parameter search space generally accelerates convergence and reduces the number of evaluations required.Bounds can exploit prior system knowledge and design choices.
- B. Restricting the parameter space: The prediction model may be constrained to be asymptotically stable when the optimal solution is assumed to use an inner controller that stabilizes the inner loop.
- B. Restricting the parameter space: Additional restrictions include matching the PID static-gain sign to the stable plant, sharing controller and prediction-model parameters, and using diagonal or grey-box models.
V. NUMERICAL EXAMPLE
The numerical example considers control of an inverted pendulum on a cart.
- The case study applies the proposed control problem to an inverted pendulum on a cart.
A. System description
The inverted-pendulum study uses a hierarchical, multi-rate controller with an inner stabilizing loop and an outer MPC whose prediction model and tuning parameters are optimized from closed-loop performance. Experiments use measured noisy outputs, input disturbances, physical constraints, and Bayesian optimization over controller, model, and horizon parameters.
- Plant and objectives: The case study controls an inverted pendulum on a cart, using cart position p and pendulum angle φ as outputs and cart force F as input.The objective is to move the pendulum to φ = 0 while limiting cart displacement, with F ∈ [−20,20] N and p ∈ [−1,1] m.
- Controller architecture: The hierarchical controller combines a fast inner PID loop, which feeds back only φ, with an outer MPC that predicts both p and φ.The inner loop operates at Ts, while the outer MPC uses a sampled prediction model at TMPC = 50 ms and enforces force and position constraints.
- Prediction model: The MPC prediction model is a two-state continuous-time model parameterized by μ, with the first column of B_M fixed to zero because p is not fed back internally.The overall MPC model is constructed from this parameterization and used to predict the inner-loop dynamics from command g to plant output y = [p φ].
- Optimization setup: Ten parameters are optimized: three inner-controller parameters θ, six prediction-model parameters μ, and the prediction horizon N_p.The real-valued parameters lie in [−500,500], while N_p ranges from 10 to 20 and N_u is set equal to N_p.
- Evaluation and optimization: The closed-loop cost evaluates angle regulation, cart displacement, and position-constraint violations over 10-second experiments, while force limits are imposed by input saturation.Measurements are sampled every 5 ms with additive output noise, and the applied force includes a random disturbance.
C. Simulation results
Bayesian optimization identified a low-cost controller during the experiment sequence, and the resulting hierarchical controller stabilized the pendulum while respecting cart-position constraints. Compared with physical-model-based baselines, the proposed approach outperformed LQG and retained stabilization where a same-rate MPC failed.
- Bayesian optimization: The optimal controller parameters were found at iteration 123, with later Bayesian-optimization trials increasingly concentrated in a low-cost region.Figure 3 reports both each trial’s cost and the best cost observed up to that iteration.
- Closed-loop validation: The designed controller stabilized the pendulum at the upright position for 20 s while respecting the cart-position constraint.The longer closed-loop experiment plotted cart position and pendulum angle trajectories.
- Baseline comparison: A physical-model MPC running at the same 50 ms rate failed to reject the disturbance and stabilize the pendulum around the upright position.The comparison attributes the contrast to the hierarchical multi-rate controller structure.
- Baseline comparison: The proposed controller achieved cost ˜J = −3.66, versus −2.41 for the LQG baseline, while stabilizing the pendulum under the experimental conditions.The LQG controller used the physical plant model and a faster 5 ms sampling rate.
VI. CONCLUSIONS AND FOLLOW-UP
The paper presents a Bayesian-optimization strategy for learning MPC-oriented models through iterative closed-loop experiments. Simulations show satisfactory controllers without system-dynamics knowledge or modeling-accuracy constraints, while theoretical analysis and real-world validation remain future work.
- Bayesian optimization was used to design iterative closed-loop experiments for learning MPC-oriented models in hierarchical control schemes.
- Simulations produced satisfactory controllers without knowledge of the system dynamics or constraints on modeling accuracy.
- The proposed framework selects the model yielding the best closed-loop performance rather than the highest input/output data fit.
- Future work will address theoretical analysis of the learning strategy and experimental validation on a real-world setup.