PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning an Approximate Model Predictive Controller with Guarantees

Michael Hertneck, Johannes Köhler, Sebastian Trimpe, Frank Allgöwer

arXiv:1806.04167v1eess.SYcs.LG

TL;DR

The paper addresses the computational cost of online nonlinear MPC and develops a supervised-learning framework for approximating its controller. Robust MPC supplies tolerance to bounded input errors, while statistical validation bounds approximation error. On a nonlinear benchmark, the learned neural-network controller achieves the stated closed-loop guarantees with 99% confidence and at least 99% probability.

  • Problem

    Online MPC optimization can be computationally intractable, while nonlinear explicit MPC formulations are not straightforward and learned approximations need stability and constraint-satisfaction guarantees.

  • Method

    The framework samples a robust MPC feedback law offline, approximates it with supervised learning, and uses Hoeffding-based validation to bound approximation error.

  • Results

    With 99% confidence, the benchmark closed-loop system is stable and satisfies constraints with probability at least 99%.

  • Takeaways & Limitations

    The framework provides an approximate MPC with lower computational requirements and statistical guarantees for nonlinear constrained control problems.

Abstract

from arXiv · show

A supervised learning framework is proposed to approximate a model predictive controller (MPC) with reduced computational complexity and guarantees on stability and constraint satisfaction. The framework can be used for a wide class of nonlinear systems. Any standard supervised learning technique (e.g. neural networks) can be employed to approximate the MPC from samples. In order to obtain closed-loop guarantees for the learned MPC, a robust MPC design is combined with statistical learning bounds. The MPC design ensures robustness to inaccurate inputs within given bounds, and Hoeffding's Inequality is used to validate that the learned MPC satisfies these bounds with high confidence. The result is a closed-loop statistical guarantee on stability and constraint satisfaction for the learned MPC. The proposed learning-based MPC framework is illustrated on a nonlinear benchmark problem, for which we learn a neural network controller with guarantees.

I. INTRODUCTION

The paper addresses MPC’s online computational burden by learning an approximate nonlinear MPC while retaining statistical guarantees for stability and constraint satisfaction.

  • MPC can handle nonlinear dynamics, hard state and input constraints, and general objective functions, but repeatedly solving online optimization may become computationally intractable.
  • The framework approximates a nonlinear MPC through supervised learning because nonlinear explicit MPC formulations are not straightforward.
  • A robust MPC tolerates bounded input errors, allowing the learned controller to retain recursive feasibility and closed-loop stability when approximation errors remain admissible.
  • Hoeffding’s Inequality validates that the learned controller’s approximation error satisfies the robust MPC’s admissible bound with statistical confidence.
  • The framework supports nonlinear constrained systems, arbitrary supervised learning or regression techniques, user-defined costs, high sampling rates, and implementation on inexpensive hardware.
  • Unlike cited approximation approaches, the proposed framework targets closed-loop guarantees for stability and constraint satisfaction of the learned controller.

II. MAIN APPROACH

This section poses the control problem and describes the proposed approach.

  • The paper formulates the control problem before presenting its proposed approach.
  • The proposed approach is described after the control problem is posed.
  • This section provides an overview rather than the detailed formulation or procedure.

A. Problem formulation

The problem concerns a nonlinear discrete-time system with polytopic state and input constraints, requiring stability, constraint satisfaction, and cost optimization from feasible initial states.

  • The system has state x(t) in R^n, control input u(t) in R^m, discrete time, and continuous dynamics satisfying f(0, 0) = 0.
  • State and input constraints are modeled as compact polytopic sets.
  • The controller must stabilize the origin, satisfy state and input constraints for all times, and optimize a cost function from feasible initial conditions.
  • The resulting controller should be implementable on inexpensive hardware for systems requiring high sampling rates.

B. General approach

The general approach designs a robust MPC, learns its feedback law from offline samples, and validates the approximation error statistically.

  • An MPC is designed to tolerate inaccurate inputs within chosen bounds, then sampled offline over the feasible state set.
  • The sampled robust MPC is approximated using supervised learning, with neural networks serving as one possible regression method.
  • Closed-loop stability is guaranteed when the approximation error remains below the robust MPC’s admissible input-disturbance bound.
  • Hoeffding’s Inequality is used to validate that the learned controller meets this error bound.

III. INPUT ROBUST MPC

The input-robust MPC replaces standard constraints with tightened sets designed to tolerate bounded additive input errors. Under stated assumptions, the resulting controller preserves feasibility and constraints while converging to a robust invariant set, with asymptotic stability possible for sufficiently small errors.

  • Robust design: The RMPC combines standard MPC with robust constraint tightening to handle bounded additive input disturbances.The disturbance satisfies d(t) ∈ W with ||d||∞ ≤ η.
  • Constraint tightening: The tightened input set ensures that applying any disturbance d within W keeps the actual input inside U.The construction uses U_t = U ⊖ W before horizon-dependent tightening.
  • Constraint tightening: Tightened state and input sets are scaled by ϵ_k, whose size depends on the disturbance bound η and decay rate ρ.The tightening parameter is defined from ϵ and ρ along the prediction horizon.
  • Guarantees: For all feasible initial conditions, the RMPC closed loop satisfies state and input constraints and converges to a robust positive invariant set around the origin.This conclusion holds under Assumptions 1–4.
  • Guarantees: If η is sufficiently small, the robust invariant set lies inside the terminal set, allowing asymptotic stability of the origin under the terminal controller.The invariant-set size depends on η.

IV. LEARNING THE RMPC

The framework learns an approximate RMPC from offline samples and validates the approximation so that the learned controller retains stability and constraint-satisfaction guarantees.

  • IV. LEARNING THE RMPC: The framework combines an MPC design robust to chosen input errors with Hoeffding-based validation of approximation error.Together, these components produce an automatic procedure for learning an approximate MPC.

A. Supervised learning

The RMPC feedback law is sampled offline and approximated with supervised learning, while the robust design converts sufficiently small approximation errors into closed-loop guarantees.

  • A. Supervised learning: The approximate controller is learned from samples of the form (x, πMPC(x)) over the feasible state set.The paper uses neural networks but permits other regression or function-approximation methods.
  • A. Supervised learning: An approximation error bounded by η can be interpreted as an admissible input disturbance for the RMPC.Specifically, πapprox(x) = πMPC(x) + d with ||d||∞ ≤ η.
  • A. Supervised learning: Neural networks can approximate sufficiently regular functions arbitrarily accurately in principle, but guaranteeing the required error bound a priori is difficult.The difficulty motivates statistical validation of the learned controller.

B. Probabilistic guarantees

The probabilistic validation method tests approximation accuracy along independently sampled trajectories and applies Hoeffding’s Inequality to obtain a high-confidence fraction of initial conditions with stability and constraint satisfaction guarantees.

  • B. Probabilistic guarantees: Each validation trajectory starts from an independently sampled initial condition, and an indicator records whether the approximation error stays below η until the terminal set is reached.The empirical risk estimates the fraction of successful trajectories.
  • B. Probabilistic guarantees: Training a neural network to the required accuracy can be difficult because the needed number of hidden units is unknown and optimization typically finds only local optima.This is identified as a practical limitation of achieving the desired approximation error.
  • B. Probabilistic guarantees: Hoeffding’s Inequality lower-bounds the true success probability by the empirical risk minus a confidence-dependent error term.The bound is applied to iid indicator variables.
  • B. Probabilistic guarantees: The validation succeeds when the empirical bound exceeds the chosen critical probability µcrit; otherwise, the sample count is increased until success or pmax is reached.A failed validation requires repeating the learning step.
  • B. Probabilistic guarantees: With confidence 1 − δh, Algorithm 1 yields closed-loop stability and constraint satisfaction for at least a fraction µcrit of initial conditions distributed according to Ω.The result requires the stated assumptions and an approximation method capable of reaching error below η.

V. EXAMPLE

The nonlinear benchmark learns an approximate MPC from offline RMPC samples and validates it statistically, achieving guaranteed closed-loop behavior with substantially lower online evaluation time than RMPC optimization.

  • Problem: The benchmark stabilizes a nonlinear continuous stirred tank reactor around an unstable steady state under state and input constraints.The transformed system uses X = [−0.2, 0.2] × [−0.2, 0.2], U = [0−ue, 2−ue], Q = I, R = 10−4, and prediction horizon N = 180.
  • Learning: A feedforward neural network is trained from 1.6 · 10^6 feasible RMPC samples generated on a uniform grid.The network has hidden layers of sizes 2, 50, and 50, uses hyperbolic tangent sigmoid activations, and is trained with the Levenberg–Marquardt algorithm.
  • Validation: With confidence of 99%, the learned AMPC is stable and satisfies constraints with probability at least 99%.Validation used δh = 0.01, µcrit = 0.99, ˜µ = 0.9987, and p = 34980 trajectories.
  • Computational demand: The algorithm required roughly 500 hours on a Quad-Core PC, although parallelizing sampling and validation could reduce this time.This is the reported total execution time for Algorithm 1.
  • Simulation results: For the marked trajectory, the LQR’s initially opposite active input constraint causes divergence and over three times higher costs than AMPC.The passage attributes the initial divergence to the different active input constraint.
  • Simulation results: The AMPC trajectories are virtually indistinguishable from the original RMPC in the benchmark simulations.The comparison includes trajectories in the feasible set and coolant-flow profiles against a discrete-time LQR.
  • Computational demand: 3 ms versus 0.71 s: the neural network evaluates the controller over 200 times faster than the online RMPC optimization.The timing comparison uses 100 random points over Xfeas; the RMPC was evaluated with Casadi and the NN in Matlab.

VI. CONCLUSION

The conclusion combines robust MPC with probabilistic validation to preserve stability and constraint satisfaction despite approximation errors. Statistical validation also supports less conservative, automated controller synthesis, while broader verification remains future work.

  • Conclusion: The proposed RMPC guarantees stability and constraint satisfaction when the approximate input remains within user-defined bounds.This robustness property is the basis for learning an approximate MPC with closed-loop guarantees.
  • Conclusion: Statistical validation avoids conservatism, supports complex controller structures, and automates controller synthesis with few design variables.The conclusion identifies these as benefits of using statistical methods.
  • Future work: Tailored learning algorithms and more general verification for higher-dimensional systems are identified as future work.The stated future direction includes different indicator functions.
Loading 1806.04167v1…