PaperlayerView exact revision on arXiv

Source-linked AI summary

Learning Model Predictive Control for iterative tasks. A Data-Driven Control Framework

Ugo Rosolia, Francesco Borrelli

arXiv:1609.01387v7eess.SYcs.LGmath.OC

TL;DR

The paper addresses repetitive control tasks without a known reference trajectory, while seeking constraint satisfaction and non-increasing performance across iterations. It proposes a reference-free LMPC that learns a safe set and terminal cost from previous trajectories. The resulting controller guarantees recursive feasibility and stability under its assumptions, and simulations demonstrate convergence and performance improvements in linear and nonlinear tasks.

Abstract

from arXiv · show

A Learning Model Predictive Controller (LMPC) for iterative tasks is presented. The controller is reference-free and is able to improve its performance by learning from previous iterations. A safe set and a terminal cost function are used in order to guarantee recursive feasibility and non-increasing performance at each iteration. The paper presents the control design approach, and shows how to recursively construct terminal set and terminal cost from state and input trajectories of previous iterations. Simulation results show the effectiveness of the proposed control logic.

I. INTRODUCTION

The paper develops a reference-free learning MPC for repetitive tasks where the reference trajectory is unknown. It uses trajectories from prior iterations to seek non-increasing cost while preserving constraint satisfaction and stability.

Motivation

  • Reference-based iterative controllers assume a known, unchanged reference and primarily minimize tracking error under disturbances.Source: §I · p. 1
  • Reference-free control is motivated by tasks whose optimal trajectory is difficult to compute because dynamics, parameters, or human inputs may be uncertain or variable.Source: §I · p. 1

Contributions

  • The proposed LMPC learns from previous iterations while keeping the initial condition, constraints, and objective function fixed.Source: §I · p. 1
  • A learned terminal safe set and terminal cost guarantee non-increasing iteration cost, recursive feasibility, and asymptotic stability.Source: §I · p. 1

Problem formulation

  • The paper defines each iteration as a closed-loop trajectory starting from the same initial state and evaluates its performance through an infinite-horizon cost.Source: §II · p. 2

A. Sampled Safe Set

The sampled safe set stores states from successful previous trajectories that can reach the terminal equilibrium under constraints. The iteration cost and terminal cost are computed from these stored trajectories for learning MPC.

Controllable and stabilizable sets

  • Stabilizable sets contain states that can reach a control-invariant target while satisfying constraints and then remain there indefinitely.Source: A. Sampled Safe Set · p. 2

Sampled Safe Set

  • Each state in SSj belongs to the maximal stabilizable set when its trajectory successfully steers the system to the terminal point xF.Source: A. Sampled Safe Set · pp. 2–3
  • The sampled safe set SSj collects state trajectories from successful iterations up to iteration j.Source: A. Sampled Safe Set · p. 3

Learning mechanism

Iteration Cost

  • The iteration cost is the infinite-horizon cost of the realized trajectory and quantifies controller performance at iteration j.Source: B. Iteration Cost · p. 3

Terminal Cost

  • Qj assigns each sampled-safe-set state the minimum cost-to-go among stored trajectories passing through that state.Source: B. Iteration Cost · p. 3

III. LMPC CONTROL DESIGN

At each time step, LMPC solves a finite-horizon constrained problem whose terminal state lies in the previous sampled safe set, then applies the first control and repeats. Under the initial feasibility assumption, this construction preserves feasibility and stability across iterations.

LMPC Formulation

  • LMPC approximates the infinite-horizon problem by repeatedly solving a finite-time constrained optimal control problem in receding-horizon form.Source: A. LMPC Formulation · pp. 3–4
  • The optimization enforces system dynamics, state constraints, and input constraints over the prediction horizon.Source: A. LMPC Formulation · p. 3
  • Its terminal constraint places the predicted terminal state in SSj−1, using successful trajectories from prior iterations as the terminal set.Source: A. LMPC Formulation · p. 4

Receding-horizon implementation

  • Only the first optimal input is applied before the problem is resolved at the next time step using the new state.Source: A. LMPC Formulation · p. 4

Recursive feasibility and stability

B. Recursive feasibility and stability

The LMPC uses previous trajectories through its sampled safe set and terminal cost to preserve feasibility across time and iterations. Under the stated assumption, it also yields asymptotic stability of the equilibrium.

Recursive feasibility

  • The sampled safe set and terminal cost constructed from prior trajectories support recursive feasibility and stability of the LMPC.The paper uses these objects to establish feasibility at every time and iteration and stability of xF.Source: B. Recursive feasibility and stability · p. 4

Stability

  • Under Assumption 1, the equilibrium point xF is asymptotically stable at every iteration while the LMPC remains feasible.Theorem 1 combines recursive feasibility with asymptotic stability for all t ≥ 0 and j ≥ 1.Source: B. Recursive feasibility and stability · p. 4

Recursive feasibility

  • Feasibility at iteration start follows from the non-empty previous safe set containing a feasible trajectory.The proof uses SS0 ⊆ SSj−1 and the common initial state to construct a feasible solution at t = 0.Source: B. Recursive feasibility and stability · p. 4
  • The receding-horizon solution supplies a feasible shifted trajectory at the next time step, preserving state and input constraints.Induction then establishes feasibility for all j ≥ 1 and t ≥ 0.Source: B. Recursive feasibility and stability · p. 4

Stability

  • The optimal finite-horizon cost is a decreasing Lyapunov function along the closed-loop trajectory.Positive definiteness of the stage cost and continuity of the optimal cost support the stability argument.Source: B. Recursive feasibility and stability · p. 5

C. Convergence properties

The paper establishes that iteration cost does not increase and analyzes the limiting trajectory as the controller learns. Under strict convexity and convergence, the steady-state trajectory is optimal for finite-horizon problems of arbitrary length; without convexity, only local properties are shown.

Iteration-cost convergence

Steady-state optimality

  • The finite-horizon comparison problem uses the same running cost, dynamics, and state and input constraints as the infinite-horizon problem.Its horizon may be longer than the LMPC horizon, while its terminal set may be smaller.Source: C. Convergence properties · p. 6

Scope and assumptions

  • Strict convexity of the infinite-horizon problem is assumed for the global optimality result.For non-convex problems, the paper states that only local optimality can be shown under additional strictness assumptions.Source: C. Convergence properties · pp. 6–7

Steady-state analysis

  • The convergence analysis assumes that the LMPC converges to a steady-state input and trajectory.Under this assumption, the sampled safe set and terminal cost also converge to steady-state objects.Source: C. Convergence properties · p. 6

Steady-state optimality

  • The steady-state trajectory is optimal for the finite-horizon problem with initial condition x∞t for every horizon T > 0.Theorem 3 extends the result from the controller horizon to arbitrary finite horizons under strict convexity.Source: C. Convergence properties · pp. 6–7

A. Constrained LQR controller

The LMPC is tested on a constrained infinite-horizon linear-quadratic regulator, using an initially feasible trajectory to construct the sampled safe set and terminal cost. Across iterations, the controller maintains feasibility, achieves non-increasing cost, and converges extremely close to the exact optimal solution.

Setup

  • The CLQR experiment initializes LMPC with a feasible trajectory, sampled safe set SS0, and terminal cost Q0(·).The feasible trajectory is generated by driving the system near the origin and then applying unconstrained LQR feedback.Source: A. Constrained LQR controller · p. 7
  • The controller uses a quadratic running cost, horizon length N = 4, and the CLQR state and input constraints.The implementation reformulates the LMPC as a mixed-integer quadratic program.Source: A. Constrained LQR controller · p. 7

Learning mechanism

  • Each subsequent closed-loop trajectory enlarges the sampled safe set used by the next iteration.The safe set is updated from the trajectories generated during learning.Source: A. Constrained LQR controller · p. 8

Results

  • After 9 iterations, the LMPC converges to a steady-state solution that is globally optimal for this CLQR instance.The converged solution saturates both state and input constraints, matching the exact solution.Source: A. Constrained LQR controller · p. 8
  • The iteration cost is non-increasing, and the LMPC improves closed-loop performance across iterations.These properties are reported for the constrained LQR experiment and summarized in Table II.Source: A. Constrained LQR controller · p. 8, Table II · p. 8
  • 1.62 × 10^-5 is the maximum deviation between the LMPC steady-state trajectory and the exact CLQR optimum.The 2-norm difference between the exact optimal cost and the converged trajectory cost is 1.565 × 10^-20.Source: A. Constrained LQR controller · p. 8

B. Dubins Car with Obstacle and Acceleration Saturation

The LMPC is applied to a minimum-time Dubins car with an obstacle and known acceleration saturation. Starting from a feasible trajectory, it learns a lower-cost steady-state maneuver that matches an independently computed locally optimal solution.

Problem setup

  • The Dubins car is controlled from xS to the unforced equilibrium xF by minimizing an infinite-horizon minimum-time objective.The state contains planar position and velocity; acceleration and steering are the control inputs.Source: B. Dubins Car with Obstacle and… · p. 8
  • The experiment includes a known acceleration saturation limit and an elliptical obstacle constraint.The saturation limit is s = 1, and the trajectory must remain outside the obstacle ellipse.Source: B. Dubins Car with Obstacle and… · p. 8

Initialization

  • A brute-force feasible trajectory initializes the sampled safe set SS0 and terminal cost Q0(·) for LMPC.The target is xF = [54, 0, 0]T, with obstacle parameters ae = 8 and be = 6.Source: B. Dubins Car with Obstacle and… · p. 9

Results

Validation

  • An interior-point nonlinear solver initialized with the LMPC trajectory produces the same locally optimal solution.This independently checks the local optimality of the learned steady-state trajectory.Source: B. Dubins Car with Obstacle and… · p. 9

C. Dubins Car with Obstacle and Unknown Acceleration Saturation

The LMPC is extended to a Dubins car whose acceleration saturation is unknown by augmenting the system with a saturation estimator and error dynamics. Across iterations, it learns the saturation coefficient while converging to the minimum-time maneuver.

Adaptive formulation

  • The unknown-saturation experiment augments the Dubins-car system with an estimated saturation coefficient and an estimation-error state.The controller simultaneously steers the vehicle to xF and estimates the unknown coefficient.Source: C. Dubins Car with Obstacle and… · p. 10
  • The augmented state includes estimated position, velocity, saturation coefficient, and estimator error; inputs include acceleration, steering, and estimate updates.The augmented dynamics and terminal constraint are incorporated into the LMPC problem.Source: C. Dubins Car with Obstacle and… · p. 10
  • Each iteration measures the system state and estimates the saturation error by inverting the system dynamics.The first element of the optimized input sequence is applied at each time step.Source: C. Dubins Car with Obstacle and… · pp. 10–11

Optimality condition

  • If the estimated error is zero for every future time step, the augmented solution is locally optimal for the original Dubins-car problem.This condition links successful saturation identification to local optimality.Source: C. Dubins Car with Obstacle and… · p. 12

Results

V. PRACTICAL CONSIDERATIONS

The sampled safe set makes the terminal constraint integer-valued, increasing computational expense. A convex-hull relaxation can reduce this burden and preserve key guarantees for linear systems with convex stage costs.

V. PRACTICAL CONSIDERATIONS

  • The sampled safe set is discrete, so the terminal constraint becomes an integer constraint.This requires solving a mixed integer programming problem at each time step, including for linear systems.Source: A. Computation · p. 12
  • Relaxing the sampled safe set to its convex hull and approximating Q(·) barycentrically can reduce computational burden.For linear dynamics and convex stage costs, the relaxed problem is convex.Source: 1) Convexifing the terminal constraint · p. 12
  • The relaxed approach preserves the properties established in Theorems 1–3 for linear systems with convex stage costs.Source: 1) Convexifing the terminal constraint · p. 12

2) Parallelize Computations:

The LMPC structure supports complexity reduction by bounding the optimal solution and restricting computation to relevant safe-set points. The framework also states deterministic guarantees and describes extensions for disturbances.

2) Parallelize Computations:

  • A subset of the sampled safe set can be used in the LMPC optimization, and the computation can be parallelized.Upper and lower bounds on the optimal solution reduce problem complexity without losing the guarantees of Theorems 1–3.Source: 2) Parallelize Computations · p. 13
  • The algorithm applies the current control action after solving the LMPC with a nonlinear optimization solver and selecting the minimizing candidate.The Dubins Car example uses Ipopt.Source: 2) Parallelize Computations · p. 13

Uncertainty

  • The theoretical guarantees are established only for the deterministic case, and disturbances cause those guarantees to be lost under the presented deterministic framework.The paper identifies stochastic and robust iterative learning MPC as extensions for future investigation.Source: B. Uncertainty · p. 13

VI. CONCLUSIONS

  • The proposed controller uses a learned safe set and terminal cost to guarantee recursive feasibility and stability while improving performance across iterations.Source: §VI · p. 13
  • The constrained infinite-horizon linear-quadratic regulator test converges to the optimal solution, while the Dubins car test estimates parameters and generates performance-improving trajectories.Source: §VI · p. 14

VIII. APPENDIX

The appendix constructs a feasible trajectory by parameterizing candidate inputs, selecting a low-cost trajectory, refining it through nonlinear optimization, and extending it into the terminal set.

VIII. APPENDIX

  • The initial guess fixes the saturation estimate at ˆs0 = 0.25 and uses a structured input parameterization.Source: §VIII · p. 14
  • The parameterized inputs use positive and negative ˜θ segments, bounded ˜a segments, zero acceleration intervals, and a terminal negative ˜a segment.Source: §VIII · p. 14
  • Trajectories generated from different parameter sets are ranked by a quantity, and the minimizing trajectory warm-starts a nonlinear optimization problem.Source: §VIII · p. 14
  • The optimized input sequence is applied to the system to compute realized trajectories and their error.Source: §VIII · p. 14
  • The resulting N-step trajectory steers the system into XF and can be used to construct the initial sampled safe set SS0.Source: §VIII · p. 14

Related papers

Loading 1609.01387v7…