PaperlayerView exact revision on arXiv

Source-linked AI summary

Bridging direct & indirect data-driven control formulations via regularizations and relaxations

Florian Dörfler, Jeremy Coulson, Ivan Markovsky

arXiv:2101.01273v2math.OC

TL;DR

The paper asks how direct regularized data-driven control relates to sequential system identification and control, and why direct methods can work well for nonlinear systems. It formulates both paradigms behaviorally, connects them through a multi-criteria problem, and shows that regularized direct formulations arise as convex relaxations of indirect control. The analysis derives a novel regularizer and associates indirect methods with variance errors and direct methods with bias errors.

  • Problem

    Indirect control separates identification from control, while direct methods require robustification because noisy data can make certainty-equivalence control arbitrarily poor.

  • Method

    The paper uses behavioral systems theory, parametric mathematical programs, and a multi-criteria identification-control formulation specialized to subspace predictive control and low-rank approximation.

  • Results

    Regularized direct data-driven control is derived as a convex relaxation of indirect control, with regularizers accounting for implicit identification and one new regularizer consistent with least-square identification.

  • Takeaways & Limitations

    The results suggest using indirect methods for variance errors and direct methods for bias errors, including nonlinear systems.

  • Takeaways & Limitations

    The results concern only open-loop predictive control, although the ultimate interest is realized performance in receding-horizon closed loop.

Abstract

from arXiv · show

We discuss connections between sequential system identification and control for linear time-invariant systems, often termed indirect data-driven control, as well as a contemporary direct data-driven control approach seeking an optimal decision compatible with recorded data assembled in a Hankel matrix and robustified through suitable regularizations. We formulate these two problems in the language of behavioral systems theory and parametric mathematical programs, and we bridge them through a multi-criteria formulation trading off system identification and control objectives. We illustrate our results with two methods from subspace identification and control: namely, subspace predictive control and low-rank approximation which constrain trajectories to be consistent with a non-parametric predictor derived from (respectively, the column span of) a data Hankel matrix. In both cases we conclude that direct and regularized data-driven control can be derived as convex relaxation of the indirect approach, and the regularizations account for an implicit identification step. Our analysis further reveals a novel regularizer and a plausible hypothesis explaining the remarkable empirical performance of direct methods on nonlinear systems.

I. INTRODUCTION

The paper bridges indirect, sequential identification-and-control methods with direct data-driven control through a multi-criteria formulation and convex relaxations. It specializes this connection to subspace predictive control and low-rank approximation, using regularization to represent implicit identification and explain differing performance across error types.

  • Motivation: Indirect control sequentially identifies a model and then designs control, whereas direct control seeks decisions compatible with recorded data.The indirect approach is modular but can be cumbersome and suboptimal, while direct methods are less tractable and rarely suited to safety-critical settings.
  • Bridge between paradigms: A multi-criteria formulation trades off identification and control objectives, with one Pareto-front tail corresponding to the indirect bi-level problem.Convex relaxation of this formulation yields regularized data-enabled predictive control formulations.
  • Concrete methods: The analysis specializes the bridge to subspace predictive control and low-rank approximation, both using Hankel-matrix-based trajectory representations.The Fundamental Lemma supports representing finite-length LTI trajectories through the column span of a data Hankel matrix.
  • Regularization: Direct regularized control is derived as a convex relaxation of indirect control by dropping LTI complexity constraints and replacing projection onto LTI systems with regularization.The regularizer therefore accounts for an implicit identification step.
  • Empirical interpretation: The paper hypothesizes that indirect methods are preferable under variance errors, while direct methods can perform better under bias errors such as nonlinear dynamics.Numerical studies are reported as supporting this bias-variance explanation for direct methods on nonlinear systems.

C. Representation-Free Estimation and Behavior Dimension

This section characterizes finite-horizon LTI behavior and its representation through data Hankel matrices. Under a rank condition, one trajectory’s Hankel matrix spans all trajectories of the chosen length.

  • Trajectory representation: A minimal state-space representation parameterizes length-L trajectories using the initial state together with input and output sequences.The initial state can be reconstructed from a prefix trajectory once the observability condition is satisfied.
  • Complexity: The complexity parameters (q,m,n,ℓ) specify signal dimension, input count, minimal state dimension, and minimal lag for the LTI model class.When only upper bounds are known, behavior dimensions and rank equalities become corresponding upper bounds and inequalities.
  • Behavior dimension: For an LTI system with m inputs, state dimension n, and lag ℓ, the length-L behavior has dimension mL+n when L≥ℓ.The result follows from a full-column-rank extended observability matrix and a corresponding state-space trajectory basis.
  • Hankel representation: The Fundamental Lemma states that the column span of a data Hankel matrix equals the restricted behavior under the rank condition rank(HL(w))=mL+n.Thus, a single trajectory can parameterize all length-L trajectories exactly when its Hankel matrix has the required rank.
  • Models versus data: The Hankel image representation is directly available from raw data but is less compressed than a parametric model and is generally limited to finite horizons.The paper therefore distinguishes data-driven representations from kernel or state-space models.

III. DIRECT AND INDIRECT DATA-DRIVEN CONTROL

The paper formulates finite-time data-driven control around trajectory tracking under plant compatibility and admissibility constraints. These formulations provide the control baseline against which indirect and direct approaches are compared.

  • Optimal control: The finite-time control problem minimizes a tracking cost over admissible trajectories compatible with a prefix and the plant behavior.The prefix trajectory implicitly sets the initial condition when its length is at least the system lag.
  • Assumptions: Convexity follows when the tracking cost is convex, the admissible set is closed and convex, and the plant behavior is a subspace.A viability condition ensures that admissible trajectories can continue for the prediction horizon.
  • Reference and feasibility: The reference trajectory need not belong to the plant behavior or the admissible set, because the control task may request tracking of non-plant behavior.Examples include tracking step references.
  • Consistency: Under the stated consistency assumptions, the plant-constrained control problem attains its minimum at the reference trajectory when that trajectory is feasible.This result establishes a ground truth for comparing alternative data-driven formulations.
  • Model-based control: A parametric plant model converts the formulation into a classical control problem, and such models are typically obtained through system identification.The paper uses this model-based formulation as the indirect-control reference point.

B. Indirect Data-Driven Control via System Identification

Indirect data-driven control first fits an LTI model to identification data and then solves a certainty-equivalence control problem on that model. The resulting bi-level formulation is optimal under consistency assumptions, but the paper deliberately abstracts from additional design levels such as model-order selection.

  • Sequential procedure: Conventional identification selects a model class, fits a model to data, and then applies certainty-equivalence control.The model class may be specified through complexity parameters such as state dimension and lag.
  • System identification: The identification problem seeks the closest LTI behavior within the chosen hypothesis class according to an identification loss.Identification can be non-convex, and existence or uniqueness depends partly on the data.
  • Certainty-equivalence control: Certainty-equivalence control minimizes a surrogate tracking error on the identified model rather than directly on the true plant.This distinction motivates the paper’s comparison between indirect and direct formulations.
  • Bi-level formulation: The bi-level problem explicitly nests model fitting inside control design, reflecting the sequential identification-then-control workflow.Under suitable consistency assumptions, the sequential approach is optimal.
  • Scope: The formulation omits further nested levels such as model-selection hyperparameters and uncertainty quantification.The paper focuses specifically on identification and control.
  • Consistency: Under assumptions (A.4)–(A.6), the bi-level problem reduces to the optimal control problem, and additional assumptions identify its minimum value with the reference solution.These assumptions are used for consistency statements rather than for the paper’s later main results and simulations.

C. Direct Data-Driven Control via the Image Representation

Direct control uses Hankel-matrix trajectory compatibility, but noisy data can make that constraint vacuous; regularization robustifies the formulation and can recover the indirect optimum under stated assumptions.

  • The Fundamental Lemma places prediction and estimation trajectories within the column span of a data Hankel matrix.
  • With Assumptions (A.1) and (A.5), the direct control problem is equivalent to the regularized formulation, sharing its minimizers and minima.
  • Direct control requires more data than identification under Assumption (A.1) because it seeks a multi-step predictor rather than a recursively applied single-step predictor.Weaving multiple trajectories can make the data lengths coincide.
  • Noisy Hankel data can have full rank, making the direct constraint vacuous and allowing realized control error to differ arbitrarily from surrogate error.Robustification addresses this gap, whereas identification can filter noisy data by projecting onto a deterministic behavior.
  • The decision variable g linearly combines Hankel columns to produce an optimal trajectory compatible with the prefix trajectory w_ini.The regularizer h(g) is weighted by a nonnegative parameter λ and may use one-, two-, squared-two-, or p-norms.
  • Regularizers admit deterministic or stochastic robust-optimization interpretations, with λ specifying the size of the assumed uncertainty set.Regularized formulations have also been used in practical nonlinear control systems.

A. Multi-Objective Data-Driven Control

A multi-criteria formulation jointly optimizes identification and control, with γ tracing their trade-off. Under partial calmness and continuity assumptions, sufficiently large γ recovers the bi-level problem, while the formulation is generally nonconvex.

  • The multi-criteria problem simultaneously optimizes identification and control objectives, with γ ≥ 0 tracing the Pareto front between them.
  • Identification and control regularize each other by fitting a model to both identification data and the reference trajectory.The identification criterion encourages adherence to observed data, while the control objective biases the identified model.
  • For any γ ≥ 0, the parametric multi-criteria problem attains a minimum under Assumptions (A.2)–(A.6).
  • If the inner identification problem has a minimum, partial calmness holds, and the control cost is continuous, sufficiently large γ makes the multi-criteria problem equivalent to the bi-level problem.The optimal values coincide up to the constant γ · ϕ.
  • Partial calmness is equivalent to the identification constraint acting as an exact penalty; later relaxations remove the requirement that γ be sufficiently large.
  • The multi-criteria problem is generally nonconvex because both the behavior model and the controlled trajectory are decision variables.In a kernel representation, the constraint contains the product of the variable operator and variable trajectory.
  • Choosing a point on the Pareto front may yield superior performance, although this possibility is presented as a research direction.

B. Bridging Towards Subspace Predictive Control (SPC)

SPC expresses the indirect identification-and-control formulation through a parametric predictor built from Hankel data. Its direct regularized counterpart is a convex relaxation that drops structural constraints and uses a projection-based regularizer induced by least-squares identification.

  • SPC formulation: SPC relates past and future trajectory blocks through a linear predictor K estimated from Hankel matrix data.For exact data, rank(Kp)=n promotes the desired LTI complexity, while lower-block triangular Kf promotes causality.
  • SPC formulation: The parametric indirect formulation is generally not equivalent to the original problem because its inner identification need not produce an LTI model.The formulation imposes rank and block-triangular structure, but the inner identification problem itself does not necessarily lead to an LTI model.
  • Regularization: Under consistency assumptions, the projection-based regularizer leaves the ground-truth control variables unchanged, whereas a conventional norm regularizer can bias them.The projection penalty affects only the homogeneous solution component and vanishes at the least-squares fit.
  • Convex relaxation: For sufficiently small λ, the regularized SPC problem is convex, contains every feasible indirect control decision, and has an optimal value no larger than the indirect problem.These properties follow from enlarging the feasible set while retaining the control objective structure.
  • Convex relaxation: The SPC direct regularized problem drops rank and block-triangularity constraints, replaces least squares with an equivalent least-norm formulation, and lifts the problem to multiple criteria.The induced regularizer is ∥(I−Π)g∥p, where Π projects onto the relevant solution subspace.
  • Regularization: The projector regularizer admits broader penalty choices, but the relaxation should not be expected to be tight because the proof qualitatively removes rank and causality constraints.The parameter λ also controls the trade-off between adherence to the data fit and the control objective.

C. Bridging Towards Structured Low-Rank Approximation

Structured low-rank approximation formulates identification by projecting noisy Hankel data onto trajectories whose Hankel matrix has bounded rank. Replacing rank and sparsity constraints with an ℓ1 regularizer yields a direct control problem that is a convex relaxation of the indirect formulation.

  • Low-rank identification: The construction assumes the identification record is much longer than the estimation and control prediction horizons.The implicit condition is T ≫ Tini+L.
  • Low-rank identification: The low-rank approach seeks the closest data sequence whose Hankel matrix has rank no greater than m(Tini+L)+n.This models identification as structured low-rank approximation of the Hankel matrix assembled from recorded data.
  • Relaxation: For sufficiently small λ, the direct problem with h(g)=∥g∥1 is convex, contains every feasible indirect trajectory, and lower-bounds the indirect optimal value.The direct formulation enlarges the feasible set through the rank relaxation.
  • Relaxation: The indirect low-rank formulation can be relaxed by dropping the rank constraint and replacing the resulting cardinality bound on g with an ℓ1-norm bound.The sequence of relaxations converts ∥g∥0 ≤ n+m(Tini+L) into ∥g∥1 ≤ α.
  • Relaxation: The ℓ1 constraint and penalized formulation are equivalent for suitable parameter choices under convexity and Slater’s condition.The corresponding λ depends on the data through the Lagrange multiplier of the ℓ1 constraint.
  • Interpretation: The resulting ℓ1 regularizer accounts for selecting model complexity, while the relaxation’s tightness is not guaranteed.The paper explicitly connects the regularizer to replacing the identification rank constraint.

D. Hybrid relaxations

The hybrid formulation combines the complexity-controlling ℓ1 regularizer with the projection-based least-squares regularizer. Under consistency assumptions, it preserves the ground-truth trajectory while providing a combined relaxation framework.

  • Hybrid regularization: The hybrid formulation blends ∥g∥1, which controls model complexity, with ∥(I−Π)g∥2, which accounts for least-squares fitting.The two regularizers combine the roles identified separately for low-rank approximation and SPC.
  • Hybrid regularization: Under the stated consistency assumptions, the hybrid problem achieves the ground-truth trajectory with ∥(I−Π)g⋆∥2=0 for any λ≥0.The result is stated as a consistency fact for the hybrid formulation.
  • Evaluation: The hybrid regularizer’s performance is evaluated numerically in Section V-B, specifically in Figure 3.The supplied passage identifies the validation location but does not state the numerical outcome.

E. Possible pitfalls of relaxations

The relaxation results are nontrivial only away from zero regularization: zero weighting can produce a low surrogate error while losing data fit and realized performance, whereas suitable positive weights preserve identification-related behavior. The regularization weight is therefore a tunable design parameter whose useful value generally cannot be fixed a priori.

  • Relaxation limits: At γ = 0, the multi-criteria problem relaxes the indirect problem: every indirect-feasible solution remains feasible, and the relaxed optimum lower-bounds the indirect optimum.The relaxation arises by dropping the inner optimality constraint.
  • Regularization limits: At λ = 0, the relaxed solution matches the reference but need not fit the identification data or belong to the true system behavior.Although this minimizes the open-loop surrogate tracking error, realized control performance can be arbitrarily poor.
  • Regularization limits: A sufficiently small but nonzero λ is required by the relaxation results to make the modeled behavior match the plant under the fitting criterion.The smallest adequate value depends on problem-specific Lipschitz constants and multipliers.

V. NUMERICAL ANALYSIS AND COMPARISONS

The numerical studies examine regularization, hybrid penalties, data length, and direct–indirect comparisons on linear and nonlinear systems. They find that projection-based regularization is robust for sufficiently large weights, hybrid regularization can improve results, and direct methods can outperform indirect methods when model-order bias dominates.

  • A. Choice of Regularization Parameter: Too-small λ produces optimistic predicted error but poor realized performance, while large λ also degrades performance, making careful tuning necessary.A broad intermediate range can nevertheless deliver similarly good results.
  • A. Choice of Regularization Parameter: For sufficiently large λ, the projection-based regularizer yields nearly constant, superior performance and is more robust than the conventional two-norm regularizer.The direct and indirect formulations become equivalent up to causality and complexity constraints, while the projection penalty enforces the least-square fit.
  • B. Hybrid Regularization: Hybrid regularization provides a minor but robust improvement, reaching up to 15% better performance than the best projection-based regularizer results.The parameter slices with λ1 = 0 and λ2 = 0 recover the separate regularizer experiments.
  • C. Effect of data length: Both direct and indirect methods are asymptotically consistent, but the indirect method is superior with little data whereas the direct method is superior under incorrect indirect model-order selection.For the case study, at least T = 59 data points are required; over-parameterizing the indirect model to n = 6 causes bias and loss of consistency.
  • C. Effect of data length: The data-length findings suggest a bias–variance trade-off between direct and indirect control methods.This trade-off motivates further analysis in the paper.
  • Direct and indirect comparisons: The comparisons target variance error on noisy linear systems and bias error on noise-free nonlinear systems, with opposite expected advantages for indirect and direct methods.Identification is expected to filter noise in the linear case, while the indirect linear model class is expected to create bias in the nonlinear case.

Comparison: Stochastic Linear System

The experiments compare direct and indirect data-driven control under varying noise, nonlinearity, and data availability. Indirect control is more robust to noise, whereas direct control retains performance as linear-model bias increases.

  • Varying noise: Both methods perform well at low noise levels, up to approximately 2% noise-to-signal ratio.The comparison uses 100 simulations for each noise-to-signal ratio and measures percentage error relative to best possible deterministic-model performance.
  • Varying noise: As noise increases, direct-method performance degrades significantly while indirect-method performance remains relatively constant.The authors attribute this pattern to identification denoising the data and associate it with lower variance error for the indirect method.
  • Varying nonlinearity: As nonlinearity increases, indirect-method performance degrades significantly while direct-method performance remains relatively constant.Both methods perform well for low nonlinearity, with ϵ ∈[0.7, 1].
  • Varying nonlinearity: The authors attribute the nonlinear-system result to bias from selecting a linear model class for indirect control, which the direct method avoids by using nonlinear-system data.The direct method does not specify LTI system complexity, allowing sufficiently complex LTI approximations without explicit rank constraints.
  • Interpretation: The paper concludes that regularized direct control is a convex relaxation of indirect control, with regularizers encoding an implicit identification step.A novel regularizer is also reported as consistent and accounting for least-square identification.
  • Interpretation: The authors suggest indirect control for variance errors and direct control for bias errors, including nonlinear systems or incorrect model-order selection.This bias-variance interpretation is presented as a partial explanation for direct data-enabled predictive control's empirical nonlinear-system performance.
  • Limitations: The reported results concern only open-loop predictive control, although realized performance in receding-horizon closed-loop implementation is the ultimate concern.The paper identifies closed-loop realized performance as an open topic for future work.

APPENDIX

The appendix formalizes partial calmness and its equivalence to exact penalization for mathematical programs. Under suitable regularity and penalty conditions, local minimizers of the penalized problem also solve the original problem.

  • Problem setup: The appendix considers a mathematical program and a perturbed version in which an equality-related condition is represented through a distance or constraint function.The general setup uses a closed feasible set C and lower semicontinuous maps f, g, and h.
  • Partial calmness: Partial calmness is defined locally around a solution of the original mathematical program across feasible perturbations.The definition uses neighborhoods around x⋆ and perturbations ϵ in δB1.
  • Exact penalization: The penalized program minimizes f(x) + µ · |h(x)| subject to g(x) ≤ 0.The penalty parameter µ controls the weight assigned to the equality-related violation.
  • Exact penalization: If partial calmness holds at x⋆, there is µ⋆ > 0 such that x⋆ locally minimizes the penalized problem for every µ ≥ µ⋆.Conversely, local minima of the penalized problem with µ > µ⋆ are also local minima of the original problem.
  • Sufficient condition: For a Lipschitz objective with constant L and a minimum-attaining original program, any local minimum of the penalized problem is also a local minimum of the original when µ > L.The stated result applies when the equality constraint is expressed as distance to a closed set.
  • Extensions: The framework extends to arbitrary norms, parametric sets, and squared merit-function penalties.These merit functions generalize distance penalties while being easier to formulate and compute.
Loading 2101.01273v2…