PaperlayerView exact revision on arXiv

Source-linked AI summary

Control of Markov Jump Linear Systems with Uncertain Lumpable Cluster Observations

Carlos A. F. Persiani, Ram Padmanabhan, Melkior Ornik, Marco H. Terra

arXiv:2610.02573v1math.OCcs.SYeess.SY

TL;DR

When a controller observes only a cluster containing the true mode, both the system dynamics and cluster-transition probabilities can be uncertain. The paper formulates robust regulation as a min-max problem and derives a cluster-dependent feedback law through a Riccati recursion. The recursion is shown to converge, and the resulting feedback is shown to yield mean-square stability under the modeled uncertainty.

  • Problem

    Cluster-only observations leave the true mode unknown and can make transitions between clusters uncertain, a source of uncertainty existing cluster-observation approaches do not account for.

    Source: §I · p. 1

  • Method

    The paper formulates min-max robust regulation and derives a closed-form cluster-dependent feedback gain using a structured uncertainty relaxation and Riccati recursion.

    Source: §I · p. 2, §IV · p. 4

  • Results

    The Riccati recursion converges to a finite set of cluster value matrices, and the resulting feedback is mean-square stable under the modeled uncertainty.

    Source: A. Convergence · p. 5, B. Stability · p. 5

  • Takeaways & Limitations

    The approach provides a cluster-dependent controller with closed-loop stability guarantees for the stated uncertain setting.

    Source: §I · p. 2, B. Stability · p. 5, §VII · p. 6

    Constructing the uncertain cluster-transition matrix from the original Markov chain and relaxing lumpability requirements are left for future work.

    Source: §VII · p. 6

Abstract

from arXiv · show

In this paper, we consider the control of Markov jump linear systems when the active system mode is not exactly known. We rather assume that this true mode is only known to belong to an observed cluster of modes, making both the dynamics and the associated transition probabilities uncertain. We construct a min-max optimal control problem in this setting, with the objective of robust regulation. Using results in regularized least mean-square optimization, we describe how the solution of this problem can be reduced to a Riccati recursion, such that the controller depends only on knowledge of which cluster the mode lies in. We subsequently establish convergence of the recursion and stability of the resulting closed-loop system. A numerical example illustrates the effectiveness of the proposed methodology.

I. Introduction

Markov jump linear systems are often studied with perfect mode information, but practical observations may reveal only a cluster containing the true mode. When lumpability does not hold, transitions between observed clusters can also be uncertain; this paper develops robust control for that setting.

I. Introduction

  • Prior cluster-observation approaches include stabilizing H2 control and formulations for detector-based observations, H∞ performance, unknown transition probabilities, and randomly timed observations –.The paper identifies their omission of naturally arising uncertainty in the stochastic transition matrix of the clusters as the gap it addresses.Source: §I · p. 1

A. System Dynamics and Clustering

  • When lumpability fails, a cluster-to-cluster transition can depend on the particular mode within the current cluster, making it uncertain to a cluster-only observer.Strong lumpability instead gives identical transition probabilities from modes in a cluster toward every other cluster.Source: §I · p. 1, A. System Dynamics and Clustering · p. 2

I. Introduction

  • The paper models both the unknown true mode within an observed cluster and uncertainty in transitions between clusters, then formulates robust optimal control for these uncertainties.The uncertain effects maximize a quadratic objective while the control minimizes it; the resulting solution is a cluster-dependent control law based on coupled Riccati equations.Source: §I · pp. 1–2

A. System Dynamics and Clustering

  • The model assumes the true mode belongs to an observed cluster and represents cluster transitions with an uncertain stochastic matrix.For a given current cluster, the transition probability toward a next cluster depends on the unknown true mode within it.Source: A. System Dynamics and Clustering · p. 2

B. Control Objective

The control objective penalizes the possible future state evolutions under uncertain modes using lifted dynamics. It is posed as a finite-horizon robust optimization and analyzed with a backward Bellman recursion, with mean-square stability as a closed-loop property of interest.

B. Control Objective

  • Because future modes are unknown, the lifted state collects the evolution associated with all true modes, and the objective weights that lifted state to account for every possible future evolution.The lifted dynamics extract the state evolution associated with each mode from the combined lifted state.Source: B. Control Objective · pp. 2–3
  • The paper poses finite-horizon robust optimal control under the lifted dynamics, with uncertain effects maximizing a quadratic cost and the control minimizing it.The Bellman equation gives a one-step optimization solved in a backward recursion.Source: B. Control Objective · p. 3
  • The uncertain output matrices depend jointly on the unknown true mode within the current cluster, so exact worst-case optimization would maximize over its admissible modes.The paper embeds these uncertainties in structured norm-bounded models to obtain a tractable robust relaxation and closed-form solution.Source: B. Control Objective · p. 3
  • The stability analysis uses mean-square stability, requiring expected squared state norm to approach zero for every initial condition and admissible uncertainty.The paper seeks convergence to a cluster-dependent stationary solution to enable closed-loop stability analysis.Source: B. Control Objective · p. 3

III. Auxiliary Lemmas

Auxiliary results in regularized least-mean-square optimization support a closed-form solution to the robust control problem. They address uncertainty in data and regularization weighting, and also provide a bound for quadratic maximization under norm-bounded uncertainty.

III. Auxiliary Lemmas

  • The auxiliary optimization treats a known matrix A, measurement b, and minimization variable x, with uncertainty in the data and regularization weighting matrix.The weighting matrices S and W are positive definite, and the modeled uncertainties are norm-bounded.Source: §III · p. 3
  • Lemma III.1 gives a unique optimal solution and corresponding optimal cost for the stated min-max regularized least-mean-square problem.Its parameter λ★ is selected by minimizing Γ(λ) over λ greater than the specified norm bound; the paper notes a practical approximation near that lower bound.Source: §III · p. 3
  • Lemma III.3 bounds the maximization of a quadratic function when the matrix uncertainty is norm-bounded by a contraction.The result is used in developing the stability analysis.Source: §III · p. 3

IV. Robust Recursive Regulation

The section relaxes the uncertain cluster-transition and mode-dependent quantities into structured norm-bounded uncertainty and uses quadratic penalties to obtain a tractable recursive optimization. Its closed-form solution yields cluster-dependent feedback, with the exact-dynamics penalty taken to infinity while the mode-uncertainty penalty remains finite.

IV. Robust Recursive Regulation

  • Quadratic penalty terms incorporate the dynamic constraints into an unconstrained one-step min-max problem for recursive solution.The section introduces positive penalty parameters μ1 and μ2.Source: §IV · p. 4
  • The relaxed uncertainty model enables a closed-form minimizing solution and cluster-dependent feedback by applying the quadratic min-max result to the recursive optimization.The uncertainty description relaxes the original vertex constraints; the recursion uses μ1 → ∞ for an exact relation while keeping μ2 > 0 finite for a relation affected by mode uncertainty.Source: §IV · pp. 3–4
  • The finite-horizon recursion is indexed backward, then reindexed for computation before deriving the cluster-dependent feedback gain.The backward indices encode successive iterates; the reversed ordering is a computational representation.Source: §IV · p. 4
  • The robust-maximization coefficients in Ψ are not valid transition probabilities; Ψ serves as a robust upper-bound weighting in the recursion.Their sum is not constrained to equal one.Source: §IV · pp. 4–5

V. Convergence and Stability 

The paper establishes convergence of the coupled Riccati recursion to stationary solutions for each cluster and uses this property to analyze closed-loop stability.

V. Convergence and Stability

  • The coupled Riccati recursion converges to a stationary solution for each observed cluster.The resulting stationary cluster-dependent control law is then used in the closed-loop stability analysis.Source: §V · p. 5
  • The stationary Riccati solutions provide the basis for analyzing stability under the resulting cluster-dependent control law.The passage states that stability is analyzed after establishing convergence.Source: §V · p. 5
  • The convergence and stability analysis concerns the coupled Riccati recursion and its associated stationary control law.The stationary solution is specified for each cluster.Source: §V · p. 5

A. Convergence

The Riccati sequences are shown to converge under stated conditions, without an additional stabilizability assumption. The paper then gives conditions for mean-square stability under the resulting cluster-dependent feedback, while noting that the stability conditions may be conservative.

A. Convergence

  • Starting from zero, each cluster's Riccati sequence is monotonically non-decreasing, bounded above, and converges to a finite limiting matrix.The proposition assumes fixed positive-definite weighting matrices, μ2 > 0, and λ_l > μ2.Source: A. Convergence · p. 5
  • No additional stabilizability assumption is required to establish convergence of the Riccati sequence.The paper proceeds from this convergence result to analyze closed-loop stability.Source: A. Convergence · p. 5

B. Stability

  • A coupled quadratic Lyapunov function and the converged Riccati equation are used to derive conditions for mean-square stability under cluster-dependent feedback.The analysis upper-bounds the Lyapunov difference under admissible uncertainties and establishes conditions for that bound to be negative definite.Source: B. Stability · p. 5
  • The closed-loop system is mean-square stable under the Riccati-derived feedback when the stated conditions and uncertainty model hold.The theorem assumes fixed positive-definite weighting matrices, μ2 > 0, and λ_l > μ2; the authors caution that its conditions may not always be satisfied and may be conservative.Source: B. Stability · p. 5

VI. Example Revisited

The example compares the proposed controller with two baselines as fault-mode dynamics degrade, while the conclusion summarizes the Riccati-based robust-control solution and its guarantees. The paper also identifies construction of uncertain cluster transitions and relaxation of lumpability as future work.

VI. Example Revisited

  • The example models partial actuator loss in fault modes and scales their state matrices by s ≥ 1 to simulate increasing deviation from nominal dynamics.The controllers use identity state and control weights and are compared without external disturbances.Source: §VI · pp. 5–6
  • As degradation increases, the proposed controller has a less severe transient response than the comparison controllers; at s = 1.00, their closed-loop behavior is similar.Figure 2 compares state norms under the same mode realization and initial condition for s = {1.00, 1.10, 1.19}.Source: §VI · p. 6, Figure 2 · p. 6
  • The min-max formulation accounts for worst-case mode transitions and dynamics; its conservatism becomes beneficial as discrepancies among modes within a cluster increase.The resulting closed-loop controller is implemented by directly evaluating the convergent Riccati recursion, without a general-purpose optimization solver.Source: §VI · p. 6

VII. Conclusion

  • The paper formulates control with lumpable cluster observations as min-max quadratic optimization, then derives a closed-form solution based on a Riccati recursion.The approach includes a lemma for the optimization problem and a numerical example with benchmark comparisons.Source: §VII · p. 6
  • The authors establish convergence and stability guarantees for the solution, and identify constructing uncertain cluster transitions from the original Markov chain and relaxing lumpability as future work.Source: §VII · p. 6

Appendix I

The appendix proofs reduce uncertainty maximizations to tractable quadratic forms, then use those results to derive the Riccati recursion and feedback gain. They do so through separate lemmas for the optimization steps and a derivation for the control theorem.

A. Proof of Lemma III.1

  • For Lemma III.1, bounding the uncertainty term reduces the maximization to a formulation solved using results from,.The proof obtains the optimizer and optimum cost before simplifying to the lemma’s result.Source: A. Proof of Lemma III.1 · p. 6

B. Proof of Lemma III.3

  • Lemma III.3 uses the contraction condition and square completion with a positive matrix to define W_λ and obtain the stated result.The proof starts from ΔᵀΔ − I ⪯ 0 and applies a positive scalar λ.Source: B. Proof of Lemma III.3 · p. 6

A. Proof of Theorem IV.1

  • Theorem IV.1’s proof incorporates transition-probability uncertainty into modified weighting matrices, then collects the quadratic-form terms to obtain the Riccati recursion and feedback gain.The gain follows from relation (i) of Lemma III.1.Source: A. Proof of Theorem IV.1 · p. 7

A. Proof of Proposition V.1

The proof establishes convergence of the Riccati recursion by showing that its matrix sequence is monotone and bounded above. The argument proceeds from initialization through an induction step to convergence.

A. Proof of Proposition V.1

C. Proof of Theorem V.3

The proof uses a Lyapunov upper bound satisfied by the convergent Riccati solution to establish a closed-loop stability result. Positive state and control weights support the decrease argument under admissible uncertainty.

C. Proof of Theorem V.3

  • The convergent Riccati solution satisfies the Lyapunov upper bound established in Lemma V.2.Source: C. Proof of Theorem V.3 · p. 8
  • With positive Q_l and R_l, the Lyapunov difference is negative for nonzero states, yielding mean-square stability of the closed-loop system.Source: C. Proof of Theorem V.3 · p. 8
Loading 2610.02573v1…