Source-linked AI summary
Optimal steering of a linear stochastic system to a final probability distribution, part II
Yongxin Chen, Tryphon Georgiou, Michele Pavon
TL;DR
The paper addresses minimum-energy steering of linear stochastic systems to prescribed finite-horizon distributions and maintenance of stationary distributions over infinite horizons. It derives coupled Riccati conditions and feasibility characterizations, showing universal finite-horizon Gaussian steering under controllability but restricted stationary covariance assignability. It also formulates optimal controls through convex semidefinite programs and illustrates the approach with inertial particles.
Problem
The paper asks how to steer a linear stochastic system to a prescribed final distribution and maintain a stationary distribution with minimum energy.
Method
The paper uses coupled Riccati equations, Lyapunov-like covariance conditions, and convex semidefinite programs to characterize and compute optimal controls.
Results
Under controllability, any Gaussian distribution can be reached over a finite interval, whereas stationary Gaussian covariances require a Lyapunov-like equation and may not be maintainable by constant feedback.
Takeaways & Limitations
Finite-horizon covariance steering is broadly feasible under controllability, while stationary covariance maintenance must satisfy an admissibility condition.
Abstract
from arXiv · showhide
We consider the problem of minimum energy steering of a linear stochastic system to a final prescribed distribution over a finite horizon and to maintain a stationary distribution over an infinite horizon. We present sufficient conditions for optimality in terms of a system of dynamically coupled Riccati equations in the finite horizon case and algebraic in the stationary case. We then address the question of feasibility for both problems. For the finite-horizon case, provided the system is controllable, we prove that without any restriction on the directionality of the stochastic disturbance it is always possible to steer the state to any arbitrary Gaussian distribution over any specified finite time-interval. For the stationary infinite horizon case, it is not always possible to maintain the state at an arbitrary Gaussian distribution through constant state-feedback. It is shown that covariances of admissible stationary Gaussian distributions are characterized by a certain Lyapunov-like equation. We finally present an alternative to solving the system of coupled Riccati equations, by expressing the optimal controls in the form of solutions to (convex) semi-definite programs for both cases. We conclude with an example to steer the state covariance of the distribution of inertial particles to an admissible stationary Gaussian distribution over a finite interval, to be maintained at that stationary distribution thereafter by constant-gain state-feedback control.
I. INTRODUCTION
The paper studies minimum-energy control of linear stochastic systems for finite-horizon distribution steering and infinite-horizon stationary covariance maintenance. It establishes contrasting feasibility results and develops Riccati- and semidefinite-program-based approaches to optimal control.
- Finite-horizon covariance assignment is possible at the interval endpoint through suitable feedback control if and only if the system is controllable.
- Stationary covariance assignment through constant state feedback is restricted to positive semidefinite matrices satisfying a Lyapunov-like algebraic equation.
- The covariance equation for constant-feedback stationary distributions is also the equation characterizing covariances generated by colored stationary input noise in open loop.
- Sufficient optimality conditions use a Schrödinger-like system of dynamically coupled equations, yielding a feedback law and corresponding optimal evolution.
- The finite-horizon problem minimizes control effort among adapted finite-energy inputs that achieve the prescribed Gaussian endpoint distribution.
- Existence of solutions to the coupled system, and hence existence of a minimizer, is not proved, although admissible controls exist and costs can approach the infimum arbitrarily closely.
B. Infinite-horizon optimal steering
The infinite-horizon problem seeks a constant state-feedback law that maintains a stationary covariance while minimizing expected input power. Feasibility is not guaranteed, and optimality can be characterized through dual Riccati/variational conditions when a stabilizing solution exists.
- Problem formulation: The stationary problem minimizes expected input power over constant state-feedback laws that preserve an invariant probability density.The admissible feedback matrix must make A − BK Hurwitz.
- Feasibility: Not every positive covariance can be maintained by state feedback, and even feasible stationary problems may lack an optimal control.The paper distinguishes feasibility from existence of an optimizer.
- Optimization formulation: The problem admits a finite-dimensional reformulation over stabilizing feedback gains K, with a quadratic objective and a covariance constraint.The gain set consists of matrices for which A − BK is Hurwitz.
- Optimality conditions: A sufficient optimality condition is obtained by setting the Lagrangian directional derivative to zero and solving for a multiplier through the dual functional.The multiplier is computed as a maximizer of a concave dual functional.
- Optimality conditions: If a symmetric Π makes A − BB′Π Hurwitz and satisfies the stated algebraic conditions, the resulting feedback is the solution to the stationary problem.The optimal gain is K∗ = B′Π∗ in the variational characterization.
- Relation to Riccati theory: The paper also relates the stationary optimization to classical Willems results by expressing the objective in an equivalent quadratic form.That reformulation separates a term independent of K from the constrained covariance optimization.
- Relation to Riccati theory: Willems’ results connect the stabilizing multiplier to the maximal solution of a corresponding algebraic Riccati equation, although Π and Q need not be unique while K is unique.The correspondence applies when the original stationary problem has a solution.
III. CONTROLLABILITY OF STATE STATISTICS
The controllability analysis asks when controlled stochastic dynamics can reach a target Gaussian distribution over a finite interval or maintain a stationary Gaussian distribution through constant feedback. The discussion restricts attention to linear state-feedback controls, with stability required in the stationary case.
- Controllability questions: The analysis studies whether controlled evolution can reach a target Gaussian distribution over [0,T] or achieve a stationary Gaussian distribution by constant state feedback.These are the finite-horizon and stationary controllability questions for state statistics.
- Assumptions: The initial condition for the finite-horizon evolution is fixed almost surely at x(0) = x0.This specifies the starting state for the steering problem.
- Assumptions: The system matrices are time-invariant, the pair (A, B) is controllable, and the controls considered are linear functions of the state.For the stationary case, K is constant and A − BK must be Hurwitz.
A. Finite-interval steering by state-feedback
The finite-interval covariance dynamics are controllable exactly when (A, B) is controllable, and controllability permits smooth positive-definite steering between arbitrary positive-definite endpoint covariances. The construction uses transformed Lyapunov dynamics and reduces the general case to a shift-matrix system.
- The covariance dynamics remain within the positive semidefinite cone regardless of the choice of K(t).A feedback transformation relates U(t) and K(t) whenever Σ(t) > 0.
- The differential Lyapunov system is controllable if and only if the pair (A, B) is controllable.This establishes equivalence between state controllability and covariance-system controllability.
- Given positive-definite Σ0 and ΣT and any Q ≥0, a smooth input U(t) can steer the covariance between the endpoints while preserving positive definiteness.The covariance satisfies Σ(0) = Σ0, Σ(T) = ΣT, and Σ(t) > 0 throughout the interval.
- The proof constructs smooth controls inductively, including arbitrary endpoint values for U(t), while maintaining positive definiteness of Σ(t).The base case uses smooth interpolation, and the induction controls matrix blocks and selects the final scalar block sufficiently large.
- After a coordinate and feedback transformation, any controllable pair can be reduced to a shift-matrix, vector-input case covered by the constructive proof.Heymann’s lemma supplies K and v so that the transformed pair is controllable and equivalent to the shift-matrix form.
Finite-interval steering via external input:
For B = B1, covariance steering by state feedback is equivalent to representing the same covariance evolution through an external input process. The resulting process shares the state statistics of the feedback-controlled system.
- When B = B1, state-feedback steering of covariance can be interpreted as covariance changes produced by an external input process.The external process is chosen so that its covariance evolution satisfies the same differential Lyapunov equation.
- The constructed process ξ(t) shares the same statistics as the feedback-controlled state x(t).Both processes have covariance paths satisfying the same equation.
B. Assignability of stationary state covariances via state-feedback
Not every positive-definite covariance is maintainable at stationarity through constant state feedback. Under the stated range condition, admissible stationary covariances are exactly those satisfying equivalent Lyapunov-like conditions and admitting a stabilizing feedback gain.
- A stationary covariance must satisfy an algebraic Lyapunov equation for some Hurwitz closed-loop matrix A − BK.Hurwitz stability is necessary for the state process to be stationary.
- A feedback gain can be recovered from a solution X of the covariance condition as K = −X′Σ−1, provided the resulting closed loop is Hurwitz.The required Hurwitz property is guaranteed when (A − BK, B1) is controllable, which follows from R(B) ⊆ R(B1).
- The stationary covariance conditions are necessary and sufficient under R(B) ⊆ R(B1).Theorem 4 characterizes positive-definite matrices assignable through suitable state feedback by equivalent statements.
- The admissible stationary-covariance condition also coincides with the characterization for covariances generated by suitable stationary stochastic inputs in the related open-loop model.This correspondence is stated for the equivalent conditions and, in the special case B = B1, connects to prior colored-noise results.
Assignability via external input:
The stationary covariance characterization for state feedback also describes which covariances can arise from a stationary external input process in the corresponding Gauss–Markov model.
- For controllable (A, B) with A Hurwitz, admissible covariances generated by stationary input processes satisfy the same characterization as feedback-assignable covariances.The paper identifies this condition with the condition in Theorem 4.
- A feedback-based implementation can be constructed separately to generate an input process whose stationary state covariance equals the target Σ.The constructed process ξ(t) has the same stationary statistics as x(t), and S = Σ satisfies the covariance equations.
IV. NUMERICAL COMPUTATION OF OPTIMAL CONTROL
The paper formulates finite- and infinite-horizon optimal-control computations as semidefinite programs, providing alternatives to solving coupled Riccati equations.
- Semidefinite programs are formulated as alternatives to solving the generalized Schrödinger system for finite-horizon and infinite-horizon optimal controls.
A. Finite interval minimum energy steering of state statistics
Finite-horizon steering is optimized over feedback gains or equivalent covariance-control variables, yielding a convex semidefinite program whose discretized solution recovers a suboptimal gain.
- The finite-horizon problem seeks a feedback gain K(t) that steers Σ0 to ΣT while minimizing expected control energy.
- The covariance dynamics become linear in U(t) and Σ(t), enabling formulation as a semidefinite program.
- After time discretization, the semidefinite program can be solved numerically and a suboptimal gain recovered as K(t) = −U(t)′Σ(t)−1.
- Stationary admissibility requires a positive definite Σ satisfying the stated Lyapunov-like condition and producing a Hurwitz closed loop.
- When R(B)̸ ⊆R(B1), stability must be checked separately; otherwise, the method guarantees only a covariance arbitrarily close to the target.
V. EXAMPLE
The inertial-particle example demonstrates steering an initial Gaussian covariance to an admissible stationary covariance, then maintaining it with constant feedback despite directional disturbance limitations.
- The example models position and velocity dynamics where disturbance directly affects position, while control reaches position only after integration.
- Although R(B)̸ ⊆R(B1), the candidate stationary covariance yields K = [1, 1] and a Hurwitz closed-loop matrix.
- Starting from Σ0 = 2I, the particles are steered to terminal covariance Σ1 at t = 1 and maintained there by constant state feedback.
- Figures 1 and 2 show optimal phase-space trajectories and time-varying feedback gains during the interval [0, 1].
- After t = 1, Figure 3 shows trajectories under K = [1, 1], while Figure 4 shows control actions across transient and stationary intervals.
VI. APPENDIX
The appendix establishes a range–null-space relationship for maps associated with the input matrix, supporting the algebraic characterization used in the control analysis.
- Lemma 6 states that the range of fB coincides with the null space of gB.
- The proof reduces the orthogonal-complement relation to showing that trace(ZX) = 0 for all X implies Z = 0.
- Self-adjointness of gB completes the equality between the relevant subspaces.