PaperlayerView exact revision on arXiv

Source-linked AI summary

No Spurious Local Minima in Full-Order Linear Quadratic Gaussian Control

Yang Zheng, Yujie Tang

arXiv:2609.32875v1math.OCcs.SYeess.SY

TL;DR

Whether the nonconvex full-order LQG cost admits suboptimal local minima remains unresolved, despite the presence of suboptimal stationary points. The paper answers the question of whether the full-order LQG cost has a suboptimal local minimum negatively and characterizes the role of non-minimal controller realizations. Every full-order local minimum is globally optimal, and any suboptimal stationary point can be converted into a strict saddle with probability one.

  • Problem

    Whether the nonconvex full-order LQG cost admits suboptimal local minima remains unresolved, despite the presence of suboptimal stationary points.

    Source: §1 · pp. 1–2

  • Method

    The paper answers the question of whether the full-order LQG cost has a suboptimal local minimum negatively and characterizes the role of non-minimal controller realizations.

    Source: §1.1 · p. 2

  • Results

    Every full-order local minimum is globally optimal, and any suboptimal stationary point can be converted into a strict saddle with probability one.

    Source: §1.1 · p. 2, §5 · p. 21

  • Takeaways & Limitations

    At any controller order, local minima with uncontrollable or unobservable realizations attain the globally optimal LQG cost across controller orders.

    Source: §1.1 · p. 2

    The results assume R ≻ 0, V ≻ 0, Q ⪰ 0, W ⪰ 0, along with the stated controllability and observability conditions on the plant.

    Source: §2 · p. 6

Abstract

from arXiv · show

This paper studies the nonconvex optimization landscape of Linear Quadratic Gaussian (LQG) control under direct state-space parameterization. Although the LQG cost may possess suboptimal stationary points, whether it admits suboptimal local minima has remained open. We answer this question negatively: Every local minimum of the full-order LQG cost is globally optimal. More generally, at any controller order, every local minimum corresponding to an uncontrollable or unobservable realization attains the globally optimal LQG cost over all stabilizing policies of arbitrary orders. We further show that any suboptimal policy can be augmented with decoupled stable controller states, so that the augmented policy admits a direction of negative curvature with probability one. Consequently, any suboptimal stationary point in full-order LQG cost can be converted into a strict saddle with probability one. One key proof idea connects the state-space Hessian to a frequency-domain global-optimality condition derived from the Youla parameterization. These results reveal a benign local-minimum landscape despite the presence of suboptimal stationary points.

1 Introduction

Direct state-space policy optimization has renewed interest, but LQG’s dynamic policies yield a more complicated nonconvex landscape, leaving open whether full-order LQG has suboptimal local minima. The paper rules out such minima and connects state-space curvature to frequency-domain optimality.

1 Introduction

  • The open question is whether the full-order LQG cost can have a suboptimal local minimum, which could trap local search.The paper answers this question negatively.Source: §1 · p. 2

1.1 Our contributions

  • Every full-order LQG local minimum is globally optimal, while at any order, local minima with uncontrollable or unobservable realizations attain the global cost across controller orders.The results do not require minimality for the arbitrary-order guarantee.Source: §1.1 · p. 2

1.2 Related work

  • An order-independent Youla-based optimality condition links a transfer-matrix residual to the state-space Hessian, connecting global optimality with local curvature.The paper states that K is globally optimal if and only if RK(s) is identically zero.Source: §1.1 · p. 3, §1.2 · p. 4
  • The paper develops a sharper connection between frequency-domain and state-space viewpoints than prior work, including an optimality condition independent of controller order.Its proof uses centered Youla parameterization and an explicit Hessian identity at augmented policies.Source: §1.2 · p. 4

2 LQG Control and Problem Formulation

The paper formulates LQG control as minimizing a quadratic performance criterion over stabilizing dynamic policies of arbitrary order. Under stated plant assumptions, a globally optimal full-order controller is available from Riccati equations, although the full-order cost remains nonconvex.

2 LQG Control and Problem Formulation

  • A policy of order q has q internal controller states, and is full-order when q equals the system state dimension n.The controller is parameterized by its state-space matrices and acts on output measurements.Source: §2 · p. 5
  • The objective Jq is the squared H2 norm of the closed-loop performance transfer function and equals the policy’s linear quadratic performance.The optimization considers stabilizing policies and seeks the best dynamic policy over arbitrary orders.Source: §2 · pp. 5–6
  • Under Assumption 1, a globally optimal controller is given by the separation principle, combining optimal LQR and Kalman filter gains obtained from Riccati equations.The controller state is the optimal Kalman estimate of the system state.Source: §2 · p. 6
  • Despite the available full-order optimum, the full-order cost is highly nonconvex, and suboptimal stationary points must be uncontrollable or unobservable.Controllable and observable stationary policies are globally optimal.Source: §2 · p. 6

3 Main Results

The main results rule out suboptimal local minima in full-order LQG control and characterize non-minimal local minima across controller orders. They also show that stable-state augmentation can expose negative curvature at suboptimal policies and stationary points.

3.1 Global optimality

  • At any controller order, an uncontrollable or unobservable local minimum attains the globally optimal cost over stabilizing policies of every order.The result does not require the controller order to equal the plant state dimension.Source: §3.1 · p. 7
  • If a policy augmented by a decoupled stable state is a local minimum, the original policy is globally optimal.The added state leaves the controller input-output map and cost unchanged.Source: §3.1 · p. 7
  • Every local minimum of the full-order LQG cost is globally optimal.This combines the result for controllable and observable policies with the result for uncontrollable or unobservable realizations.Source: §3.1 · p. 7

3.2 Suboptimal stationary points as strict saddles

  • A first-order LQG problem can have a spurious local minimum even when its feasible realizations are minimal; adding one stable state turns the example into a strict saddle.The augmented example exhibits a negative smallest Hessian eigenvalue throughout the stated stable-pole range.Source: §3.1 · p. 9, §3.2 · p. 11
  • Any suboptimal policy can be augmented with stable states to yield a strict saddle for almost every choice of the appended pole.The result applies to policies of arbitrary order; the Hessian has directions of both positive and negative curvature.Source: §3.2 · p. 9
  • A suboptimal full-order stationary point can be re-realized as a strict saddle of the same order for almost every appended stable pole, without changing its cost.The construction starts from a minimal realization and restores the original order using decoupled stable states.Source: §3.2 · p. 10

4 Technical Proofs of the Main Results

The proofs analyze local minima and suboptimal policies using Kalman canonical form, augmented-controller Hessians, and a frequency-domain condition for global LQG optimality derived from the Youla parameterization. They connect that condition to the Hessian and transfer matrix R_K to support results on global optimality and negative curvature.

4.1 Local minima in the Kalman canonical form

  • Similarity transformations preserve local minimality, so any policy can be considered in Kalman canonical form, which separates controllable-observable states from the other state blocks.The canonical-form blocks corresponding to uncontrollable or unobservable states are Hurwitz stable at a stabilizing policy.Source: §4.1 · pp. 12–13
  • Uncontrollable or unobservable local minima in Kalman canonical form admit nearby perturbations that remain local minima with positive semidefinite Hessians.The perturbations preserve the LQG cost when stabilizing, and the resulting family includes policies in augmented form.Source: §4.1 · pp. 14–15

4.2 Hessian for local minima in the augmented form

  • For augmented policies, if the Hessian quadratic form is nonnegative or nonpositive over the specified perturbation space, it is zero throughout that space.The proof strategy uses an augmented policy with the same minimal realization and stable diagonal block λI, where λ < 0.Source: §4.2 · pp. 16–17

4.3 Proofs of Theorems 3.1 and 3.3

  • A policy K is globally optimal if and only if its frequency-domain transfer matrix R_K(s) is identically zero.This criterion is one of the technical ingredients used to connect the Hessian characterization to the main theorems.Source: §4.3 · p. 18
  • For any nonempty controller-order class, K is globally optimal exactly when R_K(s) ≡ 0, because the Youla parameterization makes the LQG objective a convex quadratic.The first-order condition at Q = 0 is equivalent to R_K(s) ≡ 0, and the criterion does not depend on policy order.Source: §4.3 · pp. 19–20
  • Theorem 4.2 relates the Hessian at K ⊕ λI_p to R_K through a direct specialization of a general Hessian formula.The simplification uses the augmented controller form, the diagonal structure λI_p, and a selected direction.Source: §4.3 · p. 19
  • Any local minimum of J_q with an uncontrollable or unobservable realization is globally optimal.The proof combines Proposition 4.1 with Theorems 4.1 and 4.2.Source: §4.3 · p. 19
  • For a suboptimal K, R_K is nonzero, and the proof shows that the exceptional choices of λ form a measure-zero set, yielding negative curvature with probability one.The argument selects a nonzero rational-function entry of R_K and uses the Hessian connection for an augmented controller.Source: §4.3 · p. 20
  • The frequency-domain criterion also gives an alternative proof of the LQG separation principle, since the Riccati-constructed optimal controller satisfies R_K(s) ≡ 0.The criterion avoids the nondegeneracy condition used by the cited global-optimality results.Source: §4.3 · p. 20

5 Conclusion

The paper concludes that LQG local minima have global-optimality guarantees in the stated settings, and that suboptimal full-order stationary points can be converted into strict saddles with probability one. It identifies robust or distributed control and local search using policy augmentation as future directions.

5 Conclusion

  • Every local minimum of the full-order LQG cost is globally optimal.Source: §5 · p. 21
  • At any controller order, local minima with uncontrollable or unobservable realizations attain the globally optimal cost over controllers of arbitrary orders.Source: §5 · p. 21
  • Every suboptimal stationary point of the full-order LQG cost can be converted into a strict saddle with probability one.Source: §5 · p. 21
  • Future directions include testing whether similar results hold for robust or distributed control and developing local search algorithms using policy augmentation.Source: §5 · p. 21
  • The authors report that ChatGPT 5.6 Sol found the initial proof, which they subsequently verified, substantially revised, and rewrote.The proof idea linked Hessian characterization [4] with frequency-domain characterization [15].Source: §5 · p. 21

A Youla Parameterization and Proof of Theorem 4.1

The proof uses continuous-time, real-rational frequency-domain representations and stable transfer-matrix spaces. These spaces support an orthogonal stable–antistable decomposition used in the analysis.

A Youla Parameterization and Proof of Theorem 4.1

A.1 Youla parameterization of all stabilizing policies

The section parameterizes stabilizing controller perturbations through a Youla parameter centered at a nominal stabilizing policy. This representation covers stabilizing policies of arbitrary orders while preserving the controller’s zero direct feedthrough constraint.

A.1 Youla parameterization of all stabilizing policies

  • The perturbation structure preserves zero direct feedthrough from measurement y to control input u.Source: §A.1 · pp. 25–26
  • The perturbation can be viewed either as part of the controller connected to the plant or as feedback around a centered plant that is internally stable under the nominal policy.Source: §A.1 · pp. 25–26
  • A static perturbation changes the controller realization without changing its order, while a dynamic perturbation with p states yields an order-(q + p) realization.Source: §A.1 · p. 25
  • A perturbation ∆ internally stabilizes the plant exactly when ∆(I − M22∆)−1 is stable, and this stable transfer matrix is the Youla parameter Q.Source: §A.1 · p. 26
  • The map from admissible internally stabilizing perturbations to Q is bijective, and every stabilizing policy of arbitrary order has a representation through some Q.Source: §A.1 · p. 27

A.2 Equivalent convex reformulation in the frequency domain

The frequency-domain reformulation expresses the cost as a convex quadratic functional of the Youla parameter. It yields a first-order condition at the nominal policy that characterizes global optimality.

A.2 Equivalent convex reformulation in the frequency domain

  • The closed-loop transfer matrix under Q is Tzd(Q) = M11 + M12QM21, and the reformulation evaluates its H2 cost.Source: §A.2 · pp. 27–28
  • The frequency-domain objective is a convex quadratic functional of Q, with Q = 0 corresponding to the nominal policy and its cost.Source: §A.2 · p. 28
  • A policy is globally optimal if and only if the first-order derivative of the Youla-parameterized objective vanishes at Q = 0.Source: §A.2 · p. 28
  • The first-order residual is expressed as the linear functional L_K(Q), and its vanishing condition is the frequency-domain global-optimality test.Source: §A.2 · p. 28

A.3 Proof of Theorem 4.1

The proof characterizes global optimality through the stable component R_K of Ψ_K: K is globally optimal exactly when R_K(s)≡0. It establishes the stable–antistable decomposition by state-space computation and handles constant Youla feedthrough terms separately.

A.3 Proof of Theorem 4.1

  • Ψ_K decomposes into strictly proper stable and antistable components; R_K is stable, while the other component has poles in the right half-plane.The antistable component's poles have positive real parts, whereas R_K's poles have negative real parts.Source: §A.3 · pp. 29, 32
  • Global optimality is equivalent to R_K(s)≡0, with the converse established over the full Youla space, including admissible feedthrough terms.Global optimality makes L_K vanish on stable test functions; conversely, R_K≡0 first gives vanishing on RH2, then on the full Youla space.Source: §A.3 · pp. 29–30
  • Stable–antistable orthogonality eliminates the antistable component when testing against stable transfer matrices, leaving R_K as the optimality condition.The proof extends this reasoning to constant feedthrough terms by approximation, since a nonzero constant transfer matrix is not in continuous-time L2.Source: §A.3 · pp. 29–30
  • A Lyapunov-based coordinate transformation separates the realization of Ψ_K into stable and antistable state blocks.The transformed blocks use the Lyapunov solutions X_K and Y_K.Source: §A.3 · pp. 30–32

B Hessian Computation and Proof of Theorem 4.2

This section develops a Hessian computation used in the proof of Theorem 4.2. It applies a Hessian lemma to an augmented policy and uses Lyapunov equations to obtain and simplify the resulting expression.

B Hessian Computation and Proof of Theorem 4.2

  • The Hessian computation uses Lemma B.1 [3, Lemma 9], with X_K, Y_K, and an auxiliary Lyapunov solution X'_{K,Δ}.The auxiliary matrix is specified as the solution to a Lyapunov equation.Source: B Hessian Computation and Proof of… · p. 32
  • For the augmented policy K⊕λI_p, λ<0 keeps A_cl,K Hurwitz, so the Lyapunov equations have unique solutions.The section gives the solutions in terms of those for policy K.Source: B Hessian Computation and Proof of… · pp. 32–33
  • Applying Lemma B.1 to the augmented policy yields the Hessian expression, which the proof then simplifies using a Lyapunov equation for X'_{K,Δ}.Only the first n+q columns of X'_{K,Δ} are needed, and the derivation uses the integral identity for a Hurwitz matrix.Source: B Hessian Computation and Proof of… · pp. 33–35

Related papers

Loading 2609.32875v1…