PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimal steering of a linear stochastic system to a final probability distribution

Yongxin Chen, Tryphon Georgiou, Michele Pavon

arXiv:1408.2222v1eess.SYmath-ph

TL;DR

The paper asks how to steer a linear stochastic system between Gaussian endpoint distributions with minimum control energy, including when the diffusion may be degenerate. It constructs an explicit state-feedback solution through coupled differential Lyapunov equations and shows existence, uniqueness, and a minimum-relative-entropy Schrödinger-bridge interpretation.

  • Problem

    Existing Schrödinger-bridge methods are difficult to compute and primarily address nondegenerate diffusions, while many mechanical systems have degenerate noise.

  • Method

    The paper solves the Gaussian steering problem using state-feedback control derived from coupled differential Lyapunov equations with covariance-dependent boundary conditions.

  • Results

    Any initial Gaussian distribution can be steered to any final Gaussian distribution over finite time, with a unique optimal solution under controllability and positive endpoint covariances.

  • Takeaways & Limitations

    The optimal controlled process is a Schrödinger bridge whose law minimizes relative entropy to the uncontrolled process, extending that interpretation to possibly degenerate linear diffusions.

Abstract

from arXiv · show

We consider the problem to steer a linear dynamical system with full state observation from an initial gaussian distribution in state-space to a final one with minimum energy control. The system is stochastically driven through the control channels; an example for such a system is that of an inertial particle experiencing random "white noise" forcing. We show that a target probability distribution can always be achieved in finite time. The optimal control is given in state-feedback form and is computed explicitely by solving a pair of differential Lyapunov equations that are coupled through their boundary values. This result, given its attractive algorithmic nature, appears to have several potential applications such as to active control of nanomechanical systems and molecular cooling. The problem to steer a diffusion process between end-point marginals has a long history (Schrödinger bridges) and therefore, the present case of steering a linear stochastic system constitutes a Schrödinger bridge for possibly degenerate diffusions. Our results, however, provide the first implementable form of the optimal control for a general Gauss-Markov process. Illustrative examples of the optimal evolution and control for inertial particles and a stochastic oscillator are provided. A final result establishes directly the property of Schrödinger bridges as the most likely random evolution between given marginals to the present context of linear stochastic systems.

I. INTRODUCTION

The paper addresses optimal feedback control for stochastic mechanical systems, focusing on degenerate diffusions and the computational difficulty of existing Schrödinger-bridge solutions. It presents an explicit solution for finite-time Gaussian steering of linear systems.

  • Motivation: Feedback control is increasingly used for micro- and macro-mechanical systems, including nanodevice control, cooling, and stochastic oscillators.The introduction connects these applications to laser-driven reactions, quantum measurements, optical systems, resonant-bar detectors, and nanomechanical devices.
  • Motivation: Stochastic-oscillator cooling can be formulated as an optimal-control problem and related to Schrödinger bridges over diffusion processes.The paper notes that the infinite-time cooling problem corresponds to a special Schrödinger-bridge setting.
  • Open problem: Existing Schrödinger-bridge theory is limited for these applications because it assumes nondegenerate diffusion and generally requires coupled boundary-value partial differential equations.In stochastic oscillators, noise may act on only part of the phase-space state, while the general solution is not readily computational.
  • Contribution: The paper aims to provide a computable and implementable solution for Gauss-Markov processes, extending beyond prior discrete-time results that lacked existence and implementable control forms.The earlier discrete-time treatment also assumed nonsingular noise intensity.
  • Contribution: Any initial Gaussian distribution can be steered to any final Gaussian distribution over a finite interval using a unique minimum-energy state-feedback control.The control is obtained from two linear Lyapunov differential equations whose boundary values are nonlinearly coupled but have closed-form covariance-based expressions.
  • Paper scope: The paper derives the control construction, establishes the Schrödinger-bridge interpretation, and illustrates it with inertial particles and a thermally driven oscillator.The stated examples involve random white acceleration and Nyquist-Johnson thermal noise.

II. PROBLEM FORMULATION AND VARIATIONAL ANALYSIS

The problem seeks an adapted finite-energy control that transfers one zero-mean Gaussian endpoint distribution to another while minimizing control effort. Variational analysis yields state-feedback control, and the endpoint constraints reduce its computation to coupled matrix differential equations.

  • Problem formulation: The admissible controls are adapted finite-energy inputs that make the controlled state reach a prescribed zero-mean Gaussian target covariance at time T.The controlled dynamics use the same input channels for control and stochastic noise.
  • Problem formulation: Problem 1 minimizes the control-energy functional over all admissible controls achieving the desired endpoint probability-density transfer.The formulation asks first whether the admissible set is nonempty and then selects its minimum-energy element.
  • Problem formulation: The control authority can be extended when its input range contains the noise input range, whereas the case of weaker control authority remains an open direction.The paper explicitly identifies control authority below stochastic-noise authority as requiring further study.
  • Variational analysis: Completion of squares and Itō’s rule produce a candidate optimal control in state-feedback form based on a matrix Riccati equation.The Riccati solution must also make the controlled process attain the target density.
  • Variational analysis: The optimal controlled evolution remains a Gauss-Markov process because the feedback control depends on the current state.This condition is tied to selecting a Riccati solution that achieves the terminal density.
  • Variational analysis: Because both endpoint covariances are prescribed, the Riccati boundary value is replaced by split boundary conditions involving the evolving state covariance.The covariance and Riccati variables form a coupled system with boundary conditions at both endpoints.
  • Variational analysis: A change of variables converts the coupled Riccati equations into linear differential Lyapunov equations that remain nonlinearly coupled through their boundary values.The transformation is valid for nonsingular solutions and preserves the optimal-control construction.
  • Variational analysis: Solutions satisfying the transformed boundary conditions yield the optimal feedback and optimal Gauss-Markov evolution, provided the relevant matrices remain nonsingular.The subsequent result states that the Lyapunov system is the computational bottleneck before existence and uniqueness are established.

III. EXISTENCE AND UNIQUENESS OF OPTIMAL CONTROL FOR THE LINEAR GAUSSIAN BRIDGE

Under controllability and positive endpoint covariances, the bridge problem has a unique admissible solution obtained from coupled differential Lyapunov equations, while a second algebraic solution is non-admissible.

  • Construction: The coupled boundary-value problem consists of two differential Lyapunov equations whose boundary conditions are nonlinear and coupled.The equations arise after reducing the homogeneous Riccati equations through matrix inversion.
  • Existence and uniqueness: The admissible pair (P−(·), Q−(·)) is invertible throughout [0, T], ensuring a valid controlled covariance.Invertibility of P(t) and Q(t) is sufficient for the covariance construction.
  • Solution branches: Two solution pairs, indexed by − and +, satisfy the coupling conditions, but the + pair becomes singular and is therefore inadmissible.The − pair remains nonsingular, whereas Q+(t) and P+(t) become singular somewhere on the interval.
  • Numerical characterization: Starting from a generic Q(0), the numerical iteration converges to the − branch that generates the Schrödinger bridge.This branch also yields the least-cost evolution closest to the prior in relative entropy.
  • Solution branches: The alternative Riccati solution Π+(t)=Q+(t)−1 has a finite escape time.This behavior is consistent with the singularity of the + solution branch.
  • Existence and uniqueness: A controllable system with positive initial and target covariances admits a unique optimal solution over the finite interval.The theorem identifies the corresponding state-feedback control and least-cost bridge.

IV. MINIMUM RELATIVE ENTROPY INTERPRETATION OF OPTIMAL

The paper shows that the optimally controlled linear diffusion is the minimum-relative-entropy path law satisfying prescribed Gaussian endpoint marginals. This extends the Schrödinger-bridge interpretation to possibly degenerate linear diffusions by matching endpoint joints and pinned processes.

  • Schrödinger’s path-space problem reduces, after disintegration, to minimizing endpoint-joint relative entropy while retaining the prior’s conditional bridge laws.
  • The optimal law has the same reciprocal-class structure as the prior and is closest among path laws matching both endpoint marginals.
  • The optimally controlled and prior processes share identical pinned processes for every pair of endpoint states.
  • The controlled process minimizes relative entropy relative to the uncontrolled process among laws with prescribed Gaussian endpoint covariances.
  • Among distributions with a fixed covariance, the Gaussian uniquely minimizes relative entropy to a Gaussian reference.

V. ILLUSTRATIVE EXAMPLES

The paper illustrates optimal probability-density steering through two physical systems: inertial particles under random acceleration and an electrical circuit under resistor thermal noise.

  • Two examples demonstrate optimal probability-density steering in inertial particles and an electrical circuit driven by Nyquist-Johnson thermal noise.

A. Inertial particles

The inertial-particle example uses feedback control to compress a Gaussian phase-space distribution under random acceleration. Additional cases localize either position or velocity to compare directly and indirectly forced components.

  • A. Inertial particles: The model controls inertial particles experiencing random acceleration, with position x(t), velocity v(t), and control force u(t).
  • A. Inertial particles: The optimal feedback strategy squeezes the distribution from Σ0 = I at t = 0 to Σ1 = 1/4 I at t = 1.
  • A. Inertial particles: Figures 1 and 2 show the resulting phase-space sample trajectories and corresponding control inputs.
  • A. Inertial particles: Two additional cases use Σ1 = diag(.05, 1) and Σ1 = diag(1, .05) to localize position or velocity, respectively.

B. Nyquist-Johnson resistor noise

The circuit example models an RLC oscillator driven by resistor thermal noise and applies control voltage to reduce the noise-induced spread of current and capacitor voltage.

  • B. Nyquist-Johnson resistor noise: The circuit contains a resistor with a Nyquist-Johnson thermal-noise voltage source and uses R = L = C = 1 in compatible units.
  • B. Nyquist-Johnson resistor noise: Without active control, the RLC circuit reaches steady-state covariance 1/2 I for the state vector (iL, vC)′.
  • B. Nyquist-Johnson resistor noise: The experiment starts from Σ0 = 1/2 I and seeks a control-voltage input that reduces thermal-noise effects on the oscillator.
  • B. Nyquist-Johnson resistor noise: The corresponding control inputs are displayed in Figure 7.

VI. CONCLUDING REMARKS

The paper gives an explicit feedback-form solution for minimum-effort steering between Gaussian distributions and connects it to Schrödinger bridges, including degenerate diffusions. This formulation is positioned for active damping and cooling applications.

  • Minimum-energy steering has an explicit state-feedback solution computed from two Lyapunov equations coupled through endpoint boundary values.The controlled process also minimizes relative-entropy distance to the uncontrolled diffusion.
  • The resulting stochastic evolution is the most likely random path connecting the prescribed initial and final marginals under the uncontrolled diffusion prior.
  • The results extend corresponding minimum-energy and minimum-relative-entropy properties of classical Schrödinger bridges to possibly degenerate diffusions.
  • The explicit final-distribution control is presented as attractive for active damping of nanomechanical systems and cooling stochastic thermal fluctuations.
Loading 1408.2222v1…