PaperlayerView exact revision on arXiv

Source-linked AI summary

Optimal steering of a linear stochastic system to a final probability distribution, Part III

Yongxin Chen, Tryphon T. Georgiou, Michele Pavon

arXiv:1608.03622v1eess.SYcs.SYmath.OC

TL;DR

The paper addresses steering linear stochastic systems between endpoint distributions while extending earlier quadratic-input-cost formulations to include a quadratic state cost. It solves the problem through coupled Riccati equations, establishes a unique state-feedback solution under stated conditions, and relates the zero-noise limit to optimal mass transport.

  • Problem

    The paper addresses how to steer a linear stochastic system between prescribed endpoint distributions when the cost includes both quadratic input and state penalties.

    Source: §I · p. 1

  • Method

    It solves the distribution-steering problem using coupled matrix Riccati equations with nonlinear boundary conditions.

    Source: §I · p. 1, §II · p. 2

  • Results

    Under the stated conditions, the coupled Riccati system has a unique solution and an optimal state-feedback strategy exists.

    Source: §II · p. 2

  • Takeaways & Limitations

    In the zero-noise limit, the results give an optimal mass transport variant for particles with non-trivial dynamics and a quadratic action cost.

    Source: §V · pp. 5–6

    The analysis assumes noise and control enter through the same B matrix; the different-channel case remains open.

    Source: §V · p. 5

Abstract

from arXiv · show

The subject of this work has its roots in the so called Schroedginer Bridge Problem (SBP) which asks for the most likely distribution of Brownian particles in their passage between observed empirical marginal distributions at two distinct points in time. Renewed interest in this problem was sparked by a reformulation in the language of stochastic control. In earlier works, presented as Part I and Part II, we explored a generalization of the original SBP that amounts to optimal steering of linear stochastic dynamical systems between state-distributions, at two points in time, under full state feedback. In these works the cost was quadratic in the control input. The purpose of the present work is to detail the technical steps in extending the framework to the case where a quadratic cost in the state is also present. In the zero-noise limit, we obtain the solution of a (deterministic) mass transport problem with general quadratic cost.

I. INTRODUCTION

The introduction frames the Schrödinger Bridge Problem as steering a stochastic system between endpoint distributions, and describes its control interpretation as minimizing expected quadratic input cost. This paper extends earlier work by adding a quadratic state penalty to the steering problem.

I. INTRODUCTION

  • The Schrödinger Bridge Problem seeks the most likely evolution between empirical endpoint distributions and can be reformulated as stochastic control minimizing expected quadratic input cost.The control solution generates the process and path law sought in the original formulation.Source: §I · p. 1
  • Steering between prescribed state distributions is a form of “soft conditioning,” motivated by controls needing to handle a distribution of initial states rather than one state.The specification is on the state distribution, rather than on initial or terminal state values.Source: §I · p. 1
  • Earlier work treated linear dynamics with quadratic input cost, while the present work adds a quadratic state cost and gives a solution using coupled matrix Riccati equations.The introduction notes that prior state-cost approaches gave necessary conditions without establishing sufficiency.Source: §I · p. 1
  • The paper also studies the zero-noise limit as an optimal mass transport problem with losses and presents a numerical example.Source: §I · p. 1

II. MAIN RESULTS

The main results establish a unique solution to the coupled Riccati boundary-value problem under positive-definite endpoint covariances and uniform controllability. The proof constructs that solution through linear differential systems and identifies technical properties of the transition matrices that support it.

II. MAIN RESULTS

  • Under positive-definite endpoint covariances and uniform controllability, the coupled Riccati equations have a unique solution, yielding an optimal state-feedback strategy.The paper identifies existence under these conditions as a key contribution.Source: §II · pp. 1–2
  • For s < t, Φ12(t, s) and Φ11(t, s) are invertible, and (Φ12(t, 0)−1Φ11(t, 0))−1 decreases monotonically in the positive-definite sense.The related matrix T(t, s) is continuous and decreases with t, with left limit zero at t = s.Source: §II · pp. 2–3
  • The proof recasts the Riccati equations as linear differential systems, then determines their initial conditions through coupled boundary conditions and a symmetric matrix parameter.The parameterization leads to a quadratic equation for the symmetric matrix.Source: §II · p. 3
  • The Z− branch yields invertible X(t) and X̂(t) on [0, 1] and a well-defined Riccati solution, whereas Z+ leads to finite escape time.Source: §II · p. 3
  • When Q ≡ 0, the Riccati equations have a unique solution, recovering the earlier state-penalty-free result as a special case.Source: §II · p. 4

III. ZERO-NOISE LIMIT AND OMT WITH LOSSES

The zero-noise limit of the optimal steering problem yields an optimal mass transport problem with general quadratic cost. The section also gives a solution for zero-mean Gaussian marginals and extends the formulation to positive-definite input weights.

  • The zero-noise limit of the optimal steering problem is an optimal mass transport problem with general quadratic cost.For zero-mean Gaussian marginals, the section states the solution in Theorem 5.Source: §III · p. 4
  • As ϵ decreases to zero, the solution converges to the solution of the stated zero-noise problem.Source: §III · p. 4
  • For positive-definite input weights R(t), rescaling the control reduces the problem to (18), with feedback u(t, x) = −R(t)−1B(t)′Π(t)x.Π is specified through a Riccati equation.Source: §III · p. 5

IV. EXAMPLES

The example steers inertial particles between Gaussian marginals using the derived optimal feedback strategy. It illustrates how the state penalty and noise level affect the particle spread and paths.

  • The example targets inertial particles from Σ0 = 2I at t = 0 to Σ1 = 1/4I at t = 1 while minimizing the stated cost.The control is a force; the state contains particle position and velocity.Source: §IV · p. 5
  • With Q = I, typical phase-space paths use the derived optimal feedback, and the feedback gains and corresponding control actions are also shown.The transparent blue tube denotes the 3 σ tolerance interval.Source: §IV · p. 5
  • Increasing positive Q makes Σ(·) shrink faster, while negative Q makes it expand first as particles tend to stay away from the origin.The comparison uses Q = 10I and Q = −5I, respectively.Source: §IV · p. 5
  • For Q = I, paths at noise intensities ϵ = 10 and ϵ = 0.1 are reported to converge toward the zero-noise Problem 18 result.Source: §IV · p. 5

V. CONCLUSION

The paper extends distribution steering to include a quadratic state cost, with a same-channel assumption for noise and control. It connects the framework to relative-entropy and mass-transport interpretations while identifying limits on those connections.

  • The paper frames distribution steering as soft conditioning and as a precise variant of approximating terminal-distribution regulation through quadratic penalties.Source: §V · p. 5
  • Part III addresses a quadratic state cost, assumes noise and control enter through the same B matrix, and leaves the different-channel case open.Source: §V · p. 5
  • With the same noise and control channel, the controlled law has a relative-entropy interpretation, while differing B matrices make the relative entropy between laws infinite.The state penalty is related to creation or killing in the uncontrolled evolution in the Feynman-Kac sense,.Source: §V · p. 5
  • A separate relaxation penalizes Wasserstein distance to a target distribution; its cited work gives necessary conditions but no probabilistic or Bayesian interpretation at present.Source: §V · p. 5
  • The resulting mass-transport variant moves particles with non-trivial dynamics and uses a quadratic-Lagrangian action that need not be strictly convex in velocity.Source: §V · pp. 5–6
Loading 1608.03622v1…