Source-linked AI summary
Linear Quadratic Stochastic Differential Games: Open-Loop and Closed-Loop Saddle Points
Jingrui Sun, Jiongmin Yong
TL;DR
The paper studies a stochastic linear-quadratic zero-sum game with controls entering the state dynamics and potentially indefinite or singular weights. It characterizes closed-loop saddle-point existence through a Riccati equation with regularity conditions, while showing that solvability alone may not suffice and solutions may be non-unique.
Problem
The paper addresses how to characterize closed-loop saddle-point existence in a stochastic linear-quadratic zero-sum game with controls in the state dynamics and weights that need not be definite or nonsingular.
Source: §1 · pp. 1–2Method
The paper characterizes closed-loop saddle points through solvability of a Riccati differential equation with specified regularity conditions.
Source: §5 · p. 16, §7 · p. 26Results
Riccati solvability alone is insufficient for a closed-loop saddle point, the required L2 regularity matters, and Riccati solutions may be non-unique.
Source: §6 · p. 23Takeaways & Limitations
The closed-loop existence characterization requires attention to Riccati-solution regularity, not just solvability.
Source: §6 · p. 23, §7 · p. 26The relationship between a closed-loop strategy outcome and the corresponding open-loop control is not established by the section’s comparison.
Source: §3 · p. 11
Abstract
from arXiv · showhide
In this paper, we consider a linear quadratic stochastic two-person zero-sum differential game. The controls for both players are allowed to appear in both drift and diffusion of the state equation. The weighting matrices in the performance functional are not assumed to be definite/non-singular. A necessary and sufficient condition for the existence of a closed-loop saddle point is established in terms of the solvability of a Riccati differential equation with certain regularity. It is possible that the closed-loop saddle point fails to exist, and at the same time, the corresponding Riccati equation admits a solution (which does not have needed regularity). Also, we will indicate that the solution of the Riccati equation may be non-unique.
1 Introduction
The paper studies a stochastic linear-quadratic zero-sum game in which both players’ controls may enter the state dynamics and the weighting matrices need not be definite or nonsingular. It places the problem among prior deterministic and stochastic game work and previews its open-loop and closed-loop analyses.
1 Introduction
- Both players’ controls may enter the state equation’s drift and diffusion, while the performance weights are not required to be definite or nonsingular.The state equation is posed on a filtered probability space with Brownian motion.Source: §1 · pp. 1–2
- The game assigns Player 1 the minimizing role and Player 2 the maximizing role for the quadratic performance functional.The paper studies stochastic games and relates the deterministic game to it as a special case.Source: §1 · p. 2
- The paper examines the stochastic game from both open-loop and closed-loop perspectives, including characterizations of saddle points.Its stated plan covers open-loop saddle points by a variational method and closed-loop saddle points through a Riccati equation.Source: §1 · p. 2
- The introduction situates the stochastic game alongside deterministic zero-sum games and stochastic and deterministic linear-quadratic optimal control problems.It identifies the deterministic game as a special case of the stochastic game and the optimal-control problems as related special cases.Source: §1 · p. 2
2 Preliminaries
The preliminaries establish the stochastic linear-quadratic optimal-control setting and introduce open-loop and closed-loop strategies. They state an open-loop optimality condition and a Riccati characterization for closed-loop optimal strategies.
2 Preliminaries
- The setup uses square-integrable controls and coefficient assumptions under which the state equation has a unique solution and the cost functional is well defined.The problem asks for a control minimizing that functional.Source: §2 · p. 4
- An open-loop optimal pair is characterized by a stationarity condition involving the state, control, and an adapted BSDE solution, together with a convexity condition.The stationarity relation is necessary and sufficient when the stated convexity condition holds.Source: §2 · pp. 4–5
- A closed-loop strategy is defined by state-feedback and offset terms that are independent of the initial state and satisfy the optimality inequality for every initial state and admissible control.The closed-loop outcome is an open-loop optimal control for each fixed initial pair.Source: §2 · pp. 5–6
- A closed-loop optimal strategy exists if and only if the stated Riccati equation admits a solution in C([t, T]; S^n).The theorem’s characterization also includes associated BSDE and range conditions.Source: §2 · p. 6
3 Stochastic Differential Games
This section formulates the two-player stochastic zero-sum game and distinguishes its open-loop and closed-loop saddle-point concepts. It also relates closed-loop saddle points to open-loop saddle points for a corresponding initial state, while warning that the strategy outcomes cannot simply be equated across the two formulations.
3 Stochastic Differential Games
- Under the coefficient assumptions, the stochastic state equation has a unique solution and the quadratic performance functional is well defined for admissible control pairs.Player 1 minimizes the payoff and Player 2 maximizes it in the open-loop saddle inequalities.Source: §3 · p. 8
- A closed-loop saddle point uses state-feedback strategies and offsets that are independent of the initial state, unlike an open-loop saddle point defined for a given initial pair.The closed-loop system substitutes each player’s feedback-and-offset control into the state equation.Source: §3 · pp. 8–9
- The outcome of a closed-loop saddle point is an open-loop saddle point for the corresponding initial pair (t, X*(t)).The paper distinguishes this relation from the stochastic optimal-control case.Source: §3 · p. 9
- The section reduces unilateral deviations against a fixed opponent strategy to corresponding stochastic linear-quadratic optimal-control problems.It then uses this connection in characterizing closed-loop saddle points.Source: §3 · pp. 10–11
- The authors caution that the closed-loop strategy outcome cannot be assumed to coincide with the relevant open-loop control.The comparison leaves their relationship undetermined.Source: §3 · p. 11
4 Open-Loop Saddle Points and FBSDEs
The section characterizes open-loop saddle points through forward-backward stochastic differential equations. Under the stated convexity-concavity conditions, the FBSDE conditions characterize saddle points and connect uniqueness of the saddle point with uniqueness of the adapted solution.
4 Open-Loop Saddle Points and FBSDEs
- An open-loop saddle point is characterized by stationarity conditions involving the state, controls, and adapted BSDE variables, together with convexity-concavity conditions.The section presents the characterization in terms of an FBSDE.Source: §4 · pp. 13, 15
- A unique open-loop saddle point is equivalent, under the stated convexity-concavity conditions, to a unique adapted solution of the associated FBSDE.The theorem states both directions of this uniqueness relationship.Source: §4 · p. 15
5 Closed-Loop Saddle Points and Riccati Equations
The section characterizes closed-loop saddle-point existence for Problem (SG) through a Riccati equation and associated conditions, and gives a necessary FBSDE and stationarity condition for any such saddle point. Although the Riccati equation may have non-unique solutions, under (SG1)–(SG2) it has at most one solution satisfying (5.4)–(5.6).
5 Closed-Loop Saddle Points and Riccati Equations
- Any closed-loop saddle point yields an adapted FBSDE solution satisfying a stationarity condition almost everywhere, almost surely.The stationarity condition is B^T Y* + D^T Z* + (S + RΘ*)X* + Rv* + ρ = 0.Source: §5 · pp. 15–16
- A closed-loop saddle point exists if and only if the Riccati equation and its associated semidefinite and BSDE range conditions hold.The inequalities require R11 + D1^T P D1 ≥ 0 and R22 + D2^T P D2 ≤ 0; the BSDE condition requires B^Tη + D^Tζ + D^TPσ + ρ to lie in the range of R + D^TPD.Source: §5 · pp. 16–17
- Under the theorem’s conditions, the closed-loop saddle point and the value function each admit a representation.The sufficiency proof concludes that the constructed strategy pair is a closed-loop saddle point.Source: §5 · pp. 17, 22
- Under (SG1)–(SG2), at most one continuous solution of (5.3) satisfies conditions (5.4)–(5.6).Such a solution is called regular.Source: §5 · p. 23
- The Riccati equation (5.3) may have non-unique solutions.An example is promised in Section 6.Source: §5 · p. 23
6 Some Examples
The examples separate Riccati solvability from the regularity needed for a closed-loop optimum, show that Riccati solutions can be non-unique, and distinguish closed-loop from open-loop saddle-point existence.
Example 6.1
- The problem has no closed-loop optimal control: its unique Riccati solution yields a control that is not in L2(0, 1; R).The example states that solvability condition (5.4) holds, but the resulting control fails the required L2 condition.Source: §6 · pp. 23–24
Example 6.2
- The Riccati equation has two solutions, yet the example’s optimal control exists and its closed-loop system is well-posed.The text identifies the two solutions and states that using P2 gives a well-posed closed-loop system.Source: §6 · pp. 24–25
Example 6.3
- A unique closed-loop saddle point, (-1, -1), exists, but the open-loop saddle point does not.The example establishes the closed-loop saddle point by Theorem 5.2 and then concludes that no open-loop saddle point exists.Source: §6 · p. 26
7 Concluding Remarks
The paper characterizes closed-loop saddle-point existence through a differential Riccati equation with certain regularity. Riccati solvability and random coefficients remain open problems, with the latter raising a BSDE formulation.
7 Concluding Remarks
- Closed-loop saddle-point existence is characterized by a solution to the differential Riccati equation with certain regularity.Source: §7 · p. 26
- The solvability of the Riccati equation remains unresolved, and more complete results are desirable.The authors note relevant results in and.Source: §7 · p. 26
- The random-coefficients case is left open; its corresponding Riccati equation should be a BSDE, as in for stochastic LQ control.Source: §7 · p. 26