Source-linked AI summary
The Maximum Principle for Global Solutions of Stochastic Stackelberg Differential Games
Alain Bensoussan, Shaokuan Chen, Suresh P. Sethi
TL;DR
The paper addresses maximum principles for stochastic global Stackelberg games, where leaders commit to strategies and followers respond optimally. It uses controlled forward-backward stochastic differential equations to derive principles for both open-loop and closed-loop cases; in LQ games, open-loop Riccati solvability is established under assumptions, while closed-loop existence remains open.
Problem
The paper studies how to characterize leaders’ optimal strategies in stochastic global Stackelberg games with open-loop and closed-loop information structures.
Source: §1 · pp. 2–3Method
The paper treats the follower’s response and the leader’s optimization through controlled forward-backward stochastic differential equations, applying controlled-FBSDE optimization results to derive maximum principles.
Source: §1 · p. 3, §4 · p. 7, §4.2 · p. 11Results
The paper derives maximum principles for both stochastic global game structures and establishes existence and uniqueness for the open-loop LQ Riccati solution under appropriate assumptions.
Source: §1 · p. 3, §5.1 · pp. 14, 17Takeaways & Limitations
The closed-loop LQ Riccati equation can be derived, but its solution’s existence and uniqueness are not established.
Source: §1 · p. 3, §5.2 · p. 21For the closed-loop LQ case, the associated nonlinear FBSDE’s solution existence and uniqueness remain unavailable; the leader’s problem is also assumed well-posed in the closed-loop analysis.
Source: §4.2 · p. 11, §5.2 · p. 21
Abstract
from arXiv · showhide
This paper obtains the maximum principle for both stochastic (global) open-loop and stochastic (global) closed-loop Stackelberg differential games. For the closed-loop case, we use the theory of controlled forward-backward stochastic differential equations to derive the maximum principle for the leader's optimal strategy. In the special case of the open-loop linear quadratic Stackelberg game, we consider the follower's Hamiltonian system as the leader's state equation, derive the related stochastic Riccati equation, and show the existence and uniqueness of the solution to the Riccati equation under appropriate assumptions. However, for the closed-loop linear quadratic Stackelberg game, we can write the related Riccati equation consisting of forward-backward stochastic differential equations, while leaving the existence of its solution as an open problem.
1 Introduction
The introduction distinguishes global open-loop and closed-loop Stackelberg games by their information structures and describes time inconsistency in both. It motivates stochastic maximum principles for these games and outlines the paper’s results, including an unresolved existence question for the closed-loop LQ Riccati equation.
1 Introduction
- Both global solution types have time inconsistency because the follower’s optimal response depends on the leader’s entire strategy over the game.The leader announces the entire strategy at the start, and the follower responds to it.Source: §1 · p. 2
- Open-loop and closed-loop global games differ by whether players’ information sets include state history; the paper treats both structures in the stochastic setting.The introduction contrasts this work with prior stochastic open-loop studies.Source: §1 · pp. 2–3
- Closed-loop strategies that incorporate state history make the follower’s response difficult to determine, motivating team and maximum-principle approaches.The maximum-principle approach restricts the leader’s strategy to initial and current state information; the follower’s adjoint then involves the state derivative of that strategy.Source: §1 · p. 2
- The paper derives maximum principles for stochastic global open-loop and closed-loop Stackelberg games, using controlled FBSDE results for the leader’s optimal strategies.It also discusses linear-quadratic games and their associated Riccati equations.Source: §1 · p. 3
- For the open-loop LQ case, the associated stochastic Riccati equation has a unique solution under assumptions; the closed-loop FBSDE Riccati equation’s existence is left open.The paper derives the closed-loop equation but does not investigate existence of its solution.Source: §1 · p. 3
2 Problem formulation and definition of equilibria
The formulation distinguishes global open-loop, global closed-loop, and feedback Stackelberg games by the players’ information sets. In the global games, the leader commits to a strategy and optimizes in anticipation of the follower’s response.
2 Problem formulation and definition of equilibria
- The paper identifies global open-loop, global closed-loop, and feedback Stackelberg games according to players’ information sets.The first two are global solutions, while feedback retains the Stackelberg property at each stage.Source: §2 · pp. 4–5
- In the open-loop game, the leader’s information is the initial state and Brownian filtration, and the follower minimizes cost against the leader’s full announced strategy.The leader then chooses a strategy minimizing its own cost along the follower’s optimal-response curve.Source: §2 · p. 4
- In the closed-loop game considered here, the leader’s strategy uses the initial and current state and the filtration, rather than the full state history.The follower chooses an optimal response to that leader strategy.Source: §2 · p. 4
- Unlike global open-loop or closed-loop games, feedback Stackelberg games give the leader an instantaneous advantage and pair its current action with the follower’s instantaneous response.The feedback solution is defined by a pair of strategies.Source: §2 · p. 5
3 Stochastic open-loop Stackelberg differential games
For the stochastic open-loop game, the follower’s optimal response is characterized through a maximum principle, after which the leader faces a control problem with a forward-backward system. A controlled-FBSDE maximum principle then gives a necessary condition for the leader’s optimal strategy.
3 Stochastic open-loop Stackelberg differential games
- Assuming a unique follower response for each leader strategy, the follower’s maximum principle characterizes that response through its Hamiltonian system.The follower minimizes its cost functional subject to the leader’s strategy.Source: §3 · p. 6
- The follower’s Hamiltonian condition is assumed to define its control uniquely as a function of time, state, leader control, and follower adjoint variable.Substituting this response yields the leader’s control problem.Source: §3 · p. 6
- The leader’s controlled-FBSDE maximum principle yields an optimality condition that minimizes the leader Hamiltonian over its control.For an optimal strategy, the paper gives u*(t) as an arg min of H1.Source: §3 · pp. 6–7
4 Stochastic closed-loop Stackelberg games
The closed-loop analysis handles the leader’s state-dependent strategy by converting the resulting nonclassical control problem into a classical one. In the stochastic case, the leader’s problem is a controlled FBSDE, yielding necessary maximum-principle conditions under stated well-posedness assumptions.
4 Stochastic closed-loop Stackelberg games
- State-history dependence makes the follower’s reaction difficult to determine explicitly, while the strategy derivative enters the adjoint equation and makes the leader’s problem nonclassical.Dynamic programming is unsuitable when the follower’s value function depends on the leader’s whole strategy function.Source: §4 · p. 7, §4.1 · pp. 8–9
4.1 The deterministic case revisited
- In the deterministic case, replacing the strategy derivative with a new control yields a classical problem with the same optimal trajectory and cost as the nonclassical problem.The maximum principle for the classical problem then gives conditions for the original leader problem.Source: §4.1 · pp. 8–9
4.2 The stochastic case
- The stochastic follower’s adjoint becomes a BSDE in the leader’s state system, producing a controlled FBSDE whose controls include the strategy and its state derivative.The derivation assumes a unique follower response and a unique FBSDE solution for each leader strategy.Source: §4.2 · pp. 9–11
- The stochastic closed-loop maximum principle requires the leader’s Hamiltonian to be minimized, with an adjoint triple satisfying the associated necessary conditions.The conditions apply when the leader’s problem has an optimal strategy and the controlled FBSDE state is well posed.Source: §4.2 · pp. 9, 11
5 The linear quadratic Stackelberg games
This section studies linear-quadratic Stackelberg games in open-loop and closed-loop settings, characterizing optimal strategies through coupled forward-backward stochastic systems. It derives related stochastic Riccati equations and discusses their solvability and connections to the leader’s Hamiltonian system.
5.2 The closed-loop case
- In the closed-loop case, the leader’s nonlinear Hamiltonian system precludes an exogenous Riccati equation; assuming the forward variable is linear in the state yields an exogenous FBSDE instead.With affine leader strategies and bounded strategy derivatives, the leader’s problem is well-posed.Source: §5 · pp. 12–13, §5.2 · p. 19
5.1 The open-loop case
- The open-loop leader’s problem is well-posed, and convexity and differentiability of its objective yield a unique optimal strategy.The optimal strategy is characterized by a necessary and sufficient condition involving the coupled FBSDE.Source: §5.1 · p. 14
- For each leader control, the coupled FBSDE has a unique solution, and the maximum-principle condition is necessary and sufficient for optimality.Source: §5.1 · pp. 14–15
- Under scalar and coefficient assumptions, a linear transformation connects the derived open-loop Riccati equation to a standard one and establishes unique solvability; the Stackelberg solution has a state-feedback representation.Source: §5.1 · pp. 16–18
5.2 The closed-loop case
- The authors derive a related Riccati equation for the closed-loop problem as a forward-backward stochastic differential equation system.This formulation instead treats x as the state and represents the other variables through the Riccati-system processes.Source: §5.2 · p. 22
- If the Riccati FBSDE has a solution and x* solves the stated linear SDE, the constructed processes satisfy the leader’s Hamiltonian system.The construction sets y = ξx*, p1 = ηx*, and p2 = ζx*, with corresponding q1 and q2 processes.Source: §5.2 · p. 22