Source-linked AI summary
Linear-Quadratic $N$-person and Mean-Field Games with Ergodic Cost
Martino Bardi, Fabio S. Priuli
TL;DR
The paper asks how to characterize affine Nash equilibria and connect their distributions and strategies to a Mean Field Game limit in ergodic linear-quadratic games. It uses HJB-KFP systems and reduces quadratic-Gaussian solutions to matrix equations; under stated conditions, the large-population system has a unique quadratic-Gaussian solution. The paper also identifies limits on convergence-rate conclusions in the general case.
Problem
The paper studies how to synthesize Nash feedback strategies and equilibrium distributions for an N-player game, and how these behave as N tends to infinity in connection with Mean Field Game equations.
Source: §1 · p. 2Method
The authors seek quadratic value functions and Gaussian distributions in a reduced HJB-KFP system, reducing the solution conditions to algebraic Riccati and Sylvester equations.
Source: §1 · p. 2Results
For nearly identical players, the large-population limit yields a unique quadratic-Gaussian solution of the Mean Field Game HJB-KFP equations under the stated condition on a cost-matrix subblock.
Source: §1 · p. 2Takeaways & Limitations
The matrix equations provide conditions for existence and uniqueness and can be used to solve examples explicitly or numerically.
Source: §1 · p. 2In the general case, the analysis does not establish a convergence rate for ΣN to Σ, although an estimate is available for examples with explicit Riccati solutions.
Source: §5 · p. 24
Abstract
from arXiv · showhide
We consider stochastic differential games with $N$ players, linear-Gaussian dynamics in arbitrary state-space dimension, and long-time-average cost with quadratic running cost. Admissible controls are feedbacks for which the system is ergodic. We first study the existence of affine Nash equilibria by means of an associated system of $N$ Hamilton-Jacobi-Bellman and $N$ Kolmogorov-Fokker-Planck partial differential equations. We give necessary and sufficient conditions for the existence and uniqueness of quadratic-Gaussian solutions in terms of the solvability of suitable algebraic Riccati and Sylvester equations. Under a symmetry condition on the running costs and for nearly identical players we study the large population limit, $N$ tending to infinity, and find a unique quadratic-Gaussian solution of the pair of Mean Field Game HJB-KFP equations. Examples of explicit solutions are given, in particular for consensus problems.
1 Introduction
The paper studies affine Nash equilibria for ergodic-cost linear-quadratic games and their large-population limit. It derives algebraic conditions for quadratic-Gaussian solutions, including a unique mean-field solution under stated conditions and explicit examples such as consensus games.
1 Introduction
- For the N-player game, quadratic value functions and Gaussian invariant distributions yield affine Nash feedbacks, with existence and uniqueness characterized by algebraic Riccati and Sylvester equations.The approach uses a reduced HJB-KFP system rather than the strongly coupled N-player HJB system.Source: §1 · p. 2
- For nearly identical players, the large-population limit gives a unique quadratic-Gaussian solution of the Mean Field Game HJB-KFP system.Uniqueness among general solutions also holds under a condition on a cost-matrix subblock that means imitation is not rewarding in the limit.Source: §1 · p. 2
- Explicit formulas are obtained when diffusion and control-cost matrices are scalar multiples of the identity and the drift is symmetric or non-defective.For state dimension d > 1, the associated Riccati and Sylvester equations generally do not admit explicit solutions.Source: §1 · p. 3
- In consensus problems, the pairwise quadratic cost favors nearby player states, and a quadratic-Gaussian solution exists for each mean μ satisfying A_iμ = 0.Because imitation is rewarding in this setting, there can be infinitely many such solutions and corresponding Nash equilibria.Source: §1 · p. 3
- The paper also situates its approach among prior large-population game methods, including Nash certainty equivalence and the Lasry-Lions approach to Mean Field Games.The cited prior work includes ergodic-cost studies with different assumptions and methods.Source: §1 · p. 3
2 Preliminaries
The preliminaries establish matrix facts and admissibility criteria, then set up the Riccati framework and the reduced HJB-KFP system used for the N-player analysis.
2.1 Some properties of symmetric matrices
- The section records matrix-order and spectral facts used later, including continuity of eigenvalues and properties of products and norms of symmetric matrices.Positive semidefiniteness and positive definiteness are defined through quadratic forms.Source: §2.1 · p. 4
2.2 Admissible strategies
- An affine feedback is admissible when its closed-loop drift matrix has eigenvalues with negative real parts, yielding a unique ergodic Gaussian invariant measure.Its mean is determined by the closed-loop drift and offset, and its covariance solves a Sylvester equation.Source: §2.2 · pp. 5–6
2.3 Algebraic Riccati equations
- The Riccati preliminaries characterize solutions through invariant subspaces of an associated matrix and give existence and uniqueness properties under spectral conditions.When Q is positive definite, the relevant solution is also the unique symmetric positive definite solution.Source: §2.3 · pp. 6–7
2.4 HJB and KFP equations associated to the N–person game
- The N-player HJB-KFP system is obtained by assuming each value function depends only on its own state and integrating over the other players’ invariant distributions.This reduces the strong gradient coupling of the classical N-player HJB equations; the stated standing assumptions include invertible diffusion matrices and positive definite control-cost matrices.Source: §2.4 · pp. 7–8
3 N–person game
The N-player HJB–KFP system admits quadratic value functions and Gaussian invariant measures exactly under stated matrix-solvability conditions. When the additional uniqueness condition holds, the resulting affine feedbacks form a Nash equilibrium for every initial position.
3 N–person game
- Quadratic value functions and Gaussian measures solve the HJB–KFP system if and only if condition (E) holds.The Gaussian precision matrices satisfy Riccati equations, with further Sylvester and linear-system conditions in (E).Source: §3 · pp. 9, 13
- When the conditions hold, the affine gradient feedbacks form a Nash equilibrium for all initial positions, with each player's ergodic cost equal to λi.The equilibrium verification uses admissibility and ergodicity.Source: §3 · pp. 9, 14
- The Riccati equations determine the Gaussian precision matrices, while a linear system determines the players' means.Condition (E) supplies the Sylvester compatibility and rank condition needed to construct a solution.Source: §3 · pp. 12–13
- The quadratic-Gaussian solutions are unique if and only if condition (U) holds, which requires the matrix B to be invertible.The positive definite Riccati solutions are already unique under the standing hypotheses.Source: §3 · pp. 9, 12, 13
- The equilibrium feedbacks depend on the transformed Riccati variables rather than the noise statistics, allowing a small-noise limit under additional conditions.The paper states that the same noise-independence holds for the nearly identical-player equilibrium.Source: §3 · p. 15
4 Nearly identical players
For nearly identical players, a common quadratic value function and Gaussian distribution solve the HJB–KFP system under condition (E′). Under condition (U′), the solutions are unique and their common affine feedback is a Nash equilibrium.
4 Nearly identical players
- The players share specified dynamics, control costs, reference positions, and primary displacement costs, but their secondary displacement costs may differ.The cost symmetry condition requires each player to be influenced in the same way by the others.Source: §4 · pp. 16–17
- For nearly identical players, common quadratic value functions and Gaussian measures solve the HJB–KFP system if and only if condition (E′) holds.The shared covariance precision is the unique positive definite solution of a Riccati equation.Source: §4 · p. 18
- The common solutions are unique if and only if condition (U′) holds, and the resulting affine feedbacks form a Nash equilibrium for all initial positions.Under (U′), all players have the same feedback and invariant distribution, though their values may differ.Source: §4 · p. 19
- Differences in the secondary cost terms affect the game values λi but leave the common distribution and value function unchanged.The authors identify this result as supporting the existence of a large-population limit.Source: §4 · p. 19
5 The large population limit
The paper characterizes quadratic-Gaussian solutions of the mean-field HJB–KFP system and proves convergence of finite-player solutions under specified conditions. It also gives a criterion for uniqueness among more general solutions and identifies limits of its convergence analysis.
5 The large population limit
- The Riccati and Sylvester equations reduce the quadratic-Gaussian PDE problem to matrix solvability conditions.The equations also provide a route to explicit or numerical solutions.Source: §5 · p. 20
- Under (45), quadratic-Gaussian mean-field solutions exist exactly when (E∞) holds, and are unique within that form exactly when (U∞) holds.The covariance is determined by the Riccati equation, while the mean solves a linear system.Source: §5 · p. 21
- When the finite-player and limit conditions hold, the nearly identical players’ quadratic-Gaussian solutions converge to the mean-field solution as N tends to infinity.The value functions, invariant densities, and ergodic constants converge in the stated senses.Source: §5 · p. 21
- If ˆB is positive semidefinite, the mean-field solution is unique among solutions satisfying the stated moment, growth, and normalization conditions.The paper relates this condition to monotonicity of ˆV.Source: §5 · p. 21
- The general convergence analysis does not establish a convergence rate, and the paper cautions that its limit assumptions may not ensure the finite-player condition (E′).A rate estimate is obtained in examples with explicit Riccati solutions.Source: §5 · p. 24
6 Examples
The examples make the matrix conditions for quadratic-Gaussian solutions explicit in symmetric and non-defective systems, then apply them to consensus models. In consensus settings, uniqueness depends on the drift and cost structure, and some cases admit infinitely many Gaussian solutions.
6 Examples
- The examples derive explicit solutions by verifying the algebraic conditions for nearly identical players and the mean-field game.The consensus models include cases with infinitely many solutions.Source: §6 · p. 24
6.1 Symmetric system
- With symmetric drift, scalar-identity control costs, and a common nonnegative primary displacement cost, the nearly identical-player game has a unique affine Nash equilibrium, and its solutions converge to the mean-field solution.The convergence follows under the stated large-population scaling assumptions.Source: §6.1 · pp. 24–26
6.2 Non–defective system
- For non-defective drift, a coordinate change symmetrizes the system, extending the unique quadratic-Gaussian solution result under the corresponding transformed cost conditions.The result applies to nearly identical players; analogous conditions are given for the limit problem.Source: §6.2 · pp. 26–27
6.3 Consensus models
- For the consensus cost shown, an N-player quadratic-Gaussian solution exists exactly when its mean satisfies Aµ = 0; it is unique exactly when det A is nonzero, then µ = 0.When det A is zero, nonzero vectors in the kernel of A can yield infinitely many Gaussian solutions.Source: §6.3 · pp. 27–29
- In the mean-field consensus model, a quadratic-Gaussian solution exists exactly for means satisfying Aµ = 0.Thus the limit model can have multiple solutions when A has a nontrivial kernel.Source: §6.3 · p. 28