Source-linked AI summary
Horizon-Independent Contraction for Continuous-Time Discounted Regularized Mean-Field Games
Junji Yan, Uğur Aydın, Tamer Başar
TL;DR
Continuous-time MFG contraction conditions usually depend on the horizon, complicating long-horizon equilibrium computation. This paper combines discounting, entropy regularization, weighted metrics, and positive-operator analysis to obtain horizon-independent contraction results and related approximation bounds. It shows that sufficiently large discounting supports finite-horizon contraction, whose large-horizon condition matches the infinite-horizon non-stationary case, while also providing finite-to-infinite and discounted-to-undiscounted error bounds.
Problem
Continuous-time MFG contraction conditions typically depend on the horizon, limiting their applicability to long-horizon problems.
Method
The paper uses discounting, entropy regularization, Bielecki metrics, and positive operators to analyze contraction in finite-state, finite-action continuous-time MFGs.
Results
Sufficiently large discount rates yield horizon-independent finite-horizon contraction, matching the infinite-horizon non-stationary condition, with explicit equilibrium error bounds.
Takeaways & Limitations
Finite-horizon equilibria can provide tractable approximations to infinite-horizon equilibria, and discounted equilibria can approximate undiscounted finite-horizon equilibria under the stated conditions.
Takeaways & Limitations
The stationary continuous-time case is left for future work, and finite-to-infinite-horizon error bounds are weaker near the terminal time.
Abstract
from arXiv · showhide
We study contraction properties of non-stationary continuous-time mean-field games (MFGs) under discounting and entropy regularization. The state of the representative agent evolves according to a controlled continuous-time Markov chain, and both the state and action spaces are finite. In contrast to the undiscounted case, we show that, under a sufficiently large discount rate, finite-horizon MFGs admit a horizon-independent contraction condition, which also coincides with the corresponding infinite-horizon non-stationary contraction condition. As a byproduct, we obtain an explicit convergence rate between finite- and infinite-horizon mean-field equilibria. For each finite horizon, we further derive a refined contraction criterion from the spectral radius of a positive operator that majorizes the propagation of policy errors, and show that its large-horizon limit agrees with the horizon-independent contraction factor. Finally, we provide an explicit error bound between discounted and undiscounted finite-horizon regularized equilibria.
1 Introduction
The introduction motivates contraction-based computation for continuous-time MFGs, where horizon dependence and computational difficulty limit applicability. The paper addresses these issues using discounting, regularization, positive-operator analysis, and horizon-independent contraction results.
- MFG equilibrium computation can be computationally intractable under mere Lipschitz conditions, motivating structural conditions for efficient algorithms.
- Continuous-time MFG contraction conditions typically depend on the horizon, limiting their applicability to long-horizon problems.
- The paper derives finite-to-infinite-horizon equilibrium error bounds and discounted-to-undiscounted finite-horizon equilibrium error bounds.
- Sufficiently large discount rates yield horizon-independent contraction conditions for finite-horizon continuous-time MFGs.
- The finite-horizon horizon-independent condition coincides with the corresponding infinite-horizon non-stationary contraction condition.
- For finite horizons, a positive operator majorizes policy-error propagation, enabling spectral-radius contraction criteria and quantitative T-dependent estimates.
2 Preliminaries and Notation
The preliminaries formulate finite-state, finite-action continuous-time MFGs through controlled Markov dynamics, representative-agent optimization, and population consistency. Discounted entropy regularization and weighted policy metrics support a well-defined fixed-point contraction analysis.
- A finite-horizon MFG is specified by finite state and action spaces, running and terminal costs, a transition-rate kernel, an initial distribution, and horizon T.
- The controlled transition-rate kernel governs continuous-time Markov-chain dynamics, while the mean-field flow records the population state distribution over time.
- Given a fixed mean-field flow, the representative agent solves a finite-horizon control problem over admissible stochastic Markov policies.
- A mean-field equilibrium combines representative-agent optimality with consistency between the prescribed and policy-induced population flows.
- The model assumes generator conditions and uniform Lipschitz dependence of costs and transition rates on the mean-field argument, ensuring well-defined dynamics.
- Exponentially weighted Bielecki metrics are equivalent to standard finite-horizon metrics and are introduced to obtain horizon-uniform Lipschitz and contraction estimates.
- Discounting suppresses future perturbation accumulation, while entropy regularization makes the best response unique and smooth.
- Banach’s fixed-point theorem applied to the best-response operator yields finite-horizon equilibrium existence, uniqueness, and global Picard-iteration convergence under contraction.
3 Horizon-Independent Contraction of the MFE Operator in the Finite-Horizon Case
Discounting and entropy regularization enable a contraction analysis whose sufficient condition is independent of the finite horizon. The proof combines component-wise Lipschitz estimates, Bielecki metrics, and a positive error-propagation operator with spectral analysis.
- Horizon-independent contraction: A sufficiently large discount rate yields an eventually contractive finite-horizon MFE operator and a unique regularized equilibrium for every finite horizon.The condition uses an intermediate weight η satisfying BΛ+2LΛ < η < δ−BΛ.
- Component-wise stability: The forward mean-field operator is Lipschitz in the population flow, with coefficient aM = BΛ + 2LΛ.The estimate separates direct flow dependence from state-transition dependence and uses aM throughout the contraction analysis.
- Component-wise stability: Discounting gives a horizon-independent span bound for the value function when δ > BΛ, controlling variation across states in the HJB equation.The span is the relevant quantity because generator rows have zero total mass, so additive constants in the value function disappear.
- Component-wise stability: Entropy regularization makes the soft-min response Lipschitz, while larger α reduces sensitivity to Q-function perturbations.This estimate supplies the policy component of the MFE operator's Lipschitz analysis.
- Spectral refinement: A positive Volterra operator majorizes policy-error propagation, and its spectral radius provides a refined contraction criterion for each fixed horizon.The spectral analysis also gives a computable, generally non-closed-form horizon-dependent contraction rate.
- Spectral refinement: The best horizon-independent Bielecki contraction factor equals the large-horizon limit of the finite-horizon spectral radius, making the bound asymptotically sharp.The spectral metric depends on T, whereas the Bielecki metric uses an exponential weight independent of T.
4 Infinite-Horizon Contraction and Finite-to-Infinite Error Bounds
The infinite-horizon contraction condition matches the large-horizon finite-horizon condition, yielding unique equilibria and convergent policy iteration. The section also establishes finite-to-infinite equilibrium error bounds, with exponential decay away from the terminal time.
- Relation to finite horizons: The infinite-horizon spectral radius coincides with the asymptotic spectral radii from finite horizons, and finite-to-infinite MFE convergence rates are provided.In continuous time, the non-compactness issue that complicates the infinite-horizon discrete-time setting does not occur.
- Infinite-horizon contraction: The infinite-horizon contraction analysis uses pointwise estimates over [0, ∞), whose integrals remain finite because of discounting.The argument parallels the finite-horizon analysis, while the absence of a terminal cost removes the span(g) contribution.
- Infinite-horizon contraction: Under aM < η < δ − BΛ and κ∞δ,η,α < 1, the infinite-horizon MFE operator is a contraction.The resulting contraction factor is defined using a horizon-independent exponential Bielecki weight.
- Infinite-horizon contraction: A contraction gives a unique infinite-horizon non-stationary (δ, α)-regularized MFE, and Picard iteration converges from every initial policy.This follows from Banach’s fixed-point theorem under the stated contraction condition.
- Relation to finite horizons: The infinite-horizon contraction condition is no stronger than the corresponding finite-horizon condition with the same Bielecki weight.The difference is that the infinite-horizon span constant contains no terminal-cost term.
- Finite-to-infinite error bounds: For fixed mean-field flows, truncation estimates are lifted to equilibrium level using contraction stability and the horizon-independent Bielecki metric.Theorem 4.8 establishes existence and uniqueness of both equilibria and quantifies their comparison on [0, T].
- Finite-to-infinite error bounds: For each fixed t, the finite- and infinite-horizon MFE difference decays exponentially as T →∞, but uniform unweighted control near t = T is generally not expected.Near the terminal time, the finite-horizon equilibrium remains influenced by the imposed terminal cost g.
5 Discounted–Undiscounted Error Bounds for Finite-Horizon MFGs
The section bounds the discrepancy between discounted and undiscounted finite-horizon regularized MFGs, then lifts the operator bound to an equilibrium-level estimate under discounted contraction.
- Best-response perturbation: The discounted and undiscounted best-response maps are compared at a fixed mean-field flow, where discounting changes objective weights but not controlled state dynamics.The perturbation is first bounded at the map level before being transferred to MFE estimates.
- Operator setup: The undiscounted regularized operators are obtained by setting δ = 0 and define equilibria through coupled policy and mean-field fixed-point conditions.An undiscounted equilibrium satisfies π0,T = ΦT0(π0,T ) and µ0,T = ΓTM(π0,T ).
- Best-response perturbation: A uniform operator-difference bound depends on the horizon length T and is identified as the section’s main result.Proposition 5.1 establishes this bound under Assumption 2.2 for every η ≥ 0.
- Equilibrium-level estimate: Under contraction of the discounted MFE operator and existence of an undiscounted regularized MFE, the map perturbation bound lifts to an MFE-level error estimate.The undiscounted operator itself need not be contractive; only the discounted operator’s contraction and undiscounted equilibrium existence are required.
- Equilibrium-level estimate: The resulting discounted MFE provides a controlled approximation of a finite-horizon undiscounted MFE.This supports using discounted equilibria when the target problem has a long but finite horizon.
6 Conclusion
The conclusion develops quantitative contraction tools for discounted regularized continuous-time non-stationary MFGs and identifies a limitation in the stationary continuous-time setting.
- Main contributions: The analysis uses positive-operator theory to calculate quantitative contraction rates in the finite-horizon setting.The work studies continuous-time non-stationary discounted regularized MFGs with state evolution given by an explicit Markov kernel.
- Main contributions: The asymptotic finite-horizon contraction condition coincides with the contraction condition for the infinite-horizon non-stationary case.This establishes agreement between the large-horizon finite-horizon analysis and the infinite-horizon condition.
- Open limitation: The stationary continuous-time case remains open because solving Fπ(µ) = 0 at each step and establishing the needed Lipschitz properties introduce additional technical challenges.The authors leave the stationary case for future work.
A A Primer on Positive Operators
This appendix introduces cone-based notions of positivity and compactness, defines spectral radius through operator norms, and states the Krein–Rutman result for positive compact operators.
- Basic definitions: A cone is a closed convex set stable under nonnegative scaling and containing no nonzero vector together with its negative.A total cone additionally has a dense difference set K − K in the Banach space.
- Basic definitions: An operator is compact when it maps bounded subsets of the Banach space to relatively compact subsets.
- Basic definitions: A compact operator is positive on a total cone when it maps that cone into itself.
- Spectral radius: The spectral radius ρ(T) is defined as the limit of ∥T^n∥^(1/n), using the induced operator norm.This is identified as Gelfand’s formula.
- Krein–Rutman theorem: For a nonzero positive compact operator with ρ(T) > 0, the spectral radius is an eigenvalue with an eigenvector in the cone.This is the stated Krein–Rutman theorem.