Source-linked AI summary
Limit theory for controlled McKean-Vlasov dynamics
Daniel Lacker
TL;DR
The paper asks how optimal control of large interacting systems relates to optimal McKean-Vlasov control. It uses controlled martingale problems and relaxed controls to establish this relationship. Near-optimal empirical distributions converge to distributions supported on McKean-Vlasov optima, and every distribution on those optima can be realized as a limit.
Problem
The paper addresses the question of whether optimization commutes with the large-system limit for controlled interacting diffusions.
Method
The paper uses controlled martingale problems, relaxed controls, compactness estimates, and McKean-Vlasov limit arguments for both finite systems and limiting dynamics.
Results
Near-optimal n-state empirical distributions are precompact with limits supported on optimal McKean-Vlasov control-state laws, and every distribution supported on those laws is realizable by near-optimal controls.
Takeaways & Limitations
The results provide a rigorous bidirectional connection between large interacting controlled systems and McKean-Vlasov optimal control.
Takeaways & Limitations
The paper does not address models with common noise and requires coercivity or related assumptions for compactness and limit results.
Abstract
from arXiv · showhide
This paper rigorously connects the problem of optimal control of McKean-Vlasov dynamics with large systems of interacting controlled state processes. Precisely, the empirical distributions of near-optimal control-state pairs for the $n$-state systems, as $n$ tends to infinity, admit limit points in distribution (if the objective functions are suitably coercive), and every such limit is supported on the set of optimal control-state pairs for the McKean-Vlasov problem. Conversely, any distribution on the set of optimal control-state pairs for the McKean-Vlasov problem can be realized as a limit in this manner. Arguments are based on controlled martingale problems, which lend themselves naturally to existence proofs; along the way it is shown that a large class of McKean-Vlasov control problems admit optimal Markovian controls.
1. Introduction
The paper studies whether optimal control of large interacting systems converges to optimal McKean-Vlasov control, and establishes this connection under modest assumptions. It uses controlled martingale problems to prove limit and realization results while situating the work among related mean field control and game-theoretic models.
- Problem setup: The controlled n-state system models a central planner choosing controls for interacting state processes whose coefficients depend on empirical state distributions.This provides the finite-system counterpart to the McKean-Vlasov control problem.
- Motivation: The central question is whether the large-system limit commutes with optimization, rather than merely describing uncontrolled McKean-Vlasov convergence.The paper addresses this by analyzing empirical measure flows of optimally controlled systems.
- Main results: Empirical measure flows of optimally controlled n-state systems are tight, and every distributional limit is supported on measure flows generated by optimally controlled McKean-Vlasov states.This is the main limit result under modest assumptions on the model coefficients.
- Main results: When the McKean-Vlasov problem has a unique optimal control, the result yields proper convergence, or propagation of chaos.Uniqueness converts support of subsequential limits into convergence toward the corresponding optimal law.
- Methods: The analysis formulates both finite and limiting systems as controlled martingale problems with relaxed, measure-valued controls, combining compactification with McKean-Vlasov limit arguments.This formulation supports compactness and existence proofs for stochastic control problems.
- Positioning: The work presents a broad controlled-diffusion limit theorem relative to prior results focused on special portfolio, deterministic, or large-deviation settings.It also relates centralized McKean-Vlasov control to decentralized mean field games and Pareto-versus-competitive outcomes.
2. Model setup and main results
The paper formulates McKean-Vlasov control using relaxed controls and controlled martingale problems, then establishes existence and large-system limit results under coercivity, continuity, and regularity assumptions.
- Relaxed controls: The model uses relaxed controls represented by measures on [0,T] × A, with strict controls as their measurable-action special case.Each relaxed control has Lebesgue time marginal and can be identified with a predictable process of probability measures over actions.
- Existence results: Under assumption A, an optimal relaxed control exists for the McKean-Vlasov problem.The assumptions include continuity or semicontinuity conditions, growth bounds, moment conditions, and coercivity of the running objective.
- Existence results: Under an additional convexity assumption, an optimal Markovian control exists and can preserve the state marginals and objective value of a given control.Theorem 2.3 applies when the relevant coefficient-objective set is convex for each time, state, and measure tuple.
- Formulation equivalence: The relaxed and strong formulations have the same optimal value under stronger initial-moment and uniqueness conditions.The result requires an initial distribution with a moment of order p′′ > p′ and a unique square-integrable strong solution.
- Main limit results: For relaxed n-state ε_n-optimal controls with ε_n → 0, empirical control-state measures are precompact and every limit is supported on optimal McKean-Vlasov controls.Conversely, every distribution supported on optimal McKean-Vlasov controls can be realized as a limit of such empirical measures; under extra moment assumptions, the approximating controls can be strong.
- Main limit results: When optimal McKean-Vlasov controls are unique, empirical measures of near-optimal n-state controls converge in probability to that unique control.With the additional convexity assumption, the corresponding state-measure flows have weak limits supported on the optimal state-measure set.
3. Some first estimates
The section develops moment and compactness estimates for mean-field and n-state controls, using coercivity and stochastic-process inequalities to control states and controls uniformly.
- Mean field estimates: Coercivity converts near-optimality into uniform moment bounds for mean-field controls.The estimates use upper and lower objective bounds together with a constant-control comparison.
- n-state estimates: Burkholder-Davis-Gundy and Gronwall inequalities control state moments and stochastic-integral terms in the n-state system.Symmetry, Jensen’s inequality, and growth assumptions handle the interaction and control terms.
- n-state estimates: The same strategy yields bounds uniform in n for weakly near-optimal n-state controls.These bounds support compactness arguments for empirical control-state measures.
- Compactness: Aldous-type compactness results show that controlled processes with the stated growth bounds are precompact in Pp(Cd × V).The compactness statement allows variation in the probability space and coefficients within the prescribed bounds.
4. Proofs of existence Theorems 2.2 and 2.3
Existence is proved by restricting attention to compact sets of near-optimal controls and applying upper semicontinuity of the objective and admissibility constraints.
- Existence: The optimal value is attained because Γ is upper semicontinuous on the compact set Rε.The proof reduces existence to maximizing an upper semicontinuous function over a compact admissible set.
- Existence: The objective Γ is upper semicontinuous, and its expectation is upper semicontinuous on laws of control-state distributions.Under additional continuity assumptions, the expectation is continuous on suitable moment-controlled sets.
- Existence: The set Rε of ε-optimal controls is compact in Pp(Cd × V).Precompactness comes from controlled-process estimates, while closedness follows from upper semicontinuity and stability of the martingale problem.
- Stability: Martingale-problem constraints remain valid under limits because the associated operators are continuous and the martingale identities pass to the limit.The argument tests against smooth compactly supported functions and bounded continuous adapted observables.
- Markovian controls: Under the convexity assumptions, relaxed optimal controls can be replaced by Markovian controls without changing the relevant state marginals.A measurable selection and a mimicking theorem produce a Markovian representative.
5. Limits of n-state controls
The section identifies limits of empirical measures from n-state controls and constructs n-state controls realizing any prescribed McKean-Vlasov control law.
- Limits of n-state controls: Empirical measures of near-optimal n-state controls are precompact, and every limit is supported on optimal McKean-Vlasov controls.This is the main compactness-and-identification result for the n-state approximation.
- Compactness: Tightness of empirical measures is obtained from moment estimates and compactness criteria on the path-control space.The argument also requires tightness of the associated mean measures.
- Limit identification: Martingale arguments identify limit points by passing the n-state martingale problem to the limit through joint continuity and tightness.Orthogonality of particle martingales and dense classes of test functions complete the identification.
- Realization of limits: For every m in the McKean-Vlasov control set, there are n-state controls whose empirical laws converge to δm and whose objectives converge to Γ(m).Independent copies of a control-state pair are coupled with the interacting system, and stability estimates show convergence.
- Realization of limits: The realization proof uses trajectorial propagation of chaos to compare independent McKean-Vlasov copies with the interacting n-state processes.Lipschitz estimates, coupling, and Gronwall’s inequality control the discrepancy.
6. Proofs of the limit theorems
The proofs characterize limits of near-optimal empirical measures and show that these limits are supported on optimal McKean–Vlasov control-state pairs. The converse uses compact convexity and extreme-point approximation to realize every distribution supported on the optimal set.
- Every limit point of near-optimal empirical measures is concentrated on the set R of admissible McKean–Vlasov control-state laws.
- Theorem 2.11 supplies the inclusion from subsequential limits of near-optimal n-state controls to probability measures supported on R∗, the optimal set.
- The proof defines L as limits generated by εn-optimal controls and establishes that L is convex.
- Krein–Milman reduces the converse inclusion to showing that each extreme point δm, with m in R∗, belongs to L.
- L is closed, using neighborhoods that eventually intersect the sets of empirical measures generated by ε-optimal controls.
7. Strong versus relaxed formulations
The paper connects relaxed and strong McKean–Vlasov controls through successive approximation of control measures, martingale measures, and Wiener-driven controls. Under Lipschitz, continuity, integrability, boundedness, and uniqueness assumptions, these approximations preserve state laws and objectives.
- Approximation strategy: The construction approximates a relaxed control in three steps: bounded controls, ordinary progressively measurable controls, and strong Wiener-filtration controls.
- Approximation strategy: Truncating the relaxed control and its martingale measures yields McKean–Vlasov states Xn with E[∥Xn − X∥2] tending to zero.
- Assumptions: The arguments rely on Lipschitz well-posedness, coefficient continuity, integrability conditions, and uniform boundedness or uniform integrability.
- Relaxed controls: The first approximation gives P ◦(Xn, Λn)−1 → m and Γ(P ◦(Xn, Λn)−1) → Γ(m).
- Strong controls: The chattering lemma replaces bounded relaxed controls by bounded progressively measurable controls while retaining convergence of the induced control-state laws and objectives.
- Strong controls: Strong controls are obtained by approximation in joint law with the Wiener process, and uniqueness in law identifies every limit with the target law.
Appendix A. A note on the filtration of Cd × V
The appendix constructs countable separating classes for the natural filtration on path-control space. It does so by separately restricting state paths and control measures, then combining their separating functions.
- The natural filtration records the state path up to time t and the control measure accumulated through time t.
- A separating class is a family of measurable functions whose integrals determine probability measures on the relevant σ-field.
- For each t, the filtration admits a countable separating class of bounded continuous functions.
- The construction uses continuous restriction of state paths to [0,t] and continuous restriction of control measures to normalized measures on [0,t] × A.
- Products of separating functions for the restricted state path and control measure form a separating class for the joint filtration.