PaperlayerView exact revision on arXiv

Source-linked AI summary

Quantitative Mean-Field Limits for Recent Variants of Consensus-Based Optimization

Konstantin Haug

arXiv:2610.02901v2math.OCcs.NAmath.NA

TL;DR

Finite-particle CBO systems need quantitative evidence that they approximate their mean-field limits, which support analysis of optimization convergence. This thesis proves propagation-of-chaos estimates for three CBO variants using coupling in restricted and unrestricted settings. The estimates yield finite-horizon convergence, with rates depending on moment assumptions, but their constants grow exponentially with the time horizon.

  • Problem

    Quantitative propagation of chaos matters because it establishes whether finite-particle CBO dynamics approximate the mean-field process used as a tractable surrogate and support global convergence guarantees.

    Source: Introduction · p. 6

  • Method

    The thesis uses coupling to prove propagation of chaos for δ-CBO, consensus freezing, and consensus hopping in both restricted high-probability and unrestricted settings.

    Source: Introduction · p. 6, Propagation of Chaos Restricted on… · p. 29

  • Results

    For all three schemes, propagation of chaos holds uniformly over finite horizons; under bounded fourth moments in Wasserstein-2, the restricted rate is O(N−1/2) and the unrestricted rate is O(N−1/4).

    Source: Introduction · p. 7, Conclusion · p. 82

  • Takeaways & Limitations

    Combining the estimates with mean-field global convergence results gives high-probability guarantees for empirical means approaching invariant-measure means concentrated near the global minimizer.

    Source: Conclusion · p. 82

    The estimates cover finite time horizons, and their constants grow exponentially in T, so they do not establish uniform-in-time propagation of chaos.

    Source: §4.6.1 · p. 75, §4.7 · p. 79

Abstract

from arXiv · show

Consensus-based optimization (CBO) is a robust and flexible zero-order, multi-particle class of methods devised to provide provable solutions to high-dimensional global optimization problems, including truly nonconvex and/or nonsmooth cases. The algorithmic framework is grounded in a principled trade-off between stochastic exploration of the search space and deterministic contraction toward a consensus point. This thesis is concerned with three recently established variants of CBO. The $\delta$-CBO method is similar to the classic CBO method, with the difference that the diffusion function is constant. The consensus freezing (CF) scheme can be understood as a piecewise CBO method. The overall time horizon is divided into intervals in which the consensus point is fixed. The consensus hopping (CH) scheme can be understood as a deterministic iterative scheme that jumps at each iteration to a Gaussian distribution centered at the consensus point of the previous distribution. The central contribution of this thesis is the rigorous derivation of propagation of chaos results for all three schemes. For each variant, we establish that the empirical measure of the interacting particle system converges, as the number of particles $N$ approaches infinity, to the solution of the corresponding mean-field equation. Two regimes are treated: first, on a high-probability set on which the empirical measures are uniformly bounded; second, in the unrestricted setting, where moment bounds and a careful treatment of the inter-particle dependence yield convergence without imposing conditioning on favorable events. The propagation of chaos estimates are derived by the coupling method. Together, these results provide quantitative propagation of chaos estimates for all three CBO variants and establish that the particle systems faithfully approximate the idealized mean-field dynamics in the large-particle limit.

Introduction

This thesis studies quantitative propagation of chaos for three CBO variants, asking whether their finite-particle dynamics approximate their mean-field limits. It develops restricted and unrestricted coupling analyses and reports convergence rates, with a practical application to convergence guarantees for empirical means.

Introduction

  • The thesis establishes quantitative propagation of chaos for δ-CBO, consensus freezing, and consensus hopping, three variants with decreasing structural proximity to classical CBO.δ-CBO uses constant diffusion; consensus freezing fixes its consensus point over uniformly spaced intervals, while hopping operates in discrete time.Source: Introduction · pp. 6–7
  • The restricted analysis couples particle and mean-field systems on a high-probability set with bounded empirical moments; the unrestricted analysis avoids conditioning on that set.The unrestricted method bounds the probability of a time-dependent unfavorable set and uses an improved Wasserstein stability estimate for the consensus point.Source: Introduction · p. 7
  • In Wasserstein-2 distance under bounded fourth moments, the restricted approach achieves O(N−1/2), while the unrestricted approach yields O(N−1/4).Across all three schemes, convergence is uniform over finite time horizons; the unrestricted approach recovers O(N−1/2) under sufficiently high moment assumptions.Source: Introduction · p. 7
  • The propagation-of-chaos estimates combine with mean-field global convergence results to give high-probability guarantees for empirical means approaching invariant-measure means concentrated near the global minimizer.The thesis describes a triangle inequality separating statistical error from deterministic convergence error.Source: Introduction · p. 7

1.1 Related Work

  • Earlier classical-CBO work progressed from nonquantitative propagation-of-chaos arguments to quantitative coupling results, first restricted to high-probability sets and later unrestricted.The thesis notes that some prior finite-horizon estimates have constants growing exponentially with the horizon.Source: §1.1 · pp. 8–9

Notation

The notation section establishes conventions for probability, measures, moments, and particle systems used throughout the thesis.

Notation

  • The probability space is denoted (Ω, F, P), expectations by E[·], and Law(X) denotes the distribution of X.The notation also specifies Borel probability measures and expectations computed with respect to a measure.Source: Notation · pp. 9–10
  • For measures with finite pth moment, the pth moment is denoted M_p(μ), and empirical measures assign equal mass to the interacting particles.The empirical measure notation is abbreviated when the particle process is clear from context.Source: Notation · pp. 9–10
  • The objective f is assumed to have a unique global minimizer and satisfy (A1)–(A3); η_α denotes the associated consensus measure.For real numbers a and b, a ∨ b and a ∧ b denote their maximum and minimum.Source: Notation · pp. 9–10

Preliminaries

The preliminaries establish probabilistic and particle-system foundations for later mean-field and Wasserstein propagation-of-chaos analyses of CBO, introducing its dynamics, assumptions, coupling tools, and motivation for sharper empirical-measure bounds using CBO-specific assumptions.

2.1 Some Preliminaries from Probability Theory

  • The preliminaries introduce filtrations, martingales, stopping times, quadratic variation, Burkholder–Davis–Gundy inequalities, Wasserstein distance, and bounds for empirical means and Gaussian maxima.The empirical-mean tail bound supports unrestricted propagation-of-chaos proofs, while the Gaussian-maximum bound supports moment estimates for consensus freezing and hopping.Source: Preliminaries · p. 11, §2.1 · pp. 13–14

Preliminaries

  • The CBO preliminaries describe classical dynamics and proof challenges, then provide stability and quantitative convergence estimates for consensus points.The estimates concern Wasserstein stability between consensus points of two probability measures and convergence of empirical consensus from i.i.d. samples.Source: Preliminaries · p. 11

2.2 Interacting Particle Systems

  • Interacting particle systems are formulated in discrete and continuous time, with empirical measures assigning equal probability mass to each particle and encoding their positions.The thesis’s continuous-time systems are defined by SDEs with unique strong solutions; their noise comes from Brownian motion, while discrete-time noise uses Gaussian vectors.Source: §2.2 · pp. 14–15

2.3 Propagation of Chaos

  • Propagation of chaos describes convergence toward independent copies of a mean-field process as particle numbers grow, and coupling proves it by comparing particles with matched mean-field copies.The coupling argument seeks to make the average particle-to-counterpart distance vanish uniformly over a finite time horizon.Source: §2.3 · pp. 17–18
  • Coupling yields expected Wasserstein-p convergence of the interacting system’s empirical measure to the mean-field law, uniformly in time; pointwise estimates give pointwise convergence.For symmetric laws, empirical-measure convergence in law to a deterministic measure is equivalent to ρ-chaos.Source: §2.3 · p. 19

2.3.1 Proving Propagation of Chaos by Coupling

  • The coupling proof pairs particles with mean-field copies sharing Brownian paths and initial conditions, then controls their differences; global Lipschitz conditions would enable McKean’s theorem but do not hold here.The thesis adapts steps of Sznitman’s proof instead, under unique strong-solution assumptions for the particle and mean-field SDEs.Source: §2.3.1 · pp. 19–20

Diffusion Term

  • Drift and diffusion differences are bounded separately, their estimates combined with constants independent of N, and Grönwall’s lemma yields the desired bound.The diffusion integral becomes deterministic using the quadratic variation of Brownian motion.Source: Diffusion Term · p. 21

2.4 Consensus-Based Optimization

  • Classical CBO drives particles toward a consensus point that weights lower-objective regions more heavily, while stochastic noise promotes exploration.The section introduces classical CBO and the consensus point before turning to newer variants analyzed later.Source: §2.4 · p. 22

2.4.1 Proving Propagation of Chaos for CBO

  • The CBO coupling argument establishes Wasserstein-p propagation of chaos, but its estimate grows exponentially with T and is not uniform over infinite horizons.The sketch uses a bounded-moment condition on empirical measures; later chapters show how to remove it, with higher mean-field moments used to relax a restriction in one estimate.Source: §2.4.1 · p. 24

2.5 Assumptions and Tools for Mean-Field Approximations in CBO

  • Assumptions A1–A3 need not imply global Lipschitz continuity, but suffice for adapted coupling proofs; consensus-point stability estimates support the later propagation-of-chaos analysis.The assumptions used for Wasserstein-p propagation of chaos do not depend on p or the moments considered.Source: §2.5 · pp. 25–26
  • Generic empirical Wasserstein bounds can deteriorate when d ≫ p because they do not exploit CBO assumptions (A1), (A2), or (A3).The cited quantitative mean-field estimate can support a proof of global convergence in probability, but its rate depends on the convergence rate in (2.16).Source: §2.5 · p. 27
  • O(N^-p/2) is the improved rate obtained by a lemma exploiting CBO-specific assumptions and generalizing earlier propagation-of-chaos results.The lemma generalizes [23, Lemma 3.1] and [2, Proposition 3.11, Lemma A2], which established Wasserstein-2 propagation of chaos at rate O(N^-1/2).Source: §2.5 · p. 28

Propagation of Chaos Restricted on Good Sets

This section develops coupling-based quantitative propagation-of-chaos estimates for δ-CBO, consensus freezing, and consensus hopping on good sets with bounded empirical moments. The estimates concern finite time horizons and are expressed in Wasserstein-p distance, with an O(N^-1/2) rate stated for hopping.

Propagation of Chaos Restricted on Good Sets

  • For all three schemes, coupling on a high-probability set with bounded empirical-measure moments yields Wasserstein-p propagation-of-chaos estimates at rate O(N^-1/2), uniformly over finite time horizons.The set satisfies P(Ω_M) ≥ 1 − C/M, which tends to one as M increases.Source: Propagation of Chaos Restricted on… · p. 29

3.1 The δ-CBO Scheme

  • The δ-CBO variant [5] replaces distance-dependent diffusion with a fixed scaling parameter, and its coupling analysis uses uniform moment bounds and high-probability control of empirical moments.On the good set, shared Brownian motions cancel in the coupled difference, enabling a pathwise estimate.Source: §3.1 · p. 30, §3.1.1 · pp. 31, 34

3.1.1 Propagation of Chaos for the δ-CBO Scheme

  • For δ-CBO, empirical measures converge on the good set in Wasserstein-p distance at rate O(N^-1/2), independent of p for initial data with finite pth moments, p ≥ 2.The expected pathwise particle-to-mean-field error is of order O(N^-p/2).Source: §3.1 · p. 30, §3.1.1 · p. 36

3.2 The Consensus Freezing Scheme

  • The consensus freezing scheme [5] holds consensus fixed over short intervals, yielding Ornstein–Uhlenbeck dynamics and good-set Wasserstein-p convergence at rate O(N^-1/2).The proof combines uniform moment estimates with probabilistic empirical-moment bounds and coupling.Source: §3.2 · p. 37

3.2.1 Propagation of Chaos for the Consensus Freezing Scheme

  • For consensus freezing, explicit Ornstein–Uhlenbeck solutions give bounded moments within each interval, while coupling estimates rely on trajectory differences where consensus changes.The analysis therefore focuses on controlling empirical-measure moments at interval transitions.Source: §3.2.1 · p. 38
  • For the consensus freezing scheme, coupling on the bounded-moment set ΩM gives propagation of chaos as N approaches infinity, with a quantitative error estimate.The theorem also gives a conditional estimate when P(ΩM) > 0; the supplied passages do not expose the estimate’s rate.Source: §3.2.1 · pp. 42, 43, 46

3.3 The Consensus Hopping Scheme

  • The consensus hopping scheme updates iteratively using Gaussian distributions centered at the previous iterate and admits an interacting-particle mean-field formulation.It is analyzed as a discrete-time scheme.Source: §3.3 · p. 47

3.3.1 Propagation of Chaos for the Hopping Scheme

  • For consensus hopping, coupling proves propagation of chaos uniformly over finite horizons on ΩM, at rate O(N^-1/2) in Wasserstein-p distance.The rate holds up to constants depending on the time horizon and model parameters.Source: §3.3 · p. 47, §3.3.1 · p. 51
  • The stated result applies when P(Ω_M) > 0.The passage states this condition for every N and particle index i ≤ N.Source: §3.3.1 · p. 51
  • The consensus hopping scheme satisfies propagation of chaos on Ω_M with convergence rate O(N^-1/2).The result is restricted to the event set Ω_M.Source: §3.3.1 · p. 53
  • The hopping and freezing estimates have the same structure, differing by a factor of 2 in the freezing estimate.The passage relates this difference to the freezing scheme's convergence to the hopping scheme as s → ∞, established in Theorem 4.10.Source: §3.3.1 · p. 54

Unrestricted Propagation of Chaos

This section develops unrestricted quantitative propagation-of-chaos estimates for the CBO schemes, with rates governed by moment assumptions and finite-horizon convergence results for consensus hopping and freezing. It also relates the estimates to restricted results and mean-field global convergence, and identifies obstacles to time-uniform propagation of chaos and a translation structure in the coupling error.

Unrestricted Propagation of Chaos

  • When q≥p^2+p, all three schemes achieve O(N^-1/2); in Wasserstein-2 with q=4, the unrestricted rate is O(N^-1/4), versus O(N^-1/2) in the restricted case.The unrestricted estimates require bounded moments of order q≥p^2+p for Wasserstein-p convergence at rate O(N^-1/2), compared with bounded moments of order p in the restricted setting.Source: Unrestricted Propagation of Chaos · pp. 55, 57

4.1 Preliminaries Revisited

  • The refined Wasserstein stability estimate for the consensus point requires an explicit pth-moment bound on only one of the two measures.This improves on the earlier estimate, which required explicit pth-moment bounds for both measures.Source: unrestricted propagation of chaos · p. 57

4.2 δ-CBO

  • For δ-CBO, bounded initial moments of order q ≥ p² + p imply propagation of chaos at rate O(N^-1/2) for every p ≥ 1.Source: §4.2 · p. 61

4.3 Freezing

  • For consensus freezing, the unrestricted rate is O(N^-min{1/2,(q-p)/(2p^2)}) for 1≤p≤q/2; q≥p^2+p gives O(N^-1/2).The theorem establishes convergence for the interacting particle system and its coupled mean-field system.Source: §4.3 · p. 66

4.4 Hopping

  • For consensus hopping, the coupled particle and mean-field systems satisfy an unrestricted propagation-of-chaos estimate that vanishes as N grows, for finite horizon K and 1 ≤ p ≤ q/2.The initial distribution is required to have a finite qth moment; the result does not condition on a favorable event.Source: §4.4 · p. 66

4.5 Comparison to the Restricted Case of Chapter 3

  • Chapter 4 avoids conditioning on a fixed good set but gives rate O(N^-min{1/2,(q-p)/(2p^2)}), versus Chapter 3's O(N^-1/2); for q=4 and p=2, the rates are O(N^-1/4) and O(N^-1/2), respectively.With bounded moments of all orders, Chapter 4 recovers O(N^-1/2) without restricting to specific events; the q=4, p=2 example assumes only bounded fourth moments.Source: §4.5 · p. 70

4.6 Towards Global Convergence: Freezing and Hopping

  • The propagation-of-chaos and mean-field global-convergence estimates yield a high-probability bound placing the freezing scheme's empirical mean near the invariant-measure mean, which approaches the global minimizer as α grows.The propagation-of-chaos contribution is O(N^-1) over finite time [0,T] when the initial distribution has bounded sixth moments, while the required particle count grows exponentially with T.Source: §4.6 · pp. 71–72

4.6.1 Global Convergence in Probability: Freezing

  • For sufficiently large α, the freezing mean-field dynamics has a unique invariant measure and converges toward it, enabling a finite-horizon probabilistic convergence estimate for the particle system.The estimate assumes a common initial distribution in P6(Rd) and is not uniform in time: its propagation-of-chaos constant grows exponentially with T, so the required particle number does too.Source: §4.6.1 · pp. 72, 73, 75
  • For freezing, the empirical-mean rate is O(N^-1/2) in low dimension but deteriorates to leading order O(N^-2/d) for d > 4.At d = 4, the bound includes an additional log(1 + N) factor; the global result is stated for the empirical mean rather than the empirical measure in Wasserstein distance.Source: §4.6.1 · p. 76

4.6.2 Global Convergence in Probability: Hopping

  • As the freezing scale s tends to infinity, its mean-field dynamics approach hopping uniformly in k, with error bounded by 4R e^-sλΔt.This limit, combined with the corresponding convergence results, yields global convergence results for the hopping scheme.Source: §4.6.2 · pp. 76–77
  • For freezing and hopping, sufficiently many particles make the probability of being close to the invariant-measure mean at least 1 − ξ, but the bound constants grow exponentially with the time horizon.The propagation-of-chaos constants are the bottleneck, and uniform-in-time estimates over infinite horizons remain missing.Source: §4.6.2 · p. 78

4.7 Outlook: Uniform-in-time Propagation of Chaos

  • Uniform-in-time propagation of chaos for hopping requires a contractive consensus-point mapping; the same condition applies to freezing.The estimates established here are finite-horizon, with constants growing exponentially in T.Source: §4.7 · p. 79
  • With additive noise, centered spread approaches dδ^2/(2λ), independent of f and α, so particles do not collapse when δ > 0.The absence of centered-moment decay means a direct adaptation of the cited uniform-in-time argument does not work for these schemes.Source: §4.7 · pp. 80–81

The Coupling Error is a Translation

  • Under synchronous coupling, the particle-to-mean-field error is identical across indices, so the analysis depends on the consensus map along translations of a fixed measure.A proposed assumption on that translation behavior is left open for sufficiently rich classes of measures.Source: The Coupling Error is a Translation · p. 81

Conclusion

The thesis establishes finite-horizon quantitative propagation of chaos for three CBO schemes, using both restricted and unrestricted approaches. It also applies these estimates to convergence of empirical means, while identifying the lack of uniform-in-time estimates as a limitation.

Conclusion

  • All three schemes have finite-horizon propagation-of-chaos estimates: the restricted approach achieves O(N^-1/2), while the unrestricted rate depends on the moment assumptions.For unrestricted estimates, bounded moments of order q ≥ p^2 + p give O(N^-1/2); under bounded fourth moments in Wasserstein-2, the rate is O(N^-1/4), versus O(N^-1/2) for the restricted estimate.Source: Conclusion · p. 82
  • The unrestricted approach removes conditioning on good trajectories by splitting the expected consensus-point distance and using independence of mean-field particles to handle the complementary set.It also uses a one-sided refinement of the Wasserstein stability estimate on the selected set.Source: Conclusion · p. 82
  • Combining propagation-of-chaos estimates with global convergence results [5] gives high-probability bounds for empirical means, with an O(N^-1) statistical error floor and a deterministic term that decays exponentially in time.An analogous result for the hopping scheme follows from the connection between the freezing and hopping schemes established in [5, Theorem 4.1].Source: Conclusion · p. 82
  • The estimates are not uniform over infinite time horizons because their propagation-of-chaos constants grow exponentially in T; uniform-in-time estimates, potentially under suitable confinement assumptions, remain future work.The thesis identifies this as a central limitation of the empirical-mean convergence results.Source: Conclusion · p. 82

A Proof of Lemma 2.11

The proof develops Gaussian tail and maximum-moment estimates by reducing to independent standard normal variables and bounding their maximum. It uses a logarithmic threshold to split the tail integral and then obtains the final estimate.

A Proof of Lemma 2.11

  • Splitting the tail integral at t0 = √(2 log N) and applying a union bound controls the contribution from large values.The argument invokes a standard normal tail inequality for this estimate.Source: A Proof of Lemma 2.11 · pp. 83–84
  • The resulting bounds are combined to obtain the final estimate with a constant depending only on p.The proof states that the Gaussian maximum estimate holds for N ≥ 2 and p ≥ 2.Source: A Proof of Lemma 2.11 · pp. 83–85
  • The proof reduces Gaussian-vector maximum estimates to independent standard normal coordinates, then extends the result to vectors distributed as N(0, σ^2 I_d).It also bounds the maximum by adding further independent standard normal variables before applying the maximum estimate.Source: A Proof of Lemma 2.11 · pp. 83, 85

B Proof of Lemma 2.27

The proof uses Rosenthal's inequality to control moments of sums of independent zero-mean random variables. It applies this approach to sample means arising in the estimates and combines the resulting bounds to complete the lemma.

B Proof of Lemma 2.27

  • Rosenthal's inequality bounds moments of sums of independent zero-mean random variables in terms of the summands' moments.The proof uses the inequality for variables with finite pth moments.Source: B Proof of Lemma 2.27 · p. 85
  • The proof applies sample-mean estimates to independent, identically distributed zero-mean variables with finite pth moments.It introduces separate families of variables for the two terms being estimated.Source: B Proof of Lemma 2.27 · pp. 85, 87, 89
  • Jensen's inequality and the Rosenthal estimates are combined to bound the terms involving the consensus point and reach the final estimate.The argument treats the two parts separately before combining them.Source: B Proof of Lemma 2.27 · pp. 88, 90
Loading 2610.02901v2…