On the Optimisation Landscape of Overparameterised Weighted MMD

Motivation and setup

• MMD is used as a discrepancy because small MMD error implies low integration error for RKHS functions, and finite weighted particles provide a practical approximation when exact representation is infeasible.The paper studies both simplex weights and a relaxed formulation with unconstrained weights optimised for each particle configuration.Source: §1 · p. 1

Motivation and question

• The central gap is whether increasing particle count can eliminate poor local minima at finite N, which population-level dynamics and algorithmic work do not directly settle.The finite-particle objective is generally nonconvex in particle locations even though the measure-level objective is convex.Source: §1 · p. 2

Main contributions

• For any compactly supported target of diameter at most D, a finite threshold Nε(D) ensures every local minimum has MMD below ε, while stationary points with MMD at least ε are unstable under unweighted particle gradient flow.The guarantees hold for both simplex-constrained and relaxed weights, and computable threshold bounds are derived.Source: §1 · p. 2
• For a target supported on K points, beyond a finite threshold there are no spurious local minima, and every spurious stationary point is unstable under unweighted particle gradient flow.For relaxed weights, the threshold is sharp and simply N ≥ K.Source: §1 · p. 2
• Finite-support results also establish that every distinct particle location is active, while collisions that are spurious are neither local minima nor stable under unweighted particle gradient flow.Several structural results extend beyond the Gaussian kernel under stated kernel conditions.Source: §1 · p. 2

MMD quantisation and optimisation

• Earlier work primarily studies approximation accuracy from particular finite-particle constructions, while related optimisation papers are primarily algorithmic rather than landscape characterisations.The relaxed formulation is closely related to mean shift interacting particles, corresponding to preconditioned gradient descent on the weight-optimised objective.Source: §2 · p. 3

Optimisation landscapes

• Alon et al. (2021) establish benign landscapes for structured Gaussian parametric models, whereas this setting allows arbitrary finite target atoms with arbitrary locations and unequal weights, or merely compact support for approximate guarantees.Chen et al. (2026) instead study stationary points, superconvergence, and guarantees for computing them by MMD gradient flow.Source: §2 · p. 3

Dynamics and proof connections

• Measure-valued dynamics work examines population-level behaviour or convergence of particle approximations, while this paper studies the objective landscape for a fixed finite particle count.Proof techniques also adapt interpolation and Chebyshev-system ideas from sparse-measure recovery; signed Gaussian witness functions are distinct from positive Gaussian mixtures.Source: §2 · p. 3

Problem setup

• The objective F(w,x) is the squared MMD between the weighted atomic measure and the target, expressible as the squared RKHS norm of a witness function Δ.The normalized nonzero witness attains the variational supremum and translates stationarity into constraints on Δ.Source: §3 · p. 4
• The finite-particle approximation uses locations x=(x1,…,xN) and weights w in the simplex, defining a weighted atomic measure.The target may be arbitrary compactly supported or admit a finite atomic representation; the latter is called well specified.Source: §3 · p. 4
• The analysis uses X=R and the Gaussian kernel exp(−(x−y)^2); general bandwidths reduce to this normalization by rescaling locations and support-diameter bounds.Instability under the unweighted particle flow means arbitrarily small location perturbations can eventually leave a neighbourhood of the stationary configuration.Source: §3 · p. 4

4 RELAXED VARIANT

• At a nonzero-weight particle, weight optimality and location stationarity make both Δ and its derivative vanish, so each such location contributes a zero of multiplicity at least two.The weight-optimised relaxed objective underlies mean shift interacting particles.Source: §4 · p. 4

4.1 Structural properties of stationary points

• For ETP kernels, nonzero stationary points have Δ≤0, every distinct location is active with positive total weight, and all particles lie in the target support's convex hull.At nonzero local minima under the stated kernel assumptions, all weights are nonzero and locations distinct; collisions are unstable under the unweighted flow.Source: §4.1 · pp. 4–5

4.2 Well-specified targets

• For a well-specified target supported on K points, N≥K rules out spurious local minima and makes every spurious stationary point unstable under unweighted particle gradient flow.The zero-counting argument uses distinct, active particles and the witness function's upper bound on zeros.Source: §4.2 · p. 5

4.3 Arbitrary compactly supported targets

• For compactly supported targets of diameter at most D, a finite threshold Nε(D) ensures every local minimum has MMD below ε, and every stationary point with MMD at least ε is unstable.The argument bounds the number of active locations supported by a nonzero stationary witness, then uses collisions to establish instability.Source: §4.3 · p. 5

5 SIMPLEX VARIANT

• For sufficiently many particles, every simplex local minimum has MMD < ε for compactly supported targets, while stationary points with MMD ≥ ε are unstable under the unweighted particle flow.The threshold is uniform over targets whose support diameter is at most D.Source: §5 · p. 6

5.1 Structural properties of stationary points

• At nonzero stationary points, each distinct particle location is a strict local maximum of the witness at the common KKT level; collisions are neither local minima nor stable under the unweighted flow.These structural properties allow the argument to control the number of distinct active locations.Source: §5.1 · pp. 6–7

5.2 Arbitrary compactly supported targets

• For arbitrary compactly supported targets, a lower bound on MMD yields a positive lower bound on the KKT level, which bounds distinct active locations; excess particles then force a collision and rule out poor minima or stable stationary points.The proof relates MMD to the KKT level and then to the number of distinct active locations.Source: §5.2 · pp. 7–8

5.3 Well-specified targets

• For finitely supported targets, every spurious stationary point must miss a target atom by a fixed neighborhood independent of N, giving a positive MMD lower bound and, beyond a finite threshold, no spurious local minima and no stable spurious stationary points.The separation bound enables reuse of the KKT-level and active-location argument.Source: §5.3 · p. 8

6 CONCLUSION

• For one-dimensional Gaussian-kernel MMD, sufficiently many particles give approximate guarantees for compactly supported targets and exact guarantees for well-specified targets; the conclusion also extends the relaxed well-specified threshold to ETP kernels.The authors connect these guarantees to maximum principles and collision results, which constrain poor stationary configurations.Source: §6 · p. 8
• The strongest results are restricted to one dimension because the proofs rely on zero counting and particle-sign ordering; finite-location results also do not directly control trajectories whose particles escape to infinity.The authors identify multidimensional analogues and control of particle escape as future work.Source: §6 · p. 8

AI use statement

• The authors report using generative AI tools for proof-writing assistance, method implementation, research feedback, literature work, and formatting, while excluding them from developing theoretical models and mathematical claims.They state that assisted work was checked and that they take responsibility for the final content.Source: AI use statement · p. 9

A.1 Kernel basics

• The appendix defines RKHSs through the reproducing property and defines MMD using kernel mean embeddings.It also recalls conditions for the existence of a mean embedding.Source: §A.1 · p. 12

A.2 Total positivity, variation diminishing, Chebyshev systems

• The Gaussian kernel is TP∞, and the variation-diminishing property bounds zeros of an integral transform by the sign variations of its input under the stated assumptions.The appendix presents this property as a tool for proving earlier propositions.Source: §A.2 · p. 13
• Chebyshev systems characterize function families whose nontrivial linear combinations have at most n − 1 distinct zeros; the appendix also defines extended Chebyshev systems and ETP kernels.It states that the Gaussian kernel is ETP.Source: §A.2 · pp. 13–14

A.3 Zero counting using complex analysis and specialisation to Gaussian-type functions

• Under the stated growth bound, Jensen’s formula bounds the number of zeros of a holomorphic function inside a disc.The proposition assumes the function is nonzero at the origin and holomorphic on a domain containing a larger disc.Source: §A.3 · p. 14

B.1 Relaxed Variant

• At a nonzero stationary point under an ETP kernel, every distinct particle location is active with positive total weight, and particles lie in the convex hull of the target support.The zero-counting argument also shows that each such location is a zero of multiplicity exactly two.Source: §B.1 · pp. 14–15
• At a nonzero local minimum, all optimising weights are nonzero and particle locations are distinct under the stated analytic-universal or ETP well-specified assumptions.If coincident particles were present, their weights could be merged to create a zero entry without changing the measure or objective.Source: §B.1 · p. 15
• Under Proposition 1’s assumptions, a spurious collision is unstable under unweighted particle gradient flow.Along the considered perturbation, the displacement derivative has the same sign as the displacement.Source: §B.1 · pp. 15, 17
• Beyond a finite threshold depending on support diameter and ε, every local minimum has MMD < ε, while stationary points with MMD ≥ ε are unstable under unweighted particle gradient flow.The threshold is uniform over compactly supported targets with diameter at most D.Source: §B.1 · p. 17
• The computable threshold has asymptotic rate O(log(1/ε) log log(1/ε)); exceeding the zero-count bound forces a collision, precluding local minimality and implying instability.The collision conclusion follows from Propositions 2 and 3.Source: §B.1 · p. 17

B.2 Simplex Variant

• There exist α and β such that H(x) < 1 for all x outside {x*_1,...,x*_K}; separately, at a stationary point of F with F(w,x)>0, 0<c<1.Source: §B.2 · pp. 17–18
• At positive-MMD stationary points, every distinct particle location is active, and the discrepancy potential has value c, zero first derivative, and negative second derivative there.The potential is bounded above by c everywhere, and 0 < c < 1.Source: §B.2 · pp. 18–19
• For a finitely supported, well-specified target, every spurious stationary point misses some target atom by at least a target-dependent positive radius.The proof establishes a neighborhood around the target configuration containing no spurious stationary points.Source: §B.2 · pp. 19–20
• The cited fragments show “1 −2 log c” and state that combining “this” with (11) gives the stated bound; they do not provide its full form or conditions.Source: §B.2 · pp. 21, 23
• If a target atom is missed by a neighborhood of the radius specified for the target, the MMD has a strictly positive lower bound.The bound follows by choosing the kernel construction parameters so that (ar)^(-2m) is less than the target atom's weight.Source: §B.2 · p. 23
• A positive lower bound on KKT level c bounds the number of distinct active particle locations.Source: §B.2 · p. 24
• For a well-specified problem, beyond finite N(P*), F has no spurious local minima over simplex weights, and every spurious stationary point is unstable under unweighted particle flow.Source: §B.2 · p. 25
• For targets of support diameter at most D, N>Nε(D) ensures simplex local minima have MMD < ε; stationary points with MMD ≥ ε are unstable under unweighted particle flow.The guarantee is uniform over targets with compact support of diameter at most D.Source: §B.2 · p. 25
• Nε(D) = O(ε^-4 log(1/ε)) as ε → 0 for fixed D, the asymptotic particle-threshold rate derived for the simplex guarantees.Source: §B.2 · pp. 25–26

C PRELIMINARY ANALYSIS OF ASYMPTOTIC STATIONARITY

• The compactification retains both particle locations and pairwise distances, allowing a convergent subsequence of configurations and weights to be considered.The simplex constraints also bound the sequence of constants c_n.Source: C PRELIMINARY ANALYSIS OF ASYMPTOTIC… · p. 26
• If any particle escapes to infinity with nonzero limiting weight, nonzero-weight particles cannot remain at finite positive distances from one another.For particles escaping to the same infinity, an outermost particle in a separated group would experience an uncancelled outward force, contradicting stationarity.Source: C PRELIMINARY ANALYSIS OF ASYMPTOTIC… · p. 26
• The maximum-principle and collision conclusions extend essentially unchanged to asymptotic configurations, while Proposition 16 holds in a stronger form.These extensions are described for the simplex-constrained setting.Source: C PRELIMINARY ANALYSIS OF ASYMPTOTIC… · pp. 26–27
• Escaping locations contribute I c^2 ≤ c to limiting squared MMD, but an additional I c term can obstruct extending Proposition 14.Equal-weight particles at locations x_j^n = jn give c = 1/N and I = N, so I c = 1; limiting stationarity alone therefore does not control escape.Source: C PRELIMINARY ANALYSIS OF ASYMPTOTIC… · p. 27

• The experiments are illustrative, since finite numerical searches cannot establish that additional local minima do not exist.The observed landscapes are qualitatively consistent with the theory.Source: D NUMERICAL ILLUSTRATION · p. 27
• The experiments used 30 three-atom targets with varying atom separations and masses, plus a continuous Unif[0, 12] target, testing N = 1–9 and N = 1–16, respectively.The three-atom targets have K = 3, so Theorem 4 guarantees no spurious relaxed minima for N ≥3; uniform-target kernel integrals were evaluated analytically.Source: D NUMERICAL ILLUSTRATION · p. 27
• Each problem used 96 initialisations, sharing position starts across formulations and drawing initial weights from Dirichlet(3, . . . , 3) to reduce boundary starts.Atomic-target starts included subinterval schemes aimed at finding minima that fail to represent one of the target atoms.Source: D NUMERICAL ILLUSTRATION · p. 27
• Candidate minima were refined and retained only after stringent gradient, Newton-correction, and Hessian checks, with positive weights additionally required in the simplex case.Atomic-target candidates used 100-digit Newton refinement; retained points were rechecked at 130 digits.Source: D NUMERICAL ILLUSTRATION · p. 28
• For relaxed three-atom targets, searches retained only exact or numerically indistinguishable minima for N ≥3; simplex errors persisted for some targets, while uniform-target minima declined with N in both formulations.The figure plots the largest retained local-minimum MMD across 30 three-atom targets and the retained-minimum MMD for the uniform target.Source: D NUMERICAL ILLUSTRATION · p. 28, Figure 1 · p. 28