On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport

Lenaic Chizat; Francis Bach

Source-linked AI summary

On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport

Lenaic Chizat, Francis Bach

arXiv:1805.09545v2May 24, 2018math.OCcs.NEstat.ML

TL;DR

The paper studies how to find global minimizers when convex measure optimization is represented by finitely many particles, despite the resulting non-convexity and infinite-dimensional difficulty. It analyzes the many-particle limit of particle gradient flows and shows asymptotic global optimality under suitable initialization and structural assumptions, with favorable synthetic particle complexity.

Problem
Measure-based convex optimization captures sparse representations but remains difficult because its variable is infinite-dimensional, limiting existing approximate-minimization methods.
Method
The paper studies the many-particle limit of gradient flows, using a lifted measure formulation and Wasserstein gradient-flow analysis.
Results
The gradient flow converges asymptotically to global minimizers, while synthetic experiments find global minimizers with only slight over-parameterization.
Takeaways & Limitations
The results provide a first consistency principle for commonly used non-convex optimization methods and support the relevance of particle gradient flows.
Takeaways & Limitations
The analysis focuses on idealized continuous-time gradient flows with exact gradients and gives qualitative, non-quantitative convergence results.

Abstract

from arXiv · show

Many tasks in machine learning and signal processing can be solved by minimizing a convex function of a measure. This includes sparse spikes deconvolution or training a neural network with a single hidden layer. For these problems, we study a simple minimization method: the unknown measure is discretized into a mixture of particles and a continuous-time gradient descent is performed on their weights and positions. This is an idealization of the usual way to train neural networks with a large hidden layer. We show that, when initialized correctly and in the many-particle limit, this gradient flow, although non-convex, converges to global minimizers. The proof involves Wasserstein gradient flows, a by-product of optimal transport theory. Numerical experiments show that this asymptotic behavior is already at play for a reasonable number of particles, even in high dimension.

1 Introduction

The paper studies particle gradient descent for convex optimization over measures, showing that its many-particle Wasserstein gradient-flow limit can achieve global minimizers under suitable initialization and assumptions. It applies this framework to neural-network training and sparse deconvolution, with numerical evidence that the asymptotic regime appears for modest particle counts.

Problem setting: The framework represents sparse combinations from a parameterized set as an unknown signed measure minimizing a smooth convex loss, optionally with a convex regularizer.The parameter space is an R^d domain, and the feature map θ ↦ φ(θ) is differentiable.
Applications and experiments: The theory covers single-hidden-layer neural networks with sigmoid or ReLU activations and sparse deconvolution, while experiments indicate the asymptotic behavior for small m, even in high dimension.The applications impose conditions on the initialization pattern.
Particle gradient descent: The particle method discretizes the measure into m weighted particles and applies classical gradient descent to their positions and weights, although the resulting objective is non-convex.The paper’s goal is to determine when this practical method nevertheless inherits the convex problem’s global structure.
Many-particle limit: As m →∞, the gradient flow of the particle objective is characterized by a Wasserstein gradient flow, connecting the dynamics to optimal transport.This many-particle limit is the central intermediate object for the global convergence results.
Global optimality: Under assumptions on φ and the initialization, convergence of the Wasserstein gradient flow implies that its limit is a global minimizer, with corresponding asymptotic guarantees for particle dynamics.The results are qualitative and nonquantitative, and concern ideal continuous-time gradient flow with exact gradients.

2 Particle gradient flows and many-particle limit

This section formulates particle gradient flows for measure-valued optimization and connects them to Wasserstein gradient flows. Under the stated assumptions, empirical particle measures converge to the unique Wasserstein flow when their initial measures converge in W2.

Problem setup: The framework studies convex functionals of non-negative finite measures on Ω⊂R^d under smooth-loss, regularity, and controlled-growth assumptions.The assumptions require differentiability and local Lipschitzness of the loss differential, semiconvexity of V, and locally controlled derivatives with sublinear growth.
Particle gradient flows: A particle gradient flow evolves m positions by the subgradient flow of the particle functional, with a 1/m-scaled inner product assigning mass 1/m to each particle.The scaling is chosen to facilitate the limit m →∞, and non-smooth V leads to a subgradient flow.
Particle gradient flows: For every initialization, the particle functional admits a unique gradient flow, whose energy decreases according to the squared velocity.The particle velocities use minimal-norm subgradients; when V is non-smooth, the flow corresponds to a continuous-time forward-backward minimization scheme.
Wasserstein gradient flows: Particle dynamics extend to arbitrary probability measures through a continuity equation driven by the Wasserstein subdifferential of F.The particle velocity field is the field of minimal-norm subgradients, and atomic particle flows are recovered as Wasserstein gradient flows of their empirical measures.
Many-particle limit: If empirical initial measures converge in W2 and particle positions remain initialized in Qr0, their flows converge as m →∞ to the unique Wasserstein gradient flow from the limiting measure.Independent samples from µ0 provide an example of suitable initializations because their empirical distributions converge almost surely in W2.

3 Convergence to global minimizers

This section shows that Wasserstein gradient flows can converge to global minimizers despite stationary points being generally non-optimal. The result requires shared homogeneity and a separated initialization, and applies to both 2-homogeneous and partially 1-homogeneous settings.

General convergence principle: Stationary points of Wasserstein gradient flows are not generally global minimizers, even when R is convex.Global minimizers over nonnegative measures require a stronger characterization than stationarity.
General convergence principle: Global convergence relies on shared homogeneity of Φ and V and a separation property of the initialization support.The separation property is preserved by the dynamics and helps the flow escape neighborhoods of non-optimal points.
2-homogeneous setting: The 2-homogeneous theorem also gives lim t,m→∞F(µm,t) = min µ∈M+(Ω) F(µ), with the limits in particle number and time interchangeable.Thus convergence is not conditioned on a relative growth rate of the two parameters.
Partially 1-homogeneous setting: An analogous theorem in the partially 1-homogeneous setting yields lim t,m→∞F(µm,t) = min µ∈M+(Ω) F(µ).This setting covers lifted problems with bounded φ, including sparse deconvolution and sigmoid neural networks, under a different initialization condition.

4 Case studies and numerical illustrations

The case studies apply the theory to sparse deconvolution and neural-network training, showing global-minimizer convergence under specified conditions and favorable particle complexity. Experiments illustrate success with sufficiently many particles, while classical ReLU parameterization prevents defining gradient flows.

Sparse deconvolution: Sparse deconvolution yields a global minimizer whenever the Wasserstein gradient-flow projection weakly converges, assuming sufficient filter smoothness and support containing {0} × Θ.The loss uses an L2 data-fidelity term, absolute-weight regularization, and a signed-measure representation induced by particle parameters.
Sparse deconvolution: 5 ground-truth spikes are reconstructed experimentally on the 1-torus using a low-pass filter, noisy observations, forward-backward integration, and uniform-grid initialization.The filter is a Dirichlet kernel of order 7, and the particle flow is tested with different particle counts.
Sigmoid activation: Sigmoid networks converge to a global minimizer if the Wasserstein flow converges in W2, under finite feature moments, zero-weight initialization support, and a boundary condition.The sigmoid model uses optional absolute-weight regularization, corresponding to an ℓ1 penalty.
ReLU activation: ReLU’s classical parameterization has discontinuous dΦ, preventing gradient-flow definition even for finitely many particles.A differentiable reparameterization with signed-square coordinates and a quadratic regularizer instead yields global-minimizer convergence under stated moment, support, and regularity assumptions.
Particle complexity: Exponential particle complexity is unavoidable for convex fixed-grid optimization, whereas observed synthetic comparisons favor particle gradient flows.Figure 3 compares excess loss at convergence against particle count, with the simplest minimizer using m0 particles.

5 Conclusion … A.2 Lifting to the space of probability measures

The paper establishes asymptotic global optimality for non-convex gradient flows through Wasserstein gradient-flow analysis, while the appendices formalize the measure-theoretic framework and the lifting to probability measures. The conclusion reports favorable particle complexity on synthetic problems and motivates quantitative extensions.

5 Conclusion: The studied non-convex gradient flows achieve asymptotic global optimality through a Wasserstein gradient-flow analysis of many-particle regimes.The Wasserstein formulation simplifies the many-particle setting analogously to a mean-field limit.
5 Conclusion: Favorable particle complexity on synthetic numerical problems supports the qualitative global-optimality results.The conclusion calls for quantitative results that further exploit the observed behavior.
A Introductory facts: The appendix defines measures as finite signed measures on R^d, d ≥1, with M(X) denoting those concentrated on measurable subsets X ⊂ R^d.These measure-theoretic concepts are collected for use in the proofs.
A.1 Tools from measure theory: Signed measures admit a Jordan decomposition µ = µ+ − µ−, with variation |µ| = µ+ + µ− and total variation norm |µ|(R^d).The positive and negative parts are chosen with minimal total mass.
A.1 Tools from measure theory: Pushforwards represent the distribution of a measure’s mass after a measurable displacement, and projection pushforwards give marginals.The pushforward is defined by T#µ(B) = µ(T^-1(B)).
A.1 Tools from measure theory: Wasserstein convergence on probability measures with finite second moments is stronger than weak convergence, and P_2(R^d) is complete under W_2.The appendix also notes that the functional F is continuous for W_2 under the stated assumptions.
A.2 Lifting to the space of probability measures: The lifting converts minimization over signed measures on Θ ⊂ R^(d−1) into an equivalent minimization over probability measures on a larger space Ω ⊂ R^d.This provides the technical passage from J to F introduced earlier in the paper.
A.2 Lifting to the space of probability measures: Positive p-homogeneity implies that derivatives are positively (p −1)-homogeneous and, when differentiable away from zero, u · ∇f(u) = pf(u).These homogeneity identities are used without explicit mention in the technical development.

A.2.1 The partially 1-homogeneous case · A.2.2 The 2-homogeneous case · B Many-particle limit and Wasserstein gradient flow

The appendices exploit homogeneity to relate lifted measure problems to probability or sphere measures, establish equivalent minimization formulations, and characterize the relevant projections.

A.2.1 The partially 1-homogeneous case: Partial 1-homogeneity lets minimization over nonnegative measures be replaced by minimization over probability measures.The transformation rescales the w-coordinate by the total variation mass while preserving the objective.
A.2.1 The partially 1-homogeneous case: The projection h1 maps lifted measures on R × Θ to signed measures on Θ through their w-weighted marginals.It is defined when w is integrable under the measure.
A.2.1 The partially 1-homogeneous case: The lifting is surjective, and regularizers defined by infimizing ∫V dν over h1-preimages yield equivalent minima for F and J.If the infimum defining G is attained, a minimizer of J has a minimizing lift for F.
A.2.1 The partially 1-homogeneous case: For V(w, θ) = |w|, the lifted regularization dominates the total variation of the projected signed measure.Equality holds for lifts satisfying the stated support-separation condition, including the lift in (8).
A.2.2 The 2-homogeneous case: In the positively 2-homogeneous setting on R^d, the sphere S^(d−1) replaces Θ as the domain for the projected measure.The paper introduces h2 from nonnegative measures on R^d to nonnegative measures on S^(d−1).
A.2.2 The 2-homogeneous case: The projection h2 is well-defined exactly for measures with finite second-order moments.This condition is stated as necessary and sufficient.

B.1 Proof of Proposition 2.3

The proof establishes that F_m admits a unique gradient flow with the stated properties and shows that this flow is globally defined for all time.

Existence and characterization: F_m is locally semiconvex, so standard gradient-flow theory gives a unique flow on a maximal interval [0,T[ with the claimed properties.The velocity field is characterized by the subgradient of minimal norm.
Global existence: Lower-boundedness of F_m yields estimates showing that the flow has bounded length on every bounded time interval.The estimate is derived using Jensen’s inequality.
Global existence: Therefore, finite-time maximality would contradict compactness, and the gradient flow is globally defined with T = ∞.If T were finite, u(T) would exist, contradicting maximality.

B.2 Link between classical and Wasserstein gradient flows

This section establishes that classical gradient flows of particle parameters induce Wasserstein gradient flows of the associated atomic measures. It also notes that the fixed number of atoms can prevent convergence when no sufficiently sparse atomic minimizer exists.

Continuity equation: A time-dependent mass distribution transported by a velocity field satisfies the continuity equation ∂tµt = −div(vtµt), interpreted distributionally for nonsmooth densities.The distributional formulation uses smooth compactly supported test functions.
Classical-to-Wasserstein link: If u : R+ →Ωm is a classical gradient flow for Fm, then its empirical atomic measure µm,t is a Wasserstein gradient flow of F.The induced measure is formed from the particle trajectories as µm,t = 1/m Σ_i δ_ui(t).
Proof: The atomic trajectory is absolutely continuous for W2 and is a distributional solution of the continuity equation.The proof uses the velocity vector field defined for the particle dynamics.
Limitation: The measure keeps the same number of atoms throughout the dynamics, so convergence to a minimizer is impossible if no minimizer of F has at most m atoms.This fixed-atom constraint is an explicit limitation of the particle representation.

B.2.1 Properties of the Wasserstein gradient flow (proof of Proposition 2.5)

Under Assumptions 2.1, localized functionals F^(r) have the Wasserstein regularity and subdifferential properties needed for unique gradient flows. A support-growth bound then extends these flows to the original functional globally in time and represents them by characteristic pushforwards.

Localized functional properties: For every r > 0, F^(r) is proper and W2-continuous on its closed domain, with geodesic semiconvexity controlled by λr.The semiconvexity parameter is λr = L_dR∥dΦ∥^2_∞,r + L_dΦ∥dR∥_∞,r, and these bounds may diverge as r → ∞.
Localized functional properties: Velocity fields satisfy the Wasserstein subdifferential condition v(u) ∈ ∂(F′(µ) + ι_Qr)(u) µ-almost everywhere.The indicator ι_Qr accounts for the additional normal-cone freedom at the boundary of Qr.
Existence and uniqueness: Each localized functional F^(r) admits a unique Wasserstein gradient flow under Assumptions 2.1.Potential-term semiconvexity combines with the loss-term properties to provide strong Wasserstein subdifferentials and well-posedness.
Global well-posedness: The original flow is globally well-defined and unique because its support remains within a controlled expanding set, making the first exit times satisfy lim_{r→∞} t_r = ∞.Grönwall’s lemma yields concentration within {u ∈ Θ ; dist(u, Q_r0) ≤ (r0 + C1/C2)e^(tC2)}, so every finite time horizon is covered.
Flow representation: The Wasserstein gradient flow satisfies µ_t = (X_t)#µ_0, where X solves the characteristic ODE driven by the associated velocity field.X is uniquely defined and continuous, and X(t, ·) is Lipschitz on Qr uniformly over compact time intervals.

B.3 Proof of the many-particle limit (Theorem 2.6) · C Convergence to global minimizers

The many-particle gradient flows converge, via a direct Wasserstein compactness argument, to a global Wasserstein gradient flow. The convergence proof extends over all time, while the global-minimizer analysis uses homogeneity, projected measures, and preserved initialization support.

B.3 Proof of the many-particle limit (Theorem 2.6): The paths remain inside Q_r for a positive interval because particle velocities are bounded before the first exit time.The exit time satisfies t_r ≥ (r − r_0)/(∥dΦ∥∞,r∥dR∥∞,r + L_V,r) > 0.
B.3 Proof of the many-particle limit (Theorem 2.6): Uniform W2 equicontinuity and boundedness yield a subsequence converging weakly to a curve concentrated in Q_r.Ascoli’s theorem provides a weakly continuous limit curve, with uniform convergence in the bounded-Lipschitz norm.
B.3 Proof of the many-particle limit (Theorem 2.6): Convergence of the particle velocity fields and momenta passes the continuity equation to the limiting curve.The momentum measures E_m converge weakly to E = v_tµ_tdt.
B.3 Proof of the many-particle limit (Theorem 2.6): The exit times satisfy lim_{r→∞} t_r = ∞, so the particle limits define a Wasserstein gradient flow globally in time.Monotonic decrease of F, bounded dR on a sublevel set, and Grönwall’s inequality provide the required velocity control; uniqueness completes the theorem.
C Convergence to global minimizers: The convergence-to-global-minimizers results have separate versions for 2-homogeneous and partially 1-homogeneous settings.The proofs are more straightforward in the 2-homogeneous setting, and Figure 4 illustrates the differing level-set geometries.
C Convergence to global minimizers: The global-minimizer analysis establishes optimality conditions, escape from non-optimal stationary neighborhoods, characterization of flow limits, and preservation of initialization support.The escape and limit-characterization results apply to arbitrary initializations, while support preservation addresses the theorem assumptions.
C Convergence to global minimizers: Projected measures h_i(µ) are used because they remove homogeneity redundancy and provide a more reasonable convergence object for the optimization problem.The relevant projection indices are i = 1, 2.

C.1 Optimality conditions (proof of Proposition 3.1) · C.2 A criteria to escape from non-optimal stationary points

The sections establish first-order optimality conditions for convex functionals of measures and introduce a Wasserstein-flow criterion for escaping non-optimal stationary points. The criterion applies in both finite-particle and many-particle regimes, supporting a measure-based viewpoint even for classical gradient flows.

C.1 Optimality conditions (proof of Proposition 3.1): A first-order Taylor expansion of R provides the starting formula for analyzing variations between finite-energy measures.
C.1 Optimality conditions (proof of Proposition 3.1): For a variation σ = ν − µ, the proof decomposes σ into a component absolutely continuous with respect to µ and a singular component.
C.1 Optimality conditions (proof of Proposition 3.1): A minimizing measure must satisfy F′(µ) ≥ 0 everywhere and F′(µ) = 0 µ-almost everywhere.
C.1 Optimality conditions (proof of Proposition 3.1): Under the same first-order condition, convexity makes the condition sufficient for µ to be a minimizer.
C.2 A criteria to escape from non-optimal stationary points: A criterion is introduced for Wasserstein gradient flows to escape from non-optimal stationary points.
C.2 A criteria to escape from non-optimal stationary points: The escape criterion applies in both the finite-particle regime and the many-particle limit.
C.2 A criteria to escape from non-optimal stationary points: Its validity supports using a measure-based perspective even for finite-particle, classical gradient flows.

C.2.1 The 2-homogeneous case

In the positively 2-homogeneous case, negative values of F′ identify conic regions that drive Wasserstein gradient flows away from local minima through exponential radial growth. Any weak limit of h2(µt) must instead satisfy F′(ν)=0 ν-a.e.

Local-minimum criterion: If F′(µ) is negative somewhere, a conic region R+K based on a spherical negative sublevel set provides a direction of descent for the Wasserstein gradient flow.K is the (−η)-sublevel set of F′(µ) restricted to the sphere, with η arbitrarily close to 0.
Local-minimum criterion: Positive 2-homogeneity makes the radial velocity at least r·η, so trajectories starting in R+K grow radially as |u_t| ≥ |u_0| exp(ηt).The flow remains in R+K up to the exit time from the relevant bounded-Lipschitz neighborhood.
Local-minimum criterion: Any positive initial mass in R+K causes h2(µt)(K) to grow exponentially, forcing the flow to leave the bounded-Lipschitz ball in finite time.If the spherical restriction has no zero in its range, the same argument applies with K equal to the entire sphere, provided µ0 has mass away from the origin.
Stationary points: If h2(µt) converges weakly to ν, then the limiting first variation satisfies F′(ν)=0 ν-a.e.The proof uses convergence of the velocity fields and the finite dissipation of the lower-bounded functional to show that the limiting velocity vanishes ν-a.e.

C.2.2 The partially 1-homogeneous case … C.3.2 The partially 1-homogeneous case

The paper establishes finite-time escape from local minima and characterizes stationary limits in the partially 1-homogeneous case, then proves that support-separation properties persist under Wasserstein gradient flows in both homogeneous settings.

C.2.2 The partially 1-homogeneous case: If F′(µ) is not nonnegative, a Wasserstein gradient flow sufficiently close to µ and assigning positive mass to a suitable set escapes the local minimum.The set is defined using negative sublevel sets of F′(µ) on the positive and negative weight branches.
C.2.2 The partially 1-homogeneous case: The escape set is constructed from regular negative sublevel sets whose boundaries are differentiable and have gradients bounded away from zero.The proof selects a negative regular value arbitrarily close to zero and uses the regular value theorem.
C.2.2 The partially 1-homogeneous case: The relevant mass grows at least linearly while the flow remains close, forcing it to leave every bounded BL-norm ball and making the escape time finite.The growth follows from positive weight velocities bounded below by η/2 on the selected sublevel set.
C.2.2 The partially 1-homogeneous case: If h1(µt) converges weakly to ν, then F′(ν) vanishes ν-a.e.The proof combines convergence of the velocity fields with the finite-time disappearance of mass in negative sublevel regions.
C.3 Stability of separation properties: Topological degree theory is used to show that support separation is preserved under continuous flows, including flows with discontinuous velocity fields.The support of a pushforward under a continuous map equals the image of the original support.
C.3.1 The 2-homogeneous case: In the 2-homogeneous case, a support contained in a bounded annular separator continues separating the inner and outer spheres for every positive time.The result follows by applying an abstract separation proposition to the flow and identifying the pushforward support with the image of the initial support.
C.3.2 The partially 1-homogeneous case: In the partially 1-homogeneous case, a closed support separating the two faces of a box remains separating under the flow.The abstract argument uses topological degree, while the flow application relies on coercivity and Neumann boundary conditions.

C.4 Main theorems: proofs and generalization … D.4.2 A differentiable parameterization

The theoretical results establish global optimality when projected Wasserstein flows converge weakly, while remarks identify convergence and quantitative escape criteria as open directions. The case studies verify assumptions for losses, deconvolution, sigmoid networks, and ReLU networks, including a differentiable ReLU parameterization.

C.4 Main theorems: proofs and generalization: Lemma C.15 transfers Wasserstein-flow convergence to classical particle-flow limits when m, t →∞ under concentrated initializations converging in W2.The two iterated limits use Theorem 2.6, continuity of F, and monotonicity of F along the flow.
C.4.1 The 2-homogeneous case: Weak convergence of h2(µt) implies convergence to a global minimizer with limt→∞F(µt) = F ∗.This strengthens Theorem 3.3 because Wasserstein convergence implies weak convergence of h2(µt), but not conversely.
C.4.2 The partially 1-homogeneous case: Weak convergence of h1(µt) likewise implies that its limit is a global minimizer and limt→∞F(µt) = F ∗.The proof excludes nonminimizing stationary limits by showing that nearby flows can escape a neighborhood.
C.5 Remarks: The global-convergence theorems assume flow convergence, whose proof generally requires trajectory compactness and a Łojasiewicz inequality, while W2 compactness is especially strong.The paper relaxes the topology used to formulate convergence.
C.5 Remarks: Particle flows can escape local minima when at least one particle lies in a 0-sublevel set of the current potential F ′(µ), motivating quantitative estimates of that set.The paper studies this criterion through the many-particle limit and leaves other quantitative approaches open.
D Case studies and numerical experiments: The case studies verify the framework’s assumptions for general integral losses, sparse deconvolution, and sigmoid neural networks under stated smoothness, moment, and regularity conditions.For sigmoid networks, finite moments up to order 4 ensure a bounded Lipschitz differential, while Sard-type regularity follows under stronger moment assumptions.
D.4.1 Classical parameterization: For classical ReLU parameterization, Φ is differentiable away from θ = 0 but its differential is discontinuous at (w, 0) for w ≠ 0, so particle and Wasserstein flows are not well-defined globally.Finite second moments and a density suffice for differentiability off the singular set; the potential V(w, θ) = |w| · |θ| is nevertheless positively 2-homogeneous and semiconvex.
D.4.2 A differentiable parameterization: The signed-square ReLU parameterization is positively 2-homogeneous and differentiable under finite second moments and a density, though Sard-type regularity remains an assumption.The weaker moment requirement than for sigmoid activations follows from ReLU’s piecewise linearity.

D.5 Numerical experiments : details and additional results

The experiments detail particle-complexity evaluations for sparse-spike recovery and single-hidden-layer neural-network training, comparing particle gradient flows with weight-only minimization. Additional results show that initialization strongly affects the particle count required for global recovery, consistent with the many-particle escape mechanism in the proof.

Empirical particle-complexity setting: For sparse-spike recovery, five randomly placed, 0.1-separated spikes with weights in [0.5,1.5] are recovered from noisy filtered data using particles initialized on a regular grid and forward-backward integration.Experiments vary m and stop when improvement per iteration falls below a small tolerance.
Empirical particle-complexity setting: For a single-hidden-layer sigmoid network in dimension d = 100, data lie on a sphere and labels come from a 20-neuron network; unregularized square-loss training has global minimum 0.The particle-complexity panel reports this neural-network experiment using SGD.
Empirical particle-complexity setting: The experiments compare final excess loss versus particle count for particle flows and weight-only minimization, aggregating several random realizations with a geometric average.The weight-only problem is convex, so its minimum value is independent of the minimization method; transparent green marks loss values interpreted as optimal.
Initialization effects: Uniform-grid initialization on {0} × Θ finds a global minimizer for sparse recovery once m > 7, whereas initialization on {1} × Θ can fail with m = 15 particles.The recovery problem contains m0 = 5 spikes.
Initialization effects: The proof explains initialization sensitivity through particles entering a negative-weight escape set; reaching it from positive weights requires sliding along ridges, which is guaranteed only in the many-particle limit.High particle density around ridges of F′(ν) may be required for this mechanism.
Initialization effects: Figure 6 shows that {1} × Θ initialization fails at m = 15 but succeeds at m = 30, while ({1} × Θ) ∪ ({−1} × Θ) succeeds at m = 10.These outcomes demonstrate the effect of initialization pattern on particle complexity in partially 1-homogeneous sparse-spike recovery.