PaperlayerView exact revision on arXiv

Source-linked AI summary

Characterizing Heterogeneous Rates in Finite Mixture Estimation via Partial Optimal Transport

Dung Le, Huy Nguyen, Trang Pham, Alessandro Rinaldo, Nhat Ho

arXiv:2609.16622v1math.STstat.MLstat.TH

TL;DR

Wasserstein-based analyses capture worst-case rates but not the heterogeneous convergence of finite-mixture components. The paper introduces VPOT, using local partial transport to establish configuration-dependent rate bounds for maximum likelihood estimation. Favorable configurations attain the parametric rate, while the most singular recovers the classical worst-case rate.

  • Problem

    Wasserstein-based analyses assign a common worst-case rate, although locally isolated mixture components may converge substantially faster than competing groups.

    Source: §1 · p. 3

  • Method

    VPOT compares mixing measures in extended Voronoi neighborhoods using partial optimal transport to accommodate unequal local masses, with the discrepancy power adapted to local competition.

    Source: §1 · p. 4

  • Results

    Local rates range from n^-1/2 in favorable configurations to the classical n^-1/(4d0+2) rate in the most singular configuration.

    Source: §1 · pp. 4–5

  • Takeaways & Limitations

    The results characterize configuration-dependent parameter recovery in arbitrary fixed dimension without requiring mixing weights bounded away from zero or prior knowledge of the true component count.

    Source: §5 · p. 24

    The minimax lower bound establishes optimality for estimating the mixing measure as a whole, not separately for local POT discrepancies or individual parameters.

    Source: §3.2 · p. 17, §5 · p. 24

Abstract

from arXiv · show

Parameter estimation in finite mixture models can exhibit highly heterogeneous convergence behavior: locally isolated components may be estimated substantially faster than groups of competing components. Existing analyses based on Wasserstein distances typically characterize only the worst-case rate and therefore do not fully capture this local heterogeneity. In this paper, we introduce a Voronoi-based partial optimal transport (VPOT) framework for obtaining refined local and global convergence guarantees for the maximum likelihood estimator of the mixing measure. The key geometric idea is to localize the comparison of two mixing measures to extended Voronoi neighborhoods and use partial optimal transport to accommodate the unequal masses of their local restrictions. Within each neighborhood, the first-order POT discrepancy is raised to a power determined by the number of locally competing atoms, allowing the resulting loss to adapt to the local degree of singularity. Under suitable regularity and strong identifiability conditions, we establish uniform local and global upper bounds for a maximum likelihood estimator under the VPOT loss. These bounds reveal a configuration-dependent form of parameter estimation: less singular local configurations admit faster convergence, whereas the most singular configuration recovers the classical worst-case behavior characterized by Wasserstein-based analyses. We further establish a minimax lower bound showing that the convergence rate for estimating the mixing measure under the VPOT loss is optimal. Our results hold in arbitrary fixed dimension without requiring mixing proportions to be uniformly bounded away from zero or prior knowledge of the true number of mixture components. Overall, VPOT provides a configuration-adaptive framework for capturing heterogeneous parameter-estimation behavior in finite mixture models.

1 Introduction

Wasserstein-based analyses capture a common worst-case rate, while finite-mixture parameters can converge at different rates depending on local competition. The paper introduces VPOT and uses it to establish configuration-dependent rate bounds for maximum likelihood estimation under broader conditions.

1 Introduction

  • Earlier Wasserstein-based bounds assign parameters the same worst-case order n^-1/(4d0+2), although isolated components may be estimated substantially faster.Prior work also described heterogeneous local rates, but under one-dimensional, bounded-weight, and known-order restrictions.Source: §1 · p. 3
  • In prior local-rate interpretations, the most crowded cell has rate n^-1/(k+k∗−2k0+1), while the other k0−1 components retain the parametric rate.Because the fitted-cell count is random, these component-wise rates are interpretations of an expected-loss bound, not deterministic rates.Source: §1 · p. 3
  • The analysis studies the MLE with at most k atoms without requiring the true mixture order or weights bounded away from zero, and allows arbitrary fixed parameter dimension.These conditions broaden earlier heterogeneous-rate analyses, which assumed a one-dimensional parameter space.Source: §1 · p. 3
  • VPOT compares mixing measures within extended Voronoi neighborhoods using partial optimal transport, which handles unequal local masses while retaining location and aggregate-weight discrepancies.The local first-order POT discrepancy is powered according to the number of components in the cell, adapting the loss to local singularity.Source: §1 · p. 4
  • Local parameter rates depend on fitted and true atom counts in Voronoi cells: one fitted and one true atom yield n^-1/2, while the most singular configuration has rate n^-1/(4d0+2).Source: §1 · pp. 4–5

2 Preliminaries

The preliminaries define VPOT by comparing mixing measures locally on extended Voronoi cells, then establish its identifying properties and a uniform density-estimation bound used for parameter rates.

2 Preliminaries

  • A finite mixing measure records atom locations and weights, while restriction to a Borel region retains only atoms located there.The preliminaries also define the associated mixture density and distribution functions.Source: §2 · pp. 5–6

2.1 (Voronoi-based) Partial Optimal Transport

  • POT transports the smaller measure's full mass to a submeasure of the larger and accounts for the unmatched mass; equal-mass POT reduces to Wasserstein distance.This lets local comparisons retain both component-location differences and aggregate mixing-weight differences.Source: §2.1 · p. 6
  • Zero local POT discrepancies across a cover imply equality of the mixing measures, and the VPOT discrepancy itself is zero only when the measures are equal.The local comparisons establish equality on each covering region, which yields equality on the full support space.Source: §2.1 · pp. 6–9
  • Direct cell restriction is unstable because arbitrarily close atoms can lie on opposite sides of a boundary; nested extensions provide a boundary avoiding limiting cluster locations.Among 2k candidate extension boundaries, at least one avoids the limiting cluster locations in the sequence argument.Source: §2.1 · p. 7
  • VPOT compares measure restrictions within extended Voronoi neighborhoods using partial optimal transport, accommodating local restrictions with unequal masses.POT matches as much mass as possible and accounts separately for unmatched mass; unlike normalized Wasserstein comparisons, it retains local mass differences.Source: §2.1 · pp. 6–7

2.2 Density Estimation Rate

  • Under uniform Lipschitz and non-heavy-tail assumptions, the MLE's worst-case expected L1 density error over G≤k(Γ) decreases at rate (log n/n)^1/2.The stated conditions include standard multivariate Gaussian and Student-t families.Source: §2.2 · p. 9
  • The uniform density bound is combined with the VPOT-to-density comparison to control mixing-measure error at rates reflecting local component configurations.VPOT is constructed to account for the local configuration of mixture components.Source: §2.2 · p. 10

3 Uniform Bounds for Finite Mixture Estimation

The section develops VPOT-based upper bounds adapted to local component configurations and a minimax lower-bound argument for estimating the mixing measure as a whole. The lower bound matches the MLE upper bound at order n^-1/2, but does not establish minimax optimality for individual mixture-parameter rates.

3.1 Uniform Upper Bound

  • Assumption C(p) combines smoothness, strong identifiability, and uniform continuity of higher-order derivatives; multivariate location families have verifiable sufficient conditions.These conditions make mixture-distribution discrepancies uniformly informative about mixing-measure perturbations, even when components approach one another.Source: §3.1 · p. 11
  • A local and global L1 lower bound linking mixture densities to VPOT transfers density-estimation guarantees to mixing-measure estimation.Under the stated assumptions, the MLE has uniform VPOT convergence rate n^-1/2, up to a logarithmic factor.Source: §3.1 · pp. 12–13
  • Local rates depend on component competition within Voronoi cells: one-to-one matching gives n^-1/2, while more concentrated clusters slow convergence, and less singular cases are faster than the worst case in.The local rate is determined by cell-level component counts rather than only by the number of excess components.Source: §3.1 · pp. 13–14
  • Globally, VPOT retains heterogeneous rates over the full parameter space, ranging from n^-1/2 for isolated matches to the worst-case rate when all true atoms overlap, consistent with prior finite-mixture results [33].Components in less singular neighborhoods converge faster than those in larger local clusters, provided the corresponding transported masses are nonvanishing.Source: §3.1 · p. 15

3.2 Minimax Lower Bound

  • The minimax lower-bound argument constructs least-favorable local perturbations and uses Assumption D(p) to control their likelihood expansions; Gaussian and Student-t location families satisfy the condition for every finite p.The perturbations are designed so lower-order effects cancel and the resulting statistical models remain difficult to distinguish.Source: §3.2 · pp. 15–17
  • Assumption D(p) guarantees statistically indistinguishable local alternatives with prescribed parameter-space separation, ruling out uniformly faster convergence than the stated rate.Source: §3.2 · p. 17
  • The upper and lower bounds match at order n^-1/2 under VPOT, establishing minimax optimality for estimating the mixing measure as a whole.The result complements Theorem 1's upper bounds.Source: §3.2 · p. 17
  • The total VPOT lower bound does not establish optimality for individual parameters, whose separate local POT lower bounds are left for future work.The local terms depend on the random fitted-atom configuration and extended Voronoi construction, requiring a more delicate localization argument.Source: §3.2 · p. 17

4 Proof of Theorem 1

The proof establishes local and global bounds relating VPOT to mixture-density differences, then applies them to control the MLE’s VPOT estimation error.

4 Proof of Theorem 1

  • The proof is organized into three steps: local and global VPOT-to-density bounds, followed by their application to the MLE.Source: §4 · p. 18
  • The local argument fixes a reference mixing measure and uses coarse-graining trees, expansions, Voronoi neighborhoods, and strong identifiability to bound VPOT by the density difference.The proof treats trees whose root scale tends to zero separately from those whose root scale does not.Source: §4 · p. 18
  • The global argument extends the VPOT-to-density bound over the full mixing-measure class by contradiction, compactness, identifiability, and a tree argument with varying Voronoi cells.The reference measure and its Voronoi cells vary along the sequence in this argument.Source: §4 · pp. 18, 22
  • The proof applies the global bound to the MLE directly, while the local case requires showing the bound still holds when the estimator is not close to the true measure.Together with the density-estimation result, these bounds yield the MLE’s stated VPOT bounds.Source: §4 · p. 18

5 Conclusion

The paper introduces VPOT to capture heterogeneous convergence rates in finite-mixture estimation. Its bounds yield configuration-dependent rates and an optimal mixing-measure rate under VPOT, within the stated scope.

5 Conclusion

  • VPOT localizes comparisons to Voronoi neighborhoods, uses partial optimal transport for unequal local masses, and raises each local discrepancy to a power reflecting competing atoms.Source: §5 · p. 23
  • Up to logarithmic factors, local configurations indexed by r_i have estimation rates of order n^-1/(2r_i), while the most singular case recovers n^-1/(4d_0+2).Less singular configurations admit strictly faster rates.Source: §5 · p. 23
  • The minimax lower bound makes the rate for estimating the mixing measure as a whole under VPOT optimal.Source: §5 · pp. 23–24
  • The results apply in arbitrary fixed dimension without a uniform lower bound on mixing weights or prior knowledge of the true component count.The optimality result does not establish separate minimax lower bounds for local POT discrepancies or individual mixture parameters.Source: §5 · p. 24

A Coarse-Graining Tree Structure and Application

The coarse-graining tree encodes clusters at successive asymptotic distance scales and supports bounds on partial optimal transport between measures.

A.1 Coarse-Graining Tree and Notation

  • Nested balls form the tree: decreasing their radius yields finer clusters, and each node’s diameter describes its cluster scale.Source: §A.1 · pp. 24–25
  • The coarse-graining tree represents nested clusters of support-point sequences, with its levels recording the asymptotic scales at which clusters separate.Its combinatorial structure is fixed along the selected subsequence.Source: §A.1 · p. 25

A.2 Partial Optimal Transport Bounds

  • A recursive tree argument yields upper and lower estimates for the q-POT discrepancy between measures.Source: §A.2 · p. 25
  • The partial-transport coupling matches within each child up to the smaller local mass and transports mass across children when their masses differ.The tree-based bounds account for unmatched mass as well as transport cost.Source: §A.2 · pp. 25–26

A.3 Coarse-graining expansion

The coarse-graining expansion expresses the mixture CDF around a tree node using coefficients and a remainder, under differentiability and uniform-continuity conditions.

A.3 Coarse-graining expansion

  • When F is continuously differentiable through order 2k and its derivatives of total order 2k satisfy the stated uniform-continuity condition, it admits an order 2k coarse-graining expansion at J.The expansion consists of coefficients c_J and a remainder R(x | J).Source: §A.3 · p. 27
  • The expansion coefficients are constructed recursively by transferring child-node expansions to their parent using Taylor expansion.Source: §A.3 · p. 28
  • The proof controls the remainder as o(∥c_J∥ε_J^2k), while the second coefficient component is asymptotically negligible relative to the first.Source: §A.3 · pp. 30–31

B Proof of Theorem 2

The proof of Theorem 2 combines a local asymptotic normality construction with a two-point argument and a lower bound relating VPOT to Wasserstein distance. Together, these steps establish the stated lower-bound scale under the theorem’s conditions.

Proof of Theorem 2

  • The constructed family of experiments is locally asymptotically normal at u = 0, with Γ_n bounded away from zero and infinity.The construction uses the one-dimensional family H(u) and the theorem’s regularity assumptions.Source: B Proof of Theorem 2 · p. 32
  • ε_n = n^-1/(4d_0+2) is used to prove part (a) of the theorem.The proof constructs two mixing measures in a W1-neighborhood of G0 and applies a Le Cam two-point argument.Source: B Proof of Theorem 2 · pp. 32–33
  • The proof combines the estimator lower bound with the VPOT–Wasserstein comparison to obtain the theorem’s conclusion.The two-point argument uses contiguity, and the final step invokes Hölder’s inequality.Source: B Proof of Theorem 2 · pp. 33–34
  • For measures satisfying the stated neighborhood condition, VPOT is bounded below by a constant times W1 raised to 2d_0+1.The constant depends on Γ, G0, q, and κ.Source: B Proof of Theorem 2 · p. 34

C Proof of density estimation rate

This appendix proves the density-estimation rate using local bracketing entropy, following a standard empirical-process strategy while deriving entropy bounds for the mixture parameterization. It also fixes the model-class and entropy notation used in that argument.

C Proof of density estimation rate

  • The appendix uses a local bracketing-entropy empirical-process approach, adapting the standard strategy in to model-specific entropy bounds.The authors provide details because the entropy bounds depend on the mixture model’s parameterization.Source: C Proof of density estimation rate · p. 36

C.1 Model classes and entropy notation

  • The mixture model is represented by a probability simplex and component parameters, inducing a density class from mixing measures with at most k support points.The true mixing measure is fixed in the setup, and the corresponding density class is used in the local entropy argument.Source: §C.1 · p. 36
  • The section defines covering and bracketing numbers and their logarithmic entropies, using the L2(m) norm for density classes unless stated otherwise.Here m denotes Lebesgue measure.Source: §C.1 · p. 36

C.2 Bracket Entropy Bounds

This section bounds the entropy of the mixture-density class by constructing finite approximations from parameter nets and controlling tails. The resulting bracketing entropy grows logarithmically in inverse scale.

C.2 Bracket Entropy Bounds

  • A finite-dimensional ε-net of the parameter space yields a uniform-norm net for the mixture-density class through the component densities’ uniform Lipschitz property.The resulting log covering number is bounded by a constant times log(1/ε).Source: §C.2 · p. 37
  • The bracketing construction combines the density envelope and tail assumption to control the brackets outside a bounded region.The proof bounds the entropy using the resulting tail control and completes the bracketing estimate.Source: §C.2 · p. 37
  • The bracketing entropy of P_k(Γ) under the L1 norm is bounded by a constant times log(1/ε).The proof transfers L1 brackets to Hellinger brackets using h² ≤ ∥·∥1.Source: §C.2 · p. 38

C.3 Empirical Process Analysis of the MLE

The empirical-process analysis uses local bracketing entropy and a maximal inequality to control the likelihood estimator’s stochastic error. It derives an exponential tail bound and then an expected-error bound to prove the density-estimation upper bound.

C.3 Empirical Process Analysis of the MLE

  • A bracketing-entropy integral connects the localized density-class complexity to the empirical-process analysis.The section uses this integral as a key ingredient in controlling the MLE’s stochastic term.Source: §C.3 · p. 38
  • A maximal inequality for localized mixture densities is combined with the entropy bound to prove the upper bound in equation (6).The inequality is stated as a specialization of results in [43].Source: §C.3 · p. 39
  • The proof establishes an exponential tail bound and integrates it to obtain an expected estimation-error bound.The likelihood inequality relates the Hellinger error to the empirical-process term.Source: §C.3 · pp. 39–40

D.1 Separation Lemmas

The section establishes polynomial separation tools and uses them to show that sufficiently close mixing measures have distinct nearby matches for every true atom. It also proves a rank and norm-separation property on separated configurations.

D.1 Separation Lemmas

  • The auxiliary multivariate polynomial separation results are adapted from [45, Lemma A.8] and generalize arguments in [14, Appendix D].Source: §D.1 · p. 41
  • The matrix A(θ1, . . . , θj) has rank γ2, established by showing any vector in its kernel is zero.The proof constructs polynomials to isolate coefficients, using a derivative-vanishing lemma for factors of the form (xi − ai)^pi.Source: §D.1 · pp. 41–42
  • On the compact set of ε-separated vectors, ∥A(θ1, . . . , θj)Λ∥ is bounded above and below by constant multiples of ∥Λ∥.The lower bound follows from continuity, compactness, and the rank result for separated configurations.Source: §D.1 · pp. 41–42
  • If W1(G, G0) < πminδ/2, every true atom has a distinct estimated-atom match within δ/2.The matching map is injective under the stated separation condition.Source: §D.1 · p. 42

D.2 Proof of Proposition 3

The proof of Proposition 3 establishes identifiability for location families using characteristic functions and a linear-independence result for exponential-polynomial functions. It also derives a uniform bound on multivariate CDF derivatives of order p + 1 under the stated condition.

D.2 Proof of Proposition 3

  • A linear combination of exponential-polynomial functions with distinct real shifts can vanish for all t only when every coefficient is zero.The lemma generalizes the main component of the demonstration in [6, Lemma 3].Source: §D.2 · p. 43
  • For location families, the characteristic-function argument shows that vanishing mixture derivatives force all coefficients cj,α to be zero.The proof uses that φ is nonzero near the origin, then projects the locations onto directions with pairwise distinct projections and applies the linear-independence lemma.Source: §D.2 · p. 44
  • The argument extends the vanishing result from an open dense set of directions to all directions, then to every polynomial coefficient.Source: §D.2 · p. 45
  • Under equation (8), the multivariate CDF derivatives of order p + 1 are uniformly bounded over x and γ.The proof differentiates under the integral sign to obtain the bound.Source: §D.2 · p. 45
Loading 2609.16622v1…