PaperlayerView exact revision on arXiv

Source-linked AI summary

Minimax Rates for Conditional Generative Learning under Wasserstein Loss

Sungwon Park, Kyoung-Sook Moon, Hongjoong Kim

arXiv:2610.35490v1math.STcs.LGstat.MLstat.TH

TL;DR

Conditional generative learning requires estimating response distributions rather than only conditional means, and its Wasserstein minimax rates under unrestricted covariate marginals are established here. The analysis also treats higher loss moments and conditional Brenier maps, finding that rare unobserved covariate regions add an n^-1 term to higher-moment risk.

  • Problem

    Conditional generative learning requires estimating conditional response distributions, and matching Wasserstein minimax rates without restrictions on the covariate marginal are at issue.

    Source: §1 · pp. 1–2

  • Method

    The paper balances covariate localization with unconditional distribution estimation and uses multiscale lower bounds and transport-map stability arguments.

    Source: §1 · p. 2

  • Results

    Integrated Wasserstein loss retains rate A_n(τ) over all Borel covariate marginals, while higher moments have rate A_n(τ)^κ ∨ n^-1.

    Source: §1 · p. 2

  • Takeaways & Limitations

    Rare unobserved covariate regions can contribute an additional n^-1 term to higher-moment risk, even when response laws are simple.

    Source: §6 · p. 17

    For smooth classes with d_y > 2, computational complexity bounds for the sharp upper-bound construction remain open, and attaining the rates with efficient parameterized generators is a separate problem.

    Source: §6 · p. 17

Abstract

from arXiv · show

We study minimax rates for conditional generative learning under integrated Wasserstein loss, with H\"older continuous conditional laws and an unrestricted covariate distribution. We evaluate unrestricted generators through their conditional output distributions. For expected Wasserstein loss, the minimax rate agrees with the rate under uniform covariates and follows from balancing localization bias against unconditional distribution estimation error. Higher moments of the loss can instead be dominated by unobserved covariate regions, which force an additional risk contribution of order $n^{-1}$, where $n$ is the sample size. An adaptive localization argument gives matching upper bounds. The results cover unrestricted response laws and possibly vanishing Besov densities for $W_1$, and extend to higher Wasserstein orders under uniform response density bounds and integrability conditions. A multiscale lower bound closes the logarithmic gap in unconditional $W_1$ minimax rates in dimension two for both unrestricted distributions and Besov density classes. We also establish minimax rates for conditional Brenier maps under density, integrability, and uniform curvature conditions.

1. Introduction.

The paper establishes conditional generative learning as nonparametric estimation of conditional response distributions and derives Wasserstein minimax rates without restrictions on the covariate marginal. It also sharpens unconditional W1 rates in dimension two, develops conditional Brenier map rates, and situates these results among prior work with differing regularity and design assumptions.

1. Introduction.

  • The analysis treats conditional generative learning as nonparametric estimation of conditional output distributions, with Hölder-continuous laws and no density, doubling, or lower-mass condition on the covariate marginal.For p=1, response laws may be unrestricted or have Besov densities that vanish; for 1<p<∞, response densities are uniformly bounded above and below.Source: §1 · p. 1
  • For integrated Wasserstein loss, the minimax rate remains A_n(τ) across all Borel covariate marginals, while higher moments incur an additional n^-1 contribution from rare unobserved covariate regions.The higher-moment rate is A_n(τ)^κ ∨ n^-1.Source: §1 · p. 2
  • A multiscale lower bound establishes the missing W1 rate p log m/m in dimension two for unrestricted laws and Besov density classes.Combining perturbations across logarithmically many resolutions yields the logarithmic factor; a single resolution gives only m^-1/2.Source: §1 · p. 2
  • Uniform curvature bounds transfer squared W2 distribution error to squared L2(ρ) Brenier-map error, and inverse potentials provide a matching lower bound.The resulting map rate is stated as A_n(τ)^2.Source: §1 · p. 2

1.1. Related work.

  • The paper relates its conditional distribution rates to prior conditional density, distribution-function, and distribution-free regression work, and builds its map results on transport-map estimation and stability results.The inverse-potential construction supplies the matching conditional lower bound within the stated class.Source: §1.1 · pp. 2–3
  • Prior Wasserstein results impose differing covariate or response restrictions; this work obtains matching multivariate bounds under Hölder continuity in x over all Borel covariate marginals.For example, one cited multivariate local-estimation result assumes a uniform lower bound on the covariate law relative to Lebesgue measure.Source: §1.1 · p. 2

2. Main results.

The paper establishes minimax rates for conditional Wasserstein distribution estimation with unrestricted covariate marginals, and for conditional Brenier maps under curvature conditions. Expected loss retains the uniform-covariate benchmark, while higher moments can incur an additional n^-1 term.

2.1. Statistical model and losses.

  • Unrestricted generators and conditional distribution estimators have the same minimax risk when evaluated through their output laws.Every measurable conditional kernel can be realized using the training sample, query covariate, and independent uniform noise.Source: §2.1 · p. 4

2.2. Regularity classes.

  • The model allows arbitrary Borel covariate marginals and unrestricted W1 response laws, while higher Wasserstein orders require uniformly bounded response densities.For W1 Besov densities, no pointwise lower or upper density bound is imposed.Source: §2.2 · p. 5

2.3. Conditional distribution estimation.

  • Unconditional W1 rates are parametric in response dimension one, have a logarithmic correction in dimension two, and improve with Besov smoothness above dimension two.A multiscale lower bound establishes the dimension-two rate for unrestricted distributions and Besov density classes.Source: §2.3 · pp. 6–7
  • For higher Wasserstein orders, the conditional rate has an additional n^-1 contribution, and the localization scale remains An(τ).The n^-1 term dominates when κ exceeds κc(τ).Source: §2.3 · pp. 7–8
  • For conditional W1 risk, the minimax rate is An(τ)^κ ∨ n^-1; at κ = 1, the n^-1 term is lower order and the rate is An(τ).The additional term is forced by a two-point covariate distribution with an atom of mass n^-1; for κ > κc(τ), it dominates.Source: §2.3 · p. 7

2.4. Conditional Brenier map estimation.

  • Conditional Brenier map estimation has minimax squared L2(ρ) risk An(τ)^2 under the stated density, integrability, and uniform curvature conditions.Brenier stability transfers squared W2 error to map error, and inverse potentials provide the matching lower bound.Source: §2.4 · p. 9

3. Conditional localization with unrestricted covariate distributions.

The section develops localization bounds for conditional distribution estimation with unrestricted covariate marginals, then shows why higher loss moments are sensitive to rare covariate regions. An adaptive partition yields matching upper bounds, and a numerical example illustrates the rare-covariate effect.

3.1. Localization on a fixed grid.

  • The fixed-grid upper bound balances localization bias against unconditional distribution-estimation error and has order An(τ), uniformly over Borel covariate marginals.Pooling on cells introduces bias controlled by their diameter, while the stochastic term depends on cell sample counts.Source: §3.1 · pp. 9–10

3.2. Matching lower bound.

  • A replicated hypercube lower bound, attenuated across covariate neighborhoods, gives the matching first-moment rate at the balancing bandwidth.The construction controls the conditional W1 Hölder condition and keeps neighboring sample laws difficult to distinguish.Source: §3.2 · p. 10

3.3. Higher moments and rare covariate regions.

  • The adaptive dyadic partition uses occupancy thresholds with fixed, known regularity parameters, and its expected mass of bad terminal cells is O(n^-1).Estimation uses accepted parents of the first rejected cells; Wasserstein loss bounded on the response cube gives the same order for their risk contribution.Source: §3.3 · p. 11
  • For higher moments, aggregation over uneven cell probabilities adds an n^-1 contribution, giving rate An(τ)^κ ∨ n^-1.The two terms arise from a uniform-covariate lower bound and an unseen atom of mass n^-1; the bounds extend to fixed finite moments under the stated response-density conditions.Source: §3.3 · pp. 10–12, §3.2 · p. 10

3.4. Numerical illustration: rare covariate events.

  • The numerical illustration compares grid, occupancy, nearest-neighbor, and conditional-flow estimators for a Lipschitz two-sign family under uniform and rare covariates.It reports integrated risks for κ=1 and κ=4 across sample sizes.Source: §3.4 · p. 12
  • Fourth-moment risks decrease more slowly under rare covariates; at n=2^15, most occupancy-estimator error comes from the rare atom being observed only sparsely.The minimax benchmarks shown are n^-1/3 for the first moment, n^-4/3 for the uniform-covariate fourth moment, and n^-1 for the rare-atom scale; fixed-family estimator behavior can differ.Source: §3.4 · p. 12

4. Unconditional Wasserstein estimation and multiscale lower bounds.

This section establishes unconditional Wasserstein estimation benchmarks and a multiscale lower-bound construction. Combining perturbations across resolutions closes the two-dimensional W1 logarithmic gap, while upper bounds use empirical or truncated-wavelet estimators.

Unconditional upper bounds.

  • For smooth classes in dimensions above two, a truncated empirical wavelet expansion and approximate projection balance approximation bias against stochastic error without pointwise density bounds.The empirical measure attains the unrestricted rate in every dimension and the smooth rate in dimensions one and two; the upper bounds extend to every fixed finite moment.Source: §4 · pp. 13–14

Multiscale lower bound.

  • The multiscale potential combines perturbations across resolutions to accumulate Wasserstein separation while keeping neighboring alternatives difficult to distinguish.The construction supports the unconditional lower bounds and is attenuated across covariate neighborhoods for the conditional lower bound.Source: §4 · pp. 12, 14

Unconditional W1 minimax rates.

  • In dimension two, combining perturbations across resolutions produces a √log m separation factor, whereas a single resolution gives only m^-1/2.Each active resolution contributes at the same order, and Besov smoothness limits the finest scale but not the accumulation.Source: §4 · p. 15

Numerical illustration.

  • In the cosine-density illustration, combining levels yields greater mean transport separation than using only the finest level for J=2, 3, 4, 5 in panel (a).The tested resolutions are insufficient to determine asymptotic scaling.Source: §4 · p. 15

5. Conditional Brenier map estimation.

The section establishes conditional Brenier-map minimax rates using stability for an upper bound and inverse potentials for a lower bound. The lower construction yields squared L2(ρ) map separation of order An(τ)^2 under the stated class conditions.

5. Conditional Brenier map estimation.

  • The conditional Brenier-map result combines a stability-based upper bound with a matching lower bound from inverse potentials.The section places the result under its density, integrability, and uniform-curvature assumptions.Source: §5 · p. 15, §5.2 · p. 17

5.1. Stability and the minimax upper bound.

  • Stability transfers squared W2 error for the target distribution to squared L2(ρ) map error, enabling a plug-in map estimator for the upper bound.The estimated conditional law need not satisfy the Brenier-class curvature bounds and may be discrete.Source: §5 · p. 15, §5.1 · p. 16

5.2. Lower bound from inverse potentials.

  • The inverse-potential construction gives squared L2(ρ) map separation of order An(τ)^2 at the balanced bandwidth.Its explicit determinant target density avoids composition with an inverse map, and the construction verifies class membership, separation, and testing bounds.Source: §5.2 · pp. 16–17

6. Discussion.

Mass-weighted localization preserves the ordinary integrated Wasserstein rate without restrictions on the covariate distribution, while higher moments incur a rare-region penalty. The discussion also identifies computational and modeling conditions that remain unresolved or could potentially be relaxed.

Discussion

  • Weighting local errors by covariate-cell probabilities preserves the ordinary integrated Wasserstein rate uniformly over unrestricted covariate distributions.For higher moments, rare unobserved regions contribute an additional n^-1 term, even when response laws are simple.Source: §6 · p. 17
  • A covariate density bounded below by a fixed positive constant removes the rare-region penalty, yielding rate A_n(τ)^κ for every fixed finite κ in the considered distribution classes.The upper bound balances localization bias against estimation error at local sample size n h^d_x; the uniform-covariate lower bound still applies.Source: §6 · p. 17
  • For smooth classes with d_y > 2, the sharp upper bound uses a wavelet estimator and approximate metric projection, but its computational complexity remains open.Attaining these rates with computationally efficient parameterized generators is also a separate problem.Source: §6 · p. 17
  • Further open questions include adaptation to unknown β and τ, unbounded response spaces under suitable tail conditions, and whether the density and curvature assumptions can be relaxed without changing rates.The stated density bounds for p > 1 and curvature assumptions for Brenier maps are sufficient conditions.Source: §6 · p. 17

Appendix A. Preliminaries.

The preliminaries set wavelet conventions, define a negative Sobolev norm, and establish tools for comparing Wasserstein distance to that norm and for constructing measurable approximate projections.

Appendix A. Preliminaries

  • The boundary wavelet basis is chosen with regularity and vanishing moments exceeding K, while the lower-bound construction retains interior wavelets supported inside the response cube.The scaling spaces reproduce affine functions; detail wavelets, including boundary wavelets, have zero moments against affine functions.Source: Appendix A · p. 18
  • The negative norm ∥u∥_-1,p is the minimum Lp norm of a vector field whose divergence is the mean-zero function u and whose normal component vanishes on the boundary.The text also identifies this as the weak formulation of the divergence constraint with zero normal trace.Source: Appendix A · p. 18
  • Under a positive lower bound on density q, Wasserstein distance Wp is bounded above by a constant times q^(-(p-1)/p) and the negative Sobolev norm of the density difference.The comparison is one-sided and applies for 1 < p < ∞.Source: Appendix A · pp. 18–19
  • A measurable approximate projection selects the first candidate within δ of the infimum distance and guarantees error at most twice the preliminary estimator's error plus δ.For W1, the distance is the KR norm and the candidate family consists of rational finitely supported probability laws.Source: Appendix A · p. 19

Appendix B. Unconditional upper bounds in W1.

Appendix B develops unconditional W1 upper bounds, and Appendix C constructs lower-bound families through metric decoding and multiscale potentials. Together these tools cover unrestricted and smooth response distributions across dimensions.

Appendix B. Unconditional upper bounds in W1

  • In dimensions d ≤ 2, the empirical distribution attains the stated smooth-class moment rate, and the two-dimensional critical rate is log(em)/m.The empirical distribution also attains the unrestricted rate in every dimension; for smooth classes with d > 2, the upper-bound argument uses a truncated wavelet estimator and approximate metric projection.Source: Appendix B · pp. 19–20

C.3. A unified hypercube family

  • The response lower-bound family is built from a single compactly supported potential hypercube whose spectral sum changes with response dimension.The sum is of order 1 for d = 1, J for d = 2, and 2^J(d−2) for d > 2.Source: Appendix C · p. 22

C.1. Testing and metric decoding

  • The lower-bound decoder maps an estimator to a nearest hypercube vertex, converting metric error into coordinate-testing errors when neighboring experiments are statistically close.A uniform prior over hypercube vertices is used to average the testing errors.Source: §C.1 · p. 22

C.2. Multiscale potential estimates

  • The multiscale potential construction uses Lipschitz truncation to produce a test function with controlled exceptional-set size and matching gradients outside that set.The gradient estimates support Wasserstein separation, derivative estimates ensure positivity and Besov regularity, and a one-coordinate Hessian estimate controls neighboring Hellinger distances.Source: §C.2 · pp. 22, 24

C.3. A unified hypercube family

  • For sufficiently large m, Proposition C.4 constructs a finite localized hypercube with densities bounded between fixed positive constants and arbitrarily close to uniform; one-coordinate flips have L2 norm O(ε), yielding a Hellinger bound.Source: §C.3 · pp. 25–26

C.4. Completion of the response theorem.

The response theorem obtains matching unconditional moment bounds, while the conditional appendices use cell-probability weighting and replicated hypercubes to establish localization bounds and lower bounds.

C.4. Completion of the response theorem

  • The unconditional response theorem's upper bound follows from the established unconditional estimator and moment bounds, while nearest-vertex decoding and Jensen's inequality give a matching lower bound of order a_d,τ(m)^κ.The lower bound uses the hypercube from Proposition C.4 and a uniformly positive coordinate-error probability.Source: §C.4 · p. 27

D.1. Partition bound weighted by cell probability

  • The partition argument estimates each cell's conditional response law from observations in that cell and weights local errors by the cell probability.The argument applies when the distribution class is closed under cellwise mixtures and its unconditional estimator has the stated moment bound.Source: §D.1 · p. 28

D.2. Replication under the uniform covariate distribution

  • Replicating response hypercubes across separated covariate bumps yields an integrated W1 lower bound of order a_m, already under uniform covariates.The construction uses local sample size m = floor(A0 n h^d_x), with adjacent n-sample experiments kept close in total variation.Source: §D.2 · pp. 28–29
  • For higher moments, an adaptive dyadic partition replaces the fixed-grid argument because t ↦ t^rκ need not be concave in every regime.The adaptive construction avoids imposing regularity on the covariate distribution.Source: §D.2 · p. 30

D.3. Higher moments and adaptive localization.

Adaptive localization yields matching bounds for higher moments under arbitrary Borel covariate marginals. Complementary lower bounds capture both the uniform-covariate benchmark and risk from covariate regions that go unobserved.

D.3. Higher moments and adaptive localization.

  • Splitting the sample lets a dyadic partition select accepted pooling cells independently of the subsample used to estimate their distributions.The method selects the deepest accepted cell along each query point’s dyadic chain.Source: §D.3 · p. 30
  • Incorrect cell acceptance or rejection contributes expected covariate mass of order n^-1 to the integrated risk.The proof bounds both low-count false acceptance and high-count false rejection.Source: §D.3 · pp. 30–31
  • On correctly localized cells, the threshold and resolution balance the pooling bias against local distribution-estimation error.The proof uses local sample counts and Hölder regularity to bound both terms at the selected scale.Source: §D.3 · p. 31
  • A covariate point of mass n^-1 can remain unobserved with probability at least 1/4, forcing an additional n^-1 contribution to higher-moment risk.At that point, two separated conditional laws are indistinguishable when no sample visits it.Source: §D.3 · p. 32
  • The adaptive upper bound and complementary lower bounds establish the conditional rates for every fixed finite moment exponent κ.The lower bounds include both the uniform-covariate construction and a rare covariate atom.Source: §D.3 · pp. 31–32

E.1. Wp distribution estimation.

Wavelet bounds establish unconditional Wasserstein estimation rates, including the critical two-dimensional logarithmic factor. The section also establishes the stability and measurability needed for plug-in conditional Brenier map estimators.

E.1. Wp distribution estimation.

  • Orthonormality across wavelet resolutions controls the empirical divergence field and supplies the sharp logarithmic factor in dimension two.The proof uses a negative norm and a transport comparison to control Wasserstein error.Source: §E.1 · pp. 32, 34
  • In d=1 the upper rate is m^-1/2, while in d=2 it is {log(em)/m}^1/2; the lower bound matches a_d,τ(m)^κ under the stated density and Besov constraints.For d>2, the wavelet resolution balances the bias against the stochastic error.Source: §E.1 · p. 34

F.1. Stability and measurability.

  • Uniform curvature and density bounds yield a continuous Brenier map operator, so composing it with the measurable distribution estimator gives a measurable plug-in map estimator.Convergence of target laws in W2 implies convergence of maps in L2(ρ).Source: §F.1 · p. 35
  • The lower-bound construction parameterizes the inverse Brenier map directly, making the target density an explicit determinant.This avoids composing with an inverse map.Source: §F.1 · p. 35

F.2. Transport families from inverse potentials.

The section constructs Brenier maps from inverse potentials and uses that construction to build hard families for map estimation. Uniform curvature and density conditions are maintained in the lower-bound construction.

F.2. Transport families from inverse potentials.

  • The inverse of S_U=∇Ψ_U is a Brenier map, and change of variables gives its target density through a determinant.Eigenvalue bounds on I+D²U yield the stated determinant and curvature bounds.Source: §F.2 · p. 36
  • Inverse-potential perturbations generate a hard family satisfying the prescribed density, curvature, smoothness, and conditional regularity constraints.The construction uses separated covariate bumps and response-dimension potential families.Source: §F.2 · pp. 36–37
  • Testing separated map alternatives on covariate-bump plateaus yields a lower bound while preserving the fixed density upper bound and map curvature constant.The construction works in every response dimension.Source: §F.2 · pp. 37–38

Appendix G. Details of the numerical experiments.

The numerical appendix describes the experiment and uncertainty procedures, including limitations of the reported bounds and simulations. In the conditional-estimation experiment, it separates pooling bias from sampling error and specifies how losses and intervals are computed.

Appendix G. Details of the numerical experiments.

  • The experiments use independent repetitions and pointwise 95% percentile bootstrap intervals, preserving paired signs and estimator comparisons.Rare-covariate simulations are resampled within count strata before recombination with exact probabilities.Source: Appendix G · p. 38
  • Quadrature and resolution comparisons are sensitivity checks, not uniform error bounds; the transport lower bounds also carry a separate failure probability δ.The plotted intervals combine approximate pointwise bootstrap uncertainty with numerical bracketing.Source: Appendix G · p. 38

G.1. Conditional estimation across response dimensions.

  • For losses with numerical lower and upper bounds, the curves use mean interval midpoints and shaded bands bracket bootstrap mean endpoints.The lower-bound integral is estimated using independent target draws stratified by target cells.Source: Appendix G · p. 38, §G.1 · p. 39
  • In panel (a), the integrated pooling bias is asymptotic to 2/(15K) for K equal cells and contributes to total error.Panel (b) instead evaluates sampling error against the population law pooled over the query cell.Source: §G.1 · p. 39
  • The fixed-law simulations do not determine the full conditional minimax logarithmic exponent.The grid estimator uses the empirical response distribution in each query cell, with a uniform fallback for empty cells.Source: §G.1 · pp. 39–40

G.2. Scalar distribution estimation.

The scalar-estimation simulations examine occupancy and flow risks under rare covariates, including the contribution from an unobserved rare atom. The multiscale transport experiments find positive separation for levels J = 2, 3, 4, 5, while the constructions coincide at J = 1.

G.2. Scalar distribution estimation.

  • The grid estimator uses equal covariate cells and empirical response distributions, while the occupancy estimator selects a dyadic cell using first-half counts and estimates from second-half responses.Empty empirical-estimation cells use the uniform response distribution.Source: §G.2 · p. 40
  • Flow fourth-moment risk under uniform covariates drops as validation selects the uniform initialization less often: 53.1% at n = 256, 2.5% at n = 2,048, and none at n = 4,096.Source: §G.2 · p. 41
  • When the rare atom is unobserved, its two conditional laws are indistinguishable from the data and have W1 distance α/3; using a uniform law at that atom attains the resulting lower bound.Pooling observations from a parent cell may incur greater loss.Source: §G.2 · p. 41
  • At n = 215, the rare atom contributes 99.46% and 98.87% of occupancy-estimator and flow fourth-moment risks; unobserved-atom contributions are 8.69% and 36.39% of total risks.The figure plots the three risk contributions multiplied by n, with a dotted lower bound.Source: §G.2 · pp. 41–42, Figure G.1 · p. 42

G.3. Multiscale transport separation.

  • Exact linear-programming checks covered 6 random and 2 multiscale cases; the largest primary-run graph gap was 8.4%, and doubled-grid interval midpoints changed by at most 2.4%.All corresponding doubled-grid intervals overlapped.Source: §G.3 · p. 43
  • For J = 2, 3, 4, 5, intervals for normalized mean separation lie above zero; at J = 1, the constructions coincide and the interval width comes from separately computed bounds.Source: §G.3 · pp. 43–44

Related papers

Loading 2610.35490v1…