The Geometry of Anisotropic Dilation for Optimal Regularization

1 Introduction

• Regularizer choice is crucial for stable reconstruction, while flexible learned regularizers generally offer fewer guarantees about optimization, stability, and statistical behavior.Hand-crafted regularizers can also be limited in the structures they promote.Source: §1 · pp. 1–2
• Within the considered Gibbs class, the optimal regularizer is determined by a direction-dependent radial summary statistic that compresses information about distributional mass and radius.The statistic is defined as a weighted radial moment of the density along each direction.Source: §1 · p. 2

1.1 Our Contributions

• Anisotropic dilation rescales points along their rays by direction, preserves the directional marginal, and acts explicitly on the radial summary statistic.Any admissible target statistic can be reached from a source by a unique profile, although distributional orbits retain scale-invariant radial information.Source: §1 · p. 2, §1.1 · p. 2
• For distribution fitting, the authors derive closed-form population projections and an isotropy cost that measures the advantage of anisotropic over isotropic approximations.For centered Gaussian mixtures, the cost is zero if and only if all component covariance matrices are proportional.Source: §1.1 · pp. 2–3
• The framework extends to positive-homogeneity Gibbs energies and higher-order conditional radial exponential families, and interprets dilation as raywise optimal transport when conditional radial laws differ by scaling.Source: §1.1 · p. 3
• The paper jointly learns a base regularizer and dilation profile from samples, reporting improvements on a controlled two-dimensional problem and MNIST denoising, with accompanying sample-complexity and generalization bounds.Source: §1.1 · p. 3

1.2 Related Work

• Unlike fixed transforms and isotropic radial transports, anisotropic dilation varies across directions while preserving the angular distribution; the cited radial-transport work instead focuses on variational inference.Source: §1.2 · pp. 3–4
• The paper builds on optimal regularization for whole data distributions but shifts the question to how structured transformations act on the statistic determining the optimal regularizer.Related work also studies learned regularizers and optimal regularization within restricted classes.Source: §1.2 · p. 4

• A star body is a compact, origin-containing set whose nonzero rays meet its boundary exactly once, and its positive continuous radial function specifies it uniquely.The associated gauge is the reciprocal of the radial function on the unit sphere.Source: §2 · pp. 4–5
• A random vector is decomposed into its radius and direction; for a density, the directional marginal hP is obtained by integrating the polar-coordinate density over radius.The density in polar coordinates is r^(d−1)p(ru).Source: §2 · p. 5
• The paper assumes an absolutely continuous distribution with strictly positive continuous hP and ρP; this includes suitably decaying positive densities and compactly supported densities positive near the origin.These conditions keep both quantities bounded above and bounded away from zero on the sphere.Source: §2 · p. 5
• The radial summary statistic ρP is determined by directional mass hP and the conditional mean radius mP, separating how much mass lies in each direction from its typical distance.The conditional mean radius is the first moment of the conditional radial distribution.Source: §2 · pp. 5–6

3.1 Definition and Fundamental Properties

• Anisotropic dilations form an Abelian group of homeomorphisms: profiles multiply under composition, the identity profile gives the identity map, and the inverse profile is 1/g.In polar coordinates, Tg maps (r,u) to (g(u)r,u).Source: §3.1 · p. 6
• A profile determined by two star bodies maps one body onto the other by continuously rescaling points along each ray.Figure 1 illustrates matching star bodies via this profile.Source: §3.1 · pp. 6–7, Figure 1 · p. 7
• For a pushed-forward distribution, direction is unchanged and radius becomes g(U)R; different profiles can therefore reshape radial geometry while retaining the directional distribution.Figure 2 illustrates this effect for an isotropic Gaussian base distribution.Source: §3.1 · pp. 7–8, Figure 2 · p. 8
• Under anisotropic dilation Tg, the directional distribution U and marginal hP remain invariant, conditional mean radius transforms directionwise as mQ(u)=g(u)mP(u), and P is closed under these dilations.Source: §3.1 · pp. 8–9
• For any admissible target radial summary statistic, there is a unique profile that maps a given source distribution to it, thereby setting its corresponding optimal regularizer.This flexibility in radial summaries does not imply transitivity over distributions: scale-invariant conditional radial features remain fixed, and a standard Gaussian cannot be dilated into a Gibbs distribution.Source: §3.1 · pp. 9–10

3.2 Optimization Benefits of Matching Radial Summary Statistics

• If ρP is C2 and comes from a nonconvex star body, the original denoising objective is never weakly convex near the origin, while the transformed objective is β*-weakly convex for sufficiently large λ.Choosing g(u) = ρP(u)^−(d+1) makes the induced regularizer the ℓ2 norm; the reformulation moves nonconvexity to the quadratic data-fidelity term while retaining a convex regularizer.Source: §3.2 · pp. 11–12

4 Learning Optimal Anisotropic Dilations

• Anisotropic dilation lets the base star body determine directional mass while the profile rescales conditional mean radius, decoupling structures that general data need not couple.For Gibbs densities alone, both the directional marginal and conditional mean radius are governed by the same radial function.Source: §4 · p. 13

4.1 Population Projection onto Gibbs Orbits

• For fixed K, the unique KL-optimal profile is g*(u) = mP(u)/(dρK(u)), matching the Gibbs model’s conditional mean radius to the target’s in every direction.The result assumes the target distribution satisfies the section’s standing integrability condition.Source: §4.1 · pp. 14–15
• After radial matching, the unique optimal star body is determined solely by the target’s directional density hP and yields a Gibbs model with the same directional marginal.The outer objective reduces, up to a K-independent additive constant, to a KL divergence between directional marginals.Source: §4.1 · pp. 15–16

4.2 Provable Benefits of Anisotropic Dilation

• The fixed-star-body isotropy cost is nonnegative and vanishes exactly when the radial discrepancy ratio αK(U) is constant almost surely under the target’s directional distribution.The cost is the Jensen gap between the logarithm of the expected discrepancy and the expected logarithm.Source: §4.2 · p. 17
• The distribution-level isotropy cost compares the best isotropically and anisotropically scaled Gibbs models after optimizing the unit-volume star body separately in each family.It is not generally the minimum fixed-star-body isotropy cost, because the two model classes may have different optimal star bodies.Source: §4.2 · p. 17
• The unit-volume canonical star body is the unique isotropic optimizer, and anisotropic scaling’s cost is determined by a logarithmic Jensen gap of the diagnostic ratio.The two costs differ by a positive dimension-dependent factor; anisotropic scaling strictly improves the approximation exactly when the ratio varies across directions.Source: §4.2 · pp. 17, 19
• The isotropy cost admits a quantitative lower bound in terms of variability of the diagnostic ratio, guaranteeing improvement when that ratio varies.The bound follows by applying a Jensen-gap inequality to a normalized diagnostic ratio.Source: §4.2 · p. 19

4.3 Examples Analyzing the Isotropy Cost

• For densities generated by a star-body gauge, including centered elliptical families, the diagnostic ratio is constant and the isotropy cost is zero.The directional dependence of the conditional mean radius and radial summary statistic cancels in their ratio.Source: §4.3 · p. 20
• For non-centered elliptical distributions, the diagnostic ratio’s directional dependence is captured by a scalar projection; non-centered Gaussians therefore have strictly positive isotropy cost.For dimension d ≥2 and nonzero mean, the projection spans [−κ, κ], and the Gaussian diagnostic ratio is nonconstant.Source: §4.3 · pp. 21–22
• For centered Gaussian mixtures, the diagnostic ratio is constant exactly when all component covariance matrices are proportional.This condition characterizes when the directional dependence from mixing different covariance geometries cancels.Source: §4.3 · p. 23
• Centered Gaussian mixtures with non-proportional covariance matrices have a nonconstant diagnostic ratio, ensuring a provable benefit from anisotropic dilation.For the two-component R2 example, Δdist(Pε) = 0 if and only if ε = 1; a quantitative lower bound is also established for 0 < ε ≤ 1.Source: §4.3 · pp. 23, 25

4.4 Generalized Gibbs Orbits

• Generalized Gibbs projection matches conditional radial τ-moments while its unique optimal outer star body is the same K* as for τ = 1, independent of τ.The outer solution has radial function ρK*(u) = (d hP(u))^(1/d), matching the target directional marginal.Source: §4.4 · pp. 26–27

4.5 Orbits of Radial Exponential Families

• For fixed τ, generalized Gibbs orbits preserve the base directional marginal and give conditional radial laws whose scale depends on g(u)ρK(u), limiting their radial flexibility.This limitation motivates conditional radial exponential families built from powers of the radius, whose log-partition derivatives encode radial moments.Source: §4.5 · pp. 27–28
• A baseline conditional-moment constraint resolves the non-identifiability between the radial coefficient field η and external profile g for fixed K.Without the constraint, rescaling η and compensating in g leaves the model distribution unchanged.Source: §4.5 · pp. 28–29
• The KL projection onto the separated radial orbit decomposes into a directional term and a radial residual, with the directional term fixed when K is fixed.The stated projection analysis assumes the target’s conditional radial moments at the selected powers belong to G.Source: §4.5 · p. 29
• Under the theorem’s directional moment-matching hypothesis, the constrained objective has a unique admissible minimizer, with its optimal shape parameter determined by scaled moment-matching equations.The hypothesis requires a single coefficient vector at each direction to match all prescribed moments; for L = 1 this holds automatically, but for L > 1 finite moments alone do not ensure it.Source: §4.5 · p. 30
• The optimal transformed conditional density depends only on target moments and chosen exponents, so the remaining shape optimization uniquely selects K* with radial function rho_K*(u) = (d hP(u))^(1/d).The minimized radial term is independent of K, reducing the outer optimization to minimizing DKL(hP ∥ hK) over unit-volume star bodies.Source: §4.5 · p. 30

4.6 A Transport Perspective on Anisotropic Dilation

• When P and Q share a directional marginal and each conditional radial law of Q is a profile-scaled law of P, Tg is quadratic-cost optimal among ray-preserving transports from P to Q.The result applies to transports constrained to preserve Euclidean rays.Source: §4.6 · p. 31
• For general conditional radial laws, the best raywise scale is a least-squares slope aligning their radial quantile functions; any profile in G agreeing with it almost everywhere minimizes over G.The reversed problem uses the pointwise scale BP,Q(u)/CQ(u).Source: §4.6 · p. 33

5.1 Learning Procedure

• The learning procedure jointly fits a gauge and positive profile using an empirical objective; alternating minimization over their parameters yields stable convergence.The learned regularizer is the negative log-density of the anisotropically dilated Gibbs model, up to an additive constant.Source: §5.1 · p. 34
• A direction-dependent correction can also adapt a fixed base regularizer such as TV; its resulting energy combines the base regularizer scaled by gθ with a dimension-weighted log-profile term.This fixed-base correction is explored in the MNIST experiments.Source: §5.1 · pp. 34–35
• If the base regularizer is continuous, nonnegative, and positively homogeneous and the profile is continuous and positive, the variational objective attains a minimum.The stated existence result follows from lower semicontinuity and coercivity.Source: §5.1 · p. 35

5.2 Two-Dimensional Denoising Experiment

• Adding g to the circular baseline reduces RMSE by approximately 0.0017, 0.0148, and 0.0407 as noise increases; (K, g) is lowest among Gibbs models, with modest gains over K-only.The models were evaluated on 650 test samples.Source: Table 1 · p. 36, §5.2 · pp. 36–37
• 0.0020, 0.0075, and 0.0108 RMSE reductions follow from adding radial-shape flexibility to (K, g); (K, g, η) has the lowest test RMSE in every regime.The reductions are reported across the three noise levels.Source: Table 2 · p. 37, §5.2 · p. 37

5.3 Denoising on MNIST

• Across four noise levels, the jointly learned gauge and gθ achieve the best test PSNR; gθ gains 1.251, 1.301, 1.267, and 0.742 dB over the learned gauge alone.The gains are reported for noise levels σ = 0.10, 0.15, 0.20, and 0.25, respectively.Source: Table 3 · p. 38, §5.3 · p. 38
• Adding gθ to TV improves PSNR by 0.389, 0.784, 0.671, and 1.804 dB across four noise levels.The profile also improves the fixed TV regularizer while the combined learned model achieves the best test PSNR in every regime.Source: Table 3 · p. 38, §5.3 · p. 38

6 Statistical Learning Guarantees

• With probability at least 1 − γ, empirical and population anisotropic orbit risks converge uniformly over the hypothesis class, given finite covering numbers.The theorem holds simultaneously for all (K, g) in C × G and gives a bound in terms of sample size, class complexity, and confidence.Source: §6 · p. 39

6.1 Application: Exponentially Parameterized Families

• Linear feature profiles add an O(p log m/m) term to the uniform convergence bound, while the sparse ReLU profile class also admits a uniform convergence bound for fixed architectural parameters.The results use exponentially parameterized profiles and classes of unions of L volume-one convex bodies containing rB_d.Source: §6.1 · pp. 42–43

6.2 Estimation Error Bounds

• For a fixed star body K, population excess risk controls the squared L2(P_U) distance between a profile and the population-optimal profile.For a < b, the lower-bound constant is κ_a,b = log(b/a) + a/b − 1, which is positive.Source: §6.2 · pp. 43–44
• The theorem controls estimation error for approximate empirical risk minimizers using the uniform deviation and empirical optimization error, with the bound applying on the same high-probability event.The uniform deviation is Δm, and the empirical optimization error is εopt(g).Source: §6.2 · pp. 44–45
• A uniform sup-norm estimate follows when the profiles are Lipschitz and the directional distribution assigns at least cε^ν mass to every spherical cap of radius ε≤ε0.The bound applies when the target profile is in the Lipschitz class and the stated directional mass condition holds.Source: §6.2 · p. 44
• For d≥2, the directional mass condition holds for every P in P; a density uniformly bounded below on the sphere is sufficient.For P in P, the directional density is continuous and strictly positive on the compact sphere, so its minimum is positive.Source: §6.2 · pp. 44–45

7 Conclusion

• Anisotropic dilation reaches any admissible target radial summary statistic from a source using a unique profile, while preserving directional marginals and conditional radial shape up to scaling.The framework separates directional mass from conditional radial scale and characterizes the approximation error of restricting to isotropic scaling.Source: §7 · p. 45
• The framework also extends to homogeneous Gibbs energies and richer conditional radial families, gives a raywise transport interpretation, and provides finite-sample learning guarantees.Source: §7 · p. 45
• Experiments show that learning the scaling profile can improve handcrafted and learned regularizers in simple settings.Source: §7 · p. 45
• Quantitative reconstruction guarantees and demonstrations on higher-dimensional inverse problems remain future research directions.The conclusion also identifies broader variational-geometry results and more flexible interpretable map families as open directions.Source: §7 · p. 46

8.1 Experimental Details: Two-Dimensional Denoising

• For two-dimensional denoising, directional density and conditional radial moments are kernel-estimated, and the resulting profiles are normalized to grid mean 0.90.Scaling profiles are clipped to [0.08, 12], though no values required clipping; shape models fit log-moment targets with periodic smoothness regularization.Source: §8.1 · p. 51

8.2 Experimental Details: MNIST

• The MNIST anisotropic-profile features include proxies for each input’s spatial support and spread, standardized using means and standard deviations estimated from training data.The described training-data construction first normalizes each image by its Euclidean norm.Source: §8.2 · p. 52

8.3 Additional Proofs

• For positive C2 profiles ψ, the map Φ(z)=ψ(z/||z||2)z satisfies a defect bound proportional to ||z1||2+||z2||2−||z1+z2||2.The proportionality constant Cψ is finite.Source: §8.3 · p. 52
• The diagnostic ratio is constant on the sphere if and only if all covariance matrices in the Gaussian mixture are proportional.The converse proof assumes distinct proportionality classes, analytically continues a constancy identity to a complex covering space, and derives a contradiction from a power-series mismatch.Source: §8.3 · p. 54
• The local lift Ψ is locally biholomorphic, making its image U open in Ωx and the lifted algebraic relation identically zero there.The lift is built from a locally biholomorphic coordinate change and chosen square-root branches.Source: §8.3 · p. 57
• Nonzero holomorphic remainders yield generic-slice leading exponents d + 1 + 2p and d + 2q; their odd difference makes equality impossible and forces a contradiction.Because the exponents cannot match, dividing by the smaller power gives a nonzero limit on one side and zero on the other.Source: §8.3 · pp. 57–58
• The proof concludes that m = 1, meaning all covariance matrices lie in the same proportionality class, and thereby establishes the converse.The contradiction rules out the assumption m ≥ 2.Source: §8.3 · p. 58