Source-linked AI summary

A local-to-global principle for $f$-divergence homogenization

Aryeh Kontorovich

arXiv:2610.31164v1math.PRcs.ITmath.IT

TL;DR

The paper asks which f-divergences remain bounded when distinct independent experiments are replaced by repetitions of their average. It reduces the general comparison to a two-observation binary case, yielding a complete scalar characterization of the admissible generators.

  • Problem

    The paper asks when replacing distinct independent experiments by repetitions of their average keeps f-divergence bounded by a generator-dependent constant, uniformly over the number of observations, measures, and underlying spaces.

    Source: §1 · pp. 1–2
  • Method

    The proof reduces likelihood laws under multiplicative convolution through an ordered two-point decomposition, then extends a two-factor comparison to arbitrary products.

    Source: §4.1 · pp. 4–5
  • Results

    Homogenization for two observations on a binary alphabet is equivalent to homogenization for arbitrary product experiments and yields a complete scalar characterization.

    Source: §1 · p. 2, §2 · p. 2
  • Takeaways & Limitations

    Whenever uniform homogenization fails, a two-observation binary experiment already witnesses the failure.

    Source: §1 · p. 2

    In the asymmetric case, separate growth conditions at zero and infinity do not suffice; the criterion depends on joint behavior on both sides of one.

    Source: §2.1 · p. 3, §7.5 · p. 11

Abstract

from arXiv · show

We study the effect of replacing a family of independent statistical experiments by independent repetitions of their average. We give a simple, complete characterization of the $f$-divergences for which this operation increases divergence by at most a constant depending only on the generator, uniformly in the number of observations and the underlying measurable space. The characterization is governed by a local-to-global principle: a comparison restricted to two observations on a binary alphabet already determines the validity of the general inequality. It admits an explicit scalar formulation, which simplifies further for symmetric divergences. The result applies to arbitrary convex generators and requires no differentiability or symmetry assumptions. It recovers the homogenization inequality for total variation and identifies exactly the admissible members of the power-divergence family. In particular, whenever uniform homogenization fails, the obstruction is already present in a family of binary experiments with two observations. Moreover, the global comparison constant is controlled uniformly over generators with a common bound in the local inequality.

1 Introduction

The paper characterizes when replacing independent experiments by repetitions of their average increases an f-divergence by a generator-dependent constant. Its local-to-global result reduces this uniform comparison to two observations on a binary alphabet, with scalar criteria and proof tools developed across the paper.

2 A local-to-global characterization

  • Uniform homogenization holds exactly when the corresponding comparison holds for two observations on a binary alphabet; failure is witnessed within that class.The local bound also controls the global constant uniformly over generators sharing that bound, though no sharp dependence is asserted.Source: §2 · pp. 2–3

2.1 The scalar criterion

  • A scalar condition on 0 < r < 1 < t characterizes admissibility, simplifies for symmetric generators, and recovers total-variation homogenization.The Cressie–Read family is admissible exactly for parameters in [0, 1], with optimal constant 1 throughout.Source: §2.1 · p. 3

2.2 A thinning inequality

  • Bernoulli thinning bounds expectations of convex functions of thinned centered sums, supplying the uniformity needed near the diagonal.The estimate applies to independent centered variables and independent Bernoulli retention variables.Source: §2.2 · p. 3

3 Related work

  • Earlier total-variation results treated Bernoulli and general-measure settings; this paper extends the likelihood representation to general convex generators and characterizes when uniform homogenization holds.Unlike convolution approximation bounds, the comparison concerns divergences before and after homogenizing experiments and requires no closeness among coordinate laws.Source: §3 · p. 4

4.1 Two-point reduction and convex order

  • An ordered mixture of nested two-point mean-one laws reduces the single-law comparison, because mixed products are controlled by the square of the larger component.Integrating the two-point square bound extends it to arbitrary mean-one laws with a loss of at most a factor of two.Source: §4.1 · pp. 4–5

4.2 Hellinger coordinates and the separated regime

  • In the separated regime, Hellinger coordinates turn multiplicative convolution into additive convolution, and Klass’s estimates yield a comparison with an additive remainder.A binary contraction absorbs the remainder uniformly under the local hypothesis.Source: §4.2 · p. 5

4.3 Thinning and repeated sampling

  • Near the diagonal, thinning gives multiplicative control; Poissonization and a probability-weighted binary-tree comparison extend it to repeated sampling.The layer bounds are summable using factorial decay, and any positive power in the thinning estimate suffices.Source: §4.3 · p. 5

5.1 Divergence and data processing; 5.2 Multiplicative convolution

  • Likelihood laws convert product experiments into multiplicative convolutions, while retaining coordinate labels identifies the averaged likelihood law and data processing gives the unlabelled comparison.A homogenization inequality for equivalent pairs extends to arbitrary pairs with the same constant.Source: §5.2 · pp. 6–7, §5.1 · p. 6

6 Two-point reduction

The two-point reduction establishes equivalences among local bounds for binary experiments and a square bound for positive two-point mean-one laws. A nested convex-order decomposition extends that bound to arbitrary mean-one laws, enabling the global homogenization inequality.

6.1 Nested two-point decomposition

  • The nested components’ mixed products are dominated by the square of the larger component, controlling cross terms when the mixture is expanded.The construction matches deficits and surpluses to form two-point mean-one components with nested support intervals.Source: §6.1 · pp. 8–9
  • A two-point square bound with constant A extends to every mean-one law on [0, ∞) through a nested decomposition ordered by convex order.The resulting bound for arbitrary laws is stated in Proposition 6.3.Source: §6.1 · pp. 8–9

6 Two-point reduction

  • A Bernoulli bound K implies the square bound with A = 8K + 2, and a square bound A yields the global homogenization inequality with constant C(2A).The least constants satisfy M ≤ A ≤ M + 2 for the endpoint and square bounds.Source: §6.1 · pp. 9–10

6.1 Nested two-point decomposition

  • A Bernoulli bound K gives a two-coordinate bound of 16K + 4 on every alphabet, and the resulting global constant is controlled by a universal function.The proof removes the equivalent-pairs restriction using Lemma 5.1.Source: §6 · p. 8, §6.1 · p. 10

7 Examples

The examples sharpen the scalar criterion for symmetric divergences and identify the exact admissible power divergences. They also show the limits of endpoint-growth tests and revisit total variation and χ2.

7.1 Symmetric divergences

  • For symmetric generators, the scalar condition is equivalent to a bound with some finite constant C; symmetry yields the sufficient bound with M = 2C.The necessity direction follows by setting r = 1/t in the scalar condition.Source: §7.1 · p. 10

7.2 Power divergences

  • Homogenization holds for Cressie–Read divergences exactly when 0 ≤ α ≤ 1, with constant 1; outside this range it fails for two Bernoulli coordinates.For α > 1 or α < 0, the endpoint criterion produces an unbounded ratio.Source: §7.2 · pp. 10–11

7.3 Total variation

  • For total variation, the argument establishes that C(3/2) is admissible, alongside a direct proof with an explicit universal constant in [9].The passage applies the theorem’s universal homogenization constant to total variation.Source: §7.3 · p. 11

7.4 Failure for χ2

  • The χ2 example uses two coordinates, with one experiment having a rare event of probability ε and the other identical under both measures.In the averaged pair, repetition of the likelihood-ratio-1/ε symbol contributes to the homogenized divergence; the cited passage gives a lower bound on the ratio but not its expression.Source: §7.4 · p. 11

7.5 Insufficiency of endpoint growth bounds

  • A convex generator that is bounded at zero and linear at infinity can still fail the two-coordinate Bernoulli inequality.The example has a flat interval, and shows that endpoint growth bounds alone do not suffice; the criterion depends on joint behavior on both sides of one.Source: §7.5 · p. 11

8 Quantitative dependence

The global comparison constant is uniform across generators sharing a common two-coordinate bound, though its dependence on that bound is not claimed to be optimal.

8 Quantitative dependence

  • The modulus C(K) is uniform over generators with the same two-coordinate bound K, but the dependence on K is not asserted to be optimal.The two-point reduction loses a factor of two, while the main loss comes from controlling repeated sampling.Source: §8 · p. 12

9 Product estimates and Hellinger coordinates

Product estimates control repeated convolution through a doubling bound. Hellinger coordinates then provide an affine representation of mean-one laws and yield convexity and symmetric doubling properties for transformed quantities.

9 Product estimates and Hellinger coordinates

  • Under the standing assumptions f ≥ 0, f(1) = 0, and condition (B) with constant K, the section works first with positive finitely supported likelihood laws.Section 14 removes the finite-support and positivity restriction with unchanged constants.Source: §9 · p. 12

9.1 Product estimates

  • Iterating the product estimate gives Df(ρ⋆2^r) ≤ c^r Df(ρ); choosing r = ⌈log2 m⌉ extends control to m repeated factors.For m ≤ 2^r, ρ⋆m is a convolution factor of ρ⋆2^r, so the product estimate applies.Source: §9.1 · p. 12

9.2 Hellinger transform

  • The Hellinger transform maps positive mean-one laws affinely and bijectively onto A, with the inverse specified by the proposition.The transformed generator h is finite, nonnegative, convex, and satisfies h(0) = 0.Source: §9.2 · p. 13
  • The symmetric function Mf is nondecreasing and satisfies the symmetric doubling bound; dyadic iteration extends that bound to larger scales.The proof uses evenness and convexity of Φf(u) + Φf(−u).Source: §9.2 · p. 14
  • The reference-scale argument establishes mf > 0 for a nontrivial nonnegative convex generator, since otherwise its zero intervals would exhaust (0, ∞).Lower semicontinuity at zero would then force f ≡ 0, contrary to the standing assumption.Source: §9.2 · p. 15

10 Comparison away from zero Hellinger distance

The comparison away from zero Hellinger distance is built using growth bounds on logarithmic transforms and estimates for independent sums. These yield an average-law comparison and a divergence comparison with an additive remainder that can be absorbed at positive Hellinger distance.

10 Comparison away from zero Hellinger distance

  • Klass’s estimates give a comparison with an additive error; exponential normalization controls cancellation, and positive total squared Hellinger distance absorbs the error.This supplies the comparison strategy away from the diagonal.Source: §10 · p. 15

10.1 Growth of logarithmic transforms

  • The logarithmic transforms g+ and g− are finite, continuous, nonnegative, and nondecreasing, providing growth quantities for the later comparison.They are defined from f by g+(x) = e^-x f(e^x) and g−(x) = f(e^-x).Source: §10.1 · p. 15

10.3 Homogenization of logarithmic sums

  • For each α > 0, Theorem 10.2 gives a finite constant Rα for comparing independent sums with the average law under its stated growth condition.The theorem is stated for independent finitely supported real variables.Source: §10.3 · p. 17

10.4 Divergence comparison

  • Theorem 10.3 compares positive finitely supported mean-one families with an additive error, using the average-law result and the growth bounds.The supplied theorem statement identifies the setting and comparison, but does not include the inequality’s expression.Source: §10.4 · pp. 20–21
  • Theorem 10.4 is titled “Comparison at positive Hellinger distance” and is stated for d0 > 0; a channel mixing an independent fair bit and 1_A yields Bernoulli laws with parameters in [1/4, 3/4].Source: §10.4 · pp. 21–22

11 Thinning of centered sums

This section proves the convex thinning inequality through a sequence of comparisons for Bernoulli and shifted sums. The proof culminates in a shifted-positive-parts theorem and derives the convex thinning inequality by random partitioning.

11 Thinning of centered sums

  • A uniform comparison for shifted positive parts is the section’s main proof strategy for Theorem 2.3.Conditioning each summand on threshold exceedance reduces the comparison to centered Bernoulli variables.Source: §11 · p. 22

11.1 A Bernoulli tail comparison

  • The Bernoulli tail comparison relies on the decreasing sequence of ratios r_k and bounds obtained by iterating coefficient relations.The argument treats m ≥ 3 and m = 2 separately before concluding the stated tail estimate.Source: §11.1 · pp. 23–25

11.2 Shifted Bernoulli sums

  • The shift comparison lemma is used to compare sums with nonnegative coordinate shifts, with a further bound stated for every t > 0.Its proof identifies realizations with active sets and uses the monotonicity of the event that the shifted sum is positive.Source: §11.2 · p. 25

11.3 Shifted positive parts

  • The shifted-positive-parts theorem applies to independent centered integrable variables for every t ≥ 0.The proof reduces positive thresholds to Bernoulli comparisons and uses the t = 0 case as the L1 comparison.Source: §11.3 · pp. 26–27

11.4 Proof of the convex thinning inequality

  • Random partitioning and the positive-part representation of convex functions transfer the shifted-positive-parts theorem to the convex thinning inequality.For q = 1/m, the proof uses m random colors; other q values are handled by comparison with a nearby reciprocal integer or the unthinned sum.Source: §11.4 · pp. 28–29

12 Thinning at small squared Hellinger distance

This section transfers the thinning inequality to Hellinger measures and likelihood ratios. Completion handles zero values, while a random-translation estimate controls the centering step.

12.1 Completion and translation bounds

  • Completion transfers Theorem 2.3 to Hellinger measures; the completed variables satisfy E sinh Xi = 0, and their means and translations are controlled by total squared Hellinger distance.The section states a divergence-after-completion lemma as part of this transfer.Source: §12.1 · pp. 29–30

12.2 Centering the thinned sum

  • The random-translation estimate bounds the effect of centering with a factor of q.It is applied to nonnegative convex functions of independent centered variables and Bernoulli(q) thinnings.Source: §12.2 · p. 32

12.3 Thinning Hellinger measures

  • Theorem 12.4 gives a thinning result for Hellinger measures under a small squared Hellinger distance condition.The section also states a likelihood-ratio corollary for positive mean-one laws with summed squared Hellinger distances at most 1/4.Source: §12.3 · pp. 33–34

13 Poissonization and homogenization

The section uses Poissonization to connect thinning bounds with sampling with replacement. It then states a homogenization result under a small squared Hellinger distance condition.

13 Poissonization and homogenization

  • Poissonization represents sampling multiplicities using independent Pois(1) variables, whose level sets are Bernoulli thinnings.The levels are combined along a binary tree with depths chosen by Kraft’s inequality; factorial decay of the thinning probabilities controls accumulated comparison constants.Source: §13 · pp. 34, 36
  • Theorem 13.2 transfers thinning bounds to sampling with replacement for a mean-one family.The comparison uses the average law μ = n^-1 Σ_i μ_i.Source: §13 · p. 35
  • Corollary 13.3 states a homogenization result for small squared Hellinger distance.The proof applies the likelihood-ratio thinning corollary and Theorem 13.2.Source: §13 · p. 37

14 Completion and approximation

The section completes the finite-support argument and extends the comparison to arbitrary mean-one laws. The extension preserves the same constant, including for laws with atoms at zero and extended-valued divergences.

14 Completion and approximation

  • The global comparison constant is uniform over generators and n, and the estimate extends to arbitrary mean-one laws on [0, ∞).The extension includes atoms at zero and extended-valued divergences.Source: §14 · p. 37

14.1 Approximation and endpoints

  • The approximation principle extends a fixed-constant comparison from positive finitely supported mean-one laws to arbitrary mean-one laws on [0, ∞).The extension includes atoms at zero and extended-valued divergences, without changing the constant.Source: §14.1 · p. 38
  • Positive laws are approximated by positive finitely supported mean-one laws using conditional expectations on increasing finite sigma-fields.Almost-sure convergence of the approximations supports passage to the limit in the comparison.Source: §14.1 · p. 38

A Equivalent criteria and growth bounds

This section records consequences and reformulations of (2.5) that do not depend on the proof of Theorem 5.4.

A.1 Separate growth conditions

A.1 relates separate growth conditions on u and v to the endpoint criterion and growth bounds. A.2 expresses that criterion in posterior coordinates, including how repeated outcomes contribute.

A.1 Separate growth conditions

  • Separate doubling of u and v implies (2.5) with M = 2C; conversely, the endpoint criterion yields finite growth constants depending on f.Source: §A.1 · p. 39
  • For 0 < t ≤ 1/2, f(t) is bounded by C(1 + | log t|)^p−.Source: §A.1 · p. 39
  • Dyadic iteration bounds u and v polynomially, and substitution yields the stated growth bounds on f.Source: §A.1 · p. 39

A.2 Posterior coordinates

  • Squaring the likelihood ratio maps posterior coordinate u to T(u), giving an endpoint-criterion formulation in posterior coordinates.Source: §A.2 · p. 40
  • Under the midpoint measure, posterior values −a and b have probabilities b/(a+b) and a/(a+b), respectively.The original divergence is [bΦ(−a)+aΦ(b)]/(a+b); repeated outcomes contribute the left-hand side of (A.5) divided by (a + b)^2.Source: §A.2 · p. 40

Statements and declarations

  • Generative-AI tools were used in exploratory development and editorial preparation; the author takes responsibility for mathematical statements and references.Source: Statements and declarations · p. 40
Loading 2610.31164v1…