A Complete Characterization of Tensorizable $f$-divergences

• Tensorizable f-divergences decompose discrepancies between product measures into expressions depending only on marginal discrepancies.Source: §I · p. 1
• Their compositional formulas remain tractable for measures with many factors.Source: §I · p. 1
• Tensorizable divergences support estimation of difficult discrepancies in hypothesis testing and cumulative privacy accounting.Source: §I · p. 1
• They also underpin information-theoretic bounds in online learning and stochastic optimization.Source: §I · p. 1

• The paper fills the gap that tensorizable f-divergences had not been formally characterized.Source: A. Contributions and Methods · p. 1
• The only tensorizable f-divergences are nonnegative linear combinations of KL and reverse KL divergences or power divergences.Source: A. Contributions and Methods · p. 2
• Every admissible tensorization formula belongs to a one-parameter family of multi-affine functions.Source: A. Contributions and Methods · p. 2
• For symmetrized tensorizable divergences, the tensorization parameter and local characteristic provide two scalar identifying parameters.Source: A. Contributions and Methods · p. 2

• Unlike prior work restricted to polynomial tensorization rules, this paper identifies every admissible rule without assumptions on its formula.Source: B. Related Work · p. 2
• The analysis overlaps with axiomatic information-measure research because tensorization can be distilled into a functional equation.Source: B. Related Work · pp. 2–3
• The paper argues that tensorization alone uniquely identifies Tsallis α-relative entropy in the relevant regime.Source: B. Related Work · p. 2
• Its setting studies a broader composition notion while restricting attention to the smaller class of f-divergences.Source: B. Related Work · p. 3

• An f-divergence is defined from a convex function f with f(1)=0 and Radon–Nikodym derivatives relative to a common dominating measure.Source: §II · p. 3
• The finite-value range is R(D_f)={D_f(P∥Q):D_f(P∥Q)<∞}.Source: §II · p. 3
• If f(0) and f′(∞) are finite, the range is [0,f(0)+f′(∞)]; otherwise it is [0,∞).Source: §II · p. 3
• Tensorization requires a map τ_f that combines divergences of two factor pairs into the divergence of their product pair.Source: §II · p. 3
• The analysis applies to arbitrary measurable spaces, and tensorization bases extend to n≥2 factors inductively.Source: §II · p. 3

• 0 ∈ R(D_f) for every f-divergence.Source: 3) Marginalization · p. 4

• Df(P∥Q) is determined by the log-likelihood-ratio distribution of Q against P.Source: §III · p. 4
• Under product measures, marginal log-likelihood ratios add, so the product pair corresponds to convolution of marginal LLR distributions.Source: §III · pp. 4–5
• Tensorization makes Df(µ1 ∗ µ2) depend only on the two scalar divergences Df(µ1) and Df(µ2), upgrading mixture affinity in distributions to affinity in divergence values.Source: §III · p. 5
• Every section of the tensorization base is affine in each argument and symmetric under exchanging the arguments.Source: §III · p. 5
• τ_f(d1,d2) = d1 + d2 + γ d1d2, where γ is the single tensorization parameter.Source: §III · p. 5

• The symmetrized divergence restricted to two-point symmetric LLR distributions becomes a real-valued function g satisfying a tensorization functional equation.Source: §IV · p. 6
• The local characteristic is the finite nonnegative quadratic rate at which g(r) vanishes as r approaches zero.Source: §IV · p. 6
• For fixed tensorization parameter γ and local characteristic ℓ, at most one continuous g solves the functional equation.Source: §IV · p. 7
• The restriction g uniquely determines the symmetrized f-divergence, so matching γ and ℓ yields uniqueness of the symmetrized divergence.Source: §IV · p. 7
• A symmetrized tensorizable f-divergence must satisfy γℓ ≥ −1/8; the excluded regime produces an oscillatory, nondecreasing-incompatible solution.Source: §IV · p. 7

• The proof evaluates tensorization on the convolution of an arbitrary two-point LLR distribution and a symmetric one, producing a linear functional equation for f.Source: §V · pp. 7–8
• The relevant solution space has dimension two, allowing the complete family of continuous solutions to be identified from kernel elements.Source: §V · p. 8
• Convexity reduces all solutions to nonnegative combinations of KL and reverse KL divergences or to power divergences.Source: §V · p. 9
• When ℓ > 0 and γ = 0, f(x) = A x log x − B log x + E(x − 1), with A,B ≥ 0 and A + B = 2ℓ.Source: §V · p. 9

• The characterization shows that tensorizable f-divergences are nonnegative combinations of KL and reverse KL divergences or power divergences.Source: §VI · p. 9
• The analysis connects the adopted tensorization formalism to classical functional-equation methods rather than defining tensorization directly through additivity.Source: §VI · p. 9
• The class of tensorizable f-divergences is not closed under addition, because the simplification rule for Df+g can depend on all marginal discrepancies.Source: §VI · p. 9
• Linear combinations of tensorizable f-divergences remain a proposed direction for studying broader compositional classes.Source: §VI · p. 9
• The measure-theoretic appendix establishes invariance of the log-likelihood-ratio distribution to the dominating measure and closure of realizable LLR distributions under mixtures and convolution.Source: Appendix A · pp. 9–11
• The result applies to subclasses of measurable spaces closed under products because the proofs require tensorization for finite-support discrete distributions.Source: Appendix B · pp. 11–12

A. Existence of the Local Characteristic

• The proof studies h(r)=g(r)/r^2 along dyadic halving steps, deriving recursive relations that yield convergence as r→0+.Source: A. Existence of the Local Characteristic · pp. 12–13
• A uniform bound near the origin and continuity extend the common limit from a dense subset to an interval.Source: A. Existence of the Local Characteristic · pp. 13–14
• Two continuous solutions with the same tensorization parameter and local characteristic agree near zero, and the recursive identity propagates equality to all positive arguments.Source: B. Uniqueness from the Local… · pp. 14–15
• The kernel analysis bounds its dimension by two and shows that values on exponential lattices are determined by two initial values.Source: TENSORIZABLE f-DIVERGENCES · pp. 15–16
• The resulting f-functions include Ax log x−B log x+E(x−1), with A+B=2ℓ and convexity requiring A,B≥0.Source: TENSORIZABLE f-DIVERGENCES · p. 16
• The appendix verifies that the derived functions induce tensorizable f-divergences using logarithm and exponent identities, integral linearity, and Tonelli–Fubini.Source: TENSORIZABLE f-DIVERGENCES · p. 17