PaperlayerView exact revision on arXiv

Source-linked AI summary

Conditional Fisher-Information Central Limit Theorems under Log-Concavity with Information-Theoretic Consequences

Tong Ye, Liu-Quan Yao

arXiv:2609.07150v1cs.ITmath.ITmath.PR

TL;DR

The paper asks when conditional central limit behavior can be established in Fisher information for sums whose conditional summands are generally nonidentically distributed. Under conditional log-concavity, it proves a finite-convolution-level criterion and derives matrix convergence and operational benchmark consequences.

  • Problem

    The paper asks whether averaged conditional Fisher information converges to the Gaussian benchmark when conditioning makes the independent summands generally nonidentically distributed.

    Source: §I · p. 2

  • Method

    Under full-dimensional conditional log-concavity, the paper proves an equivalence between Fisher-information convergence and finiteness at one convolution level, with an independent scalar proof using second-order Fisher-production continuity.

    Source: §I · p. 2, A. The fixed-dimensional criterion · p. 8

  • Results

    The averaged conditional Fisher information converges to the dimension exactly when it is finite at one convolution level, while the averaged Fisher information matrix of the original sums converges in operator norm.

    Source: §I · p. 2

  • Takeaways & Limitations

    The matrix limit yields conditional relative-information and entropy convergence, and supports low-power finite-constellation and fixed-covariance Gaussian-signaling benchmark results.

    Source: §I · pp. 2–3

Abstract

from arXiv · show

We establish conditional central limit theorems in Fisher information under log-concavity in every fixed dimension. For conditionally centered normalized sums, after whitening by the averaged conditional covariance, the averaged conditional Fisher information converges to the dimension if and only if it is finite at one convolution level. The scalar criterion follows as the one-dimensional case; we also provide an independent scalar proof based on a second-order continuity theorem for Fisher production on Gaussian-smoothed, tail-controlled classes. For the original sums, the averaged Fisher information matrix converges in operator norm to the inverse averaged conditional covariance. Consequently, the conditional relative Fisher information with respect to the limiting Gaussian law vanishes, and the Gaussian logarithmic Sobolev inequality yields convergence in conditional relative entropy and conditional entropy. We give two operational consequences. For any fixed finite-constellation low-power input, the first-order conditional mutual-information slope converges to the Gaussian-noise benchmark. For Gaussian signaling at any fixed signal covariance, the mutual-information gap from that benchmark is bounded by the conditional relative Fisher deficit and hence vanishes asymptotically.

I. INTRODUCTION

The paper studies whether averaged conditional Fisher information of conditionally independent, generally nonidentically distributed sums converges to the Gaussian benchmark. Under full-dimensional conditional log-concavity, it gives a finite-at-one-convolution-level criterion and derives matrix, entropy, and channel consequences.

I. INTRODUCTION

  • Conditioning makes the summands independent but generally nonidentically distributed, and the paper proves Fisher-information convergence without an a priori Gaussian regularization.The fixed-dimensional proof identifies the conditional Gaussian limit using a conditional Lindeberg–Feller theorem, then applies Gaussian smoothing and second-order Fisher dissipation.Source: §I · pp. 2–3
  • The averaged conditional Fisher information converges to the dimension if and only if it is finite at one convolution level, and the original sums’ averaged Fisher information matrices converge in operator norm.The criterion holds under full-dimensional conditional log-concavity and preserves directional information in the matrix limit.Source: §I · p. 2
  • The Fisher-matrix limit implies vanishing conditional relative Fisher information and yields conditional relative-entropy and entropy convergence, alongside low-power and Gaussian-signaling benchmark results.The low-power result concerns fixed finite-constellation inputs; the Gaussian-signaling result concerns fixed signal covariance.Source: §I · p. 3

II. PRELIMINARIES AND NOTATION

  • The setup fixes a common probability space and dimension, and uses jointly measurable versions of conditional densities whenever they are needed.Statements on R^d take d ≥ 1 fixed.Source: §II · p. 4

A. Basic notation and moments

  • For finite-second-moment densities, the notation covers means, covariances, and a centered second-moment tail functional.In one dimension, covariance is written as variance.Source: A. Basic notation and moments · p. 4

B. Information functionals

  • Scalar Fisher information is defined in the weak Sobolev sense, allowing nonsmooth and compactly supported densities.When Fisher information is finite, the score has a weak-definition-equivalent representation.Source: B. Information functionals · p. 4

C. Conditional information functionals

  • Conditional information functionals and tail quantities are random variables determined by the conditioning state, and their conditional values are averaged over that state.The conditional Fisher information scalar is the trace of the conditional Fisher information matrix.Source: C. Conditional information functionals · p. 5

D. Conditional relative Fisher information and Gaussian reference

  • Conditional relative entropy and dimensionless conditional relative Fisher information are measured against a fixed Gaussian reference, not the random conditional covariance.These definitions assume conditional centering and use a fixed reference covariance.Source: D. Conditional relative Fisher… · p. 5

E. Log-concave densities

The paper assumes log-concave conditional laws and proves a fixed-dimensional conditional Fisher-information criterion, with scalar and information-theoretic consequences. Gaussian smoothing and conditional multivariate convergence feature in the proof strategy.

E. Log-concave densities

  • Log-concavity is preserved by affine transformations, products, convolution, marginalization, and Gaussian smoothing.The density definition is expressed through a convex potential or an equivalent log-concavity inequality.Source: E. Log-concave densities · p. 6

A. The fixed-dimensional criterion

  • The conditional fixed-dimensional theorem assumes full-dimensional log-concave conditional densities, and its Fisher-information criterion is finite information at some convolution level.The scalar criterion is stated as the d = 1 case.Source: A. The fixed-dimensional criterion · p. 7
  • Three convolutions suffice in the Bernoulli mixture of uniform conditional laws, although each original conditional density has infinite Fisher information.The example’s triple-convolution density has finite Fisher information, so the corollary applies with n0 = 3.Source: A. The fixed-dimensional criterion · pp. 7–8

B. Fisher-matrix and relative-information consequences

  • Fisher convergence is equivalent here to conditional relative Fisher convergence and implies conditional relative-entropy and entropy convergence.The stated comparison with [14] is that its finite-entropy criterion uses absolute continuity without log-concavity, whereas the present result assumes log-concavity.Source: A. The fixed-dimensional criterion · p. 8

A. The fixed-dimensional criterion

  • The scalar result also has an independent proof using continuity of second-order Fisher production K.That proof follows an entropic route and the Gaussian-smoothed continuity method.Source: A. The fixed-dimensional criterion · p. 8

B. Conditional multivariate Lindeberg–Feller convergence

C. Identification of the Gaussian-smoothed limits

The proof identifies Gaussian-smoothed limits by combining conditional weak convergence with regularity of Gaussian convolution. It then uses log-concavity and Fisher dissipation to identify smoothed Fisher limits before removing the smoothing.

C. Identification of the Gaussian-smoothed limits

  • For fixed t > 0, Gaussian convolution turns bounded-Lipschitz convergence into locally uniform convergence of densities and derivatives through order two.The convergence is locally uniform on Rd.Source: C. Identification of the… · p. 10
  • The posterior-covariance Hessian identity supplies a bound when the underlying law is log-concave.The passage states the identity for vectors with finite second moment and an independent Gaussian, then gives a bound under log-concavity.Source: C. Identification of the… · p. 10
  • For every fixed t > 0, the proof identifies the smoothed K limit under the theorem’s assumptions.Along the extracted subsequence, K converges to the Gaussian value d/(1 + t)^2.Source: C. Identification of the… · pp. 10–12
  • For every fixed t > 0, a separate proposition identifies the smoothed Fisher limit under the same assumptions.The proof invokes Fisher dissipation and integrates the smoothed K limit.Source: C. Identification of the… · p. 12

D. Proof of Theorem III.1

  • The theorem proof uses an integrable dominating function to pass from smoothed Fisher convergence to the unsmoothed limit along a dyadic subsequence.It then uses the Blachman–Stam inequality and subadditivity to extend convergence to the full sequence.Source: D. Proof of Theorem III.1 · pp. 14–15

E. Matrix Cram´er–Rao and inversion Jensen inequalities

The Fisher-matrix convergence argument uses matrix Cramér–Rao and inversion Jensen inequalities. The section reports an operator-norm convergence result for the averaged Fisher matrix.

E. Matrix Cramér–Rao and inversion Jensen inequalities

  • The matrix Cramér–Rao inequality applies to full-dimensional densities with finite covariance and finite Fisher information.The proof cites Kagan and Landsman for the multivariate covariance–Fisher information inequality.Source: E. Matrix Cram´er–Rao and inversion… · p. 15

F. Fisher matrix convergence

  • The argument establishes operator-norm convergence of the Fisher matrix after showing its entries are integrable for sufficiently large n.The passage identifies the operator-norm convergence step and its preceding integrability argument.Source: F. Fisher matrix convergence · pp. 16–17
  • The inversion Jensen inequality uses operator convexity of V ↦ V^-1 on the positive-definite cone.It is used in the Fisher-matrix convergence argument.Source: F. Fisher matrix convergence · p. 16

V. INFORMATION-THEORETIC CONSEQUENCES FOR STATE-DEPENDENT AGGREGATE NOISE

The section derives conditional Gaussianity consequences and operational results for state-dependent aggregate noise. Under the finite-level Fisher condition, finite-constellation low-SNR slopes converge to the Gaussian benchmark.

V. INFORMATION-THEORETIC CONSEQUENCES FOR STATE-DEPENDENT AGGREGATE NOISE

  • The section also states a Gaussian-signaling result at fixed signal covariance.Its introduction identifies a mutual-information gap result among the operational consequences.Source: §V · p. 17

A. Relative Fisher information controls conditional Gaussianity

  • The passage says that combining an identity, whose content is not shown here, with DΣ(Sn | ηn) → 0 proves the result labeled (III.17), without specifying that result’s content.Source: A. Relative Fisher information… · p. 18

B. Finite-constellation signaling: the low-SNR information slope

  • The conditional weak-signal expansion requires only finite expected conditional Fisher information in the weak Sobolev sense.It permits conditional densities that are nonsmooth, vanish, or have compact support; pointwise conditional remainders cannot generally be averaged over V.Source: B. Finite-constellation signaling: the… · p. 20
  • For every fixed finite-constellation input, the asymptotic conditional low-SNR slope equals the Gaussian-noise slope with covariance Σ.The first-order coefficient depends on the input covariance, not the constellation’s detailed geometry or symbol probabilities.Source: B. Finite-constellation signaling: the… · p. 21
  • The energy-constrained finite-constellation class has uniformly convergent first-order slopes under the stated finite-level condition.The section frames the optimization result through the averaged conditional Fisher information matrix.Source: B. Finite-constellation signaling: the… · p. 22

C. Gaussian signaling at fixed signal covariance

Under Corollary III.3’s assumptions, the Gaussian-signaling mutual-information gap is controlled by a Fisher-information deficit. The resulting bound is uniform over fixed signal covariances.

C. Gaussian signaling at fixed signal covariance

APPENDIX A THE MULTIVARIATE LOG-CONCAVE SECOND-ORDER FISHER INEQUALITY

The appendix establishes a multivariate log-concave convolution inequality used in the main argument, then gives an independent scalar proof whose eventual finiteness condition yields the Gaussian Fisher-information limit.

APPENDIX A THE MULTIVARIATE LOG-CONCAVE SECOND-ORDER FISHER INEQUALITY

  • Lemma A.1 provides the multivariate log-concave convolution inequality used in the main argument, derived by rewriting Toscani’s inequality [7, Theorem 6].The proof establishes the stated inequality for λ ∈ (0, 1), with endpoint cases immediate.Source: Appendix A · p. 25

APPENDIX B AN ALTERNATIVE ONE-DIMENSIONAL PROOF

  • The independent scalar proof follows the conditional entropic route of and the Gaussian-smoothed continuity method of [14], adding continuity of second-order Fisher production K.The main theorem already gives the scalar criterion by setting d = 1.Source: Appendix B · p. 26
  • Eventually finite scalar Fisher information yields the limit 1/σ2 through subadditivity and Fekete’s lemma.The proof propagates finiteness from one convolution level to later levels before identifying the limit along a dyadic subsequence.Source: Appendix B · p. 29

APPENDIX C ONE-DIMENSIONAL COMPACTNESS AND CONTINUITY AFTER GAUSSIAN SMOOTHING

Appendix C proves continuity of second-order Fisher production K on an L1-closed class of centered, Gaussian-smoothed densities with controlled tails.

APPENDIX C ONE-DIMENSIONAL COMPACTNESS AND CONTINUITY AFTER GAUSSIAN SMOOTHING

  • K is continuous under L1 convergence on the closure of the Gaussian-smoothed, tail-controlled class Ia,ℓ.The core statement proves continuity within the class first, then extends it to its closure.Source: Appendix C · p. 29
  • The continuity proof reduces to L1 convergence of the smoothed densities and uniform integrability of the integrands defining K.It uses subsequence extraction and a generalized dominated-convergence argument.Source: Appendix C · pp. 29, 30, 32

APPENDIX D PROOF OF THE CONDITIONAL MULTIVARIATE LINDEBERG–FELLER PROPOSITION

Appendix D conditions on the random state to apply the deterministic multivariate Lindeberg–Feller theorem, obtaining weak convergence of the conditional laws to a Gaussian. It also establishes locally uniform convergence of derivatives under Gaussian convolution.

APPENDIX D PROOF OF THE CONDITIONAL MULTIVARIATE LINDEBERG–FELLER PROPOSITION

  • Conditioning on the state gives independent, centered triangular arrays, allowing the deterministic multivariate Lindeberg–Feller theorem to be applied.The conditional joint distribution factors because the original pairs are independent.Source: Appendix D · p. 33
  • The conditional laws converge weakly to the standard Gaussian by the multivariate Lindeberg–Feller theorem and the subsequence criterion for convergence in probability.The argument verifies the covariance and Lindeberg conditions for the conditional triangular arrays.Source: Appendix D · pp. 33–34

A. Derivative convergence under Gaussian convolution

  • Weak convergence of measures implies locally uniform convergence of their Gaussian-convolved derivatives through order two.The derivative convergence follows from bounded-Lipschitz control and Gaussian derivative bounds.Source: A. Derivative convergence under… · pp. 34–35

B. Posterior covariance and Hessian bound

The appendix’s proof uses posterior-covariance identities and weak-Sobolev translation expansions to establish the finite-constellation weak-perturbation lemma.

B. Posterior covariance and Hessian bound

  • The derivative of the posterior mean is expressed through the conditional covariance of X given Y=y.The displayed componentwise expression gives t times the posterior covariance; the subsequent steps differentiate it to obtain a Hessian bound.Source: B. Posterior covariance and Hessian… · p. 36

APPENDIX F PROOF OF THE FINITE-CONSTELLATION WEAK-PERTURBATION LEMMA

  • The lemma proof combines first-order differentiability of translations in the weak-Fisher Sobolev space with a local KL–Hellinger comparison under a bounded likelihood ratio.The argument is given for settings that allow nonsmooth and compactly supported densities.Source: Appendix F · p. 37

A. Notation

  • The Sobolev regularity is imposed only in the Euclidean coordinate x; no differentiability in v is required.Source: A. Notation · p. 37

B. Weak Fisher information and the square-root density

C. First-order Sobolev expansion of the translated densities

  • Dominated convergence establishes the first-order expansion of translated densities, using Fourier translation and weak-derivative identities in x.The passage also establishes that the associated function a_j lies in L2(µ).Source: C. First-order Sobolev expansion of… · pp. 39–40

D. First-order cancellation in the output mixture

The proof establishes first-order cancellation in the output mixture and uses a local KL–Hellinger comparison to pass from a Hellinger expansion to relative entropy.

D. First-order cancellation in the output mixture

E. A local KL–Hellinger comparison

  • Under a bounded likelihood ratio, the local KL–Hellinger comparison supports passage from the Hellinger expansion to relative entropy.The proof establishes the needed limiting integral through an almost-everywhere subsequence argument and dominated convergence.Source: E. A local KL–Hellinger comparison · pp. 42, 44

F. Completion of the proof of Lemma V.1

The proof identifies the conditional and unconditional output densities for each constellation point, then completes the weak-perturbation lemma using a mutual-information expansion.

F. Completion of the proof of Lemma V.1

Loading 2609.07150v1…