PaperlayerView exact revision on arXiv

Source-linked AI summary

Exact Common Information and Exact Channel Synthesis for Correlated Gaussian Sources

Lei Yu

arXiv:2608.26012v1cs.IT

TL;DR

The paper addresses the unresolved exact common information of correlated Gaussian sources and the conjectured admissible region for exact channel synthesis. It establishes the conjectured exact common-information expression and confirms the conjectured channel-synthesis region, while showing a strict gap above Wyner’s common information for ρ > 0.

  • Problem

    Exact common information for Gaussian sources remains unknown, and the conjectured inner bound for exact channel synthesis was not yet known to be tight.

  • Method

    The paper proves a fundamental converse and combines it with previously conjectured inner and upper bounds to establish tight characterizations.

  • Results

    The exact common information equals 1/2 log((1 + ρ)/(1 − ρ)) + ρ/(1 + ρ), and the conjectured admissible region for shared randomness and communication rates is confirmed.

  • Takeaways & Limitations

    For every ρ > 0, exact common information is strictly larger than Wyner’s common information, with exactness penalty ρ/(1 + ρ).

Abstract

from arXiv · show

In this paper, we resolve two conjectures posed by Yu and Tan in 2020 (in two separate papers published in the IEEE Trans. Inf. Theory). Specifically, we establish that: 1) the exact common information for a pair of $ρ$-correlated Gaussian sources is given by the conjectured expression $\frac{1}{2}\log\frac{1+ρ}{1-ρ}+\fracρ{1+ρ}$; and 2) the admissible region for the shared randomness rate and the communication rate in exact channel synthesis is exactly the conjectured one. These results yield two important consequences. First, for any $ρ>0$, the exact common information of a correlated Gaussian pair strictly exceeds Wyner's common information. Second, for $ρ>0$, the exact channel synthesis of such a pair requires strictly higher rates than the total-variation version. The proof combines an exact optimal-transport representation of the worst-case Gaussian cross-entropy, Fathi's Gaussian transport inequality, and a determinant inequality arising from the covariance structure of the conditional means.

I. INTRODUCTION

The introduction distinguishes exact common information from approximate synthesis and frames exact distributed channel synthesis as a shared-randomness–communication tradeoff. Prior work characterized important special cases, but Gaussian exact synthesis remained unresolved.

  • I. INTRODUCTION: Wyner’s common information instead studies approximate generation in relative entropy and is characterized by minimizing I(X, Y; W) over X−W−Y.
  • I. INTRODUCTION: Exact common information requires generating the target joint distribution exactly using independently processed common randomness at two terminals.Its rate is the asymptotic normalized entropy of the smallest common random variable.
  • I. INTRODUCTION: Exact common information is no smaller than Wyner’s, while single-letter characterizations are known only for limited source classes and were unknown for Gaussian sources.
  • I. INTRODUCTION: Distributed channel synthesis asks for communication sufficient to generate correlated sources at separate terminals, with exact or asymptotic total-variation fidelity to the target product distribution.The sender observes X^n, communicates through a message, and the receiver generates Y^n using shared randomness and that message.
  • I. INTRODUCTION: The exact synthesis code uses shared randomness K, a sender mapping P_M|X^nK, and a receiver mapping P_Y^n|MK to induce the joint distribution.
  • I. INTRODUCTION: Prior work established the mutual-information communication rate under unlimited shared randomness, while DSBS was the first source with an explicit optimal exact rate tradeoff.

C. The Gaussian setting

For correlated Gaussian sources, prior work gave Wyner’s benchmark and conjectured tight exact-synthesis bounds. The paper focuses on proving those Gaussian conjectures, including the strict exactness penalty for positive correlation.

  • C. The Gaussian setting: The Gaussian setting considers the standard bivariate Gaussian distribution with correlation parameter ρ.
  • C. The Gaussian setting: Wyner’s common information is achieved by decomposing X and Y into a shared Gaussian component W and independent Gaussian noises.The decomposition is W ∼ N(0, ρ), X = W + N_X, and Y = W + N_Y.
  • C. The Gaussian setting: Yu and Tan proved an upper bound for exact Gaussian common information and showed it equals the ∞-Rényi common information, then conjectured the upper bound was tight.
  • C. The Gaussian setting: The second term ρ/(1+ρ) is positive for ρ > 0, so tightness would imply exact Gaussian synthesis needs more common randomness than Wyner’s approximate synthesis.
  • C. The Gaussian setting: Yu and Tan also proved an inner bound for exact Gaussian channel synthesis and conjectured that this inner bound is tight.

D. Main results

The paper establishes conjectured formulas for exact common information and exact channel synthesis of correlated Gaussian sources, then derives a strict penalty relative to Wyner’s common information.

  • Exact common information: Theorem 1’s lower bound, combined with Yu–Tan’s upper bound, confirms Conjecture 1 positively.The result applies to the correlated Gaussian law πρ = N(0, Σρ).
  • Exact common information: The exact common information has the scalar Gaussian formula 1/2 log((1+|ρ|)/(1−|ρ|)) + |ρ|/(1+|ρ|) for |ρ| < 1.For negative ρ, invariance under separate deterministic bijections reduces the expression to |ρ|.
  • Exact common information: For every ρ > 0, exact common information is strictly larger than Wyner’s common information.The exactness penalty is ρ/(1+ρ).
  • Exact common information: The gap ρ/(1+ρ) is bounded above by 1/2, and its bit-per-symbol version is bounded by approximately 0.7213.The supplied bound states ρ/(1+ρ) ≤ 1/2 and ρ/(1+ρ) log2 e ≤ 0.7213.
  • Exact channel synthesis: Applying the proof idea to exact channel synthesis yields a second contribution, and combining its outer and inner bounds confirms Conjecture 2 positively.The outer bound is combined with Yu–Tan’s inner bound in (4).
  • Proof strategy: The proof strategy introduces a maximal Gaussian cross-entropy functional and exploits conditional independence in an n-letter formulation.The construction ranges over discrete W and conditional distributions satisfying P_X^nY^n = πρ^⊗n, with h(X^n,Y^n|W) = h(X^n|W) + h(Y^n|W).

B. Evaluation of the Maximal Gaussian Cross-Entropy

The section converts maximal Gaussian cross-entropy into an optimal-transport problem, then derives an exact expression involving conditional means, covariances, and a Wasserstein term.

  • For ρ ≥ 0, maximizing cross-entropy with fixed marginals is equivalent to minimizing E⟨X^n, Y^n⟩.
  • The minimization of E⟨X^n, Y^n⟩ is transformed into minimizing E|X^n − a_w + Y^n − b_w|^2.
  • The centered conditional variables have laws determined by their conditional covariances, so the minimum is represented by a squared Wasserstein distance.
  • Proposition 2 gives the exact cross-entropy expression for every conditioning value w.

C. Fathi’s Gaussian Transport Inequality

The section applies Fathi’s Gaussian transport inequality after scaling the centered conditional distributions, reducing source-specific terms to a conditional-entropy bound.

  • Fathi’s Gaussian transport inequality is applied to centered conditional distributions with finite second moments.
  • A temporary scaling parameter t is introduced so the transport inequality matches the Wasserstein term required by the cross-entropy expression.
  • The conditional entropy is defined as H_w = h(X^n|w) + h(Y^n|w).
  • Substitution yields an upper bound containing conditional entropies, conditional mean norms, the Wasserstein term, and the correlation parameter ρ.
  • Using the source moments and conditional independence gives E[a_W · b_W] = nρ.
  • The source-specific analysis is reduced to bounding the conditional entropy E H_W.

D. Entropy Bound from the Conditional Covariance

This section establishes a determinant inequality from the covariance structure and uses Gaussian maximal entropy plus log-determinant concavity to bound conditional entropy.

  • The determinant inequality follows from positive semidefiniteness, a Schur complement, and the covariance bounds U and V ⪯ I_n.
  • For ρ > 0, the Schur complement yields V ⪰ ρ^2U^−1 and therefore ρ^2U^−1 ⪯ I_n.
  • The eigenvalues u_1, …, u_n of U are used to derive the required eigenvalue-wise bound.
  • Lemma 4 states a conditional entropy bound for every exact Gaussian code.
  • Gaussian maximal entropy bounds each conditional entropy using the determinant of its conditional covariance matrix.
  • Averaging and applying concavity of log det completes the entropy estimate.

E. Optimization over the Transport Parameter

The section optimizes the entropy bound over the transport parameter t, identifies a unique maximizer, and obtains the conjectured rate expression used in the converse.

  • For 0 < ρ < 1, differentiating the bound with respect to t gives the condition determining its optimizer.
  • The unique maximizer is t = 1 − ρ.
  • The optimized expression is 1/2 log((1 + ρ)/(1 − ρ)) + ρ/(1 + ρ).
  • The multi-letter converse applies this optimized bound to obtain the same expression as a converse rate.

III. PROOF OF THEOREM 3

The proof restricts attention to the nontrivial correlation regime ρ ∈ (0, 1), since ρ = 0 is trivial.

  • The analysis considers only correlations satisfying ρ ∈ (0, 1).The case ρ = 0 is treated as trivial.

A. A Multi-letter Bound

The paper defines an n-letter rate region for exact channel synthesis through conditional independence, mutual information, and the objective Γ_n, then proves a converse for every exact code.

  • The n-letter rate region is introduced for exact synthesis under the constraint PX^nY^n = π^⊗n.
  • The region requires R ≥ 1/n I(W; X^n) and R + R_0 ≥ 1/n Γ_n(P_W, P_X^n|W, P_Y^n|W).
  • The objective Γ_n is defined as −h(X^n|W) − h(Y^n|W) + E_W H(n).
  • The fundamental converse applies to every exact n-letter channel synthesis code, with W = (M, K) satisfying X^n — W — Y^n.The converse yields the same mutual-information and Γ_n rate lower bounds.

B. Evaluation of the Multi-letter Bound

The multi-letter bound is evaluated by controlling the eigenvalues of U, applying entropy and determinant inequalities, and showing that equal eigenvalues yield a further bound.

  • The eigenvalues u_i of U satisfy ρ^2 ≤ u_i ≤ 1.
  • Entropy, concavity of log det, and total covariance produce the intermediate bound used in the evaluation.
  • Setting all u_i to the same value leads to a further bound.
  • Defining x_i = log(1 − u_i), the proof establishes concavity and applies Jensen’s inequality under the transformed constraint.
  • The parameter t is optimized and substituted into the bound to obtain the desired result.
Loading 2608.26012v1…