Source-linked AI summary
Wasserstein continuity of entropy and outer bounds for interference channels
Yury Polyanskiy, Yihong Wu
TL;DR
The paper asks when closeness of distributions ensures closeness of entropy, in part to replace complicated non-i.i.d. interference with an i.i.d. approximation. It proves Wasserstein continuity under regularity conditions and combines it with transportation-information inequalities. This yields a new outer bound for the two-user Gaussian interference channel that settles Costa’s missing corner point.
Problem
The paper asks whether distributional closeness can guarantee closeness of differential entropy, a question relevant to replacing complicated non-i.i.d. interference with a simpler i.i.d. approximation.
Source: §1 · p. 1Method
Results
The resulting new outer bound for the two-user Gaussian interference channel settles Costa’s missing corner point.
Source: §1 · p. 2Takeaways & Limitations
The bounds support replacing complicated non-i.i.d. interference with i.i.d. approximations in multi-user information-theoretic converses.
Source: §1 · pp. 1–2
Abstract
from arXiv · showhide
It is shown that under suitable regularity conditions, differential entropy is a Lipschitz functional on the space of distributions on $n$-dimensional Euclidean space with respect to the quadratic Wasserstein distance. Under similar conditions, (discrete) Shannon entropy is shown to be Lipschitz continuous in distributions over the product space with respect to Ornstein's $\bar d$-distance (Wasserstein distance corresponding to the Hamming distance). These results together with Talagrand's and Marton's transportation-information inequalities allow one to replace the unknown multi-user interference with its i.i.d. approximations. As an application, a new outer bound for the two-user Gaussian interference channel is proved, which, in particular, settles the "missing corner point" problem of Costa (1985).
1 Introduction
The paper asks when closeness of distributions guarantees closeness of entropy, motivated by replacing complicated non-i.i.d. interference with a simpler i.i.d. approximation. It establishes Wasserstein-based entropy continuity under regularity and uses transportation-information inequalities to prove a new Gaussian interference-channel outer bound settling Costa’s missing corner point.
1 Introduction
- The missing-corner-point question is whether small divergence between the interfering input plus Gaussian noise and its Gaussian approximation implies a sublinear entropy gap after adding another independent input.This formalizes the interference-channel motivation for replacing complicated non-i.i.d. interference with an i.i.d. approximation.Source: §1 · p. 1
- Under suitable regularity conditions, entropy differences can be bounded using Wasserstein distance in both continuous and discrete settings.For the continuous results, the paper identifies moment constraints and density conditions, including conditions satisfied by Gaussian convolutions.Source: §1 · p. 2
- Fine quantization can make Wasserstein distance arbitrarily small while leaving an infinite differential-entropy gap, so regularity assumptions are necessary.Even smooth distributions can have divergence at most log 2 but differential entropies differing by Θ(n).Source: §1 · p. 2
- Transportation-information inequalities bound Wasserstein distance in terms of KL divergence, enabling the paper to establish the interference-channel entropy comparison and a new outer bound.The resulting bound settles the missing corner point of Costa (1985).Source: §1 · p. 2
- The paper also highlights a reverse-direction estimate under regularity conditions, complementing existing bounds for divergence after Gaussian convolution.The cited prior estimates measure Wasserstein distance after convolution with the Gaussian, unlike the paper’s estimate [Vil03, Chapter 9, Remark 9.4].Source: §1 · pp. 2–3
2 Wasserstein-continuity of information quantities
This section establishes Wasserstein continuity for information quantities when density regularity and moment conditions hold. Gaussian smoothing supplies regular densities, and the resulting continuity bounds are sharp in their dependence on dimension and Wasserstein distance.
2 Wasserstein-continuity of information quantities
- A coupling-based estimate compares expectations of log densities under Wasserstein distance when the relevant densities are regular.The section applies this result to bound entropy differences, with finite second moments ensuring the entropies are finite.Source: §2 · pp. 3–4
- Convolution with a Gaussian produces a regular density when the added random vector has finite first moment and is independent of the Gaussian.The proposition gives c1 = 3 log e / σ² for Gaussian noise of variance σ² per coordinate.Source: §2 · p. 4
- Under average power constraints, Gaussian-smoothed mutual information and differential entropy differ by at most a bound Δ controlled by the W2 distance.The corollary assumes independent Gaussian noise and second moments at most nP for both inputs.Source: §2 · p. 5
- The dimension dependence and linear dependence on W2 in the continuity bound are both order-optimal.The paper uses Gaussian inputs with different variances to demonstrate these optimality claims.Source: §2 · p. 5
- For bounded-norm inputs, the paper bypasses the regular-density formulation to obtain a sharpened bound useful for interference-channel applications.A subsequent corollary combines the bound with Talagrand’s inequality to relate the comparison to divergence.Source: §2 · pp. 5–6
3 Applications to Gaussian interference channels
The paper applies its information-continuity results to derive a new outer bound for the two-user Gaussian interference channel. Its new second component settles Costa’s missing corner, completing the corner-point picture under average power constraints.
3.1 New outer bound
- Under almost-sure power constraint (31), the result is stated for b ≥ 0 and C̃2 ≤ R2 ≤ C2; in both cases, as R2 approaches C2, δ tends to 0 and A to 1.Source: §3.1 · pp. 7, 9
- The new second part of the outer bound settles the missing capacity-region corner; its first part coincides with Sato’s and Kramer’s bounds.The authors note that tightening their estimate by the same method would require more information about N1(t).Source: §3.1 · p. 9
- Figure 1 evaluates the outer bound for the Z-interference case b=0 alongside a simple Han–Kobayashi inner bound using jointly Gaussian U and V.The passage does not state which bound is tighter.Source: §3.1 · p. 9
3.2 Corner points of the capacity region
- Under average power constraints, Theorem 7 completes the capacity-region corner-point picture for all a,b∈R+; the new result is proving C′1.Figure 1 identifies the upper corner as Costa’s conjectured point and the bottom corner as Sato’s established point.Source: §3.2 · p. 10, Figure 1 · p. 10
- For 0<a≤1 and b≥0, the converse uses Talagrand’s inequality and a coupling argument in place of Pinsker’s inequality.The achievability was previously known; the passage says Costa’s earlier converse proof was flawed, as pointed out in.Source: §3.2 · p. 10
4 Discrete version
This section bounds changes in discrete entropy and information using Ornstein’s distance, relates that distance to KL divergence for product distributions, and applies these results to discrete interference-channel corner rates. It verifies the channel conditions and establishes achievability of (C′1, C2) using a random MAC code.
4.1 Bounding entropy and information via Ornstein’s distance
- Under regularity conditions, the discrete entropy bound improves from superlinear to linear in Ornstein’s distance; Proposition 9 also bounds symmetric output KL divergence and mutual-information change.For a memoryless two-input channel, the bounds scale with blocklength n and the expected conditional Ornstein distance.Source: §4.1 · pp. 12–13
4.2 Marton’s transportation inequality
- For product reference distributions, Marton’s transportation inequality converts KL divergence bounds into Ornstein-distance bounds, with a one-sided improvement also available.The section also states a contraction property for channels acting coordinatewise.Source: §4.2 · pp. 14–15
- With i.i.d. interfering inputs, Proposition 9 gives a two-sided output divergence estimate that remains applicable when the comparison output is a mixture of product distributions.The output Ornstein distance is controlled using channel contraction and the product-reference transport bound.Source: §4.2 · p. 15
4.3 Application: corner points for discrete interference channels
- If I(X^n; A^n) ≤ I(X^n; B^n) + εn under the stated channel conditions, then H(X^n) ≤ I(X^n; B^n) + g(ε)n for continuous g with g(0)=0.The corollary follows from tensorization for the memoryless channel chain X^n → A^n → B^n.Source: §4.3 · p. 16
- Theorem 13 characterizes corner rates for a two-user discrete memoryless interference channel with nonuniform full-support noise and remains valid under cost constraints.The theorem considers a setting where each user’s maximum achievable rate is evaluated at the other user’s maximal rate.Source: §4.3 · p. 17
- The proof bounds I(X1; X2|Y2) by H(X2|Y2)=o(n) using Fano’s inequality.Source: §4.3 · p. 18
- Applying Corollary 12 is justified because both channels are memoryless and additive, and their noise distributions satisfy the stated conditions.PA|X has non-uniform noise P2 satisfying condition (81); PB|A has noise Q2, the maximizer of (86), and since P2 is non-uniform, Q2 is not a point mass, so PB|A satisfies (80).Source: §4.3 · p. 18
- The maximum of I(X1; X1 + ˜X2 + Z2) is nC′1, attained when X1 is uniform on {0, 1, 2}^n.Source: §4.3 · p. 18
- The rate pair (C′1, C2) is achievable by a random MAC code with X1 uniform on {0, 1, 2}^n and X2 distributed as Q2^⊗n.The code is for the channel (X1, X2) → Y2.Source: §4.3 · p. 18
A Proof of Lemma 10
The proof builds couplings from coordinate-wise optimal couplings, then uses Dobrushin contraction to bound the output Hamming distance by the input Ornstein distance.
A Proof of Lemma 10
- Independent optimal couplings of the coordinate distributions are combined to prove the tensorization inequality.The construction couples each coordinate pair optimally before forming the joint coupling.Source: A Proof of Lemma 10 · p. 18
- For joint distributions, an optimal input coupling is extended to an output coupling using the preceding inequality.The proof starts with a coupling attaining the input Ornstein distance and then bounds the output distance.Source: A Proof of Lemma 10 · p. 19
- The output bound is nη times the input Ornstein distance, where η is the largest coordinate-wise conditional total-variation distance and the intermediate step uses Dobrushin contraction.Source: A Proof of Lemma 10 · p. 19
B Proof of Proposition 11
The proof establishes strict inequality for Fc(t) at positive t using the stated channel conditions, then derives the single-letterization statement by induction from the blocklength-2 case.
B Proof of Proposition 11
- Caratheodory’s theorem reduces the optimization defining Fc to ternary-valued U, and compactness of achievable conditional-entropy pairs means Fc(t)=t would be attained.Source: B Proof of Proposition 11 · p. 19
- Under the extra conditions on PB|A and PA|X, PB|A satisfies strong data processing with η < 1; I(X; A|U) = 0 and H(X|U) = 0, implying t = 0.Source: B Proof of Proposition 11 · p. 19
- The single-letterization proof handles n=2 first, using the Markov-chain structure and Fc’s definition, concavity, and monotonicity before extending by induction.Source: B Proof of Proposition 11 · pp. 19–20