Gaussian BROJA Partial Information Decomposition in Closed Form

1 INTRODUCTION

• Figure 1 compares Tilde-PID, Thin-PID, and Eigen-PID while varying source X’s dimension and fixing source Y and target M at 1000 dimensions.Lines show median runtime across repetitions; shaded bands show the 25th–75th percentile range.Source: Figure 1 · p. 1
• Eigen-PID gives a closed-form Gaussian BROJA procedure that avoids iterative optimization and matches existing Gaussian PID methods at a fraction of their runtime.The paper calls this procedure Eigen-PID.Source: §1 · p. 1
• The paper’s roadmap runs from PID background and analytical tools to the closed-form derivation, prior-work discussion, and numerical comparisons.Appendices A–E contain identities, lemmas, proofs, additional experiments, and a real neuroimaging application.Source: §1 · p. 1

2 BACKGROUND

• BROJA PID treats unique and redundant information as functions of the two target–source marginals, while synergy may vary with the compatible coupling between sources.A compatible coupling preserves both target–source marginals while specifying dependence between the sources.Source: §2 · p. 2
• The BROJA unique-information components can be evaluated through the minimum union-information optimization over compatible couplings.Union information is the portion of information about M that is not synergistic.Source: §2 · p. 2
• For a scalar Gaussian target, Barrett showed the BROJA PID reduces to the minimum-mutual-information decomposition, but no closed-form minimum was known for multivariate targets.The passage identifies this boundary for target dimension greater than one.Source: §2 · p. 2
• High-dimensional optimization over all compatible couplings is difficult; earlier Gaussian work restricted couplings to jointly Gaussian ones and used projected-gradient optimization.That restriction initially left open whether it could increase the optimum.Source: §2 · p. 3
• Later work established that a jointly Gaussian optimizer exists for Gaussian source–target marginals and introduced Thin-PID, but the resulting finite-dimensional problem still required iterative numerical optimization.The remaining question was whether the population minimum had a closed-form solution.Source: §2 · p. 3

Main contribution

• The closed-form solution yields Gaussian BROJA union information and, consequently, the unique-information components.The result is stated in terms of the generalized eigenvalue decomposition defined from the conditional covariances of M given each source.Source: Main contribution · p. 3
• The formula evaluates the exact population minimum without initialization, convergence, or stopping criteria.This is the stated optimality advantage over iterative numerical methods.Source: Main contribution · p. 3
• More than 100× speedup is reported in high-dimensional simulations, with PID values coinciding with existing numerical methods.The comparison is against existing numerical methods.Source: Main contribution · p. 3

3 PRELIMINARIES

• The setup considers two target–source distributions with a common target marginal and uses Loewner order to compare symmetric matrices.A ⪯ B means B−A is positive semidefinite; A ≺ B means it is positive definite.Source: §3 · p. 4, §3.1 · p. 4

3.1 Gaussian Setup

• The prescribed source–target marginals are nondegenerate jointly Gaussian, meaning their joint covariance matrices are positive definite; arbitrary compatible couplings need not be Gaussian.The conditional covariances are defined by Schur complements.Source: §3.1 · p. 4

3.2 Gaussian and Matrix Tools

• The matrix tools include generalized eigenvalues and eigendecompositions, order-preserving facts, determinant inequalities, and a linear-Gaussian representation with independent noise.The Gaussian-channel representation treats a jointly distributed variable as an experiment about M.Source: §3.2 · pp. 4–5

4.1 Maximum-Determinant Optimizer

• The maximum-determinant posterior-error covariance optimizer exists uniquely and has a closed-form expression.In generalized-eigenvector coordinates, it keeps the smaller posterior variance in each target direction.Source: §4.1 · pp. 5–6

4.2 Lower Bound on the Union Information

• Proposition 2 establishes a lower bound on BROJA union information under Assumption 1.Source: §4.2 · p. 6

4.3 Achievability and the Closed-Form Solution

• Theorem 1 gives closed-form Gaussian BROJA union and unique information using generalized eigenvalues.Source: §4.3 · p. 6
• The BROJA infimum is attained by a compatible jointly Gaussian coupling.The proof constructs the coupling using a Gaussian experiment that can generate copies of both source experiments.Source: §4.3 · pp. 6–7
• The optimizer’s Gaussian experiment is at least as informative about the target as either source and is least informative among Gaussian experiments with that property.Source: §4.3 · p. 8
• Corollary 1 derives closed-form expressions for BROJA redundancy and synergy from Theorem 1’s proof.Source: §4.3 · p. 8

Secret-key agreement

• James et al. (2019) connected BROJA unique information conceptually to an elephant secret-key scenario, while Banerjee et al. (2018) showed the corresponding rate is upper-bounded by BROJA unique information for discrete variables.Source: §5 · p. 8
• In the Gaussian setting considered, the one-way secret-key rate is the exact minimum of the BROJA unique-information optimization.Source: §5 · p. 8

• For Gaussian data, the result covers multivariate targets and sources without requiring a Markov coupling, beyond the scalar-target, conditional-Markov scope of Goswami and Merkley (2024).Source: §5 · p. 8

6 IMPLEMENTATION AND NUMERICAL VALIDATION

• Eigen-PID is implemented in Python, with code, numerical examples, and usage instructions available online.Source: §6 · p. 9

Runtime comparisons.

• Up to three orders of magnitude of speedup are reported when one source dimension increases while target and other-source dimensions remain fixed at 1000.The methods use the same population covariance matrices and identical CPU resources.Source: Runtime comparisons · p. 9

6 IMPLEMENTATION AND NUMERICAL VALIDATION

• Differences between Eigen-PID and the numerical methods remain below 2.8 · 10−11 bits for Tilde-PID and 8 · 10−12 bits for Thin-PID in the gain-sweep figure.Source: Figure 2 · p. 9, §6 · p. 9

• Eigen-PID agrees with Thin-PID and Tilde-PID up to numerical error while reducing runtime by orders of magnitude in high-dimensional settings.The solution expresses each source’s unique information through generalized eigenvalues of the posterior covariance matrices and gives a closed form for Γ★.Source: §7 · p. 10

Positive-definiteness requirements

• Applying Eigen-PID requires positive-definite estimated source–target joint covariances, a condition sample covariance estimates do not meet when N ≤ d_M + max(d_X, d_Y).The paper recommends regularization or dimensionality reduction to obtain positive-definite, well-conditioned estimates.Source: §7 · p. 10

Correcting finite-sample bias

• Sample covariance plug-in estimates retain finite-sample bias even when well-conditioned; resampling-based corrections can address it.Eigen-PID makes these corrections practical for high-dimensional data; the reported fMRI analysis took 14.5 minutes versus a projected 21 days with Thin-PID, and 3.9 minutes on a GPU.Source: §7 · p. 10

Extension to non-Gaussian data

• For non-Gaussian data mapped to an approximately Gaussian latent space with normalizing flows, Eigen-PID can replace Thin-PID for the Gaussian PID stage.Source: §7 · p. 10

Differentiating through the PID computation

• The PyTorch implementation is differentiable almost everywhere, allowing exact Gaussian BROJA PID components to serve as loss terms in gradient-based optimization.It has ReLU-like kinks at λ_i = 1.Source: §7 · p. 10

AI use statement

• ChatGPT 5.6 Pro assisted exploratory mathematical work and generated preliminary versions of some proofs, whose final versions use substantially different arguments.The authors say the final proofs do not rely on critical proof ingredients from the AI-generated versions.Source: AI use statement · p. 11
• Codex assisted with software implementation from human-provided algorithmic specifications.Source: AI use statement · p. 11
• The authors say AI tools did not develop theoretical models or conceptual frameworks, supply critical ingredients for final proofs, refine hypotheses, or shape methodology, experiments, or interpretation.Source: AI use statement · p. 11
• ChatGPT provided copyediting and proofreading suggestions, and AI tools helped check consistency among numerical-result descriptions, figures, and experimental logs.Source: AI use statement · p. 11
• The authors report manually verifying and revising AI-assisted material and accept responsibility for the final text, claims, proofs, code, and other artifacts.Source: AI use statement · p. 11

B.1 Posterior-Error Covariance Lemmas

• Garbling cannot improve posterior mean-square error, and posterior-error covariance gives an information lower bound.The latter bound follows from conditional entropy and the Gaussian maximum-entropy bound.Source: §B.1 · p. 15

B.2 Linear-Gaussian Garbling Kernels

• When Γ_Z ⪯ Γ_A, an explicit linear-Gaussian kernel generates a copy A′ from Z, with M−Z−A′ and (M, Z, A′) jointly Gaussian.The construction also handles singular information matrices by reducing to a nonsingular instance.Source: §B.2 · pp. 15–17

C.1 Proof of Proposition 2

• Under Assumption 1, Proposition 2 establishes a lower bound on BROJA union information.The proof applies the bound for an arbitrary feasible source-pair law and takes the infimum over those laws.Source: §C.1 · p. 18

C.2 Closed-form redundancy and synergy for a jointly Gaussian triple

• For a nondegenerate jointly Gaussian triple under Assumption 1, BROJA redundancy and synergy have closed forms; with Theorem 1, all four PID components are closed form.Together, the unique-information formulas and these expressions give all four BROJA PID components in closed form.Source: §C.2 · p. 18

D NUMERICAL EXPERIMENTS AND COMPUTATIONAL COST

• Theorem 1 requires conditional covariances and one symmetric-definite generalized eigenproblem, with no iterations, initialization, or stopping criteria.Gradient-based methods repeatedly perform comparable linear-algebra operations during iterative optimization.Source: D NUMERICAL EXPERIMENTS AND… · p. 20

D.1 Runtime comparisons

• Across target- and source-dimension sweeps, Eigen-PID remains substantially faster than Thin-PID, with the runtime gap generally increasing at larger dimensions.Figure S1 reports median runtimes and interquartile ranges on a logarithmic scale.Source: Figure S1 · p. 21

D.2 Gain sweep

• At α = 1, the source carrying unique information about M1 switches from Y to X; along M2, Y remains more informative than X.The sweep moves from a regime where Y dominates both target directions to one where each source is more informative in a different direction.Source: §D.2 · p. 22

D.3 Initializing numerical estimators at the Eigen-PID optimum

• Initializing Thin-PID and Tilde-PID at Eigen-PID’s optimal Gaussian coupling produced no systematic reduction in final union information beyond numerical precision.Final union-information values were evaluated without regularization.Source: §D.3 · p. 23
• The Gaussian BROJA optimization leaves only Σ_XY free, and the Eigen-PID optimizer is used to construct an attaining coupling and optimal initialization for the whitened source-noise cross-covariance.The coupling replaces the (X,Y) and (Y,X) covariance blocks; † denotes the Moore–Penrose pseudoinverse.Source: §D.3 · p. 23
• Thin-PID and Tilde-PID are initialized at the Eigen-PID optimum, with their optimization procedures unchanged, to test whether either can reduce union information below the Eigen-PID value.For Tilde-PID, regularization was used during optimization but omitted when evaluating final union information; the dimensional sweep varies d_X up to 64 with d_M = d_Y = 10.Source: §D.3 · p. 23, Figure S2 · p. 23

D.4.1 Eigen-PID

• Eigen-PID provides a Python package and reproducible experiment code, using PyTorch for CPU and GPU computations.Source: §D.4.1 · p. 24

D.4.2 Baseline Implementations and Licenses

• The baselines were modified to support Eigen-PID-optimum initialization; Tilde-PID’s final evaluation omitted regularization, its upstream package declares an MIT license, and Thin-PID’s repository specified no license at access.Source: §D.4.2 · p. 24

E EIGEN-PID APPLIED TO VISUAL FMRI RESPONSES

• Eight participants’ Natural Scenes Dataset responses supplied bilateral V1 sources and left or right hV4 targets, with joint dimensions of 2,813–3,704.Responses were standardized within scan session and averaged across image repetitions; each participant had 8,779–9,841 distinct images.Source: E EIGEN-PID APPLIED TO VISUAL FMRI… · pp. 24–25
• The analysis applied Eigen-PID to fitted joint Gaussians, analytically corrected mutual-information estimates, estimated union-information bias by permutation and parametric bootstrap, then reconstructed the four components.It used 20 resamples of each type and retained negative component estimates.Source: E EIGEN-PID APPLIED TO VISUAL FMRI… · p. 25
• Redundancy was the largest mean debiased component for both targets (51.4 and 52.0 bits); ipsilateral V1 contributed more unique information in 7/8 left-target and 8/8 right-target participants.The left-target result was 7/8 participants; for the right target, it was 8/8.Source: E EIGEN-PID APPLIED TO VISUAL FMRI… · p. 25
• Permutation-only union correction, with the same analytical mutual-information correction, brought all shuffled-control mean components within 0.11 bits of zero.Under the original correction, residual component biases arose from parametric-bootstrap undercorrection of union information.Source: E EIGEN-PID APPLIED TO VISUAL FMRI… · p. 25
• Uncorrected real-data joint information averaged 242.2 bits for left hV4 and 240.7 bits for right hV4, versus corrected means of 115.1 and 116.3 bits; shuffled-control synergy was largest.The uncorrected shuffled components were all positive, with finite-sample bias particularly pronounced for synergy.Source: Figure S5 · p. 27, E EIGEN-PID APPLIED TO VISUAL FMRI… · p. 27
• The authors frame the analysis as demonstrating computational feasibility, not neuroscientific conclusions; interpretation would require dimensionality control, Gaussianity assessment, and bias-correction validation.Because the analysis is observational, assigning source and target roles does not establish causal direction or relationships.Source: E EIGEN-PID APPLIED TO VISUAL FMRI… · p. 27
• 14.5 minutes for both targets across eight participants, versus a projected approximately 21 days for Thin-PID; moving Eigen-PID to an RTX 5090 GPU reduced runtime to 3.9 minutes.The Thin-PID time was projected from timed evaluations; Eigen-PID made resampling-based bias correction practical for high-dimensional data.Source: E EIGEN-PID APPLIED TO VISUAL FMRI… · pp. 24, 28