PaperlayerView exact revision on arXiv

Source-linked AI summary

Partial Information Decomposition When Some Sources Are Known: A Source Chain Rule and a No-Go Theorem

Aaron J. Gutknecht

arXiv:2610.33743v1cs.ITmath.IT

TL;DR

The paper asks how PID should decompose information beyond sources already known, contrasting an atom-based contextual decomposition with recomputation under conditional distributions. It characterizes their agreement by a source chain rule for redundancy and proves that, with three or more sources, this rule cannot coexist with local positivity.

  • Problem

    When some sources are already known, it is unclear how to decompose the information the remaining sources provide beyond that context.

    Source: §1 · p. 2

  • Method

    The paper defines contextual and conditional decompositions of information beyond a context and characterizes their atom-by-atom agreement through a source chain rule for redundancy.

    Source: §1 · p. 2, §3 · p. 5

  • Results

    For every n≥3, the source chain rule is incompatible with local positivity, whereas every PID satisfies the rule for n≤2.

    Source: §1 · p. 3

  • Takeaways & Limitations

    The source chain rule divides established redundancy measures, so adopting or rejecting it narrows the choice of measure.

    Source: §1 · p. 2

    A PID that keeps local positivity has contextual and conditional decompositions that cannot agree for all distributions.

    Source: §7 · p. 20

Abstract

from arXiv · show

Partial information decomposition (PID) splits the information that several sources carry about a target into atoms, distinguished by which collections of sources give access to them. We ask how such a decomposition should behave when some of the sources are already known. The access structure of the atoms suggests one answer: discard the atoms that the known sources provide and regroup the rest according to which further sources unlock them. The classical construction of conditional information suggests another: recompute the decomposition under the conditional distributions and average. We show that the two answers agree, atom by atom, exactly when redundant information obeys a source chain rule, an analogue of the chain rule of mutual information: when the same sources are added to several collections, the enlarged collections share everything the added sources provide and, beyond that, exactly what the original collections share once the added sources are known. The rule divides established measures of redundant information, so that adopting or rejecting it genuinely narrows the choice of measure. Our main result is a no-go theorem: for three or more sources, the source chain rule and local positivity, the requirement that all atoms be nonnegative, cannot hold together in any PID.

1 Introduction

The paper asks how to decompose information when some sources are already known, comparing a decomposition read from the unconditional PID with one recomputed under conditional distributions. It shows these decompositions agree exactly when redundancy obeys a source chain rule, but that rule is incompatible with local positivity for three or more sources.

1 Introduction

  • PID atoms are underdetermined by consistency equations: three sources have eighteen atoms but only seven equations, and proposed measures differ in their further principles.The paper frames incompatibility results as identifying choices that a measure cannot avoid.Source: §1 · p. 2
  • The contextual decomposition discards atoms provided by known sources and regroups the rest by which further sources unlock them; the conditional decomposition averages PIDs recomputed under conditional distributions.Both decompose the same conditional mutual information.Source: §1 · p. 2
  • The two decompositions agree atom by atom for every context if and only if redundant information obeys a source chain rule.The rule says collections containing the context share its information and, beyond that, exactly what their remaining parts share once the context is known.Source: §1 · p. 2
  • The source chain rule holds for shared-exclusion, specificity–ambiguity, and common-information redundancy, but the original Williams and Beer measure violates it.The source chain rule is distinct from the target chain rule; they are logically independent, although under three elementary properties the target rule implies the source rule.Source: §1 · p. 2
  • For every n≥3, the source chain rule is incompatible with local positivity, while every PID satisfies the rule when n≤2.The counterexample uses a context that carries no target information but determines which of two other sources carries it.Source: §1 · p. 3

2 Background

The background represents PID atoms by the source collections from which they are accessible, and assigns atom values so that they sum to mutual information. Redundancy measures specify the decomposition, with Möbius inversion recovering atoms from redundancy values.

2.1 Partial information decomposition

  • A parthood distribution records which source collections provide each atom, and accessibility persists when a collection is enlarged.A PID assigns atom values that add up to the mutual information terms.Source: §2.1 · pp. 3–4
  • Parthood distributions correspond one-to-one with antichains of minimal source collections, and for two sources the four atoms are redundant, unique, unique, and synergistic information.For example, 1·23 denotes an atom accessible from source 1 alone and from sources 2 and 3 jointly.Source: §2.1 · p. 4

2.2 Redundant information

  • Consistency equations do not determine the atoms, so a PID is specified through an inducing measure, most commonly a measure of redundant information.The paper assumes the PID or inducing measure is defined for any number of sources so it can apply to a subfamily.Source: §2.2 · p. 4, §2.1 · p. 4
  • Redundancy depends only on its antichain of minimal arguments, and the redundancy values uniquely determine the atoms through Möbius inversion.The inversion recursively obtains each atom value by subtracting values for larger parthood distributions from its corresponding sum.Source: §2.2 · p. 5

3 Conditioning a PID on some of its sources

When some sources are known, the paper gives two ways to decompose information beyond that context: regroup the original PID atoms by the access patterns of the remaining sources, or recompute the PID under conditional distributions and average. The paper characterizes exactly when these decompositions agree: when redundant information satisfies the source chain rule.

3.1 The contextual decomposition

  • The contextual decomposition groups full-PID atoms that the context does not provide according to which remaining sources unlock them.For three sources with S3 as context, the five atoms accessible from S3 make up I(S3;T), while the other thirteen map to four atoms for S1 and S2.Source: §3.1 · pp. 6–7, Table 1 · p. 7
  • The contextual atoms sum to the conditional mutual information, so they decompose the information beyond the context.The proposition establishes this by relating atoms accessible after adding a collection to the difference between the information in that collection with the context and the information in the context alone.Source: §3.1 · p. 6

3.2 The conditional decomposition

  • The conditional decomposition computes a PID of the remaining sources under each conditional distribution given the context, then averages over the context.For three sources and context S3, this means computing the PID of S1 and S2 given each value of S3; the result decomposes the same conditional mutual information as the contextual route.Source: §3.2 · p. 7, B Conditional decomposition · p. 8, Figure 1 · p. 8

3.3 The source chain rule

  • The source chain rule states that the redundancy of collections enlarged by the context equals the context’s information plus their conditional redundancy.It applies for every context, all nonempty collections of remaining sources, and every joint distribution of sources and target.Source: §3.3 · p. 10
  • The two decompositions agree atom by atom exactly when redundant information satisfies the source chain rule for the context and remaining sources.If the rule fails, conditional recomputation can redistribute information among redundancy, unique information, and synergy in ways the original atoms’ access structure does not predict.Source: §3.3 · p. 11
  • For at most two sources, every PID satisfies the source chain rule; its first nontrivial cases require a nonempty context and at least two remaining sources.With an empty context the rule holds trivially, and with two sources it reduces to the mutual-information chain rule.Source: §3.3 · pp. 11–12

4 Incompatibility with local positivity

The section constructs a three-source distribution where the context selects which source carries the target, then shows that local positivity conflicts with the source chain rule. This incompatibility holds for every n≥3, while for n≤2 every PID satisfies the rule.

4 Incompatibility with local positivity

  • Local positivity is the requirement that every PID atom be nonnegative.It is a widespread requirement beyond the consistency equations.Source: §4 · p. 12

4.1 A counterexample

  • With an independent uniform context and target, S1=max(C,T), S2=min(C,T), and S3=C make one of S1 and S2 a target copy and the other constant in each context.The four outcomes are equiprobable.Source: §4.1 · p. 12, Figure 2 · p. 13

B Conditional on the context

  • Given the context, the constant source has no redundancy with the other, but without it local positivity forces at least 0.123 bits of shared information between S1 and S2.The conditional redundancy is empty, whereas the co-information lower-bounds their unconditional redundancy.Source: B Conditional on the context · p. 13

4.2 The theorem

  • For every n≥3, no PID can satisfy both local positivity and the source chain rule.The bound is sharp: for n≤2, every PID satisfies the source chain rule, including Imin.Source: §4.2 · pp. 13–14
  • The proof derives a contradiction because local positivity requires redundancy to be at least the co-information, while the source chain rule requires the contextual redundancy to vanish.The co-information is positive because the atoms accessible only from both sources are nonnegative.Source: §4.2 · p. 14

5 Measures that satisfy the source chain rule

The paper identifies established redundancies that satisfy the source chain rule, despite its incompatibility with local positivity for three or more sources. Pointwise measures satisfy a stronger realization-level identity, and common-information redundancy satisfies the rule through a maximal-common-part construction; the counterexample exposes different trade-offs among the measures.

5 Measures that satisfy the source chain rule

  • The source chain rule can be satisfied by established measures of distinct origins, but the theorem means that for three or more sources this comes at the price of negative atoms.The paper considers pointwise measures and common-information redundancy before comparing their behavior on the counterexample.Source: §5 · p. 14

5.1 Pointwise measures

  • Isx∩ and Rmin satisfy the source chain rule in a stronger pointwise form: the identity holds for every positive-probability realization, not just on average.Averaging over realizations yields the source chain rule.Source: §5.1 · p. 15

5.2 Common-information redundancy

  • For common-information redundancy, the maximal common part of the enlarged collections is the context paired with the conditional maximal common part; the mutual-information chain rule then gives the source chain rule.The conditional common part is taken separately for each context value.Source: §5.2 · p. 16

5.3 The measures on the counterexample

  • On the counterexample, Imin preserves local positivity but violates the source chain rule, while Isx∩ and I∧∩ satisfy the rule by violating different positivity requirements.Isx∩ decreases when S3 is added to both sources; I∧∩ is below the 0.123-bit co-information and has −0.123 bits of synergy.Source: §5.3 · pp. 16–17, Figure 3 · p. 17

6 Relation to the target chain rule

The source and target chain rules are logically independent, though under three elementary properties the target chain rule implies the source chain rule. Their relationship is illustrated by established redundancy measures and by chained PIDs built from locally positive base PIDs.

6 Relation to the target chain rule

  • The source and target chain rules are logically independent: common-information redundancy satisfies only the former, while locally positive chained PIDs with at least three sources satisfy only the latter.Under three elementary properties, however, the target chain rule implies the source chain rule.Source: §6 · p. 18
  • For independent fair-bit sources with T1 = S1 ⊕ S2 and T2 = S1, common-information redundancy is zero for T1 but one bit for T2 given T1.This example exhibits failure of the target chain rule for common-information redundancy.Source: §6 · p. 18
  • Any chained PID built from a locally positive base PID satisfies the target chain rule, but violates the source chain rule for n≥3.The source chain rule follows for specificity–ambiguity and shared-exclusion redundancy from the target chain rule plus the three properties.Source: §6 · pp. 18–19
  • Under three elementary properties, a redundancy measure satisfying the target chain rule also satisfies the source chain rule.The proof expands redundancy about a composite target in both orders, using invariance to constant sources, target-component swaps, and full redundancy when collections determine the target.Source: §6 · p. 18
  • The source chain rule is strictly weaker than the target chain rule under those properties: shared-exclusion redundancy satisfies the source rule but not the target rule.Specificity–ambiguity redundancy satisfies both.Source: §6 · p. 19

7 Discussion

The discussion weighs agreement between access-based and conditional decompositions against nonnegative atoms. The source chain rule requires negative atoms for three or more sources, while keeping local positivity leaves two contextual and conditional decompositions that can disagree.

Keeping the source chain rule

  • Keeping the source chain rule entails negative atoms that need interpretation; pointwise measures can read negativity as misinformation, while common-information redundancy instead denies shared information and pays with negative synergy.The authors say they are not aware of an interpretation for that negative synergy.Source: §7 · pp. 19–20
  • Shared-exclusion redundancy and specificity–ambiguity redundancy place the shared information in an atom accessible from the context, compensated by negative atoms when the context carries no target information.For specificity–ambiguity redundancy, the compensating atoms are approximately −0.35 bits each.Source: §7 · p. 19

Keeping local positivity

  • Keeping local positivity means contextual and conditional decompositions cannot agree for all distributions, and their derived redundancies and union informations can differ.For Imin in the counterexample, contextual redundancy is 1/2 bit while conditional redundancy is zero.Source: §7 · p. 20

Chain rules and positivity

  • For three or more sources, the source chain rule and local positivity cannot both hold, so a PID must give up either nonnegative atoms or chain-rule behavior.The target chain rule also conflicts with local positivity when the PID is order-independent in the two components of a composite target [21].Source: §7 · pp. 19–20

8 Conclusion

The access structure of PID atoms predicts how a decomposition should change when some sources are known, and the source chain rule requires that prediction to match conditioning. For at least three sources, that requirement is incompatible with local positivity.

8 Conclusion

  • The source chain rule holds for shared-exclusion, specificity–ambiguity, and common-information redundancies, but conflicts with local positivity whenever there are at least three sources.A PID can have nonnegative atoms or atoms whose access structure correctly predicts conditioning, but not both.Source: §8 · p. 21

Use of AI tools

The paper reports that large language model assistants contributed to argument exploration, counterexamples, numerical checks, drafting, editing, and figure preparation.

Use of AI tools

  • Large language model assistants were used for argument exploration, counterexamples, numerical checks, drafting, editing, and figure preparation.The author states that all results, proofs, and numerical values were checked and takes full responsibility for the content.Source: Use of AI tools · p. 21
Loading 2610.33743v1…