PaperlayerView exact revision on arXiv

Source-linked AI summary

Complementary Feature Domains: Information Preservation Does Not Imply Predictive-Contribution Preservation

Timothy Oladunni, Farouk Ganiyu-Adewumi

arXiv:2610.07565v1cs.LGcs.AI

TL;DR

A bijective representation recoding can preserve target information yet alter predictive contributions when the realization family is restricted. CFD quantifies this transition through a contribution defect and a compatibility bound. In a controlled ECG experiment, applying the exact inverse restored accuracies in all tested contexts.

  • Problem

    In fusion, it is not enough to know whether representations contain target information; the open question is how their contributions change across coalitions under a particular realization family.

    Source: §I · p. 1

  • Method

    CFD defines coalition-indexed contributions and a defect for their change under recoding, with the defect bounded by compatibility errors before and after adding a representation.

    Source: §I · pp. 1–2

  • Results

    A nonzero contribution defect can coexist with preserved Shannon information, while exact behavioral closure yields invariance and the ECG inverse restored all tested coalition accuracies.

    Source: §I · p. 2, §IX · p. 5, §VIII · p. 5

  • Takeaways & Limitations

    A nonzero defect need not indicate information loss; it may reveal representation–learner mismatch.

    Source: §IX · p. 5

Abstract

from arXiv · show

Complementary Feature Domains (CFD) theory characterizes predictive value as a context-indexed contribution system induced jointly by representations and their realization family. We show that Shannon-information preservation does not imply preservation of this contribution system: an invertible representation transformation can leave target information unchanged while altering predictive contribution under a restricted decision family. We formalize the resulting transition through a CFD contribution defect that measures how contextual contributions change under controlled recoding. For bounded Lipschitz utility, we show that each coalition utility shift is bounded by the behavioral distance between the attainable action sets before and after recoding; consequently, every contextual contribution defect is bounded by the sum of the corresponding coalition incompatibilities. Exact behavioral closure yields invariance, while increasingly accurate compensation yields restoration. A controlled ECG experiment illustrates the mechanism: a nonlinear bijective recoding preserves the information in a frozen time-frequency representation but changes accuracy under a fixed affine learner; applying the exact inverse restores all tested coalition accuracies. The result separates information preservation from realization-dependent contribution and provides a quantitative transition law for multi-representation prediction.

I. INTRODUCTION

An invertible recoding can preserve all target information while changing prediction when a restricted learner cannot realize the compensating map. CFD formalizes how such changes reshape a representation’s contributions across coalitions.

I. INTRODUCTION

  • A bijection preserves Shannon information, but a restricted decision family may attain different utility if it cannot implement the inverse recoding.In fusion, this can alter a representation’s marginal contribution across contexts even when every corresponding subset preserves the same information.Source: §I · p. 1
  • Existing frameworks address information content, decomposition, usability, or decision informativeness, but the paper identifies a gap in characterizing recoding’s effect on the full coalition-indexed contribution system.The authors state that, to their knowledge, prior frameworks do not directly characterize this transition under an explicit realization family.Source: §I · p. 1
  • CFD studies how contextual predictive contributions change under representation–realization interventions, defining contributions across coalitions for a representation system and realization family.The framework treats the result as a complete contribution system rather than a single importance score.Source: §I · pp. 1–2
  • Each contextual contribution defect is bounded by the Lipschitz constant times the compatibility errors for the coalition with and without the added representation.Exact behavioral closure gives invariance, while vanishing compatibility error restores the original contribution system.Source: §I · p. 2

II. CONTRIBUTION SYSTEMS

CFD defines coalition value through the best utility attainable by a declared realization family, then records each representation’s marginal utility in every context. The resulting system can differ even when full-coalition accuracy agrees.

II. CONTRIBUTION SYSTEMS

  • Coalition value is the best population-level utility attainable from that coalition under its declared family of decision rules, not the information it contains.It depends jointly on the representation and what the learner is permitted to realize.Source: §II · p. 2
  • A representation’s contribution is the utility increment from adding it to a baseline coalition, so its value can vary with context.A positive increment means it unlocks utility beyond what was attainable from the baseline coalition; zero means it adds none in that context.Source: §II · p. 2
  • The complete contribution system stacks these local increments across the coalition lattice, and two systems can have equal full-coalition accuracy but distribute predictive value differently.The system therefore preserves contextual distinctions that a single full-coalition score does not show.Source: §II · p. 2
  • When coalition value is Shannon target information, the increment for adding a representation is the target information in it not already supplied by the baseline coalition.Under unrestricted decision rules, or when the post-recoding family implements the corresponding inverse compositions, attainable decisions and the contribution system coincide.Source: §II · p. 2

III. SEPARATION UNDER RESTRICTED REALIZATION

The paper constructs finite examples where bijective recoding preserves target information but changes utility under a fixed restricted realization family. In a multi-representation construction, the change appears in contextual contributions as well.

III. SEPARATION UNDER RESTRICTED REALIZATION

  • The finite witness separates Shannon information from optimized utility under a fixed restricted realization family.The paper notes that predictive V-information addresses access to predictive information under restricted families, while CFD examines how accessibility changes propagate across representation contexts.Source: §III · p. 2
  • Both encodings contain 1 bit of target information, yet affine accuracy is 3/4 on the original XOR support and 1 on the bijectively recoded support.The original XOR labels are not linearly separable, while the recoded classes are separated by the sign of the first coordinate.Source: §III · p. 2
  • In the multi-representation construction, every subset retains equal Shannon information after recoding, but the fixed restricted realization family yields different utility for coalitions containing the recoded representation.The construction appends two independent fair nuisance bits to the XOR representation and its recoding.Source: §III · p. 2
  • Coalitions containing the recoded third representation differ in utility, while those excluding it are unchanged, so at least one contextual marginal contribution differs.The example shows that complete subset-information equivalence is insufficient for contribution-system equivalence; the missing condition is compatibility with the realization family.Source: §III · pp. 2–3

IV. COMPATIBILITY DEFECT AND STABILITY

The section defines behavioral incompatibility between attainable-action sets and uses it to track how recoding changes coalition utility and contextual contribution. Under Lipschitz utility, behavioral distance controls utility shifts and the resulting contribution defect.

IV. COMPATIBILITY DEFECT AND STABILITY

  • Behavioral incompatibility compares the attainable-action sets before and after recoding using a symmetric L1(P) Hausdorff distance.Zero distance means the sets have the same L1(P) closure; if they are closed, they coincide.Source: §IV · p. 3
  • The CFD contribution defect tracks whether an intervention changes a representation’s contextual marginal role, using pre- and post-intervention coalition utilities.A zero defect means the contextual role is unchanged for that comparison; a nonzero defect does not itself imply information loss.Source: §IV · p. 3
  • For L-Lipschitz utility, coalition utility shifts are bounded by the behavioral distance between the attainable-action sets.The two directed comparisons ask whether actions available before recoding can be approximated after it, and vice versa.Source: §IV · p. 3

ity: if the two realization families can closely reproduce one

Behavioral compatibility on both sides of a marginal comparison controls the contextual contribution defect. Exact closure yields invariance, and vanishing compatibility error restores the contribution system.

ity: if the two realization families can closely reproduce one

  • Contextual contribution is stable when the learner is behaviorally compatible both before and after adding representation i to coalition S.The defect bound uses the incompatibilities for S and S ∪{i}, and is local to those two contexts.Source: ity: if the two realization families… · p. 3
  • If behavioral errors vanish across all coalitions, the maximum contextual contribution defect converges to zero.Exact zero error for every coalition gives equality of the contribution systems.Source: ity: if the two realization families… · p. 3
  • The bound is falsifiable: if compatibility errors vanish while the population contribution defect remains bounded away from zero, the proposed mechanism cannot explain that defect.Exact inverse closure instead forces restoration.Source: ity: if the two realization families… · p. 3

V. RELATION TO EXISTING INFORMATION-THEORETIC OBJECTS

CFD studies transitions in contextual predictive contributions under representation–realization interventions, rather than redefining established information measures or attribution frameworks. With restricted realization, equal Shannon information profiles can coexist with changed decision-level contributions.

V. RELATION TO EXISTING INFORMATION-THEORETIC OBJECTS

  • CFD’s primary object is the transition of a coalition-indexed predictive contribution system under a declared representation–realization intervention.It does not redefine mutual information, PID atoms, usable information, or Blackwell ordering.Source: §V · p. 3
  • With unrestricted predictive distributions and log score, the optimal coalition value reduces to Shannon information, and its contextual marginal reduces to conditional mutual information.The table identifies analytical objects and CFD’s distinct transition question.Source: §V · pp. 3–4, Table I · p. 4
  • Under restricted realization, equal Shannon subset-information profiles need not preserve optimized coalition utilities or their contextual differences.An invertible recoding can leave PID unchanged while changing the decision-level contribution system.Source: §V · p. 4
  • Predictive V-information describes family-relative accessibility, while CFD tracks how accessibility changes alter a representation’s marginal role across contexts.Unlike Shapley values, which average marginal contributions over contexts, CFD retains the unaveraged contextual marginal vector; Blackwell comparison instead orders experiments over decision problems.Source: §V · p. 4
  • Behavioral realization equivalence implies contribution equivalence, and the stability theorem supplies a quantitative bridge as attainable sets approach one another.The theorem concerns vanishing contribution differences as behavioral sets before and after recoding become closer.Source: §V · p. 4

VI. REALIZATION-CAPACITY STRESS TEST

The capacity stress test holds target information fixed while varying realization-family expressiveness. The affine family shows the predicted utility separation, which disappears when the family can express both decision boundaries.

VI. REALIZATION-CAPACITY STRESS TEST

  • Both encodings contain exactly one bit of target information, while the tested realization families differ in capacity.Table II reports the affine population optimum and full-support fitted accuracy for nonlinear families.Source: Table II · p. 4, §VI · p. 4
  • .25 utility separation appears for the affine family, and the gap disappears once the realization family can express both decision boundaries.The capacity stress test evaluates the same one-bit XOR and bijectively recoded supports under increasingly expressive realization families.Source: §VI · p. 4, Table II · p. 4
  • For classification accuracy, the theorem applies to discrete prediction actions with the discrete metric; the experiments test the predicted invariance–failure mechanism rather than estimate its bound numerically.The reported gap vanishes once the family realizes equivalent decision behavior on both supports.Source: §VI · p. 4

VII. CONTEXT AND CAPACITY ON A COALITION LATTICE

The coalition lattice reveals that a representation’s predictive contribution depends both on its context and on the realization family. The parity example shows that affine rules capture only part of an opportunity available under Bayes realization.

VII. CONTEXT AND CAPACITY ON A COALITION LATTICE

  • The calculation compares Bayes-optimal accuracy with exact best affine-threshold accuracy across coalitions for four deterministic targets built from three independent fair bits.It uses the uniform distribution over eight states and enumerates projected dichotomies to test separability.Source: §VII · p. 4
  • For parity, D3 contributes 0 in empty or one-companion contexts but .5 after both other bits under Bayes realization; affine rules realize only .25 of that full-context opportunity.Majority, OR, and AND show no Bayes–affine discrepancy in the reported full-context marginal.Source: §VII · p. 4
  • The contribution signatures are indexed by contexts ∅, {1}, {2}, and {1, 2}, so one context-free complementarity score cannot represent their structure.CFD separates where a representation contributes across the coalition lattice from how much of that contribution a realization family can attain.Source: §VII · pp. 4–5

VIII. CONTROLLED RECODING EXPERIMENT

A bijective recoding of the ECG time–frequency representation changes coalition accuracies under a fixed learner, while applying its exact inverse restores them. The resulting defects vary with the other representations in the coalition.

VIII. CONTROLLED RECODING EXPERIMENT

  • The experiment uses frozen T, F, and TF ECG embeddings from 928 observations, split into 648 training, 143 validation, and 137 sealed test observations.It compares standardized raw embeddings, bijectively recoded embeddings, and recoded embeddings passed through the analytical inverse, using the same one-versus-rest logistic learner.Source: §VIII · p. 5
  • Applying the exact inverse restores every reported coalition accuracy, with restoration error and its bootstrap interval equal to 0 and [0, 0] in every context.The recoding intervals exclude zero for TF and F + TF.Source: §VIII · p. 5
  • The observed finite-sample defects are .1095, .0730, .1971, and .1314 for contexts ∅, {T}, {F}, and {T, F}, respectively.The intervention changes TF alone, leaving each baseline coalition S ⊆ {T, F} unchanged; thus the defect is the absolute change in the corresponding coalition containing TF.Source: §VIII · p. 5

IX. DISCUSSION AND CONCLUSION

CFD distinguishes information preservation from preservation of predictive contribution, which depends on coalition context and realization family. The ECG experiment tests this mechanism, not universal complementarity.

IX. DISCUSSION AND CONCLUSION

  • A bijection can preserve Shannon information for every coalition while changing the best utility attainable by a restricted realization family.CFD indexes contribution by coalition and realization family; its bound makes the contribution defect depend on incompatibility before and after adding a representation.Source: §IX · p. 5
  • A nonzero contribution defect need not indicate information loss; it may reveal a representation–learner mismatch.Exact closure forces invariance, while approximate closure bounds the change in contextual contribution.Source: §IX · p. 5
  • The ECG experiment tests the predicted failure–restoration pattern in a nonsynthetic multi-representation system, not universal complementarity.Finite-sample error is outside the population theorem, and broader tasks, realization families, and uncertainty guarantees remain future work.Source: §IX · p. 5
Loading 2610.07565v1…