PaperlayerView exact revision on arXiv

Source-linked AI summary

Differentiating information transfer and causal effect

Joseph T. Lizier, Mikhail Prokopenko

arXiv:0812.4373v1nlin.AOnlin.CG

TL;DR

The paper addresses confusion between information transfer and causal effect by comparing transfer entropy and information flow in local cellular automata. It finds that information flow is suited to causal structure, while transfer entropy is suited to emergent computation and can sometimes approximate causal effect.

  • Problem

    The paper addresses the unresolved distinction between predictive information transfer and causal effect and why the distinction matters for measurement.

  • Method

    It compares transfer entropy and information flow on local spatial and temporal scales in cellular automata.

  • Results

    Information flow quantifies causal effect interventionistically, while transfer entropy quantifies information transfer through conditional correlation on a causal channel.

  • Takeaways & Limitations

    Information flow should establish causal contributors first, after which transfer entropy can characterize information transfer and emergent computation.

  • Takeaways & Limitations

    Transfer entropy is an alternate causal-inference technique when intervention, underlying-dynamics knowledge, and methods such as back-door adjustment are unavailable.

Abstract

from arXiv · show

The concepts of information transfer and causal effect have received much recent attention, yet often the two are not appropriately distinguished and certain measures have been suggested to be suitable for both. We discuss two existing measures, transfer entropy and information flow, which can be used separately to quantify information transfer and causal information flow respectively. We apply these measures to cellular automata on a local scale in space and time, in order to explicitly contrast them and emphasize the differences between information transfer and causality. We also describe the manner in which the measures are complementary, including the circumstances under which the transfer entropy is the best available choice to infer a causal effect. We show that causal information flow is a primary tool to describe the causal structure of a system, while information transfer can then be used to describe the emergent computation in the system.

I. INTRODUCTION

The paper separates predictive information transfer from causal effect, arguing that transfer entropy and information flow quantify these distinct concepts. Using local cellular-automaton analyses, it assigns causal structure to information flow and emergent computation to transfer entropy.

  • Predictive transfer measures source information that improves prediction of a destination’s next state, whereas causal effect measures how changing the source alters that state.
  • Information transfer literature often conflates predictive transfer with causal effect, despite the distinction between correlation and causation.
  • The paper examines transfer entropy and information flow locally in elementary cellular automata to contrast predictive information transfer with causality.
  • Transfer entropy is linked to emergent coherent information-transfer structures such as cellular-automaton particles, while information flow identifies causal relations and influence bounds.
  • The proposed workflow establishes causal information contributors with information flow before using transfer entropy to study information transfer and emergent computation.

II. PREDICTIVE INFORMATION TRANSFER

Transfer entropy quantifies directional predictive information from a source’s past to a destination’s next state. Conditioning on destination history makes it dynamic and directional, but it remains a measure of conditional correlation rather than direct effect.

  • Transfer entropy measures the deviation from independence of a destination’s state transition given the previous state of a source.
  • It can be viewed as conditional mutual information: source information about the destination’s next state not already contained in the destination’s past k states.
  • Taking k →∞ prevents destination-history information from being mistaken for transfer, although practical estimates use finite k.
  • Transfer entropy is directional and dynamic because it conditions on the destination’s past, yet it measures observed conditional correlation rather than direct causal effect.
  • As a nonlinear extension of Granger causality, transfer entropy’s terminology may contribute to confusion between information transfer and causal effect.

B. Local transfer entropy

Local transfer entropy assigns predictive information transfer to specific spatiotemporal destinations and source channels. Apparent and complete variants separate unconditioned source contributions from contributions conditioned on other causal sources.

  • Local transfer entropy is defined at each spatiotemporal destination and for every information channel or direction in a lattice system.
  • For an ECA, local transfer entropy from a neighboring cell quantifies information contributed to the destination’s next state across that channel.
  • Complete transfer entropy conditions on other causal information contributors to prevent their influence being attributed to the source under study.
  • Apparent transfer entropy uses no conditioning on other information contributors, whereas complete transfer entropy is nonnegative in deterministic cellular automata.
  • Local destination information can be decomposed into active information storage and incrementally conditioned contributions from causal sources.
  • In ECAs, the destination’s information decomposition includes active storage and transfer contributions from the left and right neighboring sources.

III. CAUSAL EFFECT

Causal-effect measurement requires perturbing or intervening on the source and observing the destination’s response. Without intervention, directional observations still measure correlations rather than causality.

  • Measuring causal effect requires perturbation or intervention of the source to detect its effect on the destination.
  • Inferring causality without intervention leaves one measuring correlations of observations, regardless of their directionality.

A. Information flow

Information flow measures causal dependence through interventions, with its value determined by which other nodes are imposed; direct flow requires conditioning on other direct sources and blocking back-door paths.

  • A. Information flow: Information flow measures the deviation of a destination from causal independence on a source while imposing another node set S.Its definition uses interventional conditional probabilities, where imposing sets variables by intervention.
  • A. Information flow: Direct causal information flow requires S to include all other direct causal sources of the destination, including relevant destination past states.For one-cell-right flow in ECAs, S includes the destination’s immediate past and the other information-contributing cell.
  • A. Information flow: Interventional probabilities may be unavailable because interventions cannot be observed, while detailed knowledge of causal links or interaction rules is also often unavailable.When such knowledge exists, direct inferences may be possible in special cases, including complete determination or unaffected variables.
  • A. Information flow: The back-door adjustment can identify interventional probabilities from observational probabilities when its node set satisfies the back-door criteria.Those criteria include excluding descendants of the source and blocking paths containing directed links into it.
  • A. Information flow: In ECAs, back-door adjustment requires all relevant combinations to be observed and can be applied to isolated information flow without the underlying causal rules.The joint observational probabilities must be strictly positive for the relevant variable combinations.

B. Local information flow

Local information flow attributes causal effect to individual observations, but its global value is averaged over a modified interventional distribution rather than observed tuples.

  • B. Local information flow: Local information flow attributes the causal effect of source a on destination b when the imposed variables s take their observed values.It localizes causal attribution similarly to local transfer entropy, but the averaging procedure differs.
  • B. Local information flow: The global information-flow measure is not the average of local information-flow values over observations.It is averaged over a modified interventional distribution formed from interventional conditional probabilities.
  • B. Local information flow: Unobserved tuples of source, destination, and imposed variables can contribute to information flow, so averaging only observed tuples can omit relevant contributions.This distinguishes information flow from transfer entropy’s observation-based localization.
  • B. Local information flow: For cellular automata, f(i, j, n + 1) denotes local information flow into cell X_i from source cell X_i−j at the next time step.The notation represents flow across j cells to the right when j is positive.

IV. APPLICATION TO CELLULAR AUTOMATA

The paper applies local transfer entropy and information flow to ECA rule 54, whose periodic background and interacting gliders provide an emergent-computation setting for comparing the measures.

  • IV. APPLICATION TO CELLULAR AUTOMATA: ECA rule 54 contains a periodic background domain, traveling gliders, and glider collisions that form the basis of emergent intrinsic computation.The analysis focuses on transfer and flow one cell to the right per time step.
  • IV. APPLICATION TO CELLULAR AUTOMATA: For one-step-right transfer in rule 54, apparent transfer entropy averages 0.080 bits, complete transfer entropy averages 0.193 bits, and information flow averages 0.523 bits.These are averages for k = 16 and j = 1.
  • IV. APPLICATION TO CELLULAR AUTOMATA: The authors examine local values within the rule-54 results to expose differences between information transfer and causal effect.The local cases complement the reported average measures by showing how each behaves in specific regions.

A. Background domains are highly causal

In rule 54’s periodic background domain, transfer entropy vanishes while information flow remains patterned and strong, because the measures capture different perspectives on the same dynamics.

  • A. Background domains are highly causal: Local apparent and complete transfer entropies measure vanishing information transfer in the periodic background domain.The background cells’ futures are nearly predictable from their pasts, leaving little additional predictive information for transfer entropy.
  • A. Background domains are highly causal: Local information flow measures a periodic pattern of causal effect in the same background region at levels similar to those in glider and blinker regions.The result reflects causal dependence between neighboring cells despite vanishing transfer entropy.
  • A. Background domains are highly causal: The two results are both correct because transfer entropy captures computational predictability, whereas information flow captures changes under imposed source values.Long background-domain periods depend on information stored in neighbors and retrieved after several time steps.

B. Gliders distinguished as emergent information transfer

In rule 54, gliders exhibit strong predictive information transfer in their direction of motion, while local information flow detects causal effects no greater than in the background domain. These measures are both correct because they capture different perspectives: emergent computation versus direct local causation.

  • B. Gliders distinguished as emergent information transfer: Gliders show strong local transfer entropy in the direction of their motion, unlike the surrounding background domain.The effect appears for both local transfer entropy and local complete transfer entropy with history length k = 16.
  • B. Gliders distinguished as emergent information transfer: Local information flow measures causal effects in gliders at levels similar to those in the background domain.Thus, its localization does not distinguish gliders from ordinary domain dynamics.
  • B. Gliders distinguished as emergent information transfer: Predictive transfer aligns more closely with information transfer because glider states strongly predict downstream states along their motion.The same CA rules operate inside gliders and elsewhere, so imposing source values does not produce greater localized causal flow in gliders.
  • B. Gliders distinguished as emergent information transfer: Information flow emphasizes micro-level direct causal effects, whereas transfer entropy uses past context to reveal macroscopic emergent structures.Information flow cannot incorporate past-state context because intervention blocks the influence of those earlier states.

C. Information transfer to be measured from causal sources only

Apparent transfer entropy can report correlations outside a destination’s past light-cone, so transfer entropy should be applied only to causal information sources. Complete transfer entropy can approximate information flow under suitable conditioning, but requires correct causal-neighborhood knowledge and adequate observations.

  • C. Information transfer to be measured from causal sources only: Two-cell apparent transfer entropy can show a profile despite superluminal separation outside the destination’s past light-cone.The profile reflects correlation between the purported source and an actual causal source one cell away, not real information transfer.
  • C. Information transfer to be measured from causal sources only: Figure 2 compares local information flow, complete transfer entropy, and apparent transfer entropy across one or two cells in rule 54.The panels report maxima of 1.07, 1.17, 9.22, 7.93, and 6.00 bits for the displayed measures and settings.
  • C. Information transfer to be measured from causal sources only: In neighborhood-5, information flow cannot be measured from observational data alone because the dynamics do not produce all required variable combinations.The calculation therefore requires specific knowledge of the system dynamics.
  • C. Information transfer to be measured from causal sources only: Complete transfer entropy avoids the spurious two-cell profile in the correct neighborhood, yielding zero transfer because interior causal sources contain all predictive information.This agrees with the zero information-flow result, while complete transfer entropy alone can infer it from observational data in the described setting.
  • D. Complete transfer entropy as a next best inference for information flow: With k = 1, complete transfer entropy closely approximates local information flow by conditioning out other causal contributors.The reported averages are T c(j = 1, k = 1) = 0.521 bits and Ip(j = 1) = 0.523 bits; this is proposed when intervention is unavailable.
  • D. Complete transfer entropy as a next best inference for information flow: Complete transfer entropy is neither a direct nor exact causal-effect measure and requires conditioning on at least the correct neighborhood of causal sources.The paper leaves incremental source-selection testing for future work.
  • D. Complete transfer entropy as a next best inference for information flow: If crucial variable combinations are unobserved, complete transfer entropy can produce quite incorrect causal inferences.The paper illustrates this boundary with a short circuit whose causal effect on fire is unidentifiable when the relevant conditions are never observed.

V. DISCUSSION AND CONCLUSION

The paper distinguishes causal effect from information transfer and assigns information flow and transfer entropy complementary roles. Information flow establishes causal relationships where possible, while transfer entropy characterizes emergent computation or serves as an alternate causal-inference technique under constrained conditions.

  • Information flow quantifies causal effect interventionistically, whereas transfer entropy measures information transfer through conditional correlation on a causal channel.
  • Causal effect is a fundamental micro-level property, so information flow should establish and quantify causal relationships where intervention or suitable observational adjustment is possible.
  • When intervention and suitable causal knowledge are unavailable, complete transfer entropy with minimal history length k is an alternate technique for inferring causal effect.
  • Apparent transfer entropy cannot distinguish correlation from causal effect, while large-k apparent or complete transfer entropy measures predictive rather than direct causal information.
  • Both information flow and complete transfer entropy require conditioning on the correct set of other causal variables, whose identification remains future work.
  • For emergent computation, apparent and complete transfer entropy with large history length k assess information storage, transfer, and interactions among causal contributors.
Loading 0812.4373v1…