PaperlayerView exact revision on arXiv

Source-linked AI summary

Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data

Fernando E. Rosas, Pedro A. M. Mediano, Henrik J. Jensen, Anil K. Seth, Adam B. Barrett, Robin L. Carhart-Harris, Daniel Bor

arXiv:2004.08220v1q-bio.NCnlin.AO

TL;DR

Existing accounts provide few quantitative theories of emergence, especially for relating macroscopic features to the dynamics of their parts. This paper develops an information-theoretic framework for causal emergence in multivariate systems, defining downward causation and causal decoupling and deriving practical criteria. The framework is illustrated across cellular automata, flocking, and neural data, while its applicability is bounded by assumptions including full observability and Markovian dynamics.

  • Problem

    Few quantitative theories formalize how macroscopic features relate causally to the dynamics of their underlying parts.

  • Method

    The paper develops a multivariate information-theoretic theory using dynamical synergy to identify causal emergence without requiring a predefined emergent feature.

  • Results

    The framework provides practical emergence criteria and is applied to Conway’s Game of Life, Reynolds’ flocking model, and electrocorticographic neural activity.

  • Takeaways & Limitations

    The theory brings emergence closer to quantitative empirical investigation and supports formal analysis of collective neural patterns and causally decoupled psychological phenomena.

  • Takeaways & Limitations

    The framework assumes fully observable systems with Markovian dynamics, limiting direct application when variables are unobserved or dynamics are not Markovian.

Abstract

from arXiv · show

The broad concept of emergence is instrumental in various of the most challenging open scientific questions -- yet, few quantitative theories of what constitutes emergent phenomena have been proposed. This article introduces a formal theory of causal emergence in multivariate systems, which studies the relationship between the dynamics of parts of a system and macroscopic features of interest. Our theory provides a quantitative definition of downward causation, and introduces a complementary modality of emergent behaviour -- which we refer to as causal decoupling. Moreover, the theory allows practical criteria that can be efficiently calculated in large systems, making our framework applicable in a range of scenarios of practical interest. We illustrate our findings in a number of case studies, including Conway's Game of Life, Reynolds' flocking model, and neural activity as measured by electrocorticography.

I. FUNDAMENTAL INTUITIONS

Minimal examples show two distinct forms of causal emergence: causal decoupling, where collective structure predicts its own evolution without component-level predictive power, and downward causation, where the whole influences a particular part. A combined example demonstrates that both effects can coexist independently.

  • Setup: The framework represents systems as binary vectors undergoing Markovian stochastic dynamics, with features generated conditionally from the system state.Features may also be deterministic functions or aggregate properties affected by observational noise.
  • Minimal examples: Example 1 preserves the system’s parity probabilistically while individual variables have no predictive power over future variables, exemplifying causal decoupling.The complete past also lacks predictive power over any individual future element.
  • Minimal examples: In Example 2, the system’s parity predicts one future element with perfect accuracy, although no individual part accounts for that effect, exemplifying downward causation.The remaining elements evolve as independent fair coin flips.
  • Minimal examples: Example 3 combines collective-to-collective and whole-to-part effects, with both effects co-existing independently in the same system.Parity is transferred to the next state with probability γ while one element is additionally guaranteed by the system state.
  • Interpretation: The examples show that collective properties can propagate without substrate interactions, influence specific parts, or exhibit both behaviours together.These dynamical patterns motivate the formal theory developed next.
  • Conceptual distinction: Causally emergent features have predictive power beyond individual components; downward causation targets individual elements, whereas causal decoupling targets itself or other high-order features.This distinction organizes the relationships illustrated by the minimal examples.

A. Partial information decomposition

The theory uses Partial Information Decomposition to isolate information supplied collectively beyond lower-order groups and defines causal emergence through irreducible predictive influence. Dynamical synergy then provides a feature-independent criterion for whether a system can support emergence.

  • Partial information decomposition: Partial Information Decomposition separates multivariate information into atoms describing redundancy, uniqueness, and joint contributions from source collections.The framework does not require one specific functional form of PID, provided basic properties are satisfied.
  • Partial information decomposition: Kth-order synergy captures target information supplied by the whole system but absent from every group of k or fewer parts considered separately.This coarse-graining makes the decomposition more tractable as the number of sources grows.
  • Defining causal emergence: A supervenient feature exhibits causal emergence of order k when its unique information about the future, conditional on the full system, exceeds zero.The definition identifies irreducible causal influence not mediated by groups of k parts.
  • Defining causal emergence: Causal emergence is fundamentally collective: an emergent feature requires at least two system elements and cannot be perfectly predicted from a single variable.These properties follow from the formal definition’s basic consequences.
  • Defining causal emergence: A system has a causally emergent feature of order k if and only if its components exhibit positive kth-order synergy with respect to future evolution.This criterion determines emergence from system dynamics without proposing a candidate feature.
  • Defining causal emergence: System synergy upper-bounds the unique information of every supervenient feature and therefore measures the system’s capacity for causal emergence.Absence of dynamical synergy can rule out causal emergence, while the capacity depends on the chosen microscopic partition.

C. A taxonomy of emergence

The framework uses ΦID to decompose emergence capacity into downward causation and causal decoupling, distinguishing information about future subsets from information about collective properties.

  • ΦID extends partial information decomposition to multiple targets, enabling the framework to distinguish different kinds of synergy.The paper uses this fine-grained decomposition to define indices for downward causation and causal decoupling.
  • The ΦID lattice for n = 2 time series highlights the terms associated with downward causation and causal decoupling.
  • The downward causation index D(k) and causal decoupling index G(k) decompose emergence capacity into information about future k-plets and collective properties beyond k-plets.The two components correspond respectively to future subsets and future collective properties beyond those subsets.

1. Downward causation

Downward causation occurs when an emergent collective feature has predictive power over specific system parts that cannot be reduced to individual microscopic elements. The framework relates this phenomenon to D(k), while contrasting it with causal decoupling and practical approximation limits.

  • 1. Downward causation: Downward causation occurs when a collective feature has irreducible causal power over individual parts.The feature uniquely predicts a subset of future variables beyond what any particular microscopic element predicts.
  • 1. Downward causation: A system admits order-k downward causation exactly when D(k)(Xt; Xt′) > 0.
  • 1. Downward causation: Causal decoupling instead concerns irreducible causal power between collective properties; perfect decoupling requires G(k)(Xt; Xt′) > 0 and D(k)(Xt; Xt′) = 0.
  • A. Practical criteria for large systems: The practical criteria use pairwise distributions and can be data-efficient, but the full framework requires estimating high-dimensional joint distributions and ΦID atoms.For a candidate feature, simpler quantities use standard mutual information and bivariate marginals; redundancy can also be double-counted up to n times.

B. Measuring emergence via synergistic channels

The paper measures emergence through synergistic channels that transmit information about a system without transmitting information about specified subsets of its parts. ΦID-based indices then provide direct measures and theoretical conditions for emergent behavior.

  • B. Measuring emergence via synergistic channels: For a selected ΦID, D(k) and G(k) can be evaluated directly, providing a route to detect emergence without double-counting redundancy.The approach can also yield additional properties determined by the selected ΦID.
  • B. Measuring emergence via synergistic channels: A k-synergistic channel conveys information about the whole system but not about any k-element subset of its parts.Variables generated through such channels are called k-synergistic observables.
  • B. Measuring emergence via synergistic channels: The kth-order synergy is the maximum information about Xt′ extractable from a k-synergistic channel of Xt.
  • B. Measuring emergence via synergistic channels: Causal decoupling extends this construction by requiring k-synergistic channels at both time points and maximizing their mutual information.

IV. CASE STUDIES

The case studies evaluate practical emergence criteria in Conway’s Game of Life and related systems. In the Game of Life experiment, particle type satisfies the criterion and shows strong temporal information relative to synergistic information.

  • IV. CASE STUDIES: The Game of Life system meets the practical criterion for causal emergence when particle type is used as the supervenient feature.
  • IV. CASE STUDIES: The case studies evaluate practical emergence criteria in Conway’s Game of Life and Reynolds’ flocking boids model.
  • IV. CASE STUDIES: The Game of Life experiment uses 15×15 binary cell arrays, random particle-collider initial conditions, and 1000 evolution steps to obtain final states.
  • IV. CASE STUDIES: Particle type is encoded as a supervenient feature independent of particle position or orientation.Each component indicates whether a particle of a given type is present on the board.
  • IV. CASE STUDIES: Ψ(1)t,t′(V) = 0.009 ± 0.0002, versus I(Vt; Vt′) = 0.99 ± 0.02, suggesting particle dynamics may be causally decoupled from their substrate.

2. Reynolds’ flocking model

The flocking model uses the center of mass as a candidate emergent feature and tests causal emergence across avoidance regimes. Emergence appears at intermediate avoidance, while low and high avoidance fail for distinct predictive reasons.

  • The model represents each of 10 boids by its 2D position and heading, with aggregation, avoidance, and alignment governing interactions.
  • The center of mass is used as the candidate macroscopic feature, and the study varies avoidance while holding aggregation and alignment fixed.
  • Causal emergence is detected at intermediate avoidance, where flocks form and disintegrate while the center of mass traces a smooth trajectory.
  • At high avoidance, strong repulsion prevents lasting flocks and lowers the center of mass’s self-predictability; at low avoidance, self-predictability is high but remains below individual-boid mutual information.
  • Low avoidance appears dominated by increased redundancy, raising the synergy threshold needed for the criterion to detect emergence.
  • A negative Ψ value is inconclusive because whole-minus-sum estimators can fail to rule out emergence in such cases.

A. Scope of the theory

The theory focuses on synchronic emergence in dynamical systems and extends causal-emergence analysis to observational data and scalable empirical criteria. Its scope excludes several philosophical theories and remains limited by assumptions about observability, dynamics, and detection sensitivity.

  • Scope and applicability: The theory analyzes interactions between system elements and collective properties as they jointly evolve over time.It applies directly to deterministic systems with random initial conditions and stochastic systems described by Fokker–Planck equations.
  • Scope and applicability: For observational data, causality is interpreted in the Granger sense as predictive ability.The framework uses mutual information, whose interpretation depends on whether its joint distribution comes from passive observation or active intervention.
  • Limitations and open problems: The framework assumes fully observable systems with Markovian dynamics, leaving the effects of unobserved variables for future work.The paper suggests methods such as Takens' embedding theorem as possible extensions.
  • Limitations and open problems: Its practical criteria are sufficient but not necessary, so they can detect substantial emergence while missing subtler cases.The criteria are agnostic to the particular PID and ΦID choices but may overestimate microscopic redundancy.
  • Contributions and implications: The theory quantitatively formalizes causal emergence, including downward causation and causal decoupling, using multivariate statistics and information decompositions.The authors present it as a bridge toward quantitative empirical investigation, illustrated through three case studies.
  • Contributions and implications: The framework offers conceptual and practical tools for testing whether psychological phenomena could emerge from collective neural patterns.The paper leaves this conjecture for future research rather than presenting it as an established result.

Appendix A: Information decomposition in large multivariate systems

The appendix constructs kth-order information decompositions that remain exact while grouping the exponentially many PID atoms into interpretable, scalable classes. These decompositions support extensions to ΦID and the mathematical results used by the emergence theory.

  • Definitions: kth-order synergy contains target information supplied jointly by all sources but by no collection of k or fewer sources.For k = 1, it recovers standard synergy in the two-source case.
  • Definitions: kth-order unique information consists of atoms where one source of size at most k is the only such small source, although larger groups may also carry information.For n = 3, U(1)({1}) includes both the singleton source and an atom involving the complementary pair.
  • Definitions: kth-order redundancy contains information held by at least two different source groups, each of size at most k.For k = 1, it recovers standard redundancy for two sources.
  • Coarse-grained PID: Coarse-grained PID reduces the number of atoms while preserving the intuitive meanings of synergy, redundancy, and unique information.For n = 3, Figure 7 compares the standard lattice with coarse-grainings for k = 1 and k = 2.
  • Exact decomposition: The kth-order synergy, redundancy, and unique-information classes partition the PID lattice and provide an exact decomposition of mutual information.The construction separates cases with zero, one, or multiple source groups of size at most k.
  • Required properties: The framework requires non-negativity, source and target data-processing inequalities, and deterministic equality, but not a specific PID or ΦID functional form.These properties support the practical emergence criteria and their extensions to multivariate targets.
  • Structural lemmas: Lemma 3 and its ΦID corollary relate decompositions after adding a variable equal to the joint original system.These results underpin the links between kth-order information atoms and emergence measures.

Appendix C: Mathematical properties of causal emergence

The appendix proves that the theory’s information quantities characterize causal emergence, downward causation, and causal decoupling through properties of supervenient features. The whole system can itself serve as a diagnostic feature, although shorter macroscopic representations may be preferable.

  • Causal emergence: Positive kth-order synergy of the system implies causal emergence by taking the whole system as the feature.Conversely, zero system synergy makes the relevant unique-information quantity vanish for every supervenient feature.
  • Causal emergence: The whole system is sufficient for detecting whether emergent behaviour exists, but it may contain non-interesting information compared with a shorter feature.The paper notes that practical representations may seek H(V_t) < H(X_t).
  • Downward causation: A supervenient feature exhibits downward causation when its information about a k-part future subset remains beyond the whole microscopic past.The appendix derives this condition through the unique-information quantity and the properties of D^(k).
  • Causal decoupling: Causal decoupling is characterized by positive G^(k)(X_t; X_t′), with the proof reducing feature-level decoupling to the whole-system quantity.The converse uses the whole system as a supervenient feature and the ΦID extension.
  • Perfect causal decoupling: When G^(k)(X_t; X_t′) > 0 and D^(k)(X_t; X_t′) = 0, the system has at least one emergent feature without the corresponding downward-causation contribution.This follows from Syn^(k) = G^(k) + D^(k) and the theorem linking D^(k) to feature-level unique information.

Appendix D: Mathematical properties of emergence criteria

The appendix connects practical emergence criteria to the formal definitions through data-processing and decomposition properties. Positive criterion values imply the corresponding system-level emergence, downward causation, or causal-decoupling quantities.

  • Downward causation: A positive value of Δ^(k)_{t,t′}(V) implies positive D^(k)(X_t; X_t′) and therefore downward causation.The derivation combines data processing with the whole-minus-sum property of synergy and the defining properties of D^(k).
  • Causal decoupling: The appendix states that the causal-decoupling criterion is sufficient to establish the corresponding system-level condition.Its derivation is introduced alongside the criteria for emergence and downward causation.
  • Emergence criteria: A positive value of Ψ^(k)_{t,t′}(V) for a k-synergistic observable implies positive system synergy and therefore causal emergence.The proof uses stationarity, the feature Markov chain, and data processing.

Appendix E: Simulation details

The simulations instantiate the paper’s case studies with specified system sizes, initialization procedures, preprocessing, and estimators. The Game of Life and boids analyses use repeated simulations and information-theoretic computation of Ψ.

  • Game of Life: Game of Life simulations placed two randomly selected particles in a 15x15 cell array and evolved each configuration for 1000 steps.Particles were selected from three fixed types: nothing, a glider, or a lightweight spaceship.
  • Game of Life: The Game of Life particle detector used one symbol for still lifes and one for oscillators, with 2% of runs producing unrecognised particles.The analysis used 5 × 10^4 independent runs and a quasi-Bayesian estimator to reduce bias.
  • Boids: The boids simulation used N = 10 agents on a torus with side length L = 200 and random initial positions, speeds, and head angles.Each boid was updated according to the model’s specified interaction equations.
  • Boids: The boids model defines θ1 as bearing to the flock center, θ2 as bearing to the nearest boid, and θ3 as mean alignment within a 20-unit radius.The parameters a1, a2, and a3 represent aggregation, avoidance, and alignment, respectively.
  • Boids: The boids analysis averaged Ψ across 25 runs of 5000 timesteps, fixed a1 = 0.15 and a3 = 0.25, and computed information-theoretic quantities with JIDT.Trajectories were preprocessed by representing each boid by its distance to the environment’s center and first-order differencing the time series.

Appendix F: ECoG preprocessing and decoding

ECoG signals were filtered, downsampled, split into training and test sets, and reduced to a 20-component latent representation before computing Ψ on held-out data.

  • Preprocessing: ECoG data were notch-filtered, bandpass filtered from 0.1 Hz to 600 Hz, and downsampled to 120 Hz before decoding.The data were then divided into a 60/40 train/test split.
  • Decoding: The training set was standardised to zero mean and unit variance before dimensionality reduction with a 20-component Partial Least Squares representation.
  • Evaluation: Ψ was computed on held-out test data after ECoG signals were standardised and projected into the PLS latent space using training-derived parameters.
Loading 2004.08220v1…