Source-linked AI summary
Measuring integrated information from the decoding perspective
Masafumi Oizumi, Shun-ichi Amari, Toru Yanagawa, Naotaka Fujii, Naotsugu Tsuchiya
TL;DR
The paper addresses the difficulty of estimating integrated information from neural data while preserving IIT's theoretical requirements. It introduces Φ* using mismatched decoding and shows that the measure is properly bounded, with an analytical Gaussian expression for experimental application.
Problem
Estimating integrated information empirically is difficult, and previously proposed practical measures fail to satisfy required lower and upper bounds.
Method
The paper introduces Φ* by applying mismatched decoding to compare actual and hypothetical mutual information when system interactions are ignored.
Results
Φ* satisfies the theoretical requirements because it is bounded below by 0 and above by the information in the whole system.
Takeaways & Limitations
Φ* extends the original measure to empirical distributions and can support experiments testing integrated information across conscious and unconscious states.
Takeaways & Limitations
The original IIT 2.0 intrinsic and integrated information measures apply only to discrete variables because maximum entropy distributions are not uniquely determined for continuous variables.
Abstract
from arXiv · showhide
Accumulating evidence indicates that the capacity to integrate information in the brain is a prerequisite for consciousness. Integrated Information Theory (IIT) of consciousness provides a mathematical approach to quantifying the information integrated in a system, called integrated information, $Φ$. Integrated information is defined theoretically as the amount of information a system generates as a whole, above and beyond the sum of the amount of information its parts independently generate. IIT predicts that the amount of integrated information in the brain should reflect levels of consciousness. Empirical evaluation of this theory requires computing integrated information from neural data acquired from experiments, although difficulties with using the original measure $Φ$ precludes such computations. Although some practical measures have been previously proposed, we found that these measures fail to satisfy the theoretical requirements as a measure of integrated information. Measures of integrated information should satisfy the lower and upper bounds as follows: The lower bound of integrated information should be 0 when the system does not generate information (no information) or when the system comprises independent parts (no integration). The upper bound of integrated information is the amount of information generated by the whole system and is realized when the amount of information generated independently by its parts equals to 0. Here we derive the novel practical measure $Φ^*$ by introducing a concept of mismatched decoding developed from information theory. We show that $Φ^*$ is properly bounded from below and above, as required, as a measure of integrated information. We derive the analytical expression $Φ^*$ under the Gaussian assumption, which makes it readily applicable to experimental data.
Introduction
IIT links integrated information to consciousness, but estimating it empirically is difficult because the original measure relies on restrictive assumptions. The paper proposes Φ* using mismatched decoding to address these practical and theoretical challenges.
- IIT predicts that integrated information should vary with consciousness, from high levels during wakefulness to low levels during anesthesia or dreamless sleep.
- IIT defines integrated information as information generated by a system as a whole beyond the information generated independently by its parts.
- Empirical calculation is difficult because the original measure assumes a maximum entropy distribution and requires transition probabilities for all possible past states.
- Previously proposed empirical-distribution measures do not satisfy the requirements that integrated information remain nonnegative and not exceed whole-system information.
- The proposed Φ* uses mismatched decoding to compare actual mutual information with hypothetical mutual information after partitioning the system into independent parts.
Results
The paper reviews theoretical bounds for integrated information, shows that empirical-distribution measures ΦI and ΦH violate them, and introduces Φ* based on mismatched decoding. Φ* uses empirical distributions, quantifies information loss from ignoring interactions, and satisfies the required bounds, although computation remains difficult for large systems.
- Theoretical requirements: IIT 2.0 defines integrated information as whole-system mutual information minus the sum of mutual information generated independently by its parts.The original measure assumes a maximum-entropy distribution for past states and satisfies both lower and upper bounds.
- Limitations of existing measures: The original measure Φ is difficult to apply because its maximum-entropy assumption is restricted to discrete states and limits practical use with continuous neural data.Empirical distributions were proposed to remove this assumption.
- Limitations of existing measures: The empirical-distribution measures ΦI and ΦH fail theoretical requirements for integrated information.The paper evaluates their behavior in cases with no information and with correlated parts.
- Integrated information measure based on mismatched decoding: Φ* is defined as actual mutual information minus mismatched-decoding mutual information, quantifying information loss when interactions between partitioned parts are ignored.It uses empirical distributions and satisfies the lower and upper bounds because the mismatched quantity lies between 0 and whole-system information.
- Limitations of the proposed measure: Computing Φ* for large systems remains expensive because summing or integrating over all possible states grows exponentially with system size.The empirical distribution improves feasibility for neuronal recordings but does not remove this scaling problem.
- When there is no information: ΦH can be positive when whole-system mutual information is zero, so it exceeds the upper bound and reflects correlation rather than information.With no connections, ΦH increases with noise correlation while Φ* and ΦI remain zero.
- When parts are perfectly correlated: ΦI becomes negative when parts are strongly correlated, violating the lower bound, whereas Φ* decreases with correlation and reaches 0 at perfect correlation.At noise correlation c greater than approximately 0.2, ΦI is negative in the reported setting; when c is 1, Φ* is 0.
Discussion
The paper argues that Φ* satisfies the theoretical bounds required of integrated-information measures while offering an information-theoretic interpretation applicable beyond consciousness research.
- Discussion: Φ* is defined as information loss from decoding with a partitioned, mismatched distribution instead of the actual distribution.This operationalizes the loss associated with treating system parts as independent.
- Discussion: The measure is intended to have lower and upper bounds of 0 and the whole system’s generated information, respectively.These bounds follow from defining integrated information as information generated by the whole above its independently generated parts.
- Discussion: Few studies directly estimate integrated information in the brain, and Φ* is proposed for experiments comparing conscious and unconscious states.Earlier empirical work used an IIT-inspired complexity measure rather than directly measuring integrated information.
- Discussion: The study considers only a given partition, whereas IIT requires evaluating every possible partition to identify the minimum information partition.Finding the minimum information partition remains a practical difficulty.
- Discussion: Comparing integrated information across partitions remains theoretically unresolved because normalization choices may depend on partition structure and variable type.The paper specifically questions normalization for continuous variables when the proposed factor assumes discrete states.
- Discussion: Φ* measures global interaction by comparing whole-system mutual information with mutual information under hypothetical independent parts, rather than summing local interactions.The authors suggest this interpretation may support network analysis beyond consciousness research.
Methods
The methods define intrinsic information through system transitions and past-state priors, then formulate integrated information by contrasting intact and partitioned systems.
- Intrinsic and extrinsic information: Intrinsic information is evaluated from the system’s internal variables and transition probability matrix, unlike extrinsic information, which concerns external stimuli or outputs.The transition matrix specifies probabilities for transitions between past and present system states.
- Intrinsic and extrinsic information: Quantifying intrinsic information requires both a transition probability matrix and a specified prior distribution of past states.The prior is not uniquely determined by the system’s intrinsic mechanisms.
- Intrinsic and extrinsic information: IIT selects a maximum-entropy prior, which is uniform over possible past states for discrete variables.This makes every possible past state equally likely as a cause of a present state.
- Intrinsic and extrinsic information: For continuous variables, maximum entropy is not uniquely defined without constraints, limiting IIT 2.0 intrinsic and integrated information to discrete variables.A Gaussian distribution is maximum entropy only under specified mean and variance constraints.
- Intrinsic and extrinsic information: Effective information quantifies how much knowing a present state reduces uncertainty about past states from the system’s intrinsic perspective.The posterior distribution is computed using Bayes’ rule.
- Integrated information: Averaged effective information equals mutual information between past and present states, and the method uses this averaged quantity.The averaging is performed over all possible present states.
- Integrated information: Integrated information compares information between past and present states in an intact system with information after partitioning it into independent parts.The partitioned transition model supplies the hypothetical independent system.
- Integrated information: The original Φ equals the increase in uncertainty caused by partitioning, equivalently the difference between whole-system and partitioned mutual information.The formulation averages the state-dependent quantity over present states.
Supporting Information
The supporting analyses establish equivalence with the original measure under a maximum-entropy prior and examine Φ* alongside alternative measures under varying connection and noise-correlation strengths.
- Supporting Information: Φ* is equivalent to the original Φ when the maximum-entropy distribution is assumed for past states.The supporting derivation states this equivalence explicitly.
- Supporting Information: When two parts are perfectly correlated, whole-system mutual information equals each part’s mutual information and Φ* becomes 0.The example uses a two-unit system and a mismatched decoder based on one part.
- Supporting Information: The supporting figure varies connection strength a and noise-correlation strength c in a linear regression model.It displays Φ*, ΦI, ΦH, mutual information, and correlation coefficient as the parameters vary.
- Supporting Information: The figure compares Φ* with ΦI, ΦH, mutual information, and correlation coefficient under the same parameter variations.Panels A–E correspond respectively to Φ*, ΦI, ΦH, mutual information, and correlation.