Multivariate Information Theory Uncovers Synergistic Subsystems of the Human Cerebral Cortex

SIGNIFICANCE STATEMENT

• Multivariate information theory reveals synergistic higher-order brain subsystems that are invisible to pairwise graph models.The paper relates these synergies to mathematical notions of complexity and describes brain dynamics through integration, segregation, synergy, and redundancy.Source: SIGNIFICANCE STATEMENT · p. 1
• The brain combines elements of integration, segregation, synergy, and redundancy.Source: SIGNIFICANCE STATEMENT · p. 1
• Higher-order dependencies represent a large, unexplored territory for neuroscience research.Source: SIGNIFICANCE STATEMENT · p. 1

INTRODUCTION

• Pairwise networks are limited to bivariate interactions, while higher-order statistical interactions are increasingly recognized as important features of complex systems.Formal tools and computational and combinatorial difficulties have constrained their application; neuroscience work has implicated them in complexity and functional integration.Source: INTRODUCTION · p. 1
• Higher-order interactions have been proposed to relate to emergent mental phenomena and consciousness.Source: INTRODUCTION · p. 2
• Redundancy shares information across elements, whereas synergy means uncertainty is resolved only by considering multiple variables jointly.Negative O-information indicates predominantly synergistic interactions; the measure was introduced as a scalable heuristic because pure synergy is computationally costly to calculate.Source: INTRODUCTION · p. 2
• Resting-state fMRI analysis identifies abundant, widely distributed cortical subsystems with synergistic dependencies, and relates O-information analytically to Tononi-Sporns-Edelman complexity.Source: INTRODUCTION · p. 2

A. Integration, Segregation, Redundancy, Synergy

• Integration–segregation theory frames the nervous system as combining regional functional specialization with system-wide integration.The ongoing balance has been reported as regulated by distinct neuromodulatory systems and correlated with conscious awareness.Source: A. Integration, Segregation,… · p. 2
• The integration–segregation spectrum ranges from independent elements to total synchronization, with a complex regime combining independence and integration.Source: A. Integration, Segregation,… · p. 2
• The paper expands integration to include redundant and synergistic forms, yielding a non-orthogonal configuration space called an informational “morphospace”.This builds on the distinction between redundancy and synergy and the definition of O-information.Source: A. Integration, Segregation,… · p. 2

B. Information Theory and Higher-Order Information-Sharing

• Pairwise mutual information reveals dyadic functional connectivity but is insensitive to higher-order interactions among variables.The section frames multivariate generalizations as a way to study these higher-order structures.Source: B. Information Theory and Higher-Order… · p. 3
• Total correlation and dual total correlation generalize mutual information to multivariate systems, while dual total correlation captures entropy shared by at least two elements.Total correlation is high when individually entropic variables have a low-entropy joint state; dual total correlation peaks when information is shared.Source: B. Information Theory and Higher-Order… · p. 3
• O-information is the difference between total correlation and dual total correlation, with its sign indicating redundancy- or synergy-dominated dynamics.The section relates this measure to the Tononi-Sporns-Edelman complexity and a geometric interpretation of redundancy and synergy.Source: B. Information Theory and Higher-Order… · p. 3
• TSE complexity can be interpreted as the difference between expected integration at a scale and the actual integration of subsets at that scale.Its full calculation requires evaluating every possible subsystem or bipartition, making it impractical beyond small systems.Source: B. Information Theory and Higher-Order… · pp. 3–4
• Description complexity approximates TSE complexity using the average total correlation of the N subsystems formed by removing one element at a time.C(X) compares the expected decrease in integration from removing one node with the actual decrease.Source: B. Information Theory and Higher-Order… · p. 4
• The relationship between dual total correlation and description complexity permits rewriting O-information purely in terms of total correlations.The section interprets synergy as information redundantly present in large joint ensembles but absent from their subsets.Source: B. Information Theory and Higher-Order… · p. 4
• Exogenous information correlates strongly with TSE complexity and with dual total correlation across subset sizes from three to fifteen elements.The analysis uses Gaussian assumptions to assess how the relationship holds as system size increases.Source: B. Information Theory and Higher-Order… · p. 4

• For 100,000 sampled 10-node subsets, TC+DTC correlated with TSE complexity (HCP: R = 0.998; MICA: R = 0.999), while DTC alone also correlated strongly (HCP: R = 0.982; MICA: R = 0.992).The figure reports p = 0 for each correlation.Source: Figure 2 · p. 5
• For 10-node subsets, 0.41 percent of all subsets were synergistic, corresponding to 9.23×10^13 subsets; their fraction declined with size while their total number remained substantial.The full 200-node matrices were redundancy-dominated in both data sets (HCP: Ω=79.16 nats; MICA: Ω=46.69 nats).Source: §II · pp. 4–5
• O-information was positively correlated with TSE complexity in sampled HCP 10-node subsets (ρ = 0.642, p = 0), despite few sampled subsets having negative O-information.The O-information can be interpreted through the sensitivity of global integration to removal of individual elements.Source: Figure 1 · p. 5, Figure 3 · p. 6
• Absolute pairwise functional connectivity was negatively correlated with participation in synergistic subsets (HCP: ρ = −0.504; MICA: ρ = −0.485), so strongly connected pairs rarely co-occurred.The reported correlations were both p = 0.Source: §II · p. 7
• Across 100,000 sampled synergistic 10-node subsets, every cortical node participated at least once, and frontoparietal regions had the highest participation frequency for 10-node subsets.Node-participation maps were correlated across HCP and MICA (ρ = 0.579, p = 2.5 × 10−19); limbic participation dominated over frontoparietal for larger subsets.Source: §II · p. 7
• Approximately 10-node subsets achieved minimal O-information in both data sets, and simulated annealing found highly synergistic subsets for sizes 3–24 in HCP and 3–27 in MICA.No synergistic subsets were found above 27 nodes, where redundancy overwhelmed unique informational contributions.Source: §II · p. 7
• Optimized subsets recruited multiple canonical systems, but over-representation differed: visual, frontoparietal, and default mode in HCP, versus only frontoparietal in MICA.Nodal frequencies across the optimized solutions were correlated between data sets (ρ = 0.522, p = 2.2 × 10−15).Source: §II · p. 7
• No subset drawn from a single canonical functional system expressed synergy; subsets spanning 6 or 7 systems were most likely to have negative O-information.Mean O-information decreased and the fraction of synergistic subsets increased as subsets spanned more systems.Source: §II · p. 7

III. DISCUSSION

• Optimally negative O-information occurs for subsets of 8–12 nodes, while randomly sampled subsets spanning multiple canonical systems are more often synergy-dominated than within-system subsets.The latter are strongly redundancy-dominated; the optimized 10-node subsets draw participants from across the cortex.Source: Figure 5 · p. 8, §III · p. 8
• Synergistic subsystems occur across the cortex, span 3–25 regions, and are abundant in both fMRI datasets.Almost every cortical region contributes to some synergistic ensemble, though some contribute more reliably than others.Source: §III · p. 8
• O-information can be understood geometrically as integration in the whole system beyond that in subsets formed by removing each element.The authors relate this framework to Tononi-Sporns-Edelman complexity and describe synergy as information about configurations relative to the maximum-entropy case.Source: §III · pp. 7–8
• Higher-order synergies form a distributed “shadow structure” missed by pairwise network analyses, which are sensitive to redundancy-dominated dependencies.The distribution of synergies was reported as anticorrelated with functional connectivity structure, while redundancy was positively correlated.Source: §III · pp. 8–9
• The relationship between O-information and TSE complexity includes exogenous information proportional to TSE complexity for reasonably large N, supporting empirical applications.The authors note that intractable combinatorics have limited TSE complexity's empirical applicability.Source: §III · p. 9
• How information-theoretic approaches relate to parallel frameworks such as simplicial complexes, algebraic topology, and hypergraphs remains open.The authors describe a unified account of topology and statistical inference as a future possibility.Source: §III · p. 9
• Optimization of synergistic subsets is framed as dimensionality reduction that retains synergies while pruning redundant information.The authors characterize rigorous use of synergy and redundancy in brain-dynamics analysis as an active area of development.Source: §III · p. 9
• Subsystem search becomes computationally intractable as system size grows, so optimization, random sampling, or pre-filtering is needed to select subsets of interest.The significance of synergistic dependencies also remains largely unknown, and O-information is not necessarily a causal measure.Source: §III · p. 9

A. Gaussian Information Theory

• The paper focuses on higher-order information sharing in fMRI BOLD signals, modeled using multivariate Gaussian distributions.The authors cite prior work finding that more complex, highly parameterized models provide little additional benefit.Source: A. Gaussian Information Theory · p. 10
• For Gaussian variables, closed-form estimators allow calculation of joint entropy, bivariate mutual information, and total correlation.The paper uses these measures to calculate dual total correlation, description complexity, O-information, and TSE complexity.Source: A. Gaussian Information Theory · p. 10
• Gaussian bivariate mutual information is a function of Pearson correlation and is not generally sensitive to nonlinear relationships.The authors note that the Gaussian model's linearity assumption matches that of standard Pearson-correlation functional-connectivity analysis.Source: A. Gaussian Information Theory · p. 10

B. Datasets

• The study combines independent HCP and MICA resting-state fMRI datasets, using a common parcellation covering 200 cortical regions.The parcellation can be aligned with canonical resting-state networks.Source: B. Datasets · p. 10
• HCP scans used four sessions across two days, whereas MICA included 50 unrelated participants scanned in one seven-minute session.Both datasets used 3T Siemens scanners; MICA used 3 mm isotropic voxels and HCP used 2 mm voxels.Source: B. Datasets · p. 10
• HCP data came from 100 unrelated participants, with 95 retained after prespecified motion and processing exclusions.The retained sample was 56% female, mean age 29.29 ± 3.66 years, with an age range of 22–36.Source: B. Datasets · p. 10

C. Preprocessing

• HCP preprocessing included motion and distortion correction, registration, normalization, surface projection, global signal regression, detrending, and band-pass filtering.After discarding the first and last 50 frames, each scan retained 13.2 minutes or 1,100 frames.Source: C. Preprocessing · p. 11
• MICA data were processed with Micapipe, including motion and distortion correction and ICA FIX, followed by global signal regression.Time series were projected to each subject’s FreeSurfer surface.Source: C. Preprocessing · p. 11
• The covariance matrices were computed after concatenating nodal time series across subjects and runs; HCP and MICA matrices were highly correlated (R = 0.851, p = 0).Averaging single-run, single-subject matrices instead yielded virtually identical results.Source: C. Preprocessing · p. 11

D. Random Sampling and Optimization

• Random sampling cannot fully disclose the variation in informational measures across the vast repertoire of potential subsets, whereas optimization targets subsets with distinct signatures.Optimization guides the search toward specific subspaces.Source: D. Random Sampling and Optimization · p. 11
• The authors ran simulated annealing 5,000 times across subset sizes of 3–30 regions, optimizing O-information, total correlation, or dual total correlation.Each run used one functional connectivity matrix and one subset size, beginning with randomly selected regions.Source: D. Random Sampling and Optimization · p. 11
• At each step, the algorithm randomly replaced nodes and retained a new subset when its cost was lower or a temperature-dependent random test accepted it.Each run proceeded for up to 10,000 steps; the number of nodes flipped was usually one, sometimes two, and rarely three.Source: D. Random Sampling and Optimization · p. 11
• The temperature decayed from an initial value of 1 according to the iteration step and a steepness parameter, making the search progressively more deterministic.The passage identifies the current temperature as Tc, the initial temperature as T0, the steepness parameter as Texp, and the iteration step as h.Source: D. Random Sampling and Optimization · p. 11