Source-linked AI summary
Learning Dynamic Graph Representation of Brain Connectome with Spatio-Temporal Attention
Byung-Hoon Kim, Jong Chul Ye, Jae-Jin Kim
TL;DR
GNN analyses of brain functional connectivity often neglect temporal dynamics and have lacked temporal explainability, motivating a dynamic representation method. STAGIN combines graph processing with spatial and temporal attention, and reports state-of-the-art HCP performance with interpretable attention patterns.
Problem
Previous GNN-based FC analyses often omit fluctuating network dynamics, while dynamic approaches reported reduced performance and lacked temporal explainability.
Method
STAGIN processes temporal sequences of dynamic FC graphs with attention-based READOUT modules and a Transformer encoder to learn spatially and temporally explainable representations.
Results
STAGIN achieves state-of-the-art performance on HCP resting-state gender classification and task-fMRI decoding, with attention analyses yielding interpretations consistent with neuroscientific knowledge.
Takeaways & Limitations
The learned spatio-temporal attention provides model interpretations of brain regions and timepoints alongside dynamic-connectome decoding performance.
Takeaways & Limitations
The work notes privacy concerns regarding potential misuse of accurate brain-decoding methods and uses a heuristic threshold for top attended regions.
Abstract
from arXiv · showhide
Functional connectivity (FC) between regions of the brain can be assessed by the degree of temporal correlation measured with functional neuroimaging modalities. Based on the fact that these connectivities build a network, graph-based approaches for analyzing the brain connectome have provided insights into the functions of the human brain. The development of graph neural networks (GNNs) capable of learning representation from graph structured data has led to increased interest in learning the graph representation of the brain connectome. Although recent attempts to apply GNN to the FC network have shown promising results, there is still a common limitation that they usually do not incorporate the dynamic characteristics of the FC network which fluctuates over time. In addition, a few studies that have attempted to use dynamic FC as an input for the GNN reported a reduction in performance compared to static FC methods, and did not provide temporal explainability. Here, we propose STAGIN, a method for learning dynamic graph representation of the brain connectome with spatio-temporal attention. Specifically, a temporal sequence of brain graphs is input to the STAGIN to obtain the dynamic graph representation, while novel READOUT functions and the Transformer encoder provide spatial and temporal explainability with attention, respectively. Experiments on the HCP-Rest and the HCP-Task datasets demonstrate exceptional performance of our proposed method. Analysis of the spatio-temporal attention also provide concurrent interpretation with the neuroscientific knowledge, which further validates our method. Code is available at https://github.com/egyptdj/stagin
1 Introduction
Brain functional connectivity forms a time-varying network that GNN studies seek to represent for brain decoding. STAGIN addresses limited use of dynamics and missing temporal explainability while reporting strong HCP performance and ethical privacy considerations.
- Functional connectivity measures temporal correlation between brain regions, whose network organization changes over time.
- Previous GNN-based FC analyses often omit dynamic properties, while an attempted dynamic method reported lower gender-classification accuracy and no temporal explainability.
- STAGIN learns dynamic brain-connectome representations with spatio-temporal attention, using attention-based READOUT modules and a Transformer encoder.
- STAGIN achieves state-of-the-art performance on HCP resting-state gender classification and task-fMRI decoding, while its learned attention is interpreted using established analyses.
- Brain decoding may support neural-biomarker discovery but raises privacy concerns from potential misuse of accurate decoding methods.
2 Related works
Dynamic-graph representation methods and graph attention provide relevant foundations for STAGIN, but dynamic brain graphs have properties that complicate direct application of existing techniques.
- Dynamic graph methods embed graphs using time information, and attention-based approaches have also been proposed.
- Dynamic brain graphs lack node and edge additions or deletions and are sampled uniformly over time, limiting direct transfer of general dynamic-graph techniques.
- Earlier dynamic-graph studies inspire STAGIN’s temporal-information encoding and concatenation with node features.
- Graph neural network attention commonly weights edges or supports node-selection and pooling through learned relative vertex scores.
3 Theory
The theory formulates dynamic brain-graph representation as graph encoding followed by temporal integration, and interprets READOUT as a decoder whose fixed bases motivate adaptive attention.
- 3.1 Problem definition: A dynamic brain graph is a sequence Gdyn = (G(1), ..., G(T)) mapped to a length-D representation hGdyn for downstream phenotype classification.
- 3.1 Problem definition: STAGIN composes a GNN g, which produces time-specific graph representations, with a Transformer encoder q, which integrates them through self-attention.
- 3.2 Graph Isomorphism Network: GIN updates node representations by aggregating neighborhood features and applying nonlinear transformations, supporting graph classification under injectivity assumptions.
- 3.2 Graph Isomorphism Network: Conventional READOUT sums or averages updated node features using a fixed mean-pooling vector.
- 3.3 Encoder-decoder understanding of GNNs: In the encoder-decoder view, GIN produces encoded node features while mean READOUT acts as a decoder with constant bases.
- 3.3 Encoder-decoder understanding of GNNs: Because fixed decoding can restrict expressivity, STAGIN introduces adaptive attention-based READOUT functions.
4 STAGIN: Spatio-Temporal Attention Graph Isomorphism Network
STAGIN learns dynamic brain-connectome representations from temporal FC graph sequences using spatial and temporal attention. Its attention-based READOUT modules and Transformer encoder provide interpretable weighting across brain regions and timepoints.
- STAGIN learns dynamic graph representations from a sequence of brain graphs using two attention-based READOUT modules.
- 4.1 Dynamic graph definition: Dynamic FC graphs are extracted from ROI-timeseries with sliding windows, then binarized by retaining the top 30-percentile correlations as edges.Window length Γ and stride S produce T windowed FC matrices before thresholding.
- 4.1 Dynamic graph definition: Timestamp embeddings generated by a GRU are concatenated with spatial one-hot node encodings to give node features temporal information.The GRU processes ROI-timeseries up to each sliding-window endpoint.
- 4.2 Spatial attention with attention-based READOUT: GARO and SERO compute spatial attention over nodes, replacing fixed READOUT pooling with learned scaling coefficients.GARO uses key-query attention, whereas SERO uses MLP-based attention over the node dimension.
- 4.2 Spatial attention with attention-based READOUT: The spatial READOUT uses orthogonality regularization to encourage node-feature basis columns to be orthogonal while preserving their common length.The scaling term 1/m controls column normalization without fixing the specific column length.
- 4.3 Temporal attention: A single-headed Transformer encoder applies self-attention across graph features over time to produce temporal attention weights and dynamic representations.Representations from all K layers are concatenated into the final dynamic graph representation.
5 Experiment
Experiments on HCP-Rest and HCP-Task evaluated STAGIN’s classification performance and its spatial and temporal attention. STAGIN achieved strong results while its attention patterns aligned with task timing and reported neuroscientific distinctions.
- HCP-Rest: Gender classification: 87.01% and 88.20% mean accuracy were achieved by STAGIN-GARO and STAGIN-SERO, respectively, for HCP-Rest gender classification.Mean AUROC was 0.9151 for STAGIN-GARO and 0.9296 for STAGIN-SERO.
- HCP-Rest: Gender classification: Female-attended clusters showed DMN hyperconnectivity and SMN hypoconnectivity, matching the stated gender-related FC hypothesis.The clusters were identified from temporally attended dynamic FC matrices using k-means clustering.
- HCP-Task: Task decoding: 99.02% and 99.19% mean accuracy were achieved by STAGIN-GARO and STAGIN-SERO, respectively, for HCP-Task decoding.The region-based BAnD baseline achieved 98.90% accuracy, while the proposed methods outperformed previous GNN-based methods.
- HCP-Task: Task decoding: The temporal attention matrix represented working-memory subtask timing despite no supervision about subtask timing during training.The working-memory subtasks consisted of n-back performance or rest.
- HCP-Task: Spatio-temporal attention: Low-order layers emphasized SMN, VN, and SVN regions, whereas high-order layers emphasized DMN and CCN regions.The comparison used the proportion of statistically significant regions within seven intrinsic connectivity networks.
- Orthogonal regularization: The geometric analysis showed that orthogonal regularization broadens the possible graph-feature range produced from node features compared with conventional READOUT.The motivation is to make node features full rank with a good condition number.
B Detailed description of the dataset
The HCP experiments include a limitation: twin subjects were present, but shared genetic influences on functional connectivity were not modeled.
- Twin subjects comprise a large portion of HCP data, potentially introducing shared genetic influences on functional connectivity.
- Table 2 summarizes the experiment datasets used in the study.
C.1 Ablation study
Ablations indicate that orthogonal regularization, spatial and temporal attention, and timestamp encoding each contribute to STAGIN’s performance or interpretability.
- Removing orthogonal regularization, spatial attention, temporal attention, or timestamp encoding degraded STAGIN’s performance.The ablation results support the importance of each component.
- Timestamp encoding produced a larger classification-performance gain than spatio-temporal attention.
- The attention modules remain important because they provide neuroscientific explainability.
- Table 3 reports the ablation study results.
C.2 Hyperparameter experiments
STAGIN was generally robust to hyperparameter changes, with better HCP-Rest gender-classification performance at a 40% edge threshold than at 30%.
- STAGIN showed robustness to hyperparameter changes in the reported experiments.
- 40% edge thresholding yielded better HCP-Rest gender-classification performance than 30% thresholding.
- Table 4 reports the hyperparameter experiment results.
C.3 Comparative experiment of spatial attention scoring
The attention-based READOUT functions were compared with established graph-pooling methods and were associated with better, more stable graph-classification performance and lower computational overload.
- Comparative experiment of spatial attention scoring: Attention-based READOUT functions performed better and more stably than TopKPooling, SAGPooling, and ASAPooling for graph classification.
- Comparative experiment of spatial attention scoring: The reported advantage included lower computational overload for the graph-classification task.
- Comparative experiment of spatial attention scoring: The proposed READOUT functions use the globally pooled graph feature HΦmean as a prior for node-attention scoring.
- Temporal-attention analysis: Figure 6 clusters temporally attended HCP-Rest regions using five and three cluster centroids and compares unattended average FC patterns by gender.
- Temporal-attention analysis: The same DMN hyperconnectivity and SMN hypoconnectivity pattern appeared across both cluster settings, while attended-cluster frequencies differed significantly by gender.The average FC difference between female and male subjects was small but detectable in the dynamic FC graph.
- Temporal-attention analysis: Table 6 reports the chi-square test of temporal attending frequency.
D.2 Temporal and spatial attention analysis of all task types from HCP-Task
STAGIN’s Transformer encoder temporally attends to subtasks across task types without timing information during training, while spatial attention is summarized across intrinsic connectivity networks.
- Temporal attention: STAGIN’s Transformer encoder attends temporally to subtasks regardless of task type, without subtask timing information during training.
- Temporal attention: Figure 7 plots the HCP-Task temporal attention Z(k) learned by the Transformer encoder.
- Spatial attention: Figure 8 shows the proportion of statistically significant regions within seven ICNs for each HCP-Task subtask contrasted with rest or response.
E Brain plot of spatially attended regions from HCP-Rest and HCP-Task
Spatially attended HCP-Rest and HCP-Task regions are visualized on a template brain across seven intrinsic connectivity networks and four STAGIN layers, using different significance criteria.
- Brain plots: Spatial attention regions are visualized across seven ICNs and four STAGIN layers for both HCP-Rest and HCP-Task experiments.
- HCP-Rest: HCP-Rest spatially attended regions are defined as those in the top 5-percentile of attention scores.
- HCP-Task: HCP-Task spatially attended regions are defined by GLM statistical significance at p-FWE < 0.05.