BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networks

Hejie Cui; Wei Dai; Yanqiao Zhu; Xuan Kan; Antonio Aodong Chen Gu; Joshua Lukemire; Liang Zhan; Lifang He; Ying Guo; Carl Yang

Source-linked AI summary

BrainGB: A Benchmark for Brain Network Analysis with Graph Neural Networks

Hejie Cui, Wei Dai, Yanqiao Zhu, Xuan Kan, Antonio Aodong Chen Gu, Joshua Lukemire, Liang Zhan, Lifang He, Ying Guo, Carl Yang

arXiv:2204.07054v3Mar 17, 2022q-bio.NCcs.LGcs.NE

TL;DR

BrainGB addresses the lack of systematic evidence on how to design effective GNNs for brain network analysis. It standardizes functional and structural network construction and modularizes GNN designs for evaluation across diverse datasets. The experiments support general design recipes and open, reproducible resources, while small neuroimaging datasets may limit complex models’ effectiveness and generalization.

Problem
Brain network analysis lacks a public benchmark with standardized construction pipelines, fair evaluation settings, metrics, and modular baselines for GNNs.
Method
BrainGB combines functional and structural network-construction pipelines with modular GNN baselines and experiments across datasets, cohorts, modalities, and tasks.
Results
BrainGB’s connection-profile node features achieve the best performance across all datasets, with up to 33.99% improvement over degree on ABCD.
Takeaways & Limitations
BrainGB offers general GNN design recipes and reproducible resources, including preprocessing instructions, models, tutorials, source code, and an installable package.
Takeaways & Limitations
Small neuroimaging datasets may limit the effectiveness and generalization ability of complex deep learning models.

Abstract

from arXiv · show

Mapping the connectome of the human brain using structural or functional connectivity has become one of the most pervasive paradigms for neuroimaging analysis. Recently, Graph Neural Networks (GNNs) motivated from geometric deep learning have attracted broad interest due to their established power for modeling complex networked data. Despite their superior performance in many fields, there has not yet been a systematic study of how to design effective GNNs for brain network analysis. To bridge this gap, we present BrainGB, a benchmark for brain network analysis with GNNs. BrainGB standardizes the process by (1) summarizing brain network construction pipelines for both functional and structural neuroimaging modalities and (2) modularizing the implementation of GNN designs. We conduct extensive experiments on datasets across cohorts and modalities and recommend a set of general recipes for effective GNN designs on brain networks. To support open and reproducible research on GNN-based brain network analysis, we host the BrainGB website at https://braingb.us with models, tutorials, examples, as well as an out-of-box Python package. We hope that this work will provide useful empirical evidence and offer insights for future research in this novel and promising direction.

I. INTRODUCTION

BrainGB addresses reproducibility and evaluation gaps in GNN-based brain network analysis by unifying network-construction pipelines and modularizing GNN designs. It combines accessible data, standardized evaluation, empirical studies, and open resources for future research.

Brain network studies often use private datasets and undisclosed preprocessing, making experiments difficult for other researchers to reproduce.
BrainGB establishes a unified, modular, scalable, and reproducible framework with accessible datasets, standard settings, and baselines for fair evaluation.
The benchmark summarizes preprocessing and construction pipelines for both functional and structural brain networks.
Its GNN design space is decomposed into node features, message passing, attention mechanisms, and pooling strategies.
BrainGB evaluates GNN designs empirically, proposes general recipes, and provides source code, an installable package, tutorials, preprocessing instructions, models, and a leaderboard.

II. PRELIMINARIES

Brain network analysis represents each subject as a graph whose ROI nodes share a parcellation while weighted connections vary across subjects. GNNs learn node and graph representations, but brain networks require designs adapted to featureless graphs, signed weights, and fixed ROI identities.

A. Brain Network Analysis: A brain network dataset contains subject graphs and labels, with each graph representing one individual and a prediction target such as disease status.
A. Brain Network Analysis: Across subjects, the ROI node set is fixed while edge connections are represented by weighted adjacency matrices.
GNNs recursively aggregate neighborhood messages to compute node representations, then pool node states into a graph representation for prediction.
Brain networks differ from many real-world graphs through limited initial ROI features, positive or negative real-valued weights, and fixed ROI identities and ordering.

III. BRAIN NETWORK DATASET CONSTRUCTION

BrainGB standardizes MRI-derived brain network construction across functional and structural modalities while recognizing preprocessing complexity, tool variation, privacy constraints, and expert quality control.

BrainGB focuses on MRI-derived networks, using fMRI for functional networks and dMRI for structural networks.
Raw MRI data require complicated preprocessing to remove artifacts, standardize formats, and discover structure, while available tools differ in functionality.
The benchmark provides complete functional and structural example pipelines with step-by-step commands and parameter settings through its website.
The functional pipeline caption identifies preprocessing and network-construction steps alongside commonly used toolboxes, with tools ordered by usage frequency.
The structural pipeline caption similarly presents dMRI preprocessing and network construction with recommended tools, prioritizing more commonly used structural tools.

1) Functional Brain Network Construction:

Functional brain network construction preprocesses fMRI signals, defines ROI nodes, extracts representative regional time series, and computes connectivity, while structural pipelines use dMRI processing and tractography.

1) Functional Brain Network Construction:: Functional connectivity preprocessing commonly includes detrending, demeaning, and whitening of fMRI BOLD time series.
1) Functional Brain Network Construction:: A brain atlas or ROI set defines nodes, and representative BOLD series are obtained by averaging or applying SVD within each node.
1) Functional Brain Network Construction:: BrainGB provides an ABCD example using the ABCD-HCP BIDS pipeline for anatomical and functional preprocessing before applying brain parcellation.
2) Structural Brain Network Construction:: Structural construction preprocesses dMRI, models local diffusion patterns, and uses tractography to reconstruct fiber connections between brain regions.
2) Structural Brain Network Construction:: A PPMI example uses TrackVis, FSL, and FreeSurfer, then computes networks from fiber counts within ROIs after removing false positives.

C. Discussions

BrainGB covers multiple ways to construct brain-network edges and initializes node features for featureless graphs using positional or structural information.

Brain networks can use directional, hybrid, or dynamic functional edges in addition to mainstream connectivity constructions.
For featureless brain networks, BrainGB categorizes node initialization methods as positional or structural.
Identity features assign each ROI a unique one-hot vector, allowing the GNN to learn relative positions by memorizing k-hop neighbors.
Eigen features use the top k eigenvectors of the weighted connectivity matrix to encode node positions with global graph information compressed into low dimensions.
Degree and degree-profile features capture local structural information, while connection profiles retain each node’s connections to all other regions.

B. Message Passing Mechanisms

BrainGB modularizes message passing around neighbor aggregation and compares edge-weighted, binned, concatenated, and node-preserving message designs.

Each GNN layer aggregates neighbor messages and updates node embeddings using neighborhood structure, edge weights, and an update function.The update may optionally include the node’s previous embedding.
BrainGB compares message functions because the benefit of incorporating brain-network connections into graph-level prediction remains unresolved.
Edge-weighted messages multiply each neighbor representation by its connection strength, matching standard GCN message passing when wij = 1/Ni.
Bin concat maps edge weights into T equally divided buckets and concatenates each learned bucket representation with the neighbor embedding before an MLP.The experiment searches T over.
Edge weight concat scales d · wij to the node-feature dimension before concatenating it with the neighbor representation, preserving original edge information.Here d equals the node-feature dimension.
Node edge concat combines central-node, neighbor-node, and edge representations, reinforcing the previous central-node embedding and potentially alleviating over-smoothing.
Node concat retains the node representations in the message design while omitting edge weights.

C. Attention-Enhanced Message Passing

Attention-enhanced message passing adds learned neighbor-specific weights to brain-network aggregation, with variants that either exclude, multiply, or sum connection weights.

Attention updates brain-region representations in a data-driven way by adding adjustable neighbor weights alongside edge-weight signals.The paper emphasizes that conventional graph attention often omits edge attributes, although brain-network connection weights carry biomedical information.
Attention weighted applies the original GAT mechanism without incorporating edge attributes.
Edge weighted w/ attn multiplies the neighbor representation by both the connection weight wij and attention score αij.
Attention edge sum adds wij and αij before using their sum as the impact factor on the neighbor embedding.
Node edge concat w/ attn multiplies the neighbor embedding by αij before concatenating it with node and edge representations and applying an MLP.
Node concat w/ attn similarly applies αij to the neighbor representation before concatenation, but uses the node-concat design.

D. Pooling Strategies

Pooling converts node embeddings into permutation-invariant graph representations, with BrainGB comparing mean, sum, concat, and DiffPool readouts.

A pooling strategy R computes a feature vector for the whole graph from its node vectors.
Pooling is required to be invariant to permutations of the node vectors.
Mean pooling averages node features to form the graph-level representation.
Sum pooling adds all node features to form the graph-level representation.
Concat pooling forms the graph-level representation by concatenating the features of every node.
BrainGB includes DiffPool to compare basic pooling operators with a more complex pooling method.

V. EXPERIMENTAL ANALYSIS AND INSIGHTS

The experiments evaluate 375 GNN architectures formed by varying design dimensions across four modules, while acknowledging that the design space can expand with new dimensions.

375 architectures are evaluated by varying design dimensions across the four modular components.The study does not aim to cover every possible combination, but to identify a relatively effective design.

A. Experimental Settings

The benchmark combines four neuroimaging datasets spanning fMRI and dMRI modalities, diverse atlases, cohorts, and prediction tasks, with shallow and deep baselines evaluated under standardized settings.

Four datasets cover fMRI studies HIV, PNC, and ABCD, plus the dMRI PPMI study.The datasets use different brain atlases and preprocessing procedures.
HIV and PPMI classify patients versus healthy controls, whereas PNC and ABCD predict gender.These tasks provide disease-related and demographic prediction settings.
The benchmark compares modular GNN designs with shallow methods M2E, MPCA, and MK-SVM, alongside BrainGNN and BrainNetCNN.Shallow graph-level embeddings are evaluated with logistic regression classifiers.
Models are trained with Adam for 20 epochs using a 1e-3 learning rate, 1e-4 weight decay, cross-entropy loss, and automatically selected hyperparameters.Implementation uses PyTorch and PyTorch Geometric, with NNI for automated hyperparameter selection.

3) Implementation Details:

Across module-specific experiments, connection-profile node features and concat pooling perform strongly, attention generally improves message passing, and the best modular design outperforms BrainGNN across datasets.

1) Node Feature: Connection-profile node features achieve the best performance across all datasets, improving up to 33.99% over degree on ABCD.Structure-based features generally outperform positional features.
2) Message Passing: Node concat message passing has the highest AUC across four datasets, while node edge concat has sometimes slightly better accuracy and F1 scores.The authors associate these methods with reinforcing the central node’s self-representation during message passing.
3) Attention Enhanced Message Passing: Attention-enhanced message passing improves outcomes most of the time, with up to 5.23% relative improvement over corresponding non-attention architectures.Node concat with attention and node edge concat with attention alternate as the strongest variants.
4) Pooling Strategies: Concat pooling consistently outperforms the other two pooling methods across all four datasets.It retains final representations from all brain regions, whereas the alternatives produce lower-dimensional graph embeddings.
5) Other Baselines: The best modular design outperforms BrainGNN on HIV, PNC, PPMI, and ABCD, while matching BrainNetCNN in most cases.GNNs also exceed shallow models by over 5 percent on HIV and about 2 percent on ABCD.

5) Other Baselines:

The results indicate that network sparsity influences GNN hyperparameter choices, while the benchmark’s broader paradigm remains limited by unresolved brain-network structures and small neuroimaging datasets.

PPMI structural networks contain approximately 22.64% of possible edges on average, whereas functional ABCD networks are fully connected.Functional and structural brain networks therefore differ substantially in sparsity.
PPMI reaches maximum performance with hidden dimension 64, while ABCD performs best with hidden dimension 256.The authors relate denser inputs to a need for more learnable parameters.
BrainGB provides an installable package, tutorials, examples, trained models, and preprocessing instructions for functional and structural network construction.The resources are released through the BrainGB website and open-source repository.
The effective graph structures beyond pairwise connections, including communities and subgraphs, remain unknown for brain networks.This is identified as a limitation of current GNN aggregation mechanisms in this setting.
Small neuroimaging datasets may limit the effectiveness and generalization ability of complex deep learning models.The paper proposes neurology-driven designs and pre-training or transfer learning as future directions.