PaperlayerView exact revision on arXiv

Source-linked AI summary

LGGNet: Learning from Local-Global-Graph Representations for Brain-Computer Interface

Yi Ding, Neethu Robinson, Chengxuan Tong, Qiuhao Zeng, Cuntai Guan

arXiv:2105.02786v3cs.NEcs.LGeess.SPq-bio.NC

TL;DR

LGGNet addresses the challenge of modeling EEG activity within and among brain functional areas for BCI cognitive classification. It combines multi-scale temporal convolutions with neurophysiologically defined local-global graph filtering, and under nested cross-validation achieves significantly higher accuracies and F1 scores than other methods in most experiments.

  • Problem

    Existing EEG methods do not effectively model both localized activity within functional areas and complex relations among those areas.

  • Method

    LGGNet combines multi-scale 1D temporal convolutions and attentive fusion with local and global graph filtering based on neurophysiologically meaningful EEG graphs.

  • Results

    LGGNet achieves significantly higher accuracies and F1 scores than other methods in most experiments across four cognitive tasks and three public datasets.

  • Takeaways & Limitations

    The results support incorporating neuropsychological knowledge into neural-network design for task-specific EEG neural activations.

Abstract

from arXiv · show

Neuropsychological studies suggest that co-operative activities among different brain functional areas drive high-level cognitive processes. To learn the brain activities within and among different functional areas of the brain, we propose LGGNet, a novel neurologically inspired graph neural network, to learn local-global-graph representations of electroencephalography (EEG) for Brain-Computer Interface (BCI). The input layer of LGGNet comprises a series of temporal convolutions with multi-scale 1D convolutional kernels and kernel-level attentive fusion. It captures temporal dynamics of EEG which then serves as input to the proposed local and global graph-filtering layers. Using a defined neurophysiologically meaningful set of local and global graphs, LGGNet models the complex relations within and among functional areas of the brain. Under the robust nested cross-validation settings, the proposed method is evaluated on three publicly available datasets for four types of cognitive classification tasks, namely, the attention, fatigue, emotion, and preference classification tasks. LGGNet is compared with state-of-the-art methods, such as DeepConvNet, EEGNet, R2G-STNN, TSception, RGNN, AMCNN-DGCN, HRNN and GraphNet. The results show that LGGNet outperforms these methods, and the improvements are statistically significant (p<0.05) in most cases. The results show that bringing neuroscience prior knowledge into neural network design yields an improvement of classification performance. The source code can be found at https://github.com/yi-ding-cs/LGG

I. INTRODUCTION

LGGNet addresses limitations in EEG spatial modeling by combining temporal convolutions with neurophysiologically defined local-global graphs. It evaluates this design across four cognitive BCI tasks and compares it with established deep-learning and graph-based methods.

  • EEG-based BCI supports direct brain–machine communication and applications including robot control, stroke rehabilitation, and emotion regulation.
  • Prior EEG methods learned temporal information effectively but often modeled global or local spatial information separately, limiting spatial representation.
  • Neuropsychological evidence motivates modeling both activity within functional areas and complex relations among those areas.
  • LGGNet combines multi-scale temporal convolutions and attentive fusion with local and global graph-filtering layers for EEG representation learning.
  • Its local-global graphs define fully connected local regions and global connections among functional areas, with general, frontal, and hemisphere variants.
  • The method is evaluated against eight state-of-the-art baselines on three public datasets covering attention, fatigue, emotion, and preference classification.

B. Graph Neural Networks

LGGNet combines temporal convolutions and graph learning to represent EEG dynamics and brain-structured relations. Its temporal block uses multi-scale kernels whose outputs are concatenated before further processing.

  • B. Graph Neural Networks: Graph neural networks generate node representations by aggregating node features with neighboring features through graph structure.The graph representation uses nodes, edges, node attributes, and an adjacency matrix.
  • B. Graph Neural Networks: LGGNet contains temporal-learning and graph-learning blocks for processing EEG representations.The temporal block includes temporal convolution and kernel-level attentive fusion; the graph block includes local and global graph-filtering layers.
  • B. Graph Neural Networks: Multi-scale 1D temporal kernels operate in parallel to learn dynamic temporal and frequency representations.Kernel lengths use different ratios of the sampling rate, with three kernel levels used in the reported study.
  • B. Graph Neural Networks: The temporal convolution applies squaring, average pooling, and logarithmic activation to filtered EEG signals to learn power features.The convolution acts as a digital filter, while pooling estimates averaged power over shorter segments.
  • B. Graph Neural Networks: Outputs from all temporal-kernel levels are concatenated along the feature dimension before kernel-level fusion.The concatenation produces a multi-scale representation for subsequent processing.

2) Kernel-level attentive fusion:

The kernel-level attentive fusion module combines representations learned by different temporal kernels into a downsampled fused representation used to build EEG-graph node attributes.

  • 2) Kernel-level attentive fusion:: A one-by-one convolution fuses features learned by different temporal kernels after their concatenation.Batch normalization is applied before and after fusion, followed by Leaky ReLU and average pooling.
  • 2) Kernel-level attentive fusion:: The fusion module uses batch normalization, Leaky ReLU activation, and average pooling to transform the fused representation.The average-pooling kernel and step sizes are both (1, 2).
  • 2) Kernel-level attentive fusion:: The resulting fused tensor provides the attributes of EEG-channel nodes in the EEG-graph representation.The fused representation is reshaped for graph learning.

B. Graph Learning Block

The graph-learning block organizes EEG channels into neurophysiologically meaningful local graphs, aggregates activity within each region, and represents relations among regions through graph filtering.

  • 1) Defining local-global graphs of EEG:: LGGNet constructs general, frontal, and hemisphere local-global graphs from neuroscience findings and EEG channel organization.The general graph follows functional areas in the 10-20 system; frontal and hemisphere variants add symmetrical subgraphs.
  • 1) Defining local-global graphs of EEG:: Each electrode is treated as a graph node whose learned dynamic temporal representation serves as its node attribute.The graph representations are built from the fused temporal features.
  • 1) Defining local-global graphs of EEG:: Channels are reordered so electrodes within each local graph are adjacent for local graph-filtering aggregation.This ordering enables aggregation operations to be applied within local regions.
  • B. Graph Learning Block: The local graph-filtering layer attentively aggregates learned electrode representations within each local graph.Its two stages are local graph filtering and local representation aggregation.
  • B. Graph Learning Block: Electrodes within one local graph are fully connected through an all-ones local adjacency matrix.The construction reflects the assumption that electrodes in a subgroup capture similar activity from the corresponding functional area.
  • B. Graph Learning Block: The local aggregation function produces hidden embeddings for the local graphs from their filtered node representations.LGGNet selects average aggregation, while the formulation allows functions such as maximum, minimum, or average.

3) Global graph-filtering layer:

LGGNet learns relations among local brain graphs through a global graph whose connections are based on local-graph similarity and refined by trainable attention. The resulting global representation is normalized, filtered with a GCN, and transformed into the final classification output.

  • Global graph construction: Global graph edges represent relations among local graphs, using dot products between their representations for each EEG instance.The construction is motivated by evidence that activating one brain region can accompany activation of other regions involved in high-level cognition.
  • Global graph construction: A symmetric trainable attentive mask emphasizes important connections in the instance-level similarity adjacency matrix.The mask remains symmetric because the global graph is undirected.
  • Global graph filtering: The final global adjacency matrix applies the masked similarity graph, ReLU, and identity self-loops before normalized global graph filtering.Self-loops are added after masking to preserve their strength, while ReLU makes adjacency values non-negative.
  • Global graph filtering: One global GCN layer filters aggregated local-graph embeddings, after which batch normalization, flattening, and a linear layer produce predictions.The output bias has one entry per class, with two classes used in this paper.

A. Datasets

LGGNet is evaluated on three publicly available EEG datasets spanning attention, fatigue, emotion, and preference classification. The datasets differ in task and acquisition setting, including sustained attention driving and affective-state measurement.

  • Dataset-task mapping: Three datasets support four tasks: attention, fatigue, emotion, and preference classification.The attention dataset supports attention classification, the fatigue dataset supports fatigue classification, and DEAP supports emotion and preference classification.
  • Attention dataset: The attention dataset includes 26 subjects and uses a discrimination/selection response task for cognitive attention classification.Each session contains 40-second attention periods and 20-second rest periods.
  • Fatigue dataset: The fatigue dataset records 32-channel EEG during a 90-minute virtual-reality driving task involving induced lane-departure events.Twenty-seven subjects participated, and signals were sampled at 500 Hz.

B. Pre-processing

Pre-processing converts EEG recordings into neural-network inputs through dataset-specific filtering, artifact handling, resampling, segmentation, and labeling. Trial-wise splitting and nested cross-validation are used to limit leakage and reduce evaluation bias.

  • General preprocessing: Pre-processed EEG signals, rather than hand-crafted features, are used as neural-network inputs.The datasets undergo preprocessing before entering the models.
  • Dataset-specific preprocessing: Attention data are filtered from 0.5–50 Hz, cleaned with ICA-based EOG removal, downsampled to 200 Hz, and segmented into overlapping 4-second trials.Only the first half of each attention trial is retained to balance attention and rest samples.
  • Dataset-specific preprocessing: Fatigue trials use preprocessed EEG with additional 1–50 Hz filtering, artifact removal, downsampling to 128 Hz, and 3-second windows before lane-departure events.Fatigue labels are derived from local and global reaction-time thresholds.
  • Evaluation split: Trial-wise shuffling keeps adjacent segments from the same trial out of both training and test folds, while nested cross-validation uses outer subject-specific folds and inner 3-fold selection.The outer fold counts are 10 for DEAP and fatigue and 6 for attention.

D. Two-stage Training

The study trains and evaluates LGGNet under standardized settings shared with baseline methods, using two-stage optimization within nested cross-validation. Results are reported with accuracy, F1 score, and statistical testing, with LGGNet-G strongest in most experiments.

  • Two-stage training: Two-stage training selects models through inner-fold validation and then trains using the available training data.The first stage identifies the best-performing candidate across inner folds before subsequent training.
  • Optimization settings: Training uses PyTorch, cross-entropy loss, Adam optimization, dropout of 0.5, and early stopping.The first and second stages allow up to 200 and 20 epochs, respectively.
  • Baseline comparison: LGGNet is compared with CNN-, RNN-, and GNN-based baselines under generalized evaluation settings designed to avoid data leakage.The baseline set includes DeepConvNet, EEGNet, TSception, R2G-STNN, HRNN, RGNN, AMCNN-DGCN, and GraphNet.
  • Results: LGGNet-G achieves the highest classification results in most experiments, with statistically significant accuracy improvements across all attention-dataset comparisons.The reported accuracy gains range from 5.56% to 13.28% over the listed baselines.

2) Fatigue Classification Task:

On the fatigue dataset, LGGNet-F achieved the best accuracy and F1 score, with most improvements over the compared baselines statistically significant. Ablation results also show that removing temporal fusion or local graph filtering reduced performance.

  • Fatigue classification results: 90.76% ACC was achieved by LGGNet-F for fatigue detection, exceeding the compared baselines by 1.97%–23.9%.Most accuracy improvements were statistically significant; the comparison with RGNN was not significant (p = 0.231).
  • Ablation studies: The ablation study removed kernel-level attentive fusion, local graph filtering, and global graph filtering from LGGNet-H on DEAP preference classification.The study used DEAP because it had more data and subjects than the other datasets.
  • Ablation studies: Removing kernel-level attentive fusion reduced accuracy from 63.07% to 60.95%, a 2.12% decrease.The F1 score decreased by 4.08%, indicating a contribution from attentive fusion.
  • Ablation studies: Removing local graph filtering reduced accuracy from 63.07% to 60.93%, a 2.14% decrease.The F1 score decreased by 3.17%.

3) The Contribution of The Global Graph Filtering:

Removing global graph filtering lowered both accuracy and F1 score more than removing local graph filtering, indicating a larger contribution from global graph filtering in LGGNet. Neuroscience-informed local-global graphs improved classification across most cognitive tasks, while task-specific graph variants differed without significant performance differences in several tasks.

  • Global graph-filtering contribution: Removing global graph filtering caused a 4.01% accuracy drop and a 5.45% F1-score drop.These decreases exceeded those observed after removing local graph filtering.
  • Global graph-filtering contribution: Global graph filtering contributed more to LGGNet performance than local graph filtering in the ablation study.
  • Effects of local-global graphs: Neuroscience-informed local-global graphs significantly improved classification for all four cognitive tasks, except attention with frontal and hemisphere graphs.For the general graph, accuracy and F1 improvements over the no-local-global-graph baseline were reported for attention, fatigue, emotion, and preference tasks.
  • Graph variants: LGGNet-G achieved the highest attention and emotion results, LGGNet-F the highest fatigue results, and LGGNet-H the highest preference results.Performance differences among variants were not significant for fatigue, emotion, and preference classification.
  • Saliency-map analysis: Saliency analysis identified task-related EEG regions, including temporal and parietal areas for attention and frontal areas for fatigue.Emotion and preference classification also relied strongly on temporal regions, with frontal involvement for emotion.

2) Learned Global Connection Visualization:

LGGNet’s learned adjacency matrices and attentive masks reveal task-related relations and important local brain regions across four cognitive tasks. The authors also note that fully connected local areas may not capture complex within-area activity, motivating further model improvement.

  • Visualization procedure: The visualizations average normalized attentive masks and adjacency matrices across subjects to show learned relations for attention, fatigue, emotion, and preference.Negative attentive-mask values were set to zero before normalization.
  • Attention: Attention emphasizes frontal-parietal connections and higher self-weights in frontal, parietal, and temporal regions.Connections between AF–PO and FC–PO are observed, while CP and T-l receive stronger self-loop attention.
  • Fatigue: Fatigue shows relatively strong connections among frontal, occipital, and motor-related regions, with higher self-weights in frontal, parietal, and occipital areas.Reported cross-region links include Fp-r–O, F-l–CP, Fp-l–C, and C–Z.
  • Emotion: Emotion exhibits more frontal, occipital, and temporal interconnections than attention and preference, with higher self-weights in frontal and temporal regions.The reported links include AF–FC, AF–O, CP–O, and P–T-r.
  • Preference: Preference contains fewer inter-area connections, emphasizes temporal self-weights, and also highlights selected frontal and occipital regions.The highlighted regions include T-l, T-r, AF-r, F-r, FC-r, PO-l, and PO-r.
  • Interpretation and limitation: The authors report that most learned relations are task-related but caution that fully connected local areas may not reflect complex activity within functional regions.They call for further analysis and improved network or loss-function design.
Loading 2105.02786v3…