Source-linked AI summary
PGCN: Pyramidal Graph Convolutional Network for EEG Emotion Recognition
Ming Jin, Enwei Zhu, Changde Du, Huiguang He, Jinpeng Li
TL;DR
EEG emotion-recognition GCNs may underuse long-range electrode dependencies, while shallow networks have limited receptive fields. PGCN aggregates features at local, mesoscopic, and global scales using electrode relationships and prior-informed brain regions. The authors report the best results on all three public datasets and describe a balance between expanded receptive fields and over-smoothing.
Problem
EEG emotion-recognition GCNs may underuse electrode relationships, particularly long-range dependencies, while shallow networks have limited receptive fields.
Source: §I · p. 1Method
PGCN aggregates EEG features across local, prior-informed mesoscopic, and global scales, using GCNs and attention to model electrode relationships.
Source: §I · p. 2Results
PGCN achieves the best results on all three publicly available datasets.
Source: §VI · p. 10Takeaways & Limitations
The authors describe PGCN as balancing expanded GCN receptive fields with the consequent over-smoothing problem.
Source: §VI · p. 10
Abstract
from arXiv · showhide
Emotion recognition is essential in the diagnosis and rehabilitation of various mental diseases. In the last decade, electroencephalogram (EEG)-based emotion recognition has been intensively investigated due to its prominative accuracy and reliability, and graph convolutional network (GCN) has become a mainstream model to decode emotions from EEG signals. However, the electrode relationship, especially long-range electrode dependencies across the scalp, may be underutilized by GCNs, although such relationships have been proven to be important in emotion recognition. The small receptive field makes shallow GCNs only aggregate local nodes. On the other hand, stacking too many layers leads to over-smoothing. To solve these problems, we propose the pyramidal graph convolutional network (PGCN), which aggregates features at three levels: local, mesoscopic, and global. First, we construct a vanilla GCN based on the 3D topological relationships of electrodes, which is used to integrate two-order local features; Second, we construct several mesoscopic brain regions based on priori knowledge and employ mesoscopic attention to sequentially calculate the virtual mesoscopic centers to focus on the functional connections of mesoscopic brain regions; Finally, we fuse the node features and their 3D positions to construct a numerical relationship adjacency matrix to integrate structural and functional connections from the global perspective. Experimental results on three public datasets indicate that PGCN enhances the relationship modelling across the scalp and achieves state-of-the-art performance in both subject-dependent and subject-independent scenarios. Meanwhile, PGCN makes an effective trade-off between enhancing network depth and receptive fields while suppressing the ensuing over-smoothing. Our codes are publicly accessible at https://github.com/Jinminbox/PGCN.
I. INTRODUCTION
The introduction identifies gaps in EEG emotion-recognition GCNs, including limited exploration of electrode connectivity and shallow networks that miss long-range dependencies. PGCN addresses these gaps by aggregating local, mesoscopic, and global features using electrode topology and neuroscience-informed regions.
I. INTRODUCTION
- Existing GCN methods explore structural and functional electrode connectivity only to a limited extent, and shallow networks miss long-range dependencies.The introduction also notes that deeper GCNs are prone to over-smoothing.Source: §I · p. 1
- PGCN combines local, mesoscopic, and global aggregation, using spatially based GCNs, prior-informed virtual centers, and attention-based global connectivity.It fuses multiscale features with the original features before a three-layer fully connected classifier.Source: §I · p. 2, Figure 1 · p. 2
- PGCN incorporates absolute, relative, and numerical node relationships and uses neuroscience-informed regions to construct virtual mesoscopic centers.The authors describe the network as progressively extending its perceptual field from local to mesoscopic and global.Source: §I · p. 2
- The authors report state-of-the-art results on three public emotion-recognition datasets.They also state that visualization results demonstrate the method’s effectiveness.Source: §I · p. 2
II. RELATED WORK
Related work covers EEG emotion-recognition approaches and graph convolutional networks. The paper describes graph convolution as a way to model relationships among nodes and aggregate their features.
A. EEG Emotion Recognition
- EEG emotion-recognition methods include signal preprocessing and feature extraction, traditional machine learning, and deep learning models that use spatial, temporal, or frequency information.Examples include CNNs, RNNs, and recurrent models with attention.Source: A. EEG Emotion Recognition · pp. 2–3
B. Graph Convolutional Networks
- A graph represents nodes and their edges with an adjacency matrix, while a feature matrix collects node features; the simplified GCN formulation uses trainable weights.The paper presents the simplified GCN as a reduction of the more complex spectral formulation.Source: B. Graph Convolutional Networks · p. 3
C. GCN in EEG emotion recognition
The paper situates EEG GCNs among methods that learn electrode relationships and draws on brain-network findings about local structure and long-range connections. PGCN’s overview combines those perspectives in a local-to-global architecture.
C. GCN in EEG emotion recognition
- Prior EEG GCNs learn electrode relationships through dynamic adjacency, regularization, local and global filtering, or self-attention-based updates.The cited methods address discriminative features, cross-subject variation, brain-region activity, or learnable electrode relationships.Source: C. GCN in EEG emotion recognition · p. 3
D. Emotion and Human Brain
- Brain-network research describes small-world structure, modular brain regions with core nodes, and long-range connections that can support faster, more direct, and less noisy information transport.The cited findings include core nodes mainly distributed in frontal, temporal, and occipital lobes.Source: D. Emotion and Human Brain · p. 3
A. Overview
- PGCN’s overview combines spatial-distance-based local aggregation, prior-informed mesoscopic centers, and attention-based sparse global connectivity before final classification.The fused features are passed to a three-layer fully connected network.Source: A. Overview · p. 3, Figure 2 · p. 4
B. Local Feature Aggregation
The local module aggregates neighboring electrode features with a two-layer GCN and fuses them with original features. Its graph construction uses spatial relationships while addressing limitations of binary, fixed, sparse adjacency.
B. Local Feature Aggregation
- The local module builds sparse graph relations from relative electrode positions, applies a two-layer GCN, and fuses the output with original features to suppress over-smoothing.The adjacency construction aims to retain sparsity while reflecting local connections.Source: B. Local Feature Aggregation · p. 4
1) Sparse Graph Relation Reasoning:
- The authors identify three problems with fixed adjacency: uneven electrode distribution makes simple 0/1 relationships inaccurate, over-sparsity can lose critical connections, and cross-subject or cross-session differences cause under-optimization.They construct an initial adjacency from inverse-square 3D spatial distance, with δ=9 giving the best result in their experiment; A is learnable and updated through gradient backpropagation.Source: 1) Sparse Graph Relation Reasoning · p. 4
2) GCN for Local Representation Aggregation:
The local GCN aggregates neighboring electrode features while limiting depth to reduce over-smoothing. The mesoscopic stage uses prior-informed brain-region divisions to expand the perceptual field through virtual centers.
2) GCN for Local Representation Aggregation:
- A two-layer local GCN transports information between neighboring nodes, with cross-layer connections introduced to prevent over-smoothing.The layer representations are combined with the original input features for the local output.Source: 2) GCN for Local Representation… · p. 4
C. Mesoscopic Feature Aggregation
- Virtual mesoscopic centers let regional features converge without node-to-node propagation, increasing the perceptual field while avoiding over-smoothing.The regions are constructed with reference to prior brain-science knowledge.Source: C. Mesoscopic Feature Aggregation · p. 5
1) Graph Coarsen for Mesoscopic Regions:
- The first mesoscopic partition follows cortical anatomy, subdividing temporal electrodes in more detail and adjusting regions for structural and functional connectivity.The cortex is divided into frontal, parietal, temporal, and occipital lobes; the temporal electrodes named are FT7, T7, TP7, FT8, T8, and TP8.Source: 1) Graph Coarsen for Mesoscopic Regions · p. 5
- A second mesoscopic partition groups regions according to the brain’s two hemispheres.The passage notes that the hemispheres have different functions and interact through the corpus callosum.Source: 1) Graph Coarsen for Mesoscopic Regions · p. 5
2) Mesoscopic Node Relation:
Within each mesoscopic region, self-attention models functional connectivity among node features. Those attention weights determine virtual-center features and positions, which are then fused with the original electrode nodes.
2) Mesoscopic Node Relation:
- Self-attention over the features within each mesoscopic region is used to focus on functional connectivity.The regional input is a set of node features, with N nodes and F features per node.Source: 2) Mesoscopic Node Relation · p. 5
3) Virtual Mesoscopic Center:
- Row-summed attention weights represent each node’s functional connection weight to all others and are used to calculate virtual-center features and locations.The electrode position matrix supplies the absolute positions used in calculating the centers.Source: 3) Virtual Mesoscopic Center · p. 5
- The virtual mesoscopic centers are imputed alongside the original electrode nodes to produce fused features and locations containing local and mesoscopic attributes.The two regional divisions yield center matrices with 7 and 2 rows, respectively.Source: 3) Virtual Mesoscopic Center · p. 5
D. Global Feature Aggregation
The global stage addresses the limited reach of mesoscopic partitions by constructing attention-based relationships among nodes using features and absolute positions. A global GCN output is concatenated with mesoscopic features for emotion recognition.
D. Global Feature Aggregation
- Expanding mesoscopic partitions can increase the perceptual range but risks information loss as the number of nodes grows.Source: D. Global Feature Aggregation · p. 5
- An attention-based, absolute-position-dependent adjacency matrix models long-distance node correlations and removes weak connections before global graph convolution.Source: D. Global Feature Aggregation · p. 5
1) Global Node Relation:
- The global relation stage adds spatial position encodings to original and virtual-node features, uses six-head attention, and retains the top 20% of connections to keep the adjacency sparse.A learnable weight vector fuses the attention heads into the global relation matrix.Source: 1) Global Node Relation · pp. 5–6
2) GCN for Global Representation Aggregation:
- Global graph convolution operates on the attention-based adjacency, and its output is concatenated with mesoscopic features before a three-layer emotion-recognition network.Source: 2) GCN for Global Representation… · p. 6
IV. EXPERIMENTS
Across three datasets, the experiments evaluate PGCN in subject-dependent and subject-independent settings. Results vary by dataset and comparison, with strong accuracy and improvements over several baselines.
A. Datasets and Protocol
- Experiments used SEED, SEED-IV, and SEED-V, with subject-dependent training and testing on the same person and subject-independent training and testing across people.The reported baseline results were extracted from cited references.Source: §IV · p. 6, A. Datasets and Protocol · p. 6
B. Experiment on SEED
- 96.93% accuracy was reported for subject-dependent SEED emotion recognition, with PGCN best on all-frequency, theta, beta, and gamma bands but marginally behind V-IAG on delta and alpha.Source: B. Experiment on SEED · p. 6
- 2.67%: PGCN led RGNN without its domain-adaptive module in subject-independent SEED; RGNN with that module was 0.71% ahead of PGCN.The comparison notes that transfer learning uses unlabeled test-set data for training, which may be unavailable for practical online recognition.Source: B. Experiment on SEED · p. 6
C. Experiment on SEED-IV
- 2.87%: PGCN accuracy exceeded RGNN in subject-dependent SEED-IV under the same setting.More than two-thirds of subjects exceeded 80% accuracy, and one-third exceeded 90%.Source: C. Experiment on SEED-IV · p. 6
- Over 3%: PGCN improved on all subject-independent SEED-IV baselines except RGNN; it led RGNN without domain adaptation by 2.04% and trailed RGNN with it by 0.15%.Source: C. Experiment on SEED-IV · p. 6
D. Experiment on SEED-V
On SEED-V, PGCN exceeds the previous best MD-AGCN by 0.92% in subject-dependent recognition and reaches 61.78% accuracy in subject-independent recognition. Ablations report gains from the local, mesoscopic, and global modules, with mesoscopic regions adding intermediate perceptual fields.
D. Experiment on SEED-V
- PGCN improves 0.92% over MD-AGCN on subject-dependent SEED-V recognition and reaches 61.78% accuracy on the subject-independent task.Table IV covers subject-dependent and subject-independent classification accuracy on SEED-V.Source: Table IV · p. 7, D. Experiment on SEED-V · p. 7
A. Ablation Study
- The three modules raise accuracy from 92.34% to 96.93% on SEED and from 75.94% to 82.24% on SEED-IV.The authors report that each module appears to contribute positively.Source: A. Ablation Study · p. 8
- Reported Shapley values are about 2% for local, 3%–3.5% for mesoscopic, and 1% and 2% for global modules on SEED and SEED-IV, respectively.Source: A. Ablation Study · p. 8
- The mesoscopic module improves performance by 2.57% on SEED and 4.77% on SEED-IV compared with the baseline.The authors speculate that it extracts discriminative mesoscopic features using prior knowledge and fuses them with the backbone output.Source: A. Ablation Study · p. 8
- Adding the global module atop the local module improves recognition by 0.39% on SEED but brings no improvement on SEED-IV.The authors hypothesize that deeper GCNs can suffer severe over-smoothing.Source: A. Ablation Study · p. 8
- Local-plus-global stacking can stagnate or reduce fitting ability, while adding 7-region and 2-region mesoscopic modules improves fitting ability.The mesoscopic modules add intermediate perceptual fields and virtual centers.Source: A. Ablation Study · p. 8
B. Why PGCN works? Network Architecture Analysis
PGCN retains accuracy as its receptive field expands, while vanilla GCN performance declines as layers are added. The analysis links this difference to how each architecture balances feature aggregation against over-smoothing.
B. Why PGCN works? Network Architecture Analysis
- PGCN node smoothness rises slightly at the local layer, stays nearly constant or decreases at the mesoscopic layer, then rapidly increases at the global layer.Node smoothness is the mean cosine similarity between output-feature nodes.Source: B. Why PGCN works? Network… · p. 8, Figure 5 · p. 8
- 96.93% at five layers was PGCN’s accuracy peak, compared with vanilla GCN’s 92.34% peak at two layers.PGCN accuracy then began to decline as layers continued to increase.Source: B. Why PGCN works? Network… · p. 8, Figure 6 · p. 9
- PGCN outperforms vanilla GCN by 1.2% on shallow networks (layer ≤2), while sparse, prior-guided mesoscopic aggregation helps extend the receptive field without excessive feature similarity.As layers continue to stack, over-smoothing eventually prevails and PGCN performance declines.Source: B. Why PGCN works? Network… · pp. 8–9
C. Representation Visualization
Learned adjacency weights and connections often emphasize frontal, temporal, and occipital electrodes, though connectivity varies across subjects. PGCN uses local, mesoscopic, and global modules and achieves the best results on three public datasets.
C. Representation Visualization
- Across SEED and SEED-IV, learned node weights are greater in frontal, temporal, and occipital lobes, with stronger frontal activation reported for SEED-IV.The paper notes that this dataset difference may reflect inductive bias between three- and four-class tasks.Source: C. Representation Visualization · p. 9, Figure 7 · p. 10
- Top learned connections vary considerably across subjects, but most involve electrodes in frontal, temporal, and occipital lobes across both datasets.Figure 8 omits diagonal elements when displaying the top-10 connections.Source: C. Representation Visualization · p. 9, Figure 8 · p. 10
- Despite subject and dataset variability, frontal, temporal, and occipital nodes have greater self-loop weights and strong connections to other nodes.The authors associate these learned patterns with the network’s emotion-recognition performance.Source: C. Representation Visualization · p. 10
VI. CONCLUSION
- PGCN achieves the best results on all three publicly available datasets using separate local, mesoscopic, and global modules.The modules target small-world properties, prior-defined brain-region connections, and sparse global connections, respectively.Source: §VI · p. 10
APPENDIX A
The appendix reports implementation settings and dataset-specific subject-dependent and subject-independent evaluation protocols for SEED, SEED-IV, and SEED-V.
APPENDIX A
- PGCN was implemented in PyTorch on a 2080TI GPU, and performance was evaluated using average accuracy and standard deviation.Training used AdamW, warm-up, a 1e-2 learning rate, and batch size 64.Source: Appendix A · p. 12
1) SEED Dataset:
- SEED contains 15 subjects and three emotion classes; subject-dependent evaluation trained on nine trials and tested on six, while subject-independent evaluation used leave-one-subject-out cross-validation.The subject-dependent protocol calculated mean and standard deviation across subjects on two sessions.Source: 1) SEED Dataset · p. 12
2) SEED-IV Dataset:
- SEED-IV contains 15 subjects and four emotion classes; subject-dependent evaluation trained on 16 trials and tested on the last two trials per emotion, while subject-independent evaluation left one subject out.Average accuracy was calculated across subjects over three sessions.Source: 2) SEED-IV Dataset · p. 12
3) SEED-V Dataset:
- SEED-V contains 16 subjects and five emotion classes; subject-dependent evaluation used the first ten trials for training and last five for testing, while subject-independent evaluation used leave-one-subject-out.Accuracy and standard deviation were calculated across subjects over three sessions for both experiments.Source: 3) SEED-V Dataset · p. 12