Source-linked AI summary
A Temporally Augmented Graph Attention Network for Affordance Classification
Ami Chopra, Supriya Bordoloi, Shyamanta M. Hazarika
TL;DR
Affordance-related EEG classification must handle complex spatial interactions and temporally non-uniform neural signals, while existing graph models largely rely on static or implicit temporal aggregation. EEG-tGAT augments GATv2 with temporal attention and temporal dropout for temporally robust spatiotemporal modeling. It achieved 71.55% ± 1.58% mean accuracy, exceeding the reported GATv2 baseline of 69.91% ± 1.40%.
Problem
Affordance-related EEG classification is challenged by non-stationary signals, inter-subject variability, complex spatiotemporal processes, and graph models that treat temporal relevance too uniformly.
Method
EEG-tGAT augments graph attention with temporal attention and temporal dropout while jointly modeling spatial relationships among EEG channels and informative temporal segments.
Results
71.55% ± 1.58% mean classification accuracy was achieved by EEG-tGAT, compared with 69.91% ± 1.40% for GATv2.
Takeaways & Limitations
The reported results support explicitly encoding temporal importance and temporal robustness as inductive biases for affordance-driven EEG data.
Abstract
from arXiv · showhide
Graph attention networks (GATs) provide one of the best frameworks for learning node representations in relational data; but, existing variants such as Graph Attention Network (GAT) mainly operate on static graphs and rely on implicit temporal aggregation when applied to sequential data. In this paper, we introduce Electroencephalography-temporal Graph Attention Network (EEG-tGAT), a temporally augmented formulation of GATv2 that is tailored for affordance classification from interaction sequences. The proposed model incorporates temporal attention to modulate the contribution of different time segments and temporal dropout to regularize learning across temporally correlated observations. The design reflects the assumption that temporal dimensions in affordance data are not semantically uniform and that discriminative information may be unevenly distributed across time. Experimental results on affordance datasets show that EEG-tGAT achieves improved classification performance compared to GATv2. The observed gains helps to conclude that explicitly encoding temporal importance and enforcing temporal robustness introduce inductive biases that are much better aligned with the structure of affordance-driven interaction data. These findings show us that modest architectural changes to graph attention models can help one obtain consistent benefits when temporal relationships play a nontrivial role in the task.
I. INTRODUCTION
Affordance-related EEG classification is difficult because neural signals are non-stationary, variable across subjects, and shaped by complex spatiotemporal processes. Existing methods incompletely capture distributed spatial relationships and non-uniform temporal relevance, motivating temporally enhanced graph attention.
- Affordance-related EEG classification involves non-stationary signals, inter-subject variability, and complex processes spanning perception, motor imagery, and action execution.
- Handcrafted EEG pipelines depend on expert-defined frequency bands, channel heuristics, and task-specific assumptions, limiting generalization across subjects, sessions, and conditions.
- CNN-based models learn temporal and spatial filters but assume grid-like structure, restricting explicit representation of non-Euclidean functional interactions between electrodes.
- Temporal attention weights different EEG segments, but existing models can retain constrained spatial relationships through predefined graphs or local convolutions.
- Existing GAT-based EEG models often use recurrent modules or fixed aggregation, implicitly treating time as uniformly relevant despite evolving affordance phases.
- The proposed temporally enhanced GAT explicitly combines temporal attention and temporal dropout for affordance-related EEG classification.
II. METHODOLOGY
The methodology represents EEG channels as graph nodes and learns attention-weighted spatial interactions to capture distributed cortical dynamics. Temporal convolution, temporal self-attention, and graph attention are integrated to decode affordance-related activity.
- EEG channels are represented as graph nodes so learned attention can model adaptive spatial dependencies between channels.
- The EEG-GAT framework integrates temporal convolution, temporal self-attention, and graph attention to model coordinated neural dynamics.
A. Experimental Design and Data Acquisition
EEG data were collected during an affordance-driven Action Observation and Motor Imagery paradigm designed to evoke coordinated perceptual and motor cortical activity.
- The experiment used an affordance-driven Action Observation and Motor Imagery paradigm to elicit coordinated perceptual and motor cortical responses.
B. EEG Data Preprocessing
The preprocessing pipeline used MNE-Python and standardized EEG recordings before task-epoch extraction. It included metadata cleanup, filtering, and resampling across subjects and trials.
- Raw EEG recordings were loaded with MNE-Python after removing non-essential metadata to support consistent parsing and stable data loading.
- A 50 Hz notch filter was applied to suppress power-line interference before subsequent signal processing.
- Signals were resampled at 256 Hz to establish uniform temporal resolution across subjects and trials.
C. Graph-based EEG Representation
The model represents each one-second EEG segment as a graph whose nodes are EEG channels and whose node features contain channel-specific temporal signals. Temporal convolutions extract multiscale features before unified graph learning combines temporal and spatial representations.
- Graph construction: Each one-second EEG segment is represented as a graph with EEG channels as nodes and their corresponding temporal signals as node features.The graph is fully connected and undirected, allowing channel interactions within each segment.
- Graph construction: The one-second segments are treated as independent graph samples during training, increasing the effective number of training examples.
- Temporal feature extraction: Three successive temporal convolutional layers with kernels 1 x 128, 1 x 64 and 1 x 32 process channel signals at multiple temporal resolutions.The multiscale processing is designed to capture low- and high-frequency neural oscillations relevant to motor imagery and related activity.
E. Graph Attention-Based Spatial Modeling
GATv2 layers model adaptive spatial dependencies between EEG channels, and global mean pooling converts channel-level representations into a segment-level graph embedding for classification.
- Graph attention: Channel-wise vectors from temporal processing are fed into two successive GATv2 layers to model spatial dependencies between EEG channels.The first layer uses multi-head attention with feature concatenation, while the second uses a single head for compact node representations.
- Graph readout: Global mean pooling across EEG channels produces a graph-level embedding that summarizes distributed cortical activity for each segment.The resulting embedding is passed to a multilayer perceptron classifier.
G. Model Training and Evaluation
Training uses AdamW, label-smoothed cross-entropy, mini-batch optimization, early stopping, and learning-rate reduction on plateau. Evaluation uses five-fold cross-validation with multiple classification metrics.
- Training: The model is trained with AdamW using an initial learning rate of 3 × 10^-4 and weight decay of 1 × 10^-3.Cross-entropy with label smoothing, mini-batch stochastic optimization, and early stopping based on validation loss are used during training.
- Training: A Reduce-on-Plateau scheduler adaptively reduces the learning rate during training.
- Evaluation: Five-fold cross-validation uses exclusive training and test sets in each fold, with model selection by early stopping on validation loss.
- Evaluation: Performance is reported with accuracy, precision, recall, F1-score, and Cohen’s Kappa.
- Evaluation: A full ablation study was not performed because temporal attention and temporal dropout are closely integrated and retraining is costly.The evaluation instead emphasizes overall effectiveness, stability, complementary quantitative results, and visual analyses.
A. Performance Analysis
Across five folds, the model shows stable and balanced classification performance and exceeds the reported baseline accuracies. Ablation results indicate performance declines when temporal dropout or temporal attention is removed.
- Cross-fold performance: 71.55% ± 1.58% average accuracy was achieved across five folds, with closely matched precision, recall, and F1-score.The corresponding values were 71.61% ± 1.60% precision, 71.52% ± 1.55% recall, and 71.51% ± 1.56% F1-score.
- Cross-fold performance: The close correspondence between precision and recall indicates balanced class-wise performance and consistent behavior across folds.
- Baseline comparison: 71.55% ± 1.58% accuracy outperformed EEGNet at 54.88% ± 1.04%, CNN-LSTM at 70.36% ± 5.46%, and spatial GATv2 at 69.91% ± 1.40%.
- Ablation study: 70.53% ± 2.58% accuracy followed removal of temporal attention, while removing both temporal components yielded 69.91% ± 2.49%.The reported results describe complementary contributions from temporal attention and temporal dropout.
D. Confusion Matrix Analysis
Across five folds, confusion matrices show balanced errors between the two classes, while higher-accuracy folds exhibit less off-diagonal confusion but persistent overlap.
- Confusion matrices across all five folds show true positives and true negatives dominating the diagonals, with false positives and negatives fairly balanced.
- Similar precision and recall indicate that predictions are not particularly biased toward either class.
- Higher-accuracy folds, such as Fold 5, show less off-diagonal confusion, suggesting better temporal and relational discrimination.
- Misclassification remains non-negligible across folds, reflecting overlap between classes and the difficulty of noisy, temporally distributed EEG affordance signals.
E. Stability Analysis of Cross Fold Stability
Accuracy remains stable across folds, with limited variation indicated by the distribution's small interquartile range and lack of extreme outliers.
- A box plot of classification accuracy across five folds shows a small interquartile range and no extreme outliers.
- The results are described as consistent and reproducible gains across folds, supporting the effectiveness of temporally informed graph attention for affordance classification.
IV. DISCUSSION
The discussion attributes EEG-tGAT's gains to explicitly modeling spatiotemporal EEG dynamics rather than treating time as noise. Five-fold evaluation reports stable performance and supports temporally augmented graph attention over spatial-only variants.
- EEG-tGAT explicitly handles spatiotemporal dynamics by augmenting graph attention with temporal modeling for affordance-related EEG signals.
- Treating time as meaningful directs learning toward stable neural correlates of affordance-related cognition rather than superficial correlations.
- 71.55% ± 1.58% mean classification accuracy was achieved in five-fold cross-validation, with closely matched precision, recall, and F1-score.
- The conclusion reports more robust and discriminative representations than spatial-only GAT variants while maintaining training stability.