Source-linked AI summary

EEG-ITNet: An Explainable Inception Temporal Convolutional Network for Motor Imagery Classification

Abbas Salami, Javier Andreu-Perez, Helge Gillmeister

arXiv:2204.06947v1cs.LGcs.AIq-bio.NC

TL;DR

End-to-end EEG models avoid handcrafted feature extraction, but interpreting their learned features remains a challenge. EEG-ITNet combines inception modules and causal dilated convolutions with a feature-visualization method. Across two motor-imagery datasets and three evaluation scenarios, it significantly improves classification in most scenarios relative to existing end-to-end architectures.

  • Problem

    Interpreting learned features remains a challenge for end-to-end EEG architectures, motivating methods that make their patterns more comprehensible.

    Source: §I · p. 2, §V · p. 12
  • Method

    EEG-ITNet combines inception modules and causal dilated convolutions with Fourier-transform and Savitzky-Golay-filtering methods for visualizing learned features.

    Source: 4) Classification block · p. 6, §V · p. 12
  • Results

    Across two motor-imagery datasets and three scenarios, EEG-ITNet significantly improved classification in most scenarios compared with EEG-Inception, EEGNet, and EEG-TCNet.

    Source: §VI · p. 12
  • Takeaways & Limitations

    Visualizations connect learned spectral filters and scalp maps to motor-imagery patterns, while gamma-band activity may reflect attention to the visual cue.

    Source: E. VISUALISATION · pp. 10–11

    Performance may improve with scenario-specific hyperparameter optimization, and visualizing temporal features and dependencies remains future work.

    Source: §VI · p. 12

Abstract

from arXiv · show

In recent years, neural networks and especially deep architectures have received substantial attention for EEG signal analysis in the field of brain-computer interfaces (BCIs). In this ongoing research area, the end-to-end models are more favoured than traditional approaches requiring signal transformation pre-classification. They can eliminate the need for prior information from experts and the extraction of handcrafted features. However, although several deep learning algorithms have been already proposed in the literature, achieving high accuracies for classifying motor movements or mental tasks, they often face a lack of interpretability and therefore are not quite favoured by the neuroscience community. The reasons behind this issue can be the high number of parameters and the sensitivity of deep neural networks to capture tiny yet unrelated discriminative features. We propose an end-to-end deep learning architecture called EEG-ITNet and a more comprehensible method to visualise the network learned patterns. Using inception modules and causal convolutions with dilation, our model can extract rich spectral, spatial, and temporal information from multi-channel EEG signals with less complexity (in terms of the number of trainable parameters) than other existing end-to-end architectures, such as EEG-Inception and EEG-TCNet. By an exhaustive evaluation on dataset 2a from BCI competition IV and OpenBMI motor imagery dataset, EEG-ITNet shows up to 5.9\% improvement in the classification accuracy in different scenarios with statistical significance compared to its competitors. We also comprehensively explain and support the validity of network illustration from a neuroscientific perspective. We have also made our code open at https://github.com/AbbasSalami/EEG-ITNet

I. INTRODUCTION

EEG-ITNet is proposed as an end-to-end motor-imagery classifier intended to combine classification performance with interpretable learned features. The introduction motivates this goal by contrasting end-to-end models with handcrafted pipelines and reviewing existing architectures.

I. INTRODUCTION

  • End-to-end models avoid expert-provided prior information and handcrafted feature extraction, unlike traditional pipelines that transform signals before classification.EEG signals are also described as having low signal-to-noise ratios due to physiological interference, electrode detachment, and volume conduction.Source: §I · pp. 1–2
  • EEG-ITNet targets the interpretability gap by pairing a low-parameter architecture with a methodology for visualizing learned patterns as topographic maps.The authors report classification gains over other end-to-end architectures across different scenarios.Source: §I · p. 2
  • EEG-ITNet replaces conventional convolutions with inception modules and depthwise causal convolutions with dilation to seek higher performance at lower complexity.The authors also propose transforming the model into intuitive topographic visualizations.Source: §I · p. 2

II. RELATED WORK

  • Earlier EEG architectures trade compactness, interpretability, and performance differently: EEGNet is compact, while EEG-Inception and MI-EEGNET have interpretability shortcomings.The cited discussion says EEGNet's visualizations were not very intuitive, EEG-Inception lacked interpretability, and MI-EEGNET's parameter count made interpretation challenging.Source: §II · p. 2

III. METHOD

The method combines parallel multiscale convolutions for frequency sub-bands with causal dilated convolutions for temporal context. Residual blocks extend temporal coverage while retaining causal sequence processing.

1) Inception:

  • Parallel convolutional layers with varied filter sizes split multichannel EEG into informative frequency sub-bands, avoiding a single fixed-length kernel.The inception approach is presented as useful for small training datasets or limited computational capacity, where deeper, more complex networks may overfit or exceed resources.Source: 1) Inception · p. 3

2) Dilation:

  • Causal convolutions make each time-step output depend only on earlier input time steps, while dilation expands the receptive field and leading zero padding preserves sequence length.The receptive field depends on layer count, dilation base, and kernel size.Source: 2) Dilation · pp. 3–4
  • The temporal convolutional network uses residual blocks with increasing dilation rates to extract temporal features over a longer history.The design also uses activation functions and dropout in residual blocks and batch normalization after convolutional layers.Source: 2) Dilation · p. 4

B. PROPOSED DEEP ARCHITECTURE

The proposed architecture begins with parallel inception branches that filter EEG across time and electrodes. Their outputs represent sources in different frequency sub-bands.

B. PROPOSED DEEP ARCHITECTURE

  • EEG-ITNet has four main blocks: inception, temporal convolution, dimension reduction, and classification.These are the blocks shown in the general architecture schematic.Source: B. PROPOSED DEEP ARCHITECTURE · p. 4

1) Inception block:

  • The inception block uses parallel time-axis convolutions for frequency filtering, followed by depthwise convolutions spanning all electrodes for spatial filtering.The depthwise kernel matches the dataset's electrode count to combine electrodes and identify brain-activity sources.Source: 1) Inception block · p. 4
  • Varying kernel sizes let the network learn filters for multiple frequency sub-bands, and the resulting tensor represents signals from sources in those bands.Longer kernels are more likely to learn low-frequency features; nonlinear activation and dropout follow the inception operations.Source: 1) Inception block · p. 4

2) Temporal convolution (TC) block:

EEG-ITNet extracts temporal features separately for each source and frequency sub-band, then reduces the feature dimensions before classification. Its visualization method examines learned frequency filters and spatial sources.

2) Temporal convolution (TC) block:

  • The TC block uses residual depthwise causal convolutions to extract each source’s temporal information separately, with average pooling to reduce dimensions and avoid overfitting.The temporal features incorporate the history of the time series; kernel length, dilation base, and residual-block count can be selected according to the intended history.Source: 2) Temporal convolution (TC) block · pp. 4–5

3) Dimension reduction (DR) block:

  • The DR block combines temporal features from different frequency spectrums with a 1 × 1 convolution and controls the number of features used for classification.It also uses an activation function, dropout, and average pooling to reduce tensor dimensions.Source: 3) Dimension reduction (DR) block · p. 5

4) Classification block:

  • The classification block flattens its input before a fully connected layer with softmax; the block can be modified for different outputs and problems.The architecture figure is captioned as showing details of its different blocks.Source: 4) Classification block · p. 5, Figure 4 · p. 5
  • The inception block applies parallel 2D convolutions over time with varying kernel sizes, acting as frequency filters that extract informative sub-bands.The initial convolutions operate on each electrode’s time series with same padding.Source: 4) Classification block · pp. 5–6
  • Fourier transforms of the learned convolutional kernels identify selected frequency sub-bands, and Savitzky-Golay filtering smooths the transforms for visualization.The paper describes this smoothing as least-squares polynomial approximation over fixed-length windows.Source: 4) Classification block · p. 6
  • The depthwise convolution linearly combines electrode signals to map the EEG from the surface domain to sources, while the mixing matrix—not the unmixing filters—indicates source locations.The convolution is applied separately in each convolutional channel, corresponding to a different frequency sub-band.Source: 4) Classification block · p. 6

A. DATASET AND EVALUATION SCENARIOS

The evaluation uses BCI competition IV dataset 2a and the OpenBMI motor imagery dataset. The paper evaluates within-subject, cross-subject, and cross-subject with fine-tuning scenarios, without using the unlabeled second-session data for training or validation.

A. DATASET AND EVALUATION SCENARIOS

1) BCI competition Ⅳ dataset 2a:

  • Dataset 2a contains four-class motor imagery EEG from nine participants, recorded with 22 electrodes at 250 Hz across two sessions of 288 trials each.Signals were downsampled to 125 Hz, standardized to zero mean and unit variance, and epoched to a three-second window after cue onset; the second session was reserved for competition testing.Source: 1) BCI competition Ⅳ dataset 2a · p. 6

2) OpenBMI motor imagery dataset:

  • The OpenBMI evaluation uses first-session motor imagery recordings from 54-subject dataset, selecting left- and right-hand grasp imagery and 20 motor-cortex electrodes.The selected session has balanced train and test sets of 100 trials each; signals were downsampled to 125 Hz, standardized, and epoched to three seconds after cue onset.Source: 2) OpenBMI motor imagery dataset · p. 7

1) Within-subject:

The section describes three evaluation scenarios and identifies EEGNet as a compact comparison architecture. It also names EEG-TCNet and EEG-Inception among the comparison models.

1) Within-subject:

  • Within-subject evaluation trains and validates a separate network for each subject using that subject’s first-session recordings, then tests on the test session.Results are reported for each subject.Source: 1) Within-subject · p. 7

2) Cross-subject:

  • Cross-subject evaluation trains on other subjects’ data, without using any data from the test subject, and obtains unique parameters for each individual subject.The training subjects are divided into training and validation sets.Source: 2) Cross-subject · p. 7

3) Cross-subject with fine-tuning:

  • In the fine-tuning scenario, each target subject’s labelled training-session recordings fine-tune the system developed for that subject in the cross-subject scenario.Source: 3) Cross-subject with fine-tuning · p. 7

B. COMPARISON MODELS

1) EEGNet:

  • EEGNet is a compact CNN that applies temporal and spatial convolutions followed by a separable convolution.Source: 1) EEGNet · p. 7

1) EEG-TCNet:

This section describes the comparison architectures and EEG-ITNet’s design and training selections. It explains how convolutional filter sizes and extra training epochs were chosen.

1) EEG-TCNet:

  • EEG-TCNet extends EEGNet by adding a temporal convolutional network after its feature-extraction layers.Source: 1) EEG-TCNet · p. 7

3) EEG-Inception:

  • EEG-Inception was designed for ERP-based spellers and comprises two inception modules and an output module; its implementation followed the cited architecture table and matched the data’s sampling frequency after downsampling.Source: 3) EEG-Inception · p. 7

SELECTION

  • EEG-ITNet’s inception block uses 16-, 32-, and 64-sample kernels, with larger filters selected to capture lower frequencies relevant to motor imagery.At 125 Hz, the 64-sample kernel can capture components as low as approximately 2 Hz.Source: SELECTION · p. 7
  • Training used cross-validation with early stopping, followed by up to 50 extra epochs on combined training and validation data; extra epochs were selected by examining loss curves to avoid overfitting.The cross-subject setting used 150 rather than 500 epochs per fold, and patience of 15 rather than 100.Source: SELECTION · pp. 8–9

D. PERFORMANCE EVALUATION

Across the reported evaluation settings, EEG-ITNet outperformed comparison deep models in mean classification accuracy, with statistically significant improvements reported for several comparisons. Fine-tuning and OpenBMI results extend the evaluation beyond the within- and cross-subject settings.

  • Within-subject BCI IV 2a evaluation found EEG-ITNet superior in mean accuracy across subjects, with the highest accuracy for five of nine subjects and second-highest for three.The authors used a one-sided Wilcoxon signed-rank test at a 0.05 significance level to assess improvement.Source: D. PERFORMANCE EVALUATION · p. 9
  • In cross-subject fine-tuning, EEG-ITNet significantly outperformed the other deep models; it improved over within-subject results for 7 of 9 subjects, averaging 2%.Shallow EEGNet improved for 5 of 9 subjects, averaging 0.23%; EEG-Inception improved for 7 of 9, averaging 1.54%.Source: Table IV · p. 9, D. PERFORMANCE EVALUATION · pp. 9–10
  • EEG-ITNet had the highest mean accuracy in the cross-subject BCI IV 2a setting, significantly outperforming EEG-Inception, EEGNet, and EEG-TCNet.Models were trained without data from the target subject; the reported results were lower in this setting than in within-subject evaluation.Source: D. PERFORMANCE EVALUATION · p. 9
  • EEG-ITNet showed superior mean classification accuracy in all evaluated OpenBMI cases, with significance assessed using a right-tailed paired t-test after checking assumptions.The authors report checking normality with the Shapiro–Wilk test.Source: Table V · p. 10, D. PERFORMANCE EVALUATION · p. 10

E. VISUALISATION

The visualisation section interprets EEG-ITNet’s learned spectral and spatial filters in relation to motor imagery and spatial attention. It also identifies possible eye-related patterns and frequency-range caveats.

  • The visualisations show learned spectral and spatial filters from EEG-ITNet’s inception block for one subject in the within-subject BCI IV 2a setting.The authors present the example as illustrating frequency responses and scalp locations of sources that activate across motor-imagery classes.Source: E. VISUALISATION · p. 10, Figure 10 · p. 11
  • Larger filters capture relatively lower frequencies, while smaller filters tend toward higher frequencies; alpha and beta desynchronisation patterns are associated with contralateral hand imagery.The paper links parieto-occipital alpha activity to spatial attention and cautiously associates vertex patterns with imagined foot movement.Source: E. VISUALISATION · pp. 10–11, Figure 10 · p. 11
  • Parieto-occipital alpha activity may reflect covert spatial attention, while increased gamma power may relate to attending to the visual cue during the analysed time window.The authors caution that visualisations above 60 Hz are invalid after downsampling to 125 Hz.Source: E. VISUALISATION · p. 11, Figure 10 · p. 11
  • Frontal low-frequency activations below 5 Hz most likely represent eye blinks or eye movements, which may be captured as discriminative patterns during the tasks.The authors state that eye blinks are unlikely to play a functional role, while eye movements may relate to how subjects attended to and performed the tasks.Source: E. VISUALISATION · p. 11, Figure 10 · p. 11

V. DISCUSSION

The discussion presents EEG-ITNet as a compact, explainable architecture whose dilated residual temporal blocks support classification and whose learned features are visualized and interpreted neuroscientifically. The conclusion reports statistically significant gains in most evaluated scenarios, while identifying further visualization and model-tuning work.

V. DISCUSSION

  • EEG-ITNet combines multiscale inception modules and causal dilated residual blocks to learn EEG patterns and temporal dependencies.The authors describe causal dilated convolutions as an alternative to RNNs and say the residual blocks extract discriminative features.Source: §V · p. 12
  • Fourier transform plus Savitzky-Golay filtering yields smoothed EEG-ITNet source spectra, contrasted with EEGNet’s single-frequency temporal-kernel interpretation.The figure caption gives an EEGNet example of 8 cycles in 0.25 seconds, corresponding to 32 Hz.Source: §V · p. 12, Figure 11 · p. 12
  • EEG-ITNet has fewer trainable parameters than EEG-Inception and EEG-TCNet, and the authors discuss its spatial filters from a neuroscientific perspective.The authors argue that motor imagery in their selected experiment relies on lateralized spatial-attention changes toward the target limb and imagined motor execution.Source: §V · p. 12

VI. CONCLUSION

  • EEG-ITNet showed statistically significant classification improvement over other end-to-end architectures in most scenarios across two motor-imagery datasets.The evaluation covered within-subject, cross-subject, and cross-subject-with-fine-tuning scenarios.Source: §VI · p. 12
  • The authors identify scenario-specific hyperparameter optimization, visualizing temporal features in the TC block, and testing network depth as future work.They state that these investigations may further improve performance or interpretability.Source: §VI · p. 12
Loading 2204.06947v1…