EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces

Vernon J. Lawhern; Amelia J. Solon; Nicholas R. Waytowich; Stephen M. Gordon; Chou P. Hung; Brent J. Lance

Source-linked AI summary

EEGNet: A Compact Convolutional Network for EEG-based Brain-Computer Interfaces

Vernon J. Lawhern, Amelia J. Solon, Nicholas R. Waytowich, Stephen M. Gordon, Chou P. Hung, Brent J. Lance

arXiv:1611.08024v4Nov 23, 2016cs.LGq-bio.NCstat.ML

TL;DR

Existing EEG-based BCI CNNs have largely targeted single paradigms, leaving unclear whether one compact architecture can generalize across different EEG tasks. EEGNet uses EEG-specific depthwise and separable convolutions and generalizes across four paradigms with limited data while producing interpretable features.

Problem
It remains unclear whether a single compact CNN can generalize across BCI paradigms with distinct EEG feature characteristics and dataset sizes.
Method
EEGNet applies depthwise and separable convolutions to encode EEG feature-extraction concepts while reducing trainable parameters, and evaluates one architecture across four BCI paradigms.
Results
EEGNet generalized across P300, ERN, MRCP, and SMR datasets with limited data and learned interpretable features across ERP- and oscillatory-based BCI tasks.
Takeaways & Limitations
EEGNet provides a compact architecture whose learned filters can overlap with established EEG representations, supporting interpretable cross-paradigm EEG classification.
Takeaways & Limitations
Comparisons omitted dataset-specific meta features and replaced the original xDAWN+RG ensemble with a single model, limiting direct equivalence to original reference implementations.

Abstract

from arXiv · show

Brain computer interfaces (BCI) enable direct communication with a computer, using neural activity as the control signal. This neural signal is generally chosen from a variety of well-studied electroencephalogram (EEG) signals. For a given BCI paradigm, feature extractors and classifiers are tailored to the distinct characteristics of its expected EEG control signal, limiting its application to that specific signal. Convolutional Neural Networks (CNNs), which have been used in computer vision and speech recognition, have successfully been applied to EEG-based BCIs; however, they have mainly been applied to single BCI paradigms and thus it remains unclear how these architectures generalize to other paradigms. Here, we ask if we can design a single CNN architecture to accurately classify EEG signals from different BCI paradigms, while simultaneously being as compact as possible. In this work we introduce EEGNet, a compact convolutional network for EEG-based BCIs. We introduce the use of depthwise and separable convolutions to construct an EEG-specific model which encapsulates well-known EEG feature extraction concepts for BCI. We compare EEGNet to current state-of-the-art approaches across four BCI paradigms: P300 visual-evoked potentials, error-related negativity responses (ERN), movement-related cortical potentials (MRCP), and sensory motor rhythms (SMR). We show that EEGNet generalizes across paradigms better than the reference algorithms when only limited training data is available. We demonstrate three different approaches to visualize the contents of a trained EEGNet model to enable interpretation of the learned features. Our results suggest that EEGNet is robust enough to learn a wide variety of interpretable features over a range of BCI tasks, suggesting that the observed performances were not due to artifact or noise sources in the data.

1 Introduction

The introduction motivates EEG-based BCIs as a promising noninvasive communication technology and presents EEGNet as a compact, interpretable CNN designed to generalize across BCI paradigms. Its EEG-specific design uses depthwise and separable convolutions to capture established EEG features while reducing trainable parameters.

Motivation: BCIs enable direct communication with machines through brain signals and increasingly use noninvasive EEG approaches beyond traditional medical applications.Traditional applications include neural control of prosthetic artificial limbs, while newer research targets performance enhancement in healthy users.
Motivation: Deep learning, especially CNNs, has reduced reliance on manual feature extraction and achieved state-of-the-art results in computer vision and speech recognition.CNN success in image classification has motivated their growing use in BCI systems.
Contribution: EEGNet is introduced as a compact CNN for EEG-based BCI classification and interpretation.The model is designed specifically for EEG rather than a single narrowly defined paradigm.
Contribution: Depthwise and separable convolutions construct an EEG-specific network that captures optimal spatial filtering and filter-bank construction while reducing trainable parameters.These convolutions were previously used in computer vision and are adapted here to encapsulate well-known EEG feature extraction concepts.
Evaluation: EEGNet’s generalizability is evaluated on EEG datasets collected from four different BCI paradigms.The manuscript compares the model with other BCI models, including CNN- and non-CNN-based approaches.

2 Materials and Methods · 2.1 Data Description

The study describes event-related and oscillatory EEG-based BCI signals, then introduces four datasets spanning P300, feedback ERN, MRCP, and SMR paradigms. These datasets differ in signal characteristics, task design, participants, recording setups, and preprocessing requirements.

2.1 Data Description: BCIs are categorized as event-related or oscillatory according to the EEG feature of interest.Event-related potentials are time-locked, high-amplitude, low-frequency responses with stereotyped waveforms, whereas oscillatory BCIs use power in specific frequency bands and are generally asynchronous.
2.1 Data Description: Oscillatory BCIs are more difficult to train because they have lower signal-to-noise ratios and greater variation across subjects.When time-locked to stimuli, oscillatory signals can also be represented using event-related spectral perturbation analyses.
2.1.2 Dataset 2: Feedback Error-Related Negativity (ERN): The ERN dataset included 26 healthy participants performing a P300 speller task to classify correct versus incorrect feedback.The participants comprised 16 for training and 10 for testing, using randomly flashing letters arranged in a 6×6 grid.
2.1.3 Dataset 3: Movement-Related Cortical Potential (MRCP): The MRCP dataset examined voluntary finger movements, whose EEG contains both event-related and oscillatory components.MRCP activity is observed over central and midline electrodes, including a 0-5Hz readiness potential and 10-12Hz early desynchronization before movement onset.
2.1.3 Dataset 3: Movement-Related Cortical Potential (MRCP): The MRCP experiment recorded 13 subjects performing self-paced left- and right-hand finger movements with a 256-channel system sampled at 1024 Hz.The data underwent PREP preprocessing because of extensive signal noise, and trials were used for binary left-versus-right-hand classification.
2.1.4 Dataset 4: Sensory Motor Rhythm (SMR): The SMR dataset contained four imagined-movement classes from 9 subjects, recorded with 22 electrodes at 250 Hz and resampled to 128 Hz.The classes involved imagined movements of the left hand, right hand, feet, and tongue; SMR reflects mu and beta desynchronization over contralateral sensorimotor cortex.

2.2 Classification Methods

EEGNet is a compact, interpretable CNN designed to classify EEG across multiple BCI paradigms with limited data. Its architecture combines temporal, depthwise, separable, pooling, and softmax classification operations, and is evaluated against CNN and paradigm-specific traditional baselines.

EEGNet architecture: EEGNet begins with temporal convolution to learn frequency filters, followed by depthwise convolution to learn frequency-specific spatial filters.The temporal kernel captures frequency information at 2Hz and above, while depthwise filters operate on individual feature maps.
EEGNet architecture: EEGNet applies average pooling and maximum-norm spatial-filter regularization before passing features directly to an N-unit softmax, omitting a dense aggregation layer.Pooling reduces the sampling rate from 128Hz to 32Hz, and each spatial filter uses a maximum norm constraint of 1.
EEGNet architecture: A separable convolution then summarizes each feature map independently and merges them with pointwise convolutions, reducing parameters while decoupling within- and across-feature-map learning.Block 2 uses a (1, 16) depthwise convolution followed by F2 (1, 1) pointwise convolutions.
Reference CNNs: The study compares EEGNet with DeepConvNet and ShallowConvNet, adapting their temporal-kernel and pooling lengths for 128Hz EEG signals.DeepConvNet uses five convolutional layers, whereas ShallowConvNet uses temporal and spatial convolutions, squaring, average pooling, and logarithmic nonlinearities.
Traditional baselines: Traditional comparisons use xDAWN + RG for P300, ERN, and MRCP, and an OVR filter-bank common spatial pattern algorithm for SMR.The xDAWN + RG pipeline combines spatial filtering, channel selection, Riemannian tangent-space projection, L1/L2 normalization, and Elastic Net classification.

2.3 Data Analysis

The analysis distinguishes within-subject and cross-subject classification, using blockwise cross-validation and paradigm-specific statistical responses. Class weighting addressed imbalance in P300 and ERN, while SMR used a shorter temporal kernel because signals were high-passed.

Classification analyses: Within-subject models train on each subject’s data, whereas cross-subject models use other subjects’ data to train subject-agnostic models.Within-subject performance generally tends to exceed cross-subject performance across tasks.
Evaluation procedure: Within-subject evaluation used four-fold blockwise cross-validation with two training blocks, one validation block, and one testing block.Repeated-measures ANOVA modeled AUC for P300, MRCP, and ERN and classification accuracy for SMR, with subject number and classifier type as factors.
Class imbalance: Class weighting was applied when trial counts were imbalanced, using inverse training proportions with the majority class set to 1.This procedure was applied only to P300 and ERN, the datasets with significant class imbalance.
Class imbalance: In the P300 dataset, 5.6:1 non-target-to-target odds produced class weights of 1 for non-targets and 6 for targets.When odds were fractional, the next highest integer was used for the minority-class weight.
SMR implementation: SMR used a 32-sample temporal kernel instead of the 64-sample length in Table 2 because the data were high-passed at 4Hz.The shorter kernel was specific to the SMR analysis.

2.4 EEGNet Feature Explainability

EEGNet feature explainability is examined through three complementary approaches: hidden-unit activations, convolutional kernel weights, and single-trial feature relevance. These methods support validation that learned features are interpretable rather than driven by noise or artifacts.

Validation purpose: Feature-explainability methods are intended to verify that EEGNet classification performance depends on relevant, interpretable features rather than noise or artifacts.The paper presents explainability as part of robust model validation.
Feature explainability approaches: EEGNet features are summarized through hidden-unit activations after depthwise convolution, linking spatial filters to narrow-band frequency activity.Because spatial filters are tied to particular temporal filters, the activations provide insight into spatial localization.
Feature explainability approaches: Convolutional kernel weights are visualized to interpret temporal convolutions as narrow-band features.Depthwise and separable convolutions limit layer connectivity, making the weights more interpretable.
Feature explainability approaches: Single-trial feature relevance is calculated with DeepLIFT using the Rescale rule to assess each feature’s contribution to the classification decision.Positive relevance supports the outcome, whereas negative relevance provides evidence against it.

3 Results

EEGNet performed comparably across configurations and competitively with reference methods across BCI paradigms, with performance depending on paradigm and evaluation setting. Feature visualizations and ablations showed that EEGNet learned interpretable neurophysiological patterns and distributed, task-relevant representations.

Within-subject results: EEGNet-4,2 and EEGNet-8,2 showed no significant performance difference across within-subject paradigms (p > 0.05).This suggests the increased number of temporal filters in EEGNet-8,2 did not produce a significant within-subject advantage.
Within-subject results: For within-subject SMR classification, EEGNet-8,2 performed similarly to FBCSP, while DeepConvNet performed significantly worse than all other models (p < 0.05).ShallowConvNet and FBCSP also produced very similar performances.
Cross-subject results: Cross-subject results again showed no significant difference between EEGNet-4,2 and EEGNet-8,2 across datasets (p > 0.05).On P300, all CNN-based models significantly outperformed xDAWN+RG, while on MRCP EEGNet-8,2 and DeepConvNet significantly outperformed ShallowConvNet.
Cross-subject results: For cross-subject ERN classification, xDAWN + RG significantly outperformed all CNN models (p < 0.05).Cross-subject SMR results showed no significant difference across models, with CNN-based models slightly outperforming FBCSP.
Feature interpretation: EEGNet visualizations revealed distinct P300 filters, including low-frequency power increases approximately 500ms and 350ms after image presentation and alpha desynchronization.One filter appeared to show a theta-beta relationship, while another corresponded to the P300 response.
Feature interpretation: Removing any single EEGNet filter minimally affected test performance, with the largest decrease occurring when Filter 4 was removed.The study iteratively ablated all combinations of the four filters by replacing selected filters with zeros.
Feature interpretation: DeepLIFT relevances highlighted contralateral motor cortex for high-confidence MRCP predictions, whereas low-confidence predictions showed broader temporal and spatial relevance.ERN relevance analysis also examined the different timing of positive ERP peaks for correct and incorrect feedback trials.

4 Discussion

EEGNet generalized across four BCI paradigms with limited data while producing interpretable features. Its learned filters aligned with FBCSP, and its comparative performance varied with paradigm, subject-analysis setting, and training-set size.

Contributions: EEGNet generalized across P300, ERN, MRCP, and SMR datasets while supporting interpretable feature analysis under limited-data conditions.The study evaluated EEGNet against state-of-the-art ERP and oscillatory BCI approaches across four EEG datasets.
Interpretability: Three visualization approaches helped interpret EEGNet features despite the challenges of non-stationary EEG signals and noise artifacts.The approaches included analyzing spatial-filter outputs, visualizing learned features, and examining single-trial feature relevance.
Interpretability: EEGNet’s approximately 12Hz spatial filters significantly correlated with FBCSP filters in the SMR 8-12Hz filter bank.Although EEGNet minimizes cross-entropy and CSP optimizes variance separation, overlapping filters suggested similar learned representations; depthwise convolutions enabled this comparison.
Comparative performance: DeepConvNet and EEGNet performed similarly in cross-subject analyses, while DeepConvNet was generally weaker within subjects except on P300.Cross-subject training sets were about 10-15 times larger than within-subject sets, suggesting training-data quantity may explain the discrepancy.
Comparative performance: ShallowConvNet tended to underperform on ERP datasets but perform well on SMR, whereas DeepConvNet showed the opposite pattern.The discussion attributes this contrast to ShallowConvNet’s design for log bandpower features, which may be less suitable when ERP signal amplitude dominates.
Limitations: ShallowConvNet’s severe within-subject MRCP underperformance was not explained by training-set size, because MRCP had approximately 700 trials versus SMR’s 192.Both datasets were evenly split between left and right finger movements.

5 Appendix

The appendix specifies the DeepConvNet and ShallowConvNet architectures in terms of channels, time points, and classes. It also defines ShallowConvNet’s square and log activations, including log-input clipping for numerical stability.

DeepConvNet architecture: Table 5 defines the DeepConvNet architecture using C for channels, T for time points, and N for classes.C, T, and N denote the number of channels, time points, and classes, respectively.
ShallowConvNet architecture: Table 6 defines the ShallowConvNet architecture using C for channels, T for time points, and N for classes.The architecture uses the same notation for channels, time points, and classes as DeepConvNet.
ShallowConvNet activations: ShallowConvNet uses square and log activations defined as f(x) = x2 and f(x) = log(x), respectively.The log function is clipped so its minimum input is ϵ = 10e−7 for numerical stability.