Source-linked AI summary
Compact Convolutional Neural Networks for Classification of Asynchronous Steady-state Visual Evoked Potentials
Nicholas R. Waytowich, Vernon Lawhern, Javier O. Garcia, Jennifer Cummings, Josef Faller, Paul Sajda, Jean M. Vettel
TL;DR
SSVEP decoding commonly relies on hand-crafted frequency references and calibration, limiting use when stimulus information is unavailable or asynchronous. This paper evaluates a compact convolutional neural network that learns from raw EEG, substantially outperforming CCA-based methods while revealing phase- and amplitude-related features. The method shows promise for asynchronous BCI applications and for analyzing diagnostic SSVEP representations.
Problem
Conventional high-accuracy SSVEP decoding depends on hand-crafted frequency information and calibration, limiting decoding when such stimulus information is unavailable.
Method
The paper applies a compact convolutional neural network to broadly filtered raw EEG for automatic feature extraction without user-specific calibration.
Results
The Compact-CNN substantially outperforms CCA and Combined-CCA and reveals phase- and amplitude-related features in learned SSVEP representations.
Takeaways & Limitations
The Compact-CNN shows promise for asynchronous BCIs and for analyzing diagnostic features of SSVEP signals.
Takeaways & Limitations
The experiment did not include a non-control state, so efficacy with such a state in a closed-loop setting remains for future investigation.
Abstract
from arXiv · showhide
Steady-State Visual Evoked Potentials (SSVEPs) are neural oscillations from the parietal and occipital regions of the brain that are evoked from flickering visual stimuli. SSVEPs are robust signals measurable in the electroencephalogram (EEG) and are commonly used in brain-computer interfaces (BCIs). However, methods for high-accuracy decoding of SSVEPs usually require hand-crafted approaches that leverage domain-specific knowledge of the stimulus signals, such as specific temporal frequencies in the visual stimuli and their relative spatial arrangement. When this knowledge is unavailable, such as when SSVEP signals are acquired asynchronously, such approaches tend to fail. In this paper, we show how a compact convolutional neural network (Compact-CNN), which only requires raw EEG signals for automatic feature extraction, can be used to decode signals from a 12-class SSVEP dataset without the need for any domain-specific knowledge or calibration data. We report across subject mean accuracy of approximately 80% (chance being 8.3%) and show this is substantially better than current state-of-the-art hand-crafted approaches using canonical correlation analysis (CCA) and Combined-CCA. Furthermore, we analyze our Compact-CNN to examine the underlying feature representation, discovering that the deep learner extracts additional phase and amplitude related features associated with the structure of the dataset. We discuss how our Compact-CNN shows promise for BCI applications that allow users to freely gaze/attend to any stimulus at any time (e.g., asynchronous BCI) as well as provides a method for analyzing SSVEP signals in a way that might augment our understanding about the basic processing in the visual cortex.
1 Introduction
SSVEP-based BCIs require accurate decoding of stimulus-related frequency information, but conventional methods depend on hand-crafted domain knowledge and calibration. The paper proposes a Compact-CNN that learns task-relevant features directly from broadly filtered raw EEG and outperforms CCA-based approaches without user-specific calibration.
- Motivation: SSVEPs are robust EEG oscillations evoked by temporally flickering visual stimuli and can serve as BCI control signals.They arise in visual cortex and support communication links that bypass neuromuscular pathways.
- Limitations of conventional methods: CCA-based decoding matches EEG against hand-crafted sinusoidal references, while Combined-CCA improves references using user calibration data.A pooled transfer variant can remove calibration for the current user, but retains the template-based approach.
- Limitations of conventional methods: CNN approaches for SSVEPs often use FFT representations, restricting learning to frequency features that may miss other task-relevant information.This domain-specific representation is used to reduce the training data needed by deep models.
- Proposed approach: The Compact-CNN operates on broadly filtered EEG, automatically extracting task-relevant features while remaining suitable for relatively small datasets.The method uses a previously collected 12-class SSVEP dataset without user-specific calibration and achieves substantially better accuracy than CCA and Combined-CCA.
- Implications: The learned representation reveals additional phase- and amplitude-related SSVEP features and supports asynchronous BCI applications in which users can freely attend to stimuli.The paper also presents the model as a tool for examining representations relevant to visual-cortex processing.
2 Methods
The study evaluates a Compact-CNN and two CCA-based methods on a 12-class SSVEP dataset using subject-independent validation. It also visualizes learned representations to examine how the methods encode stimulus-related EEG information.
- Data and procedure: Ten participants viewed 12 flickering numeric stimuli, with EEG recorded during 4 s fixation trials.Stimulus frequencies ranged from 9.25 to 14.75 Hz in 0.5 Hz steps.
- Validation: Leave-one-subject-out validation pooled nine participants for training and held out the tenth, producing user-independent models without test-participant training data.Each participant served as the test participant in turn.
- Classifiers: The compared methods were Compact-CNN, calibration-free CCA using sinusoidal references, and a transfer-learning variant of Combined-CCA.Combined-CCA combines traditional references with prototype responses averaged from other participants.
- Representation analysis: The Compact-CNN representation was projected with t-SNE to visualize how network layers separate and cluster individual SSVEP trials.The projections were used to inspect diagnostic features learned by the network.
- Compact-CNN: Compact-CNN learns temporal filters from broadly filtered EEG and uses convolutional operations for automatic feature extraction.Its architecture is designed for raw EEG classification when only limited data are available.
- Standard CCA: CCA selects the stimulus frequency whose sinusoidal reference signals produce the maximum canonical correlation with multichannel EEG.Each reference set contains the fundamental stimulus frequency and its harmonics.
- Architecture notation: Table 1 defines the Compact-CNN dimensions using C for channels, T for time points, F1 and F2 for temporal and separable filters, D for spatial filters, and N for classes.Here, F1 equals F2.
3 Results
Compact-CNN outperformed CCA-based methods on the 12-class SSVEP task and learned representations aligned with stimulus-related frequency, class, phase, and amplitude structure. Visualization analyses further showed within-class variation associated with trial segments and phase differences across classes and channels.
- 3.1 Compact-CNN outperforms CCA and Combined-CCA: Compact-CNN improved classification relative to CCA and Combined-CCA, particularly for participants with poor CCA-based performance.The comparison used four-second trials divided into one-second segments in a 10-fold cross-validation procedure.
- 3.1 Compact-CNN outperforms CCA and Combined-CCA: Compact-CNN significantly outperformed Combined-CCA and CCA in paired tests, with p < 0.0001 for both comparisons.The reported statistics were t(9) = −10.5 versus Combined-CCA and t(9) = −8.7 versus CCA.
- 3.2 Compact-CNN extracts narrow-band frequency activity: The learned temporal kernels captured narrow-band activity near the stimulus frequencies, including approximately 9 Hz and 14 Hz components.The kernels included both slow-wave and fast-wave activity, suggesting extraction of task-relevant oscillatory features.
- 3.3 Compact-CNN reveals differences among classes: Layer-3 t-SNE projections formed coherent clusters by stimulus class but not by participant or trial order.This pattern indicates that the learned representation separated the 12 keypad classes without clustering by participant identity or time-on-task.
- 3.4 Characterizing features of within-class clusters: Within-class clusters corresponded to the four one-second segments of each original four-second trial, showing that the learner captured features beyond class-level differences.The segments were 0–1 s, 1–2 s, 2–3 s, and 3–4 s.
- 3.4 Characterizing features of within-class clusters: Trial segments showed separable phase patterns, while amplitudes were more similar in the 12.25 Hz Oz analysis; related phase and amplitude patterns also appeared across classes and scalp locations.Figure 6 reports these patterns for the 9.25 Hz, 12.25 Hz, and 14.75 Hz classes across electrodes.
4 Discussion
The Compact-CNN outperformed CCA-based approaches in simulated asynchronous and calibrationless SSVEP classification while learning interpretable class- and phase-related EEG representations. These findings suggest potential for flexible BCI operation and for discovering intrinsic phase structure in EEG data.
- 4 Discussion: The Compact-CNN outperformed both CCA and Combined-CCA, including for individuals with particularly poor conventional-method performance.Layer-3 activations differentiated the 12 stimulus classes without separating participants or trial order.
- 4 Discussion: Within-class activation clusters revealed robust phase variability across the four 1 s segments of the original 4 s trials.The phase differences were robust across channels and almost all stimulus classes, despite not being required for the classification task.
- 4 Discussion: The 1 s segmentation both increased training trials and exposed phase variability that impaired Combined-CCA under asynchronous operation.The authors link this impairment to assumptions about particular stimulus frequency and phase.
- 4.2 Potential benefits of Compact-CNN-based asynchronous classification: The Compact-CNN showed promise for asynchronous SSVEP classification, where users can freely gaze or attend to any stimulus at any time.The 1 s segments simulated a discretized asynchronous paradigm with classifications made every second.
- 4.2 Potential benefits of Compact-CNN-based asynchronous classification: Using leave-one-subject-out transfer, the Compact-CNN outperformed baseline CCA approaches without user-specific calibration.The learned features were subject invariant, supporting transfer across subjects.
- 4.3 Detecting frequency and phase information with the Compact-CNN: The network learned frequency- and phase-related information from broadly filtered EEG through temporal convolution, pooling, and convolutional phase properties.The model’s phase features were invariant to subject and trial time-on-task differences.