PaperlayerView exact revision on arXiv

Source-linked AI summary

A Transformer-based deep neural network model for SSVEP classification

Jianbo Chen, Yangsong Zhang, Yudong Pan, Peng Xu, Cuntai Guan

arXiv:2210.04172v1q-bio.NCcs.AI

TL;DR

Subject-specific calibration makes SSVEP classification burdensome, motivating methods that transfer across subjects. This paper proposes Transformer-based SSVEP classifiers using spectral input, with a filter-bank variant, and reports stronger results than baseline methods on two public datasets. The findings support the feasibility of Transformer models for calibration-free inter-subject classification, while evaluation remains offline and uses only two datasets.

  • Problem

    Conventional SSVEP spatial-filtering methods depend heavily on subject-specific calibration data, motivating inter-subject methods that can reduce the calibration burden.

    Source: §1 · p. 4, §4.1 · p. 20

  • Method

    SSVEPformer classifies complex SSVEP spectra with a Transformer, and FB-SSVEPformer adds filter-bank processing to use harmonic information.

    Source: §1 · p. 6, §2.2 · p. 9

  • Results

    Across both datasets, FB-SSVEPformer outperforms three popular baselines; at 1 s it reaches 88.37% accuracy and 112.45 bits/min ITR on Dataset 1, and 83.19% accuracy and 157.65 bits/min ITR on Dataset 2.

    Source: §1 · p. 6, §5 · p. 26

  • Takeaways & Limitations

    The results validate the feasibility of Transformer-based models for calibration-free SSVEP classification in inter-subject settings and suggest potential to alleviate calibration in practical BCI systems.

    Source: §4.1 · p. 21

    Evaluation used only two public datasets, required substantial training data from existing subjects, and was conducted offline.

    Source: §4.4 · pp. 24, 26

Abstract

from arXiv · show

Steady-state visual evoked potential (SSVEP) is one of the most commonly used control signal in the brain-computer interface (BCI) systems. However, the conventional spatial filtering methods for SSVEP classification highly depend on the subject-specific calibration data. The need for the methods that can alleviate the demand for the calibration data become urgent. In recent years, developing the methods that can work in inter-subject classification scenario has become a promising new direction. As the popular deep learning model nowadays, Transformer has excellent performance and has been used in EEG signal classification tasks. Therefore, in this study, we propose a deep learning model for SSVEP classification based on Transformer structure in inter-subject classification scenario, termed as SSVEPformer, which is the first application of the transformer to the classification of SSVEP. Inspired by previous studies, the model adopts the frequency spectrum of SSVEP data as input, and explores the spectral and spatial domain information for classification. Furthermore, to fully utilize the harmonic information, an extended SSVEPformer based on the filter bank technology (FB-SSVEPformer) is proposed to further improve the classification performance. Experiments were conducted using two open datasets (Dataset 1: 10 subjects, 12-class task; Dataset 2: 35 subjects, 40-class task) in the inter-subject classification scenario. The experimental results show that the proposed models could achieve better results in terms of classification accuracy and information transfer rate, compared with other baseline methods. The proposed model validates the feasibility of deep learning models based on Transformer structure for SSVEP classification task, and could serve as a potential model to alleviate the calibration procedure in the practical application of SSVEP-based BCI systems.

1. Introduction

SSVEP offers a high-signal-to-noise, low-training-time BCI paradigm, but calibration-dependent methods and cross-subject variability complicate classification for new users. The paper introduces Transformer-based SSVEP classifiers, including a filter-bank variant that uses harmonic information.

1. Introduction

  • SSVEP frequencies match stimulus frequencies and their harmonics, enabling BCI targets to be encoded by distinct flicker frequencies.Filter-bank CCA uses multiple subbands and weights their correlation coefficients to distinguish fundamental and harmonic components.Source: §1 · pp. 2–3
  • Inter-subject classification trains on existing subjects and tests on new subjects, addressing the time-consuming and laborious collection of subject-specific SSVEP data.SSVEP variability across individuals and over time challenges this transfer setting, and traditional machine-learning performance degrades relative to intra-subject classification.Source: §1 · pp. 4–5
  • Transformer models had been applied to EEG classification, but the authors state that Transformer had not yet been used for SSVEP classification.The paper also notes that inter-subject deep-learning models still need stronger generalization and interpretability investigation.Source: §1 · p. 6
  • The proposed SSVEPformer uses SSVEP spectra as input, while FB-SSVEPformer adds filter-bank processing to use harmonic information; both are evaluated for inter-subject classification.The study evaluates two public datasets, with 12 classes and 10 subjects in Dataset 1 and 40 classes and 35 subjects in Dataset 2.Source: §1 · p. 6
  • With a 1 s window, FB-SSVEPformer reaches 88.37% accuracy and 112.45 bits/min ITR on Dataset 1, and 83.19% accuracy and 157.65 bits/min ITR on Dataset 2.These results are reported for the inter-subject classification scenario.Source: §1 · p. 6

2.1. Datasets

The evaluation uses two public SSVEP datasets with different numbers of subjects and visual targets. Their acquisition protocols differ in trial duration, electrodes, and sampling setup.

2.1. Datasets

  • The methods were evaluated using two public SSVEP datasets.Source: §2.1 · p. 7
  • Dataset 1 contains 12 targets and 10 subjects, with EEG recorded from eight occipital electrodes across 15 blocks of 4-second trials.The stimuli span 9.25–14.75 Hz in 0.5 Hz steps; data were sampled at 2048 Hz and downsampled to 256 Hz.Source: §2.1 · p. 7
  • Dataset 2 contains 40 targets and 35 subjects, with six blocks of 40 trials per subject and 6 seconds per trial.EEG was sampled at 1000 Hz, downsampled to 250 Hz, and notch-filtered at 50 Hz.Source: §2.1 · pp. 7–8

2.2. Data preprocessing

Preprocessing prepares EEG time series and their frequency-domain representations for classification. For the proposed models, the input preserves both real and imaginary spectral components.

2.2. Data preprocessing

  • Time-domain models receive fourth-order Butterworth-filtered EEG with an 8–64 Hz passband, using dataset-specific windows shifted after stimulus onset.The window begins at 0.135 s for Dataset 1 and 0.64 s for Dataset 2, and extends for the selected duration d.Source: §2.2 · p. 8
  • For frequency-domain models, FFT converts preprocessed EEG from the time domain into the frequency domain.The input signal x is preprocessed time-domain EEG.Source: §2.2 · p. 8
  • The proposed models use the complex spectrum, concatenating real and imaginary components to preserve magnitude and phase information.The paper selects the spectrum from 8 to 64 Hz; zero padding yields 0.25 Hz resolution for Dataset 1 and 0.2 Hz for Dataset 2.Source: §2.2 · p. 9
  • Magnitude-spectrum input discards phase, whereas complex-spectrum input retains both magnitude and phase, which prior studies associate with SSVEP classification performance.The authors cite prior evidence that phase information has a role in classification and that complex-spectrum input outperforms magnitude-spectrum input in comparative experiments.Source: §2.2 · p. 9

2.3. The baseline methods

The baseline methods include TRCA, EEGNet, and CCNN, which use different spatial-filtering, convolutional, and spectral-feature approaches for SSVEP classification.

2.3.1. TRCA

  • TRCA learns spatial filters and class reference signals from pooled data of the other subjects to classify a held-out subject.The projected test sample is compared with the reference signals for each stimulus frequency.Source: §2.3.1 · pp. 9–10

2.3.2. EEGNet

  • EEGNet combines channel-wise filtering, depth-wise spatial filtering, separable convolution, and a fully connected classification layer.It was applied to inter-subject SSVEP classification in prior work.Source: §2.3.2 · p. 10

2.3.3. CCNN

  • CCNN uses the complex spectrum to extract frequency and phase information, with convolutional layers for filtering and feature extraction and a fully connected output.The passage reports that its complex-spectrum representation can benefit SSVEP classification.Source: §2.3.3 · p. 10
  • Figure 2 depicts SSVEPformer as a six-block model, including complex-spectrum representation, channel combination, an encoder, and an MLP head.The encoder contains two successive identical sub-encoders.Source: §2.3.3 · p. 11, Figure 2 · p. 11

2.4. The proposed SSVEPformer

SSVEPformer processes complex-spectrum EEG through channel combination, a Transformer-style encoder, and an MLP head to produce class predictions. Its encoder retains channel MLPs while replacing attention with CNN modules.

2.4. The proposed SSVEPformer

  • SSVEPformer has six blocks, from input and complex-spectrum representation through channel combination and encoding to the MLP head and output.Its complex-spectrum representation transforms the input EEG, with N denoting classes, C channels, and F spectrum length per channel.Source: §2.4 · p. 11

2.4.1. Channel combination block

  • The channel-combination block learns 2 × C spatial filters with 1-length convolutions to form weighted combinations of EEG channels.Multiple combinations are used because different combinations can focus on different classification information or suppress different noise.Source: §2.4.1 · pp. 11–12

2.4.2. SSVEPformer encoder block

  • The encoder replaces the Transformer attention module with CNN modules while retaining channel MLP modules.It uses two successive sub-encoders, each with CNN and channel MLP modules and residual connections.Source: §2.4.2 · pp. 12–13

2.4.3. MLP head block

  • The MLP head flattens upstream features and uses two fully connected layers to map them to dataset classes.The layers refine extracted features before producing the final result.Source: §2.4.3 · p. 13

2.5. FB-SSVEPformer

FB-SSVEPformer applies separate SSVEPformer subnetworks to filter-bank subbands and learns to fuse their outputs. The chosen setting uses three subbands to balance accuracy and model size.

2.5. FB-SSVEPformer

  • FB-SSVEPformer filters EEG into S subbands, processes each with an SSVEPformer subnetwork, and weights and fuses the subnetwork outputs.The filter bank is intended to use harmonic information for classification.Source: §2.5 · p. 14
  • Learned convolutional weights fuse subband results, unlike FBCCA's fixed weights obtained by grid search.The fusion parameters are trained toward the best classification result and eventually stabilize.Source: §2.5 · p. 15
  • Three subbands were selected: four slightly improved accuracy but greatly increased parameters, while two had lower accuracy than three.The comparison used 1-second data from Dataset 1; the lower cutoff was m × 8 Hz for Dataset 1 and m × 9 Hz for Dataset 2, with an 80 Hz upper cutoff.Source: §2.5 · p. 15

2.6. Experimental settings and performance evaluation

The evaluation compares the proposed models with EEGNet, CCNN, and TRCA using classification accuracy and information transfer rate. The supplied settings describe implementation and training details, but do not give comparative outcome values.

2.6. Experimental settings and performance evaluation

  • The deep-learning models were implemented with PyTorch, and the convolutional and linear parameters were initialized from a normal distribution with mean 0.Source: §2.6 · p. 15
  • FB-SSVEPformer subnetworks are trained on their respective frequency bands for 100 epochs before the full model is trained for 20 epochs.SSVEPformer is trained for 100 epochs; the reported batch size is 128 and dropout rate is 0.5.Source: §2.6 · p. 16
  • EEGNet, CCNN, and TRCA serve as baselines, with preprocessing matched across methods and an additional complex-spectrum transform for CCNN.Source: §2.6 · pp. 16–17
  • Classification accuracy and ITR in bits/minute are the evaluation metrics.The ITR calculation adds 0.5 seconds of gaze-movement time; with 1-second data, T is set to 1.5 seconds.Source: §2.6 · p. 17

3. Result

On both datasets, SSVEPformer and FB-SSVEPformer outperformed the compared baselines in inter-subject classification, with FB-SSVEPformer leading average accuracy and ITR.

3. Result

  • Across both datasets, SSVEPformer and FB-SSVEPformer outperformed the baselines, while FB-SSVEPformer led average accuracy and ITR.The figures report average results across subjects.Source: Figure 5 · p. 18, §3.1 · p. 18, Figure 6 · p. 19

3.1. Dataset 1

  • On Dataset 1, SSVEPformer outperformed the three baselines especially when data length was shorter than 1 s.FB-SSVEPformer achieved the best average accuracy and ITR on this dataset.Source: §3.1 · p. 18

3.2. Dataset 2

  • On Dataset 2, SSVEPformer also outperformed the baselines, and FB-SSVEPformer achieved the best average accuracy and ITR.The reported comparison covers data lengths from 0.5 s to 1 s.Source: §3.2 · p. 19

4. Discussion

The discussion presents SSVEPformer and its filter-bank variant as approaches to inter-subject SSVEP classification, where calibration-data collection and differences among subjects pose challenges. It reports performance gains with subject adaptation, visualizes learned features and model decisions, and identifies limitations in dataset scope, training data, and offline evaluation.

4.1. A feasible method in inter-subject scenario for SSVEP classification

  • SSVEP data collection is time-consuming, and differences among subjects challenge inter-subject classification; the authors propose SSVEPformer and FB-SSVEPformer for calibration-free classification.The discussion describes using data from existing subjects to train a classifier for new subjects.Source: §4.1 · p. 20
  • 84.16% and 80.40% accuracies for SSVEPformer, versus 88.37% and 83.19% for FB-SSVEPformer, on 12-class and 40-class tasks, respectively.The models were evaluated on two public datasets in the inter-subject setting.Source: §4.1 · p. 21

4.2. Enhancing the performance with subject-adaptive strategy

  • 11.80% and 10.33% gains followed subject-adaptive training with one data block per frequency for SSVEPformer and FB-SSVEPformer, respectively.Both models exceeded 90% accuracy after subject adaptation; increasing the blocks from one to five added 2.72% and 2.36%, respectively.Source: §4.2 · p. 22

4.3. Model visualization and interpretation

  • For a representative Dataset 1 subject, the proposed models’ learned features had smaller intra-category and larger inter-category distances than those of baseline methods.The visualization compared EEGNet, CCNN, SSVEPformer, and FB-SSVEPformer.Source: §4.3 · pp. 23–24
  • Grad-CAM weights were higher at stimulation frequencies and harmonics than at other frequency points, for both real and imaginary input parts.The visualizations covered representative subjects from both datasets and were used to examine the SSVEPformer encoder’s input relevance.Source: §4.3 · p. 24

4.4. Limitation

  • The evaluation used only two public datasets and offline experiments, while training still requires substantial data from existing subjects.The authors identify reducing training-data needs while maintaining performance and evaluating online efficiency and effectiveness as future work.Source: §4.4 · pp. 24, 26

5. Conclusion

The conclusion reports that SSVEPformer and its filter-bank variant outperform three popular baselines on two public datasets in inter-subject tests. It also describes grad-CAM visualizations and presents the models as having potential for practical SSVEP-based BCI applications.

  • The proposed models outperformed three popular baseline methods on both datasets under inter-subject conditions, and FB-SSVEPformer achieved the best results.Experiments used data lengths from 0.5 s to 1 s.Source: §5 · p. 26
  • Grad-CAM visualizations showed the impact of different input locations on the model’s output, which the authors cite as demonstrating interpretability.Source: §5 · p. 26
  • The authors state that the models hold promising potential to promote practical applications of SSVEP-based BCI systems.Source: §5 · p. 26

Related papers

Loading 2210.04172v1…