Source-linked AI summary
EEG-based Cross-Subject Driver Drowsiness Recognition with an Interpretable Convolutional Neural Network
Jian Cui, Zirui Lan, Olga Sourina, Wolfgang Müller-Wittig
TL;DR
Calibration-free EEG drowsiness recognition is difficult because signals vary across subjects and sessions, while deep-learning models offer limited insight into learned features and noise effects. The paper introduces an interpretable, compact CNN with separable convolutions and sample-wise analysis, achieving 78.35% mean accuracy in leave-one-out cross-subject recognition and identifying biologically meaningful EEG patterns.
Problem
Calibration-free EEG drowsiness recognition remains difficult because subject variability and noise complicate cross-subject modeling, while existing deep-learning systems provide limited insight into learned EEG characteristics.
Method
InterpretableCNN combines a compact spatial-temporal CNN using separable convolutions with an interpretation technique that highlights locally important input regions for each prediction.
Results
78.35% mean accuracy was achieved on 11 subjects for leave-one-out cross-subject drowsiness recognition, exceeding the compared conventional and deep-learning baselines.
Takeaways & Limitations
The interpretation results show biologically meaningful features, including Alpha spindles and Theta bursts for drowsiness and eye-related and peripheral-channel Beta features for alertness.
Takeaways & Limitations
The authors report that preprocessing must be carefully designed for different artifacts and noise, and that sample labeling could be further improved.
Abstract
from arXiv · showhide
In the context of electroencephalogram (EEG)-based driver drowsiness recognition, it is still challenging to design a calibration-free system, since EEG signals vary significantly among different subjects and recording sessions. Many efforts have been made to use deep learning methods for mental state recognition from EEG signals. However, existing work mostly treats deep learning models as black-box classifiers, while what have been learned by the models and to which extent they are affected by the noise in EEG data are still underexplored. In this paper, we develop a novel convolutional neural network combined with an interpretation technique that allows sample-wise analysis of important features for classification. The network has a compact structure and takes advantage of separable convolutions to process the EEG signals in a spatial-temporal sequence. Results show that the model achieves an average accuracy of 78.35% on 11 subjects for leave-one-out cross-subject drowsiness recognition, which is higher than the conventional baseline methods of 53.40%-72.68% and state-of-the-art deep learning methods of 71.75%-75.19%. Interpretation results indicate the model has learned to recognize biologically meaningful features from EEG signals, e.g., Alpha spindles, as strong indicators of drowsiness across different subjects. In addition, we also explore reasons behind some wrongly classified samples with the interpretation technique and discuss potential ways to improve the recognition accuracy. Our work illustrates a promising direction on using interpretable deep learning models to discover meaningful patterns related to different mental states from complex EEG signals.
I. INTRODUCTION
EEG-based drowsiness recognition aims to detect driver fatigue early, but subject variability, noise, and limited model interpretability complicate calibration-free systems. The paper addresses these challenges with an interpretable CNN for cross-subject EEG analysis.
- Driver drowsiness reduces attention, vigilance, and cognitive performance, making continuous monitoring important for driving safety.
- Cross-subject EEG recognition is difficult because signals vary across subjects and sessions and contain task-irrelevant activity and noise.
- InterpretableCNN uses a compact architecture with separable convolutions for spatial-temporal EEG processing and a visualization technique that highlights locally important input regions.
- EEG offers high temporal resolution for monitoring brain activity and can support earlier drowsiness detection than behavior-based measures.
- Existing deep-learning approaches improve representation learning from raw EEG but provide limited insight into which signal characteristics drive classification.
A. Data preparation
The study derives labeled EEG segments from a sustained-driving dataset, applies filtering and artifact rejection, and constructs subject-level datasets for cross-subject classification. Both unbalanced real-world-like data and a balanced training dataset are retained.
- The source dataset contains 27 subjects performing a sustained-driving task in virtual reality, with drowsiness reflected by reaction speed to lane-departure events.
- EEG was recorded from 30 electrodes at 500 Hz, bandpass filtered from 1–50 Hz, artifact-rejected, down-sampled to 128 Hz, and segmented into 3-second windows.
- Each extracted EEG sample has 30 channels and 384 sample points, with labels based on local and global reaction times relative to session-specific alert reaction time.
- After session and class-selection procedures, the study retained 2952 samples from 11 subjects in an unbalanced dataset.
- A balanced dataset was created by selecting representative majority-class samples using the shortest alert and longest fatigue local reaction times.
B. Network design
InterpretableCNN incorporates spatial demixing and temporal feature learning through a compact separable-convolution architecture for EEG drowsiness recognition. Its interpretation-oriented design distinguishes alert and drowsy outputs while reducing redundancy and over-fitting.
- Spatial processing: The first convolution generates N1 new channels from the m input EEG channels through learned linear combinations.The implementation sets N1=16 to reduce redundancy and encourage convergence.
- Core idea: The model combines pointwise convolution for signal demixing with depthwise convolution for independent temporal-feature learning.This processing sequence incorporates a demixing-and-feature-extraction pipeline into the CNN.
- Separable convolution: InterpretableCNN processes EEG signals in a spatial-temporal sequence using separable convolutions.The sequence differs from typical separable convolution in operation order.
- Network structure: The network contains seven layers: pointwise and depthwise convolutions, activation, batch normalization, global average pooling, dense, and Softmax layers.Global average pooling reduces parameters and can help prevent over-fitting.
- Temporal processing: The second layer applies depthwise convolutions independently to the generated signals, using 2N1 nodes with kernel length l=64.The resulting output has dimensions (32, 321) for the described input.
- Classification: The Softmax output uses c=0 for alert and c=1 for drowsy states, while the interpretation technique motivates the name InterpretableCNN.The model is presented as offering an interpretability advantage over other state-of-the-art deep learning models.
C. Interpretation technique
The interpretation technique produces sample-specific heatmaps by tracing highly activated class locations through InterpretableCNN to influential regions of the original EEG input. It adapts CAM because spatial channel mixing prevents direct application of the original method.
- Purpose: The technique addresses the need to determine whether EEG classification is driven by relevant features and to discover neurophysiological patterns.It is presented as a procedure for validating model behavior and interpreting learned EEG characteristics.
- CAM adaptation: Original CAM localizes class-discriminative input regions, but cannot be directly applied to this model because pointwise convolutions mix input channels.The proposed method therefore begins with CAM and modifies its tracing procedure.
- Heatmap objective: For a classified EEG sample, the method seeks a class-specific heatmap showing regions important for the prediction.The class label c represents either alert or drowsy state.
- Location tracing: It ranks activation-map values and traces the top N discriminative locations through the network to corresponding centers in the input signal.The traced locations identify influential time-channel regions associated with high class activations.
- Input contribution: A discriminative activation is linked to the strongest contributing input episode, formed from a weighted sum across EEG channels over a local temporal window.This provides the local signal basis for tracing activations back to the input.
D. Methods for comparison
The evaluation compares InterpretableCNN with established deep-learning and conventional EEG baselines, alongside model ablations that isolate the contributions of its architectural components.
- Comparison design: The study compares the proposed model with state-of-the-art deep-learning methods, conventional baselines, and model variations.The variations assess how individual components influence overall performance.
- Deep-learning baselines: EEGNet-4,2 and EEGNet-8,2 serve as deep-learning baselines for cross-subject driver drowsiness recognition.EEGNet uses convolutional filtering followed by spatial and temporal depthwise convolutions.
- Deep-learning baselines: Sinc-ShallowNet provides a second deep-learning baseline using sinc and depthwise convolutions in a temporal-spatial sequence.Its sinc layer uses two cutoff frequencies to form band-pass filters.
- Conventional baselines: The conventional baselines include relative band power, log band power, band-power ratios, wavelet entropy, and four entropy features.These methods represent feature-extraction approaches used for EEG drowsiness or fatigue recognition.
- Ablation models: The 1DConv variation replaces the pointwise and depthwise layers with one standard convolutional layer containing 32 kernels of length 64.NoDepthwise, NoPointwise, and NoBatchNorm remove corresponding components to assess their contributions.
E. Implementation details
The implementation compares the models and conventional methods using a Python and PyTorch/TensorFlow software stack on a specified desktop platform. Batch-normalization behavior was modified after initial tests revealed accuracy drops.
- Computing environment: Comparisons ran on a 64-bit Windows 10 desktop with an Intel i7-6700 CPU and NVIDIA GeForce GTX 1080 GPU.The code used Python 3.6.6.
- Software: InterpretableCNN, Sinc-ShallowNet, and their variations were implemented in PyTorch, while EEGNet models used TensorFlow with Keras.The conventional methods used SciPy and scikit-learn implementations.
- Batch normalization: Initial tests found fast convergence followed by a significant accuracy drop when the default batch-normalization settings were used.The comparison models were modified by disabling running mean and variance estimation.
A. Model comparison results
Under leave-one-subject-out cross-validation, InterpretableCNN outperformed conventional and deep-learning baselines on balanced and unbalanced EEG datasets. Its separable-convolution design and batch normalization contributed to performance, while individual-subject results exposed variability across classifiers.
- 78.35% peak mean accuracy was reached by InterpretableCNN after 11 epochs, exceeding benchmark deep-learning models across the first 50 epochs.EEGNet models peaked at 71.75% and 71.88%, Sinc-ShallowNet at 72.42%, and Conv-ShallowNet at 75.19%.
- Approximately 2% separated InterpretableCNN from its 1DConv variation, while removing batch normalization substantially affected performance.Removing pointwise or depthwise convolution produced performance similar to 1DConv, indicating contributions from separable convolution and batch normalization.
- 53.40%-72.68% was the mean-accuracy range for conventional baseline methods, with LogPower+GNB achieving the highest value of 72.68%.RelativePower, PowerRatio, and FourEntropies performed best with SVM at 68.64%, 64.24%, and 66.49%, respectively.
- 77.70% was the highest mean accuracy among five models on the unbalanced dataset, with InterpretableCNN outperforming the compared baselines overall.The comparison included EEGNet-4,2, Conv-ShallowNet, RelativePower+SVM, and LogPower+GNB under leave-one-subject-out evaluation.
B. Interpretation on the learned characteristics from EEG
The interpretation analysis identifies EEG patterns associated with correct drowsy and alert classifications, while also revealing sensor noise, EMG activity, and ambiguous subject-specific signals behind errors.
- Correctly classified drowsy samples: High-confidence drowsy classifications commonly rely on Theta-rich signals or Alpha spindles localized to informative episodes.The model identifies rhythmic Theta bursts and spindle-like Alpha structures as evidence for drowsiness.
- Correctly classified drowsy samples: Alpha spindles are narrow-band peaks that the model detects across signal episodes as indicators of early drowsiness.These spindle-like structures have been reported as strong drowsiness indicators in driving simulator studies.
- Correctly classified alert samples: Alert classifications rely on peripheral-channel Beta activity or frontal large-amplitude, low-frequency waves, although these features include EMG and eye-movement artifacts.Peripheral Beta activity can mix cortical Beta with EMG, while frontal Delta power can result from eye blinks and eye movements.
- Wrongly classified samples: For Subject 2, residual sensor noise and alert samples resembling drowsy EEG patterns contribute to misclassification.The model falsely treated TP7 fluctuations as alert evidence and localized Alpha spindles in ambiguous alert samples.
- Wrongly classified samples: EMG activity can mislead the model across subjects, overriding apparent drowsiness-related Alpha spindles in peripheral channels.Representative errors involved EMG episodes from channels such as F7 and FT7.
V. DISCUSSION AND FUTURE WORKS
The paper uses interpretable deep learning to identify meaningful EEG patterns across subjects and examines misclassifications to improve cross-subject drowsiness recognition. It highlights both useful physiological signals and constraints in interpretation and preprocessing.
- The compact model uses separable convolutions for spatial-temporal EEG processing and an interpretation technique to reveal important local sample regions.
- Sample-wise analysis identified Alpha spindles and Theta bursts as drowsiness evidence, while eye movements, blinks, and peripheral-channel Beta or EMG activity indicated alertness.
- Peripheral-channel EMG activity, commonly treated as artifact, could help distinguish alert from drowsy EEG signals.
- Misclassification analysis indicates that strong sensor fluctuations should be removed, whereas eye movements, blinks, and EMG activity may require discriminative treatment.
- Only a small portion of samples was interpreted because EEG signals are intrinsically harder to interpret than images or natural language.
- Future work targets improved interpretation and methods supporting calibration-free brain-computer interfaces.
VI. CONCLUSION
The conclusion presents a compact interpretable CNN for discovering cross-subject EEG patterns and analyzing model decisions in driver drowsiness recognition. The model outperforms both conventional and state-of-the-art deep-learning baselines while identifying biologically meaningful features.
- The proposed compact CNN uses separable convolutions to process EEG signals in a spatial-temporal sequence and highlights input regions relevant to classification.
- The model achieves better cross-subject drowsiness-recognition performance than conventional baseline methods and state-of-the-art deep-learning models.
- Interpretation shows that the model learned biologically meaningful features, including Alpha spindles, to distinguish drowsy from alert EEG signals.
- The study also analyzes wrongly classified samples and discusses potential solutions for these errors.