Source-linked AI summary
AI Generalisation Gap In Comorbid Sleep Disorder Staging
Saswata Bose, Suvadeep Maiti, Shivam Kumar Sharma, Mythirayee S, Tapabrata Chakraborti, Srijitesh Rajendran, Raju S. Bapi
TL;DR
Healthy-trained EEG sleep-staging models may not generalize reliably to stroke patients with disrupted sleep. This paper introduces iSLEEPS and combines deep learning with Grad-CAM and clinical analysis, finding poor cross-domain performance and physiologically irrelevant attention.
Problem
Healthy-trained sleep-staging models have limited reliability in neurologically impaired populations, while pathology-specific EEG datasets and tailored models remain scarce.
Method
The paper introduces iSLEEPS and evaluates a single-channel EEG SE-ResNet–Bi-LSTM model using Grad-CAM and clinician-supported analyses.
Results
Cross-domain staging achieved 55.1% accuracy and 51.9% MF1, while Grad-CAM showed frequent attention to physiologically irrelevant EEG regions in stroke patients.
Takeaways & Limitations
Clinical sleep staging should use subject-aware or disease-specific models and medical supervision when applying healthy-trained models to stroke patients.
Takeaways & Limitations
Healthy-data models generalize poorly to stroke patients because sleep architecture differs substantially between the cohorts.
Abstract
from arXiv · showhide
Accurate sleep staging is essential for diagnosing OSA and hypopnea in stroke patients. Although PSG is reliable, it is costly, labor-intensive, and manually scored. While deep learning enables automated EEG-based sleep staging in healthy subjects, our analysis shows poor generalization to clinical populations with disrupted sleep. Using Grad-CAM interpretations, we systematically demonstrate this limitation. We introduce iSLEEPS, a newly clinically annotated ischemic stroke dataset (to be publicly released), and evaluate a SE-ResNet plus bidirectional LSTM model for single-channel EEG sleep staging. As expected, cross-domain performance between healthy and diseased subjects is poor. Attention visualizations, supported by clinical expert feedback, show the model focuses on physiologically uninformative EEG regions in patient data. Statistical and computational analyses further confirm significant sleep architecture differences between healthy and ischemic stroke cohorts, highlighting the need for subject-aware or disease-specific models with clinical validation before deployment. A summary of the paper and the code is available at https://himalayansaswatabose.github.io/iSLEEPS_Explainability.github.io/
1. INTRODUCTION
Sleep staging supports diagnosis and management of sleep disorders, but healthy-trained models may fail in neurologically impaired populations with severe abnormalities. The paper addresses this clinical gap with iSLEEPS, a stroke-specific PSG dataset and pathology-aware sleep-staging framework.
- Clinical motivation: Sleep staging is critical for diagnosing and managing disorders such as obstructive sleep apnea and hypopnea, which impair cognition and quality of life.PSG is described as the clinical gold standard, using EEG, EOG, and EMG for classification.
- Generalisation gap: Severe neurological and electrophysiological abnormalities in stroke patients can make models trained on healthy cohorts unreliable and clinically misleading.The passage identifies epilepsy, altered thalamocortical coupling, and asymmetric cortical activity as relevant abnormalities.
- Proposed framework: The paper presents the first stroke-specific sleep-staging framework, combining deep learning, medically interpretable analysis, and a curated PSG dataset of stroke patients with comorbid sleep disorders.The framework is intended to support pathology-aware and clinically deployable sleep assessment for neurologically impaired populations.
- Dataset and contributions: iSLEEPS is introduced as a new PSG dataset containing 100 ischemic stroke patients with severe sleep disorders.The dataset is identified as accessible through the paper’s listed URL.
- Dataset and contributions: The paper benchmarks a deep learning model on iSLEEPS and reports limited generalization for models trained on healthy cohorts.Its statistical and explainability analyses, validated by clinician feedback, examine this cross-cohort limitation.
2. METHODOLOGY
The method stages sleep from sliding windows of consecutive 30 s EEG epochs using a SE-ResNet feature extractor and stacked bidirectional LSTMs for temporal context. Grad-CAM provides class-specific explanations by highlighting influential time or frequency regions for physiological interpretability.
- Model architecture: The model uses consecutive 30 s EEG epochs in a sliding window and predicts the sleep stage of the central epoch.The architecture is characterized by window size w and stride length s.
- Model architecture: A SE-ResNet block extracts spectral–temporal features by enhancing relevant frequency–amplitude patterns while suppressing noise.The block is designed to emphasize discriminative signal structure.
- Model architecture: Stacked Bi-LSTM layers model bidirectional temporal dependencies and long-range sleep-stage transitions before fully connected layers.The encoded features are passed through fully connected layers for the final prediction.
- Explainability: Grad-CAM generates class-specific heatmaps from target-class gradients with respect to the final convolutional layer.For signal classification, the heatmaps highlight influential time or frequency regions, including potentially meaningful features such as sleep spindles or alpha rhythms.
3. EXPERIMENTS
The experiments evaluate raw single-channel C4:M1 EEG across three public sleep datasets and the iSLEEPS ischemic stroke dataset. The datasets differ in cohort size, scoring standards, and population composition.
- Datasets: Experiments use raw single-channel EEG from SleepEDF-20, SleepEDF-78, SHHS, and iSLEEPS.The EEG channel is C4:M1.
- Datasets: SleepEDF-20 contains 39 PSG recordings from 20 healthy adults aged 25–34, scored under R&K staging.Stages include W, N1, N2, N3 [N3+N4], and REM; Movement and UNKNOWN are excluded.
- Datasets: SleepEDF-78 extends the SleepEDF cohort to 78 participants, while SHHS includes 6,441 AASM-scored subjects.SHHS analysis is restricted to 329 typical sleepers.
- Datasets: The iSLEEPS dataset comprises 100 ischemic stroke participants.
4. RESULTS AND DISCUSSION
The model performs competitively across healthy and patient datasets but generalizes poorly from healthy subjects to ischemic stroke patients. Statistical and computational analyses show substantial differences in sleep architecture between the cohorts, motivating medically supervised clinical use.
- Benchmarking: The proposed model performs at or above existing literature benchmarks across healthy and patient-based sleep-staging datasets.Tables 1 and 2 compare accuracy, Macro F1, Kappa, and class-wise F1 performance across SleepEDF-20, SleepEDF-78, SHHS, and iSLEEPS.
- Cross-domain generalization: 55.1% accuracy and 51.9% MF1 were obtained on 100 iSLEEPS patients using the SHHS-trained model, sharply below its healthy-dataset performance.GradCAM analysis found frequent attention to physiologically irrelevant EEG regions, despite some clinically meaningful epochs.
- Cross-domain generalization: Healthy-trained models often misdirect attention, miss clinically relevant cues, and produce systematic staging errors when evaluated on stroke patients.Representative epochs showed correct predictions with physiologically irrelevant attention as well as attention closer to relevant patterns.
- Sleep architecture analysis: Sleep architectures differed significantly between SleepEDF-78 and iSLEEPS across stage frequency, order, and transitions.The analysis used 22,430 SleepEDF-78 epochs and 23,192 iSLEEPS epochs; transition analysis reported average KL Divergence ≈0.17, Chi-Square Statistic ≈1831.4552, and 13 significant transitions out of 20.
- Statistical lens: N2→Wake transition probability was 0.354 for stroke patients versus 0.164 for healthy individuals, indicating greater sleep instability.The transition analysis associated this pattern with frequent arousal rather than progression into deep restorative sleep.
- Computational lens: Average Run Length perfectly separated patient and healthy cohorts among 27 engineered structural and sequential EEG features.Binary classifiers included Random Forest, Logistic Regression, Decision Tree, and SVM.
5. CONCLUSION
Healthy-data-trained sleep-staging models generalize poorly to patient sleep, with an ≈30% accuracy drop and strong dataset bias. The paper calls for variability-aware models, expert review before clinical use, and the iSLEEPS dataset to address this gap.
- Limitations: Models trained only on healthy sleep data show an ≈30% accuracy drop on patient sleep staging, indicating strong dataset bias.The passage also notes that no public benchmarks exist for multimorbid sleep disorders.
- Future needs: The paper calls for models that handle interpatient and structural variability, including hierarchical approaches separating healthy and patient data.These approaches are presented as an urgent need for improving generalization across populations.
- Clinical deployment: Mandatory expert review is recommended before clinical use because models may rely on non-pathological features.This recommendation addresses the clinical risk associated with uninformative model behavior in patient data.
- Dataset contribution: The paper introduces iSLEEPS to address the lack of public benchmarks for multimorbid sleep disorders.The supplied passage ends while introducing iSLEEPS, so no further dataset details are available here.
7. ETHICAL COMPLIANCE
The study was conducted at NIMHANS in Bengaluru, India, with approval from its Institutional Ethics Committee and adherence to ethical requirements.
- 7. ETHICAL COMPLIANCE: The study was conducted at NIMHANS, Bengaluru, India, under Institutional Ethics Committee approval [No. NIMHANS/34th IEC (BS&NS DIV.)/2022 dated 05.02.2022].Ethical considerations were strictly followed throughout the study.