MM-Net: A Multimodal Multi-Task Network for Sleep Staging and Sleep Apnea Assessment After Ischemic Stroke

1. Introduction

• Sleep-stage scoring relies mainly on neural and muscular activity, whereas respiratory-event identification draws on ventilation, effort, oxygenation, and related physiological responses.Their concurrent analysis therefore requires learning from heterogeneous signals with task-specific relevance.Source: §1 · p. 1
• Joint sleep-stage and respiratory-event modeling has mainly been evaluated in general sleep-disorder, suspected-apnea, or elderly cohorts, while stroke-specific studies have focused on staging.Stroke physiology may alter sleep spindles, slow-wave activity, and hemispheric EEG characteristics.Source: §1 · p. 2
• Multimodal models must integrate neural and cardiorespiratory evidence while retaining task-relevant features; controlled channel interventions help distinguish complementary relationships from reliance on event-scoring signals.Source: §1 · p. 2
• Cross-fitting generates out-of-sample training embeddings, and ridge-based alignment maps them into the deployed encoder’s space to address representation inconsistency.Independent adapted encoders can occupy different feature spaces, so both operations are needed for consistent downstream inputs.Source: §1 · p. 2
• MM-Net combines sleep-adapted LaBraM embeddings and physiological descriptors with cardiorespiratory waveforms for joint staging and event detection, with patient-level AHI estimation as an additional output.The study evaluates patient-independent performance, controlled signal interventions, and transfer beyond the stroke population.Source: §1 · pp. 2–3

2.1. Automatic Sleep Staging

• Sleep staging has progressed from convolutional and recurrent models to attention and pretrained EEG representations, with architectures also combining raw signals, spectra, and sequence modeling.Source: §2.1 · p. 3

2.2. Multimodal and Joint PSG Analysis

• Multimodal PSG models use modality-specific encoding and fusion, while prior joint stage-and-respiratory studies mainly evaluated suspected-apnea, elderly, or general sleep populations.The cited joint work includes Huttunen et al. and later multi-task approaches.Source: §2.2 · pp. 3–4

2.3. Respiratory-Event Detection

• Respiratory-event detection commonly uses airflow, effort, oxygen saturation, and cardiac measures, while joint models bring event prediction closer to sleep staging’s 30-second epoch scale.Source: §2.3 · p. 4

2.4. Sleep and Sleep-Disordered Breathing After Stroke

• Stroke can alter sleep-related neural patterns, and existing stroke-specific automated studies have concentrated primarily on sleep-stage classification rather than joint epoch-level modeling of respiratory disturbance.Source: §2.4 · p. 4

3. Datasets and Preprocessing

• Four public PSG datasets serve distinct roles: iSLEEPS is the primary development cohort, while Sleep-EDF, ISRUC-Sleep, and UCD support reference, independent evaluation, or transfer.UCD is used for transfer only, and external cohorts do not select the reported iSLEEPS configuration.Source: §3 · p. 4, Table 1 · p. 5

3.1. iSLEEPS: Primary Stroke Cohort

• The analyzed iSLEEPS cohort contains 99 patients and 92,560 epochs; respiratory-event onsets occur in 16.0% of epochs, with patient-level burden ranging from 0 to 60.5%.The median patient-level burden is 13%, and 80 of 99 patients have events in more than 5% of their epochs.Source: §3.1 · pp. 4–5, Table 2 · p. 5, Figure 1 · p. 5

3.2. Sleep-EDF: Healthy-Sleep Reference Cohort

• Sleep-EDF uses 40 Sleep Cassette recordings from subjects 0–20, split by subject; its EEG-only signals support staging evaluation but not respiratory-performance evaluation.Source: §3.2 · p. 5

3.3. ISRUC-Sleep: Independent Sleep-Disorder Cohort

• ISRUC-Sleep supports evaluation of staging, respiratory events, and subgroup-I AHI across sleep-disorder and healthy populations.The selected signals are harmonized to iSLEEPS inputs; 30 recordings receive a header-based EEG/EOG scaling correction.Source: §3.3 · p. 6

3.4. UCD: Independent Sleep-Apnea Cohort

• UCD contains 25 suspected sleep-disordered-breathing recordings and is used only for transfer; its EEG, EOG, and EMG are scaled per recording without labels.Source: §3.4 · p. 6

3.5. Signal Selection and Harmonization

• The model retains 15 signals across neural and cardiorespiratory groups, excludes snoring from respiratory prediction, and zero-fills unavailable selected inputs.Source: §3.5 · p. 6

3.6. Preprocessing and Epoch Labels

• Respiratory labels mark event onset within a 30-second epoch, and patient-level AHI is derived from predictions; iSLEEPS signals are aligned to retained scored-sleep epochs.Neural signals are resampled for descriptors and LaBraM, cardiorespiratory inputs to 25 Hz, and features are standardized per recording without external labels.Source: §3.6 · p. 6

4.1. Problem Formulation

• Each night is represented as a sequence of epochs, with a neural vector combining 188 engineered descriptors and a 200-dimensional LaBraM embedding alongside eight raw cardiorespiratory channels.The model scores respiratory-event onset by epoch and predicts sleep stages.Source: §4.1 · p. 7, Figure 2 · p. 7

4.2. Physiological Feature Representations

• Neural descriptors summarize spectral power, entropy, spectral edges, Hjorth and time-domain measures, spindle and slow-wave activity, ocular movement, and tonic EMG.The 14 cardiorespiratory descriptors include oxygenation, pulse, ECG, airflow, respiratory effort, and thoraco-abdominal synchrony; the learned branch instead reads raw channels.Source: §4.2 · pp. 7–8

4.3. Sleep-Adapted EEG Encoder

• Cross-fitting embeds each training patient with an encoder that did not see that patient; ridge regression maps these embeddings into the full-fold encoder’s space.The map is fitted using paired embeddings from training patients only, so downstream training inputs use deployment-space coordinates.Source: §4.3 · p. 8

4.4. Two-Stream Encoders

• The neural stream uses a two-layer perceptron, while the cardiorespiratory stream uses a convolutional encoder that emits the same-width vector as the feature perceptron it replaces.LayerNorm is used instead of BatchNorm because recurrent windows are short and batches mix patients.Source: §4.4 · pp. 8–9

4.5. Fusion

• Cross-attention was selected with uniform class weights and the sleep-adapted encoder, and is the fusion operator in final MM-Net.The alternative concatenation operator was used in the frozen-encoder configuration.Source: §4.5 · p. 9

4.6. Temporal Decoder and Two Heads

• The forward and backward states are concatenated into h_t, which feeds a staging head and a respiratory head that also reads the per-epoch cardiorespiratory embedding.The staging head applies softmax to h_t.Source: §4.6 · p. 9

4.7. Training and Decoding

• The two tasks use equal-weight loss, with uniform staging weights in final MM-Net and positive-weighted respiratory loss matching the corpus event rate of about 16%.The respiratory weight is approximately 5.3; this weighting shifts scores monotonically, leaving AUC and average precision unaffected.Source: §4.7 · p. 9
• At inference, a Viterbi pass uses training-label transition counts and network scores to regularize the staging sequence; respiratory outputs are not smoothed.The decoder is discriminative rather than using generative likelihoods as emissions.Source: §4.7 · p. 9

4.8. AHI Estimation

• Respiratory-event onset scores are summed over decoded sleep and divided by decoded sleep time to estimate event rate.A fold-specific scale is fitted on validation patients and applied unchanged to test patients; external-cohort transfer uses the median iSLEEPS scale of 0.459.Source: §4.8 · p. 10

5.1. Protocol and Metrics; 5.2. Implementation and Hyperparameters; 5.3. Compared Methods

• Evaluation used ten-fold patient-independent splits, three seeds for MM-Net, and published training settings for reimplemented baselines.Models shared the same folds; the final training configuration was shared across folds and cohorts.Source: §5.1 · p. 10, §5.3 · p. 10, Table 3 · p. 11

5.4. Staging Benchmark

• 0.763 three-seed accuracy and 0.756 single-model accuracy exceeded the 0.60–0.72 range of retrained staging architectures on iSLEEPS.The three-seed model was ahead of each of eight reimplemented and ported architectures in all ten folds (Holm p=0.016).Source: §5.4 · p. 11, Table 4 · p. 12

5.5. Integrating the Sleep-Adapted Encoder

• Aligning cross-fitted sleep-adapted embeddings raised three-seed accuracy from 0.755 to 0.769 (p=0.020), whereas cross-fitting without alignment changed it by only +0.002 (p=0.63).With alignment, the adapted encoder raised accuracy in eight of ten folds; without alignment, it fell below the frozen encoder in five folds.Source: Table 6 · p. 14, §5.5 · p. 14, Figure 5 · p. 15

5.6. Respiratory-Event Detection

• 0.825 AUC and 0.487 average precision for respiratory-event detection exceeded gradient boosting by 0.152 AUC and 0.197 average precision.Detection AUC was 0.810 for hypopneas, 0.898 for obstructive apneas, and 0.946 for central apneas.Source: §5.6 · p. 15, Table 7 · p. 16, Figure 7 · p. 16

5.7. Joint Training and Stream Contributions

• Joint training improved staging around respiratory events and in severe apnea, while also improving respiratory detection in the frozen-encoder configuration.Removing EEG lowered staging accuracy by 0.050; removing the cardiorespiratory stream lowered respiratory AUC by 0.118.Source: §5.7 · pp. 16–17, Table 8 · p. 17, Table 9 · p. 17

5.8. AHI Estimation

• Summed respiratory scores estimated annotated AHI with Spearman ρ=0.765, mean absolute error 10.4 events/h, and 54% correct AASM severity-band classification.Staging accuracy showed no significant ordered trend across clinical AHI severity groups (p=0.20).Source: §5.8 · p. 17, Table 10 · p. 18, Table 11 · p. 18

5.9. Staging Across Stroke Characteristics

• Staging accuracy was unrelated to measured stroke severity, disability, independence, or time since stroke, and encoder gains were larger in more affected patients.For NIHSS ≥5, the encoder gain was +0.028, versus +0.004 in minor strokes.Source: §5.9 · p. 18, Table 12 · p. 19

5.10. External Validation on Independent Cohorts; 5.11. Model Complexity and Capability

• On ISRUC-I, MM-Net reached 0.803 staging accuracy and 0.889 respiratory AUC; on Sleep-EDF, it reached 0.864 staging accuracy, above reimplemented single-channel architectures.Without retraining, respiratory AUC was 0.833 on ISRUC-I and 0.841 on UCD. The model has 2.85 M trainable parameters, plus a separately trained 5.8 M-parameter encoder, and produces staging, respiratory-event, and AHI outputs in one pass.Source: Table 13 · p. 19, §5.10 · pp. 19–20, Table 14 · p. 20

6. Discussion

• MM-Net stages iSLEEPS at 0.763 (κ = 0.656), ahead of every same-fold retrained architecture, and reaches 0.825 respiratory AUC, 0.152 above engineered-descriptor boosting.The respiratory AUC is also 0.092 above a raw-signal network on seven cardiorespiratory channels.Source: §6 · pp. 20–21
• Joint training improves staging where disordered breathing disrupts sleep, improves event detection and AHI estimation versus two single-task models, and respiratory effort is the strongest individual respiratory input.The respiratory output uses the staging timescale, while AHI is derived without a separate counting rule.Source: §6 · pp. 20–21
• The respiratory score estimates AHI with ρ = 0.765, exceeding the corpus-recorded clinical index agreement of ρ = 0.687.Respiratory-event onset is scored per 30-second epoch, and the expected-count formulation normalizes by decoded sleep time.Source: §6 · p. 21
• On ISRUC-I, MM-Net reaches 0.803 staging accuracy and 0.889 respiratory AUC, ahead of every retrained baseline on the same folds.Without retraining, the iSLEEPS-trained model reaches respiratory AUCs of 0.833 on ISRUC-I and 0.841 on UCD.Source: §6 · p. 21
• Across populations, MM-Net reaches 0.864 staging accuracy on Sleep-EDF and 0.763 in the stroke cohort; nasal-pressure contribution depends most on acquisition system.The discussion cautions that population, montage, and acquisition conditions change together; the seven-channel variant is appropriate for transfer.Source: §6 · p. 21

6.1. Limitations

• The single-center iSLEEPS evaluation limits stroke-specific replication; staging requires a completed recording, and respiratory scoring does not yet provide event onset and duration boundaries.The authors identify a second public stroke PSG cohort and event-level scoring as useful extensions.Source: §6.1 · p. 21

7. Conclusion

• MM-Net integrates sleep-adapted EEG representations through cross-fitting and embedding alignment, improving sleep-stage discrimination while supporting concurrent respiratory-event detection.The approach targets joint characterization of sleep architecture and respiratory disturbances in subacute ischemic stroke, where altered neural physiology and heterogeneous cardiorespiratory information challenge conventional approaches.Source: §7 · p. 22
• The findings address training–inference mismatch for pretrained representations and show that shared temporal modeling accommodates physiologically specialized prediction tasks.Controlled interventions reveal distinct roles for neural and cardiorespiratory information.Source: §7 · p. 22
• Patient-independent evaluation and experiments across independent sleep cohorts support applicability beyond a single recording population.The authors present the framework as a basis for further investigation of representation consistency and multimodal generalization in clinical sleep analysis.Source: §7 · p. 22

Data and Code Availability

• The iSLEEPS corpus is publicly available through Zenodo and the india-data.org portal, as described by Maiti et al..The passage gives DOI 10.5281/zenodo.14873844.Source: Data and Code Availability · p. 22
• ISRUC-Sleep, the UCD database, and Sleep-EDF are public datasets.Source: Data and Code Availability · p. 22
• The authors release code, per-fold configurations, engineered features, external-cohort corrections, and reproduction notebooks.The passage provides the project GitHub repository.Source: Data and Code Availability · p. 22

Ethics

• The iSLEEPS data collection was approved by the NIMHANS Institutional Ethics Committee, and this secondary analysis required no additional enrollment.The passage specifies approval number NIMHANS/34th IEC (BS&NS DIV.)/2022, dated 05.02.2022.Source: Ethics · p. 22

• Suva contributed to conceptualization, methodology, software, formal analysis, investigation, and original-draft writing.Source: CRediT Authorship Contribution Statement · p. 22
• Abha contributed to methodology, software, validation, and original-draft writing; Sajin contributed to software, investigation, data curation, visualization, and original-draft writing.Source: CRediT Authorship Contribution Statement · p. 22
• Mobin contributed to conceptualization, supervision, project administration, and review and editing; Uddin contributed to conceptualization, supervision, and review and editing.Source: CRediT Authorship Contribution Statement · p. 22

• No specific grant supported this research.Source: Funding · p. 22