PaperlayerView exact revision on arXiv

Source-linked AI summary

SeqSleepNet: End-to-End Hierarchical Recurrent Neural Network for Sequence-to-Sequence Automatic Sleep Staging

Huy Phan, Fernando Andreotti, Navin Cooray, Oliver Y. Chén, Maarten De Vos

arXiv:1809.10932v3cs.LGeess.SPstat.ML

TL;DR

Manual sleep staging is labor-intensive, time-consuming, costly, and prone to human errors, motivating automation. SeqSleepNet frames staging as sequence-to-sequence classification with hierarchical recurrent modelling and achieves strong performance on the MASS dataset.

  • Problem

    Manual sleep staging is labor-intensive, time-consuming, costly, and prone to human errors, while reliable automatic staging is needed for scalable sleep monitoring.

    Source: §I · p. 1

  • Method

    SeqSleepNet jointly classifies multiple PSG epochs using hierarchical recurrent modelling with learned filterbanks, attention-based epoch processing, sequence-level recurrent modelling, and end-to-end training.

    Source: §I · p. 2, §VII · pp. 9–10

  • Results

    87.1% overall accuracy, 83.3% MF1, and 0.815 Cohen’s kappa were achieved on the MASS dataset, with a 0.7% absolute accuracy gain over E2E-DeepSleepNet.

    Source: 1) Sleep stage classification… · p. 7

  • Takeaways & Limitations

    Long-term sequential modelling improves automatic sleep staging, with SeqSleepNet averaging a 3.4% absolute accuracy gain over the E2E-ARNN baseline.

    Source: 1) Sleep stage classification… · p. 8

    SeqSleepNet was trained on the MASS dataset, a cohort of healthy subjects, and may not directly work well on other subject populations.

    Source: §VI · p. 9

Abstract

from arXiv · show

Automatic sleep staging has been often treated as a simple classification problem that aims at determining the label of individual target polysomnography (PSG) epochs one at a time. In this work, we tackle the task as a sequence-to-sequence classification problem that receives a sequence of multiple epochs as input and classifies all of their labels at once. For this purpose, we propose a hierarchical recurrent neural network named SeqSleepNet. At the epoch processing level, the network consists of a filterbank layer tailored to learn frequency-domain filters for preprocessing and an attention-based recurrent layer designed for short-term sequential modelling. At the sequence processing level, a recurrent layer placed on top of the learned epoch-wise features for long-term modelling of sequential epochs. The classification is then carried out on the output vectors at every time step of the top recurrent layer to produce the sequence of output labels. Despite being hierarchical, we present a strategy to train the network in an end-to-end fashion. We show that the proposed network outperforms state-of-the-art approaches, achieving an overall accuracy, macro F1-score, and Cohen's kappa of 87.1%, 83.3%, and 0.815 on a publicly available dataset with 200 subjects.

I. INTRODUCTION

Automatic sleep staging is clinically important but traditionally labor-intensive and often framed around isolated or context-limited epoch decisions. SeqSleepNet instead formulates staging as many-to-many sequence-to-sequence classification with hierarchical recurrent modelling and end-to-end training.

  • Manual scoring of 30-second PSG epochs is labor-intensive, time-consuming, costly, and prone to human errors.Source: §I · p. 1
  • Home-based sleep monitoring requires reliable automatic staging methods alongside comfortable, long-term, clinical-grade wearable EEG devices.Source: §I · p. 1
  • Prior schemes classify one epoch to one label, multiple epochs to one label, or one epoch to multiple labels.Source: §I · p. 1, Figure 1 · p. 2
  • SeqSleepNet maps multiple input epochs to all target labels at once, framing automatic sleep staging as sequence-to-sequence classification.Source: §I · p. 2
  • SeqSleepNet combines filterbank preprocessing, an epoch-level attention-based bidirectional RNN, and a sequence-level bidirectional RNN in an end-to-end architecture.Source: §I · p. 2

II. MONTREAL ARCHIVE OF SLEEP STUDIES (MASS) DATASET

The study evaluates SeqSleepNet on the MASS dataset using multichannel PSG recordings represented as time-frequency images. The preprocessing retains EEG, EOG, and EMG information while the model processes epoch sequences.

  • MASS contains whole-night recordings from 200 subjects aged 18–76 years across five subsets.Source: §II · p. 3
  • Experts manually labelled MASS epochs using AASM or R&K standards, with annotations converted into five stages: W, N1, N2, N3, and REM.Source: §II · p. 3
  • The evaluation used the entire MASS dataset and included EEG, EOG, and EMG channels to complement EEG for automatic sleep staging.Source: §II · p. 3
  • Channel-specific filterbank layers preprocess the input image, concatenate channels in the frequency direction, and produce epoch features for sequential modelling.Source: §III · p. 3
  • Each 30-second PSG epoch is transformed into a multichannel time-frequency image using STFT-derived log-power spectra with F = 129, T = 29, and C = 3.Source: A. Time-Frequency Image Representation · p. 3

B. Filterbank Layers

SeqSleepNet uses learned, channel-specific frequency-domain filterbanks to preprocess PSG inputs before attention-based bidirectional recurrent modelling of epoch features.

Filterbank design

  • Channel-specific filterbank layers learn frequency-domain preprocessing for EEG, EOG, and EMG inputs.The filters emphasize task-relevant subbands and attenuate less important ones.Source: B. Filterbank Layers · p. 3
  • Each filterbank layer is a fully connected transformation from F frequency bins to M filters, with M < F.Its weight matrix has dimensions F × M.Source: B. Filterbank Layers · p. 3

Filterbank constraints

  • Sigmoid-constrained weights and a triangular matrix enforce non-negative, band-limited, regulated, frequency-ordered filters.The triangular filterbank matrix is combined with the learned weights by element-wise multiplication.Source: B. Filterbank Layers · p. 3

Filterbank output

  • Filtering a C-channel spectral image produces a smaller C-channel output in the frequency dimension, followed by frequency-wise channel concatenation.The concatenated representation has frequency size MC × T.Source: B. Filterbank Layers · p. 3

Short-term sequential modelling

  • The filtered image is interpreted as a sequence of T feature vectors for an attention-based bidirectional RNN.The RNN processes feature vectors in opposite temporal directions, and attention combines recurrent outputs into one epoch representation.Source: C. Short-term Sequential Modelling · p. 4
  • GRU cells provide the recurrent hidden-layer function, using fewer parameters and greater computational efficiency than LSTM cells.The forward and backward recurrent layers compute hidden-state sequences in opposite directions.Source: C. Short-term Sequential Modelling · p. 4

D. Long-term Sequential Modelling

The sequence-level bidirectional RNN models the ordered sequence of epoch representations to encode long-term information across epochs, then classifies every sequence position.

Long-term sequential modelling

  • A sequence-level bidirectional RNN receives the epoch-wise attentional feature vectors and models long-term dependencies across epochs.Its forward and backward recurrent layers use GRU cells.Source: D. Long-term Sequential Modelling · p. 5

Sequence outputs

  • The sequence-level recurrent outputs form a sequence O = (o1, o2, . . . , oL), with one output vector for each epoch position.Each output vector is computed from the forward and backward hidden states at that position.Source: D. Long-term Sequential Modelling · p. 5
  • A softmax layer converts every output vector into a sequence of sleep-stage classification outputs.Each ŷl is a probability distribution over all sleep stages.Source: D. Long-term Sequential Modelling · p. 5

E. Sequence Loss

SeqSleepNet uses a sequence loss that penalizes misclassification across all labels in an input sequence and includes an ℓ2 regularization term during training.

Sequence loss

  • The sequence loss penalizes misclassification of every element in an input sequence.It compares the ground-truth one-hot labels with the corresponding sequence of classification outputs and is normalized by sequence length L.Source: E. Sequence Loss · p. 5

Optimization

  • The network minimizes the sequence loss over N training sequences.The loss is optimized using the training data’s sequence-level examples.Source: E. Sequence Loss · p. 5
  • λ controls the trade-off between error terms and the ℓ2-norm regularization term.The regularization term is included in the objective alongside the sequence error.Source: E. Sequence Loss · p. 5

F. End-to-End Training Details

End-to-end training is enabled by folding and unfolding inputs at different levels of SeqSleepNet’s hierarchy so each module receives the representation it expects.

Hierarchical data flow

  • The filterbank, epoch-level RNN, and sequence-level RNN operate on spectral columns, epoch images, and sequences of attentional epoch vectors, respectively.These input units define the different levels of the network hierarchy.Source: F. End-to-End Training Details · p. 5

End-to-end training

  • The training procedure adaptively folds and unfolds minibatch data across the hierarchy to train the complete network end-to-end.The minibatch consists of S sequences, each containing L epochs, with each epoch represented as a T × F time-frequency image.Source: F. End-to-End Training Details · p. 5

IV. ENSEMBLE OF DECISIONS AND PROBABILISTIC AGGREGATION

SeqSleepNet fuses multiple decisions for each epoch by aggregating posterior probabilities from overlapping input sequences, then selects the most likely sleep-stage label.

  • Each epoch receives an ensemble of L decisions when the input sequence advances by one epoch during testing.This ensemble excludes recording-end epochs from the full L-decision overlap.Source: §IV · p. 5
  • Multiplicative aggregation combines the posterior probabilities assigned to each candidate sleep stage across overlapping sequences.The candidate stages are W, N1, N2, N3, and REM.Source: §IV · p. 5
  • Aggregation is performed in the logarithm domain to avoid numerical problems when the ensemble becomes large.Source: §IV · p. 5
  • The final predicted label is the sleep stage with maximum aggregated likelihood.Source: §IV · p. 5

V. EXPERIMENTS

Experiments use 20-fold cross-validation on 200 MASS subjects and compare SeqSleepNet with newly developed end-to-end baseline networks.

  • 20-fold cross-validation split 200 MASS subjects into 180 training, 10 validation, and 10 test subjects per fold.Outputs from all folds were pooled to compute overall sleep-staging performance.Source: A. Experimental Setup · p. 6
  • SeqSleepNet experiments varied sequence lengths of 10, 20, and 30 epochs, corresponding to 5, 10, and 15 minutes.Source: B. Network Parameters · p. 6
  • Three novel end-to-end baseline networks were developed to assess SeqSleepNet’s efficiency against existing approaches.Source: C. Baseline Networks · p. 6

Baseline Networks

  • E2E-ARNN combines filterbank layers with an attention-based bidirectional RNN for short-term, one-to-one sleep-stage classification.Its filterbank learning and sleep-stage classification are jointly trained end to end.Source: C. Baseline Networks · p. 6
  • The E2E-DeepSleepNet baseline replaces DeepSleepNet’s separate training stages with an end-to-end variant.Source: C. Baseline Networks · p. 6

D. Experimental Results

Experiments on MASS show that SeqSleepNet benefits from both short- and long-term sequential modelling, achieving strong overall performance and especially improving the difficult N1 stage.

Impact of short-term sequential modelling

  • E2E-ARNN improves overall accuracy by 0.9% over the best single-output CNN while retaining the standard one-to-one setup.It outperforms other single-output CNN architectures by 2.9% to 5.7%.Source: 1) Sleep stage classification… · p. 6

Single output vs multiple output

  • 87.1% overall accuracy, 83.3% macro F1-score, and 0.815 Cohen’s kappa establish SeqSleepNet’s state-of-the-art performance on MASS.The comparison includes the proposed model, developed baselines, and published systems using overall and class-wise metrics.Source: 1) Sleep stage classification… · pp. 6–7
  • 0.7% absolute accuracy gain over E2E-DeepSleepNet makes SeqSleepNet the strongest competitor comparison on the entire MASS dataset.E2E-DeepSleepNet is identified as the best competing system.Source: 1) Sleep stage classification… · p. 7

Benefits of long-term sequential modelling

  • 3.4% absolute average accuracy gain over E2E-ARNN demonstrates the benefit of long-term sequential modelling.The gain is averaged over all experimented sequence lengths.Source: 1) Sleep stage classification… · p. 8
  • 17.2% improvement in N1 accuracy is the clearest class-level effect of long-term sequential modelling, while other stages change subtly.The achieved N1 accuracy also exceeds values reported in previous works.Source: 1) Sleep stage classification… · p. 8

Error analysis

  • 34.5% of transitioning epochs are misclassified, compared with 8.7% of non-transitioning epochs, identifying stage transitions as substantially harder.Transitioning epochs constitute 16.6% of the data, whereas non-transitioning epochs constitute 83.4%.Source: 1) Sleep stage classification… · p. 8

3) Influence of the sequence length and the network’s depth:

Sequence lengths of at least 10 have little effect on performance, and increasing SeqSleepNet’s recurrent depth likewise does not materially change accuracy.

Influence of the sequence length

  • 0.1% is the negligible accuracy variation for sequence lengths L = {10, 20, 30} in both SeqSleepNet and E2E-DeepSleepNet.The results indicate minimal performance impact once the sequence length reaches 10.Source: 3) Influence of the sequence length… · p. 8

Hypnogram

  • SeqSleepNet-20 produces an output hypnogram alongside ground truth, with errors marked and posterior probabilities shown by sleep stage.The visualization concerns subject 22 from the MASS dataset’s SS1 subset.Source: Figure 5 · p. 9

Influence of the network’s depth

  • Increasing recurrent depth from one to two layers does not change SeqSleepNet’s accuracy when the sequence length is considered.The additional experiment stacked GRU cells in both epoch-level and sequence-level recurrent layers.Source: Table III · p. 8, 2) Hypnogram · p. 9

4) Visualization of the learned attention weights:

SeqSleepNet’s learned attention weights highlight stage-relevant signal regions, while its performance gains are concentrated in N1 and REM. The method remains constrained for real-time use and for direct application to sleep-disordered cohorts.

Attention-weight visualization

  • Attention weights emphasize high brain activity and muscle tone in Wake epochs and ocular activity in REM epochs.These patterns correspond to signal characteristics used to distinguish those stages.Source: 4) Visualization of the learned… · p. 9
  • SeqSleepNet’s approximately 1% overall improvement over DeepSleepNet is uneven, with stronger gains on N1 and REM than on N2 and Wake.The authors describe the N1 and REM gains as potentially clinically meaningful for narcolepsy and REM-Sleep Behavior Disorder assessment.Source: §VI · p. 9
  • The sequence-to-sequence design requires access to entire multi-epoch sequences, which can delay online and real-time sleep monitoring.The authors also identify complementary class-wise behavior between SeqSleepNet and DeepSleepNet as a motivation for exploring combinations.Source: §VI · p. 9
  • SeqSleepNet jointly classifies epoch sequences and models both intra-epoch and inter-epoch information with hierarchical recurrent layers.The architecture combines filterbank preprocessing, epoch-level attention-based bidirectional recurrence, and sequence-level bidirectional recurrence.Source: §VII · pp. 9–10

Related papers

Loading 1809.10932v3…