Deep Feature Learning for EEG Recordings

• EEG has low signal-to-noise ratio and spatial resolution, despite millisecond time resolution for recording brain dynamics.Noise can arise from electrical interference, muscle activity, eye movements, or blinks.Source: §1 · p. 1
• EEG datasets are scarce because collection requires specialized equipment and substantial effort, limiting data available to deep-learning researchers.Unlike internet-derived photos or text, EEG data are costly to collect and rarely shared beyond individual labs and institutes.Source: §1 · p. 2
• The paper addresses EEG analysis challenges with proposed techniques and experiments, then discusses results and conclusions.The appendix provides further feature illustrations and implementation details.Source: §1 · p. 2

• Deep-learning applications in cognitive neuroscience, particularly EEG processing, had been limited, while MRI applications had recently demonstrated potential (Plis et al., 2014).Source: §2 · p. 2
• Prior EEG studies used CNNs or DBNs for seizure prediction, epilepsy anomalies, and sleep-stage classification, often on one-second segments.Raw-data classification was reported as comparable to hand-chosen features for seizure-related tasks and close to classification using 28 selected features for sleep staging.Source: §2 · p. 2
• Other neural-network applications included BCI detection of SSVEPs and P300 waves, EEG emotion recognition, and music rhythm and origin classification.The authors' prior work used stacked denoising auto-encoders and CNNs for rhythm perception and distinguishing individual rhythms (Stober et al. (2014b); Stober et al. (2014a)).Source: §2 · p. 2

• OpenMIIR contains EEG recordings from 10 subjects listening to and imagining 12 short music fragments, each 7–16 seconds long.The fragments came from well-known pieces and were selected across genres and musical dimensions.Source: §3 · pp. 2–3
• The experiments analyzed perception trials: five randomized blocks of 12 stimuli yielded 60 trials per subject.Stimuli were normalized in volume, kept as similar in length as possible, and began with complete musical phrases.Source: §3 · p. 3
• EEG was recorded at 512 Hz with 64 channels, filtered to 0.5–30 Hz, and processed to remove bad channels and eye-blink artifacts.One participant was excluded because coughing caused substantial movement artifacts; channels were finally normalized to zero mean and range [−1, 1].Source: §3 · p. 3

4.1 SUPERVISED CNN TRAINING BASELINE

• The supervised baseline compared two-layer CNNs on raw EEG at 64 Hz or 512 Hz with corresponding SVCs.The comparison tested whether the higher timing precision of 512 Hz justified its greater processing time and memory requirements.Source: §4.1 · p. 4

4.2 LEARNING BASIC COMMON SIGNAL COMPONENTS

• For basic common components, a CAE trained on combined subjects initialized separate subject-specific auto-encoders to adapt the shared filters.A weight-change penalty was optional but not needed; the cross-subject validation set was used for early stopping, leaving model-selection bias.Source: §4.2 · pp. 4–5
• Common convolutional auto-encoder features had reconstruction errors similar to PCA and substantially lower than ICA, and supported SVC and CNN classifiers.The authors describe the representation as common across subjects while accommodating individual differences.Source: §4.2 · p. 5

4.3 CROSS-TRIAL ENCODING

• Cross-trial encoding trains a CAE to reconstruct a different trial from the same class, targeting components stable across trials.Pairing trials uses class-label knowledge, while the objective remains reconstruction error; eligible input-target pairs increase training examples.Source: §4.3 · p. 5
• Cross-trial training proceeds from within-subject pairs to cross-subject pairs, with hydra-net pathways selecting encoder and decoder weights by input and target subject.Weights are tied within subjects, and the resulting classifier accuracies are reported in Table 1 for IDs K–O.Source: §4.3 · p. 5

4.4 SIMILARITY-CONSTRAINT ENCODING

• Similarity-constraint encoding trains on triplets so a reference trial is more similar to a same-class trial than to an other-class trial.The network compares dot-product similarities and learns by predicting which paired trial belongs to the reference trial's class.Source: §4.4 · p. 6
• The authors contrast relative triplet comparisons with pairwise same-or-different classification and hypothesize that local constraint contributions yield smoother, more gradual learning.They focus on triplets, although higher-order tuples could add comparisons and make the task harder while providing multiple constraints per example.Source: §4.4 · p. 6
• Across tested configurations, similarity-constraint filters showed the same pattern; more than one filter generally did not improve constraint-encoding performance.At 64 Hz, widths greater than three samples left further time-step channel weights close to zero; classifier accuracies are reported for IDs Q–Y.Source: §4.4 · p. 6, Figure 2 · p. 6

5 DISCUSSION

• At 64 Hz (28,160 dimensions) or 512 Hz (225,280 dimensions), all CNN classifiers beat chance; median cross-validation accuracy stayed below 40%, though some subject-specific accuracies exceeded 50%.The classifiers were trained on a 12-class problem with only 4 examples per class and subject.Source: §5 · p. 7
• Averaging filters across the nine cross-validation models performed without a significant difference from majority voting and produced a single, easier-to-analyze CNN.Source: §5 · p. 7
• Similarity-constraint features improved SVC model T over the raw-EEG baseline at p=0.05; a pretrained spatial filter cut optimization from over a week to a few hours.The average CNN model Y improvement over its baseline was significant at p=0.061.Source: §5 · p. 7
• The experiments used global similarity-constraint filters; the current Hydra-net implementation substantially increases processing time when adapting them to individual subjects.The authors describe this as a technical implementation limitation.Source: §5 · p. 8
• Models R and W matched much larger models in accuracy while remaining simple enough for domain-expert interpretation.The learned filters highlighted electrodes over the auditory cortex and showed similar temporal patterns for tempo-matched song recordings.Source: §5 · p. 8
• Model W reached perfect accuracy for certain binary stimulus pairs despite being trained on the 12-class problem.The authors identify a brain-computer interface setting as an eventual application.Source: §5 · p. 8

6 CONCLUSIONS

• Similarity-constraint encoding yielded the most significant improvements over raw EEG input, while the learned features remained interpretable to domain experts.The conclusion identifies relevant brain regions and musically meaningful temporal characteristics, such as down-beat position.Source: §6 · p. 8

A ADDITIONAL INFORMATION ABOUT THE OPENMIIR MUSIC STIMULI

• Table 2 reports tempo, meter, and length for the OpenMIIR music stimuli.Source: Table 2 · p. 10

B VISUALIZATIONS OF COMMON COMPONENTS

• Figure 3 compares PCA, ICA, and learned auto-encoder channel weights; Figure 4 shows cued-imagination components described as very similar to those from perception trials.Source: Figure 4 · p. 11, Figure 3 · p. 11

C FURTHER VISUALIZATIONS OF CROSS-TRIAL ENCODERS

• Figures 5 and 6 show global filters trained on within-subject trial pairs and individual hydra-net filters trained with cross-subject trials, used by CNNs J/K and N/O, respectively.Figure 5 shows one spatial filter; Figure 6 shows four.Source: Figure 5 · p. 12, Figure 6 · p. 12

D DETAILED CNN CLASSIFIER PERFORMANCE

• Table 3 overviews the best CNN classifiers identified by hyper-parameter optimization.Source: Table 3 · p. 13

E MODEL VISUALIZATIONS

• Figures 7 and 8 visualize CNNs R and W, which process raw EEG at 64 Hz and 512 Hz, respectively, with layer 1 pre-trained using similarity-constraint encoding.Source: Figure 7 · p. 14, Figure 8 · p. 14

• Figure 9 presents the 12-class confusion matrix for CNN W averaged across the nine cross-validation fold models.Source: Figure 9 · p. 15
• Figure 10’s binary matrices use only the 18 test trials belonging to either of the two stimulus classes being compared.Source: Figure 10 · p. 15
• Figure 10’s inset gives chance-likelihood p-values for the binary classification rates, although CNN W was trained only for the 12-class problem.Source: Figure 10 · p. 15

• The code for building the network structures and running the experiments is shared as open source in the deepthought library.Source: G IMPLEMENTATION DETAILS · p. 16
• The implementation is based on Pylearn2 and Theano, with custom Layer and Dataset classes.Source: G IMPLEMENTATION DETAILS · p. 16
• Custom classes support on-the-fly generation of trial tuples and their classification targets during iteration.Source: G IMPLEMENTATION DETAILS · p. 16

• Convolutional auto-encoders encode inputs into compressed representations and decode them toward the original space by minimizing reconstruction error.Source: §G.1 · p. 16
• MSRE measures squared input–reconstruction distance averaged over samples, while MCC averages channel-wise Pearson correlations; MSRE was the superior cost function in this setting.Source: §G.1 · p. 16
• Filters span time in samples and may span frequency bands; width-1 linear auto-encoders are closely related to PCA, while wider or nonlinear designs can capture more complex components.Source: §G.1 · p. 16
• Trials of unequal length were zero-padded for equal-length network inputs, but padded trials should not be used for supervised training because length could become a distinctive feature.Variable length is not problematic for unsupervised basic auto-encoding, whose objective is reconstruction.Source: §G.1 · p. 16
• The deconvolution filters shared weights with convolution filters; tanh was selected to match the input range, and biases were omitted because they did not improve results.Source: §G.1 · p. 16
• Auto-encoder filters learned from perception trials were used to initialize subject-specific auto-encoders; component relations remained evident across subjects and changed only slightly for cued-imagination trials.Source: §G.1 · p. 16
• The current Pylearn2-based implementation limits convolutional auto-encoders with variable-length input, although such auto-encoders can in principle operate on variable-length input.Source: §G.1 · p. 16

G.2 IMPLEMENTING A HYDRA-NET

• Hydra-net layers create individualized copies of an internal network and select the pathway using metadata supplied with each input.Individualization can span one layer or multi-layer subnetworks, provided their input and output formats match.Source: §G.2 · p. 17
• Mini-batch processing either computes every pathway and masks outputs by selector, or processes instances individually through only their selected pathways.The individual-processing strategy uses Theano's scan and lazy ifelse selection.Source: §G.2 · p. 17
• Despite unnecessary computation, the all-pathways strategy performed slightly better in the experiments than the scan-based strategy.The first strategy's overhead worsens as the number of selector values increases; the second incurs a scan-function penalty.Source: §G.2 · p. 17

G.3 TRAINING A CROSS-TRIAL ENCODER WITH CROSS-SUBJECT TRIAL PAIRS

• The encoder selects pathways using the input trial's subject, while the decoder uses the paired trial's subject, with weights shared for each subject.The architecture was used for the cross-trial encoders, with filter shape and number varying between models.Source: §G.3 · p. 17
• Training initialized general filters from 1296 within-subject pairs, continued subject-specific encoders, then trained the hydra-net on 15120 cross-subject pairs.The pair counts derive from 432 training trials; paired trials and selector metadata were generated on demand from a base dataset.Source: §G.3 · p. 17

G.4 CONSTRUCTING AND TRAINING A SIMILARITY-CONSTRAINT ENCODER

• Similarity-constraint encoders process tuples through shared-weight pipelines, compare each reference trial with the others, and predict the trial with the highest similarity.For k-tuples, the method produces k −1 similarity scores; the selected measure is the dot product, matching classifier-output computation.Source: §G.4 · pp. 17–18
• Maximizing similarity only for paired trials is ill-posed because mapping every input to one representation would also make different-class trials maximally similar.The passage also notes that specifying target similarities for every possible trial pair would be impractical.Source: §G.4 · p. 17
• 57024 within-subject triplets pair each training trial with same-class and same-subject positives and different-class negatives.The passages describe tuple construction and the triplet count, but do not provide classifier-performance outcomes.Source: §G.4 · pp. 18–19

G.5 SIMILARITY-CONSTRAINT ENCODERS WITH HYDRA-NET LAYERS

• On synthesized signals, adding a hydra-net first layer produced more specific spatial filters and correctly learned individual channel masks.The signals were embedded in 64-channel Gaussian noise, with the relevant channel varying across nine subjects.Source: §G.5 · p. 19
• Real-data training was harder, and hydra-net processing overhead should be addressed before attempting the 5987520 cross-subject triplets.The passage contrasts these with 57024 within-subject triplets and says training sets of this size remain feasible.Source: §G.5 · p. 19

H CLASSIFIER PERFORMANCE DURING HYPER-PARAMETER OPTIMIZATION

• The plots compare estimated cross-validation error with actual test-set error for averaged models and nine-fold majority votes.Selected models are labeled by their Table 1 IDs; the supplied passages do not state a performance winner.Source: H CLASSIFIER PERFORMANCE DURING… · p. 20