A Survey on Deep Learning-based Non-Invasive Brain Signals:Recent Advances and New Frontiers

1.1. General Workflow

• Brain-signal analysis proceeds from collection and preprocessing through feature extraction and classification; it becomes a BCI when decoded signals control equipment.Brain-signal analysis can also support further information analysis without a specific device.Source: §1.1 · pp. 2–3

1.2. Why Deep Learning?

• Artifacts, low signal-to-noise ratio, domain-dependent feature extraction, and rapidly changing data challenge conventional brain-signal systems.The survey describes deep learning as able to work directly on raw signals and capture high-level features and latent dependencies.Source: §1.2 · p. 3

1.3. Why this Survey is Necessary?

• Existing surveys often cover only partial EEG subtypes or specific applications, while few studies examine deep learning across brain signals.The authors position their work among the first comprehensive surveys of recent deep-learning advances in brain-signal analysis.Source: §1.3 · p. 3

1.4. Our Contributions

• The survey reviews brain signals and deep-learning techniques, offers signal-subtype-specific model-selection guidance, and examines applications and future research topics.It summarizes over 200 contributions, most published in the last five years.Source: §1.4 · p. 4

2. Brain Imaging Techniques

• The taxonomy includes EEG, fNIRS, fMRI, and MEG, while the survey focuses mainly on EEG and its subcategories.The section introduces typical non-invasive imaging techniques and refers readers to Appendix A for further details.Source: §2 · p. 4
• EEG is divided into spontaneous signals and evoked potentials; evoked potentials split into ERP and SSEP, with visual, auditory, and somatosensory types.Spontaneous EEG includes sleep, motor imagery, emotional, and mental-disease signals.Source: §2 · pp. 4–5
• The taxonomy omits dashed categories without deep-learning studies, places P300 under ERP, and excludes non-EEG visual or auditory tasks lacking such studies.Source: Figure 2 · p. 5

• Deep learning uses layered information-processing stages for pattern classification and feature or representation learning.Source: §3 · p. 5
• Discriminative models learn distinguishing features and classify inputs into known labels; examples include MLP, RNN, and CNN.LSTM and GRU are identified as mainstream RNNs.Source: §3 · pp. 5–6, Figure 3 · p. 6
• Representative models extract features but do not classify; common examples are AE, RBM, and DBN.Source: §3 · p. 6
• Generative models learn the joint distribution of input data and target labels and are mostly used to generate brain-signal samples for training-set enhancement.VAE and GAN are examples used in brain-signal analysis.Source: §3 · pp. 6–7, Figure 3 · p. 6
• Hybrid models combine more than two deep learning models, often using representation methods for feature extraction and discriminative methods for classification.A DBN combined with softmax remains representative rather than hybrid because softmax is not counted as an algorithmic component.Source: §3 · p. 7

4.1.1. Spontaneous EEG

• Sleep EEG studies use CNN, RNN/LSTM, and DBN-RBM models, while motor-imagery work spans discriminative, representative, and hybrid methods, including a CNN-RNN model with 72.22% average accuracy on a public dataset.Sleep-stage pipelines commonly filter EEG and segment it into 30-second windows.Source: §4.1.1 · pp. 7, 9
• 77.88% mean accuracy was reported for person-independent emotional EEG recognition using subspace alignment autoencoding to address distribution mismatches across subjects or sessions.Emotion studies also use CNN/RNN discriminative models, DBN-RBM representations, and hybrid models combining feature extraction and classification.Source: §4.1.1 · p. 9
• CNNs are widely used for EEG disease detection, with reported accuracies including 93.5% and 96.0% for depression using left- and right-hemisphere signals, and 81–83% for early-stage Creutzfeldt-Jakob disease.Other approaches include representative DBN/autoencoder models and CNN-LSTM architectures for seizure detection.Source: §4.1.1 · pp. 9–10
• GAN-based seizure-data augmentation raised classification accuracy from 48% to 82%; spontaneous-EEG models also address workload, driver fatigue, and eye-state detection.Reported examples include 95.48% accuracy for mental-workload classification and 98.9% for eye-state detection.Source: §4.1.1 · pp. 10–11
• ERD/ERS describes event-related EEG power decreases and increases, but its use is limited by unstable cross-subject accuracy; analyses use relative power changes against baseline and indicate that energy dynamics matter.The section describes ERD in the mu band (8–13 Hz) and ERS in the beta band (13–30 Hz).Source: §4.1.1 · p. 11

4.1.2. Evoked Potential

• For visual evoked potentials and P300 tasks, studies use CNN, DBN-RBM, autoencoder, and hybrid CNN-LSTM approaches, with reported P300-speller accuracy reaching 95.5%.A concealed-information test using DBN-RBM features and an SVM reported 97.3% accuracy on a local dataset.Source: §4.1.2 · p. 12
• AEP studies found CNNs effective at extracting temporal and spatial features, while RSVP CNN performance was good cross-task but unsatisfactory cross-subject; reported RSVP AUC reached 86.1%.For one RSVP comparison, MLP, CNN, and DBN achieved AUCs of 81.7%, 79.6%, and 81.6%, respectively.Source: §4.1.2 · p. 12
• For SSVEP, a CNN-RNN hybrid achieved 93.59% accuracy, a compact CNN had approximately 80% cross-subject mean accuracy, and independent CNN and LSTM models achieved 69.03% and 66.89%.The cited studies include direct analysis of raw signals and frequency-filtered, FFT-processed signals.Source: §4.1.2 · pp. 12–13
• SSVEP models included sparse-AE classification at 97.78%, DBN-RBM and DBN-AE at 86.9% and 86.01%, and a dry-electrode CNN at 96% on a local dataset,, [193].Dry-electrode signals were more challenging because of lower SNR; another MLP study examined fatigue and wheelchair-command switching among manual, semi-autonomous, and autonomous modes [193],.Source: §4.1.2 · p. 13

4.2. fNIRS

• fNIRS classifiers included MLPs reporting 96.3% accuracy for mental arithmetic versus rest and 66.48% for three mental states, while robot control reached 82% offline and 66% online,,.The studies used manually extracted or hand-crafted features,.Source: §4.2 · p. 13
• fNIRS denoising-autoencoder features supported gender detection at 81% average accuracy, and fNIRS was also combined with EEG features for left/right motor-imagery classification,.The survey passage notes that fNIRS has higher time resolution and is more affordable than PET and fMRI.Source: §4.2 · p. 13

4.3. fMRI

• fMRI CNNs supported tumor segmentation and disease recognition, while one three-class brain-tumor model achieved an F1 score of 88%,,.Havaei et al. used differently sized filters to capture local and global features; CNN-extracted features were also classified by SVM for epileptic-seizure detection, and CNNs were described as promising for fMRI analysis.Source: §4.3 · p. 13
• A CNN that completed PET patches from fMRI improved classification to 92.87%, versus 91.92% with fMRI alone.Koyamada et al. used a nonlinear MLP to extract common features across subjects on an HCP dataset [202].Source: §4.3 · p. 13
• Representative fMRI models included targeted autoencoders, DBNs, and multi-view learning for neurological-disease classification,,,,.Reported results included 87.5% accuracy for targeted-AE Alzheimer’s diagnosis, average F1 above 90% across four public datasets for DBN-RBM, 97.9% for DBN-AE versus 95.4% for DBN-RBM, and 72.58% accuracy for MCI detection; another approach combined MRI and PET representations with voting across SVM classifiers.Source: §4.3 · pp. 13–14
• fMRI GANs were used to reconstruct visual stimuli, generate synthetic multi-sequence scans, and recover images; DCGAN outperformed WGAN for synthetic fMRI generation,,.The synthetic scans were proposed for data augmentation and physician training.Source: §4.3 · p. 14

4.4. MEG

• MEG deep-learning studies addressed artifact removal, single-word decoding, and visual-object recognition; artifact-refinement CNNs achieved 85% sensitivity and 97% specificity locally,,,.The single-word sparse-autoencoder approach helped some subjects but did not improve overall decoding accuracy.Source: §4.4 · p. 14

5.1. Health Care

• Healthcare applications use brain signals to detect sleep disorders, Alzheimer’s disease, epileptic seizures, and other conditions, often using EEG or fMRI.Sleep-stage studies commonly use spontaneous EEG from healthy participants; Ruffini et al. reported 85% average accuracy for REM Behavior Disorder recognition, while several fMRI Alzheimer’s studies exceeded 90% accuracy.Source: §5.1 · p. 14

5.2. Smart Environment

• Brain-controlled robots and exoskeletons could support smart-home assistance and help people with lower-limb motor impairments walk and manage daily activities.The authors identify brain-controlled appliances for older adults and people with disabilities in smart homes and hospitals as a future research direction.Source: §5.2 · p. 15

5.3. Communication

• P300 spellers and deep learning can translate users’ brain signals into text, enabling communication for people who have lost most motor abilities.The passage describes detecting P300 segments and focused characters, as well as an RNN-CNN-AE model that recognizes intended letters from MI EEG.Source: §5.3 · p. 15

5.4. Security

• EEG biometrics offer a spoof-resistant alternative for identifying or authenticating people, with systems using VEP, RSVP, and MI signals.The passage explains that EEG signals are difficult for imposters to mimic and also describes a hybrid EEG-and-gait dual-authentication system.Source: §5.4 · p. 15

5.5. Affective Computing

• Deep learning supports EEG-based affective computing, including emotion classification and recognition of arousal, valence, dominance, or liking dimensions.Yin et al. reported arousal of 77.19% and valence of 76.17% on DEAP using an ensemble classifier of autoencoders.Source: §5.5 · p. 15

5.6. Driver Fatigue Detection

• EEG-based driver-fatigue detection is reported at 82%–95% accuracy and is presented as feasible with portable, affordable headsets.The authors identify keeping drivers alert during self-driving requests to intervene as a future use for brain-signal fatigue detection.Source: §5.6 · p. 19

5.7. Mental Load Measurement

• EEG and fNIRS deep-learning systems measure mental workload, supporting operator-performance monitoring and potentially helping prevent accidents.Reported results include 96.3% accuracy for fNIRS binary mental-task classification, 92% overall accuracy for single-trial EEG workload recognition, and 88.9% for binary workload classification across multiple tasks.Source: §5.7 · p. 19

5.8. Other Applications

• Beyond the main application areas, brain signals are explored for recommender systems, emergency braking, and visual-object recognition for guilty-knowledge or concealed-information tests.Cecotti et al. aimed to build a common RSVP target-recognition model that works across subjects rather than for a specific subject.Source: §5.8 · p. 19
• fNIRS cerebral blood-flow patterns differed by gender, while spontaneous EEG CNNs identified gender and highlighted fast beta activity (20 ∼25 Hz).Source: §5.8 · p. 19

5.9. Benchmark Datasets

• The survey catalogs public brain-signal datasets with download links across most signal types, including five BCI-C IV datasets accessible through one link.Source: §5.9 · p. 19
• Table 6 reports subject count, class count, sampling rate, and channel count; channel listings focus on brain signals, excluding biometric channels such as ECG.Source: §5.9 · pp. 19–20, Table 6 · p. 20

6.1. Brain Signal Acquisition

• EEG dominates the surveyed non-invasive signal studies, including 133 spontaneous-EEG publications, while fNIRS, fMRI, and MEG receive less attention.fNIRS studies rarely use deep learning beyond simple MLPs; 23 fMRI classification papers commonly use CNNs, whereas only a few MEG studies are reported.Source: §6.1 · pp. 20–21

6.2. Selection Criteria for Deep Learning Models

• Discriminative models are most frequent, and CNNs or variants account for more than 70% of them.The survey relates CNN use to discriminative feature and spatial-dependency learning, prior success and available code, and the image-like form of some signals.Source: §6.2 · p. 21
• CNNs can process signals already represented as images or EEG arranged as channel-by-time matrices, while single-channel EEG can use 1-D CNNs.The survey describes converting each time point into a spatial image or arranging a segment as a matrix whose rows and columns represent channels and time points.Source: §6.2 · p. 21
• RNNs appear in about 20% of discriminative-model studies, and the survey’s preliminary experiments found them over 20 times slower to train than CNNs on long sequences.DBN-RBM was a popular feature extractor, especially in work published before 2016; more recent studies increasingly combine feature learning and classification in CNNs or hybrid models.Source: §6.2 · pp. 21–22
• Generative models are rarely used alone; GANs and VAEs mainly support data augmentation and image reconstruction for EEG and fMRI.The survey reports more competitive classifier performance after augmentation and identifies 53 hybrid-model publications, including CNN-RNN combinations and representative-model-plus-classifier designs.Source: §6.2 · p. 22

6.3. Application Performance

• Sleep-stage, disease-diagnosis, smart-environment, security, and affective-computing applications are accompanied by practical data and study constraints.The survey notes only a few drive-fatigue studies, citing expensive experimental costs and a lack of accessible datasets.Source: §6.3 · pp. 22–23
• Sleep-stage models generally exceed 85% accuracy, while Alzheimer’s fMRI classification exceeds 90% and epileptic-seizure detection often exceeds 95%.For sleep-stage classification, CNN-LSTM hybrids offer only incremental improvement; seizure studies attribute high accuracy partly to binary classification being easier than multi-class classification.Source: §6.3 · p. 22
• Brain-signal smart environments appear in few publications, while security applications offer high fake-resistance but face expensive equipment and inconvenient EEG-headset use.Affective-computing studies largely use spontaneous EEG; on SEED, discriminative models appear to outperform representative-model pipelines by a small margin.Source: §6.3 · pp. 22–23

7.1. Explainable General Framework

• A general framework should handle varied channels, sample dimensions, and stimuli through attention and latent-feature capture, while learned-feature interpretation may matter more than prediction.The authors suggest combining CNN feature extraction with attention mechanisms; attention scores or reinforcement learning could identify valuable input regions.Source: §7.1 · p. 23

7.2. Subject-Independent Classification

• Person-independent classification remains a future goal because most tasks train and test on the same individual; proposed routes include transfer learning and subject-independent features.A hybrid multi-task model could jointly perform person identification and class recognition.Source: §7.2 · p. 23

7.3. Semi-supervised and Unsupervised Classification

• Expensive manual labeling limits training data; semi-supervised and unsupervised learning are presented as better suited to settings with little ground truth.Suggested strategies include crowd-sourcing labels and aligning source and target signal distributions through unsupervised domain adaptation.Source: §7.3 · pp. 23–24

7.4. Online Implementation

• 10% lower accuracy is reported generally for online brain-signal systems than offline counterparts; post-processing can improve decoding but adds latency.EEG live signals are noisier and less stable, while online evaluation is relatively less challenging for fNIRS and fMRI.Source: §7.4 · p. 24

7.5. Hardware Portability

• Hardware options trade portability, signal quality, comfort, and cost; the proposed direction is cheaper, more comfortable wireless ear-EEG.The survey gives targets below $200 and more than three hours of use without discomfort.Source: §7.5 · p. 24

8. Conclusion

• The survey reviews deep-learning techniques, applications, challenges, and future directions for non-invasive brain-signal analysis, and offers guidance on selecting models for signal categories.It contrasts deep learning with traditional machine learning by describing automatic high-level feature learning and less dependence on domain knowledge.Source: §8 · p. 24

32 Appendices

• Non-invasive brain imaging methods use electrical, magnetic, or metabolic collection and include EEG, fNIRS, fMRI, and MEG.Source: A. Non-invasive Brain Signals · p. 32

A.1. Electroencephalography (EEG)

• EEG electrodes measure neuronal voltage fluctuations on the scalp, typically using the 10-20 placement system; each channel records a signal stream over time.EEG has low spatial resolution due to volume conduction, while its temporal resolution exceeds 1000 Hz.Source: §A.1 · p. 32, Figure 5 · p. 32
• EEG generally has poor SNR, while portable headsets and high temporal resolution are advantages compared with other non-invasive techniques.Its SNR is affected by environmental noise, skull and tissue obstruction, stimulation, and subject state; EEG also avoids exposure to high-intensity magnetic fields.Source: §A.1 · p. 32
• EEG contains Delta, Theta, Alpha, Beta, and Gamma frequency bands, whose patterns have corresponding characteristics and intra-band correlations with behavioral states.Source: §A.1 · p. 32, Table 7 · p. 33
• EEG is divided into spontaneous EEG and evoked potentials, with the latter split into event-related and steady-state potentials by stimulus frequency.Several potential subtypes are listed for systematic completeness but excluded from the survey because few deep-learning studies address them.Source: §A.1 · p. 32

A.1.1. Spontaneous EEG

• Spontaneous EEG records activity without external stimulation, including sleep, mental tasks, disorders, motor imagery, and emotion; image-gazing is treated as a state without changing stimuli.These systems are challenging to train because of lower SNR and greater variation across subjects.Source: §A.1.1 · p. 33

A.1.2. Evoked Potential (EP)

• ERP follows isolated discrete stimuli, while SSEP responds to periodic stimulation; their typical stimulus frequencies are below 2 Hz and 3.5–75 Hz, respectively.Evoked potentials are time-locked to external stimuli and generally have higher amplitude and lower frequency than spontaneous EEG.Source: §A.1.2 · pp. 33–34

A.2. Functional Near-infrared Spectroscopy (fNIRS)

• fNIRS uses near-infrared light to measure oxygenated and deoxygenated hemoglobin, but cannot measure cortical activity deeper than 4 cm.Its temporal resolution is lower than electrical or magnetic signals; spatial coverage depends on emitter-detector pairs.Source: §A.2 · p. 35

A.3. Functional Magnetic Resonance Imaging (fMRI)

• fMRI measures BOLD response magnetically, with greater penetration and spatial resolution than fNIRS but similar temporal resolution; its scanners are expensive and poorly portable.For fMRI image quality, the survey prefers time-series CNR over (t)SNR.Source: §A.3 · p. 35

A.4. Magnetoencephalography (MEG)

• MEG records magnetic fields from naturally occurring brain electrical currents, offering temporal resolution of at least 1000 Hz but relatively low spatial resolution.Its equipment is expensive and nonportable, limiting real-world deployment.Source: §A.4 · p. 35

B. Basic Deep Learning in Brain Signal Analysis

• The survey defines T(x, x′) as the weighted sum of two inputs plus their respective biases, using weights and biases for each variable.T(x) is also introduced as a single-input operation.Source: B. Basic Deep Learning in Brain Signal… · p. 36

B.1. Discriminative Deep Learning Models

• Discriminative models map brain-signal observations to their corresponding labels and require both observations and ground-truth labels for supervised learning.The survey motivates its model overview as support for readers from non-computer-science areas, including biomedicine.Source: §B.1 · p. 36, B. Basic Deep Learning in Brain Signal… · p. 36

B.1.1. Multi-Layer Perceptron (MLP)

• An MLP has multiple fully connected hidden layers, unlike the basic fully connected network with one hidden layer, which the survey does not generally regard as deep.MLP nodes connect to nodes in adjacent layers, not to other nodes in the same layer.Source: §B.1.1 · p. 36, Figure 7 · p. 37
• The MLP feeds observations forward through hidden layers, applying a nonlinear activation such as sigmoid, before producing predictions.The section uses a two-hidden-layer network to illustrate the data flow.Source: §B.1.1 · p. 36
• Training compares one-hot predictions with ground-truth labels, back-propagates the loss, and adjusts model weights and biases until convergence.The survey names Euclidean-distance error as one example and lists cross-entropy, negative log likelihood, and mean square estimation among common losses.Source: §B.1.1 · p. 36
• ANN and DNN are often used broadly or synonymously, but MLP refers specifically to fully connected networks; ANN may be shallow, whereas DNN excludes shallow networks.The survey notes that terminology is not strictly distinguished in the literature.Source: §B.1.1 · p. 36

B.1.2. Recurrent Neural Networks (RNN)

• RNNs capture temporal dependencies by combining each input with a hidden state that carries information from earlier time steps.The survey describes the hidden state as a form of node memory.Source: §B.1.2 · p. 37
• LSTM controls memory with four gates and a transmittable hidden state, while GRU uses two gates and receives temporal information through the previous output.The survey reports comparable performance in literature, with GRU faster to train and requiring less data for generalization; LSTM generally works better with sufficiently large training datasets.Source: §B.1.2 · p. 37

B.1.3. Convolutional Neural Networks (CNN)

• CNNs target spatial information in brain signals; convolution extracts features, pooling progressively reduces spatial size and computation, and fully connected layers connect prior activations.The survey identifies fMRI images and spontaneous EEG as examples of inputs whose spatial dependencies CNNs can exploit.Source: §B.1.3 · p. 38

B.2. Representative Deep Learning Models

• Representative models such as autoencoders, restricted Boltzmann machines, and deep belief networks learn dense input representations without ground-truth labels.The survey describes representation learning as producing inputs more suitable for downstream tasks such as classification, generation, or clustering.Source: §B.2 · p. 39

B.2.1. Autoencoder (AE)

• Autoencoders encode observations into a latent code and decode it back to the original space, training by minimizing reconstruction error against the input.Source: §B.2.1 · p. 39
• Because the input itself supplies the reconstruction target, an AE can learn without ground-truth labels.Source: §B.2.1 · p. 39
• Deep-AE generally uses 2n + 1 hidden layers, with n encoder layers, a shared code layer, and n decoder layers.Source: §B.2.1 · p. 39

B.2.2. Restricted Boltzmann Machine (RBM)

• An RBM is a two-layer stochastic network with no within-layer connections and tied encoder and decoder weights; deep-RBM stacks hidden layers for encoding and reconstruction.Source: §B.2.2 · p. 40

B.2.3. Deep Belief Networks (DBN)

• DBN-AE and DBN-RBM stack component networks by passing one hidden layer into the next; unlike deep AE/RBM models, DBNs train each component greedily after the preceding one converges.Source: §B.2.3 · p. 40

B.3. Generative Deep Learning Models

• Generative models produce similar brain-signal samples to augment training data before discriminative classification, particularly when training data is limited.Source: §B.3 · p. 41

B.3.1. Variational Autoencoder (VAE)

• VAE forces latent representations into a continuous prior-distributed space, allowing random sampling and interpolation that standard AE representations do not support.Source: §B.3.1 · p. 41

B.3.2. Generative Adversarial Networks (GAN)

• GANs jointly train a generator and discriminator; after convergence, the generator can create brain signals to enlarge the training set for classification.Source: §B.3.2 · p. 42

B.4. Hybrid Model

• Hybrid models combine at least two deep model types; most brain-signal studies target classification, while a few non-classification hybrids reconstruct signals from fMRI.Source: §B.4 · p. 42