Source-linked AI summary
A Survey on Deep Learning for Neuroimaging-based Brain Disorder Analysis
Li Zhang, Mingliang Wang, Mingxia Liu, Daoqiang Zhang
TL;DR
Brain-disorder neuroimaging analysis needs methods that can handle complex images and limitations in existing diagnostic workflows. This review surveys deep-learning architectures and their applications to four brain disorders, finding state-of-the-art performance across them while identifying data, training, and evaluation challenges. It outlines future directions including human-machine collaboration.
Problem
Complex neuroimages and limited training data challenge computer-aided brain-disorder analysis, while schizophrenia lacks a widely used neuroimaging dataset.
Method
The paper reviews deep-learning architectures and recent neuroimaging studies covering Alzheimer’s disease, Parkinson’s disease, Autism spectrum disorder, and Schizophrenia.
Results
Deep-learning models have achieved state-of-the-art performance across the four brain disorders using brain images.
Takeaways & Limitations
The review identifies human-machine collaboration as a future direction for medical diagnosis and treatment as pathogenesis, techniques, and open datasets develop.
Takeaways & Limitations
Deep-learning methods require many training samples, which are difficult to acquire for neuroimaging data.
Abstract
from arXiv · showhide
Deep learning has been recently used for the analysis of neuroimages, such as structural magnetic resonance imaging (MRI), functional MRI, and positron emission tomography (PET), and has achieved significant performance improvements over traditional machine learning in computer-aided diagnosis of brain disorders. This paper reviews the applications of deep learning methods for neuroimaging-based brain disorder analysis. We first provide a comprehensive overview of deep learning techniques and popular network architectures, by introducing various types of deep neural networks and recent developments. We then review deep learning methods for computer-aided analysis of four typical brain disorders, including Alzheimer's disease, Parkinson's disease, Autism spectrum disorder, and Schizophrenia, where the first two diseases are neurodegenerative disorders and the last two are neurodevelopmental and psychiatric disorders, respectively. More importantly, we discuss the limitations of existing studies and present possible future directions.
1 Introduction
Medical imaging supports disease detection and diagnosis, but interpreting large volumes of complex images is time-consuming and vulnerable to human bias and fatigue. This review focuses on deep learning architectures and their applications to four brain disorders, while discussing limitations and future directions.
- Medical imaging provides anatomical and functional representations that support disease detection, diagnosis, treatment, and research across modalities including MRI and PET.
- Human interpretation of large medical-image collections is time-consuming and susceptible to expert bias and fatigue, motivating computer-assisted diagnosis systems.
- Traditional machine-learning approaches require handcrafted feature selection and have limited representational power for complex medical-image structures.
- Deep learning automatically learns informative representations without domain-specific feature engineering, making it a methodology of choice for medical image analysis.
- Deep learning supports classification, detection or localization, segmentation, registration, retrieval, generation, enhancement, and image-report combination tasks.
- The review surveys architectures and deep-learning studies for Alzheimer’s disease, Parkinson’s disease, Autism spectrum disorder, and Schizophrenia, then discusses limitations and research directions.
2 Deep Learning
This section introduces neural-network foundations used in medical image analysis, from single-layer networks to multilayer feed-forward architectures trained by back-propagation.
- The reviewed deep-learning models include feed-forward networks, stacked auto-encoders, belief networks, Boltzmann machines, GANs, CNNs, GCNs, and RNNs.
- A single-layer network connects input and output layers, whereas a multilayer perceptron adds hidden layers between them.
- Neighboring layers are fully connected in the feed-forward architecture, with no connections between units within the same layer.
- Back-propagation propagates output errors toward the input, computes gradients across layers, and updates parameters until convergence or a preset iteration limit.
2.2 Stacked Auto-Encoders
Stacked auto-encoders deepen auto-encoder representations by composing multiple encoding and decoding stages. Greedy layer-wise pretraining learns successive layers unsupervised and can use unlabeled samples.
- Auto-encoder structure: An auto-encoder learns latent representations unsupervised and reconstructs the input, while stacking multiple auto-encoders forms a deeper SAE.
- Hierarchical representations: Stacking improves representation power and enables lower layers to capture simpler patterns while higher layers capture more complicated ones.
- Architecture: Figure 2 depicts the SAE’s encoding and decoding stages using blue and red dotted boxes, respectively.
- Training: Greedy layer-wise training pretrains each layer in sequence, feeding one hidden layer’s output into the next layer as input.
- Training: Unsupervised pretraining can increase the effective training data by incorporating unlabeled samples and avoid the poor local optima associated with back-propagation.
2.3 Deep Belief Networks
Deep Belief Networks (DBNs) stack multiple restricted Boltzmann machines into a deep generative architecture with directed lower layers and an undirected top pair. Training proceeds through layer-wise pre-training followed by fine-tuning.
- DBNs stack multiple RBMs to construct a deep architecture with one visible layer and multiple hidden layers.
- The lower DBN layers form directed generative models, whereas the top two hidden layers form an undirected RBM.
- The DBN joint distribution combines conditional distributions between successive hidden layers with the joint distribution of the top hidden layers.The conditional term represents each hidden layer given the layer above it, while the top-layer term represents the joint distribution of the final two hidden layers.
- DBN training uses layer-wise RBM pre-training to find the parameter space, followed by fine-tuning to optimize the network parameters.Each layer receives the output representation of the preceding layer as its training input during pre-training.
2.4 Deep Boltzmann Machine
Deep Boltzmann Machines (DBMs) stack RBMs into an entirely undirected model. Each hidden layer uses information from both neighboring layers, increasing robustness to noisy observations while making inference more complex than in DBNs.
- DBMs are constructed by stacking multiple RBMs, but all layers form an entirely undirected model.
- Each DBM hidden layer is conditioned on its two neighboring layers, h(l−1) and h(l+1).The hidden units within the layers are mutually independent under this conditional formulation.
- Using information from both lower and upper hidden layers makes DBM representations more robust to noisy observed data.
- DBM conditional probabilities are more complex than DBN probabilities because DBMs incorporate both neighboring hidden layers.DBNs condition on the upper hidden layer, whereas DBMs condition on the lower and upper hidden layers.
2.5 Generative Adversarial Networks
Generative Adversarial Networks (GANs) use a generator and discriminator in a two-player minimax game. With sufficient capacity, training can make generated and real data distributions coincide, but vanilla GANs face training and evaluation challenges.
- GANs consist of a generator G that maps random noise z to samples and a discriminator D that receives generated or real inputs.
- With sufficient capacity, the generator and discriminator can reach pg = pdata, where D(x) = 0.5 and real and generated samples are indistinguishable to D.
- Vanilla GANs face challenges including model collapse and saddle points, motivating variants such as WGAN and DCGAN.
2.6 Convolutional Neural Networks
Convolutional Neural Networks (CNNs) preserve neighboring pixel or voxel structure by directly processing 2D or 3D images. Their training commonly addresses over-fitting through regularization techniques such as dropout and batch normalization.
- CNNs retain structural information among neighboring pixels or voxels better than vector-input SAE, DBN, and DBM models.They directly accept two-dimensional or three-dimensional images and require minimal preprocessing.
- A CNN is a sequence of layers in which each layer transforms one volume of activations into another through a differentiable function.
- Over-fitting arises from limited training samples relative to the large number of learnable parameters in deep models.
- Dropout randomly removes a fraction of units or connections during each training iteration to help avoid over-fitting.
- Batch normalization regularizes training using running mean–variance statistics from mini-batches and improves generalization performance.The passage also states that batch normalization drastically speeds up training.
2.7 Graph Convolutional Networks
Graph convolutional networks adapt convolutional ideas to non-Euclidean graph data, using graph operations and pooling for graph classification. They include spectral-based and spatial-based approaches, with spatial methods aggregating features from neighboring nodes.
- GCNs adapt CNN-inspired architectures to analyze complex non-Euclidean graph data.The illustrated pipeline uses graph convolution and graph pooling for graph classification.
- A simple GCN transforms traditional data into graph data before applying graph operations and pooling.
- Spectral-based GCNs define convolution through graph-signal filters, whereas spatial-based GCNs aggregate features from spatially close neighbors.
2.9 Open Source Deep Learning Library
Open-source deep learning toolkits make it easier to build deep models, including for users without extensive prior familiarity with deep learning. The section presents popular toolkits and their principal software features.
- Open-source toolkits enable users to build deep models even without extensive familiarity with deep learning.
- Table 2 lists popular deep learning toolkits together with their creators, licenses, platforms, languages, and interfaces.
3 Applications in Brain Disorder Analysis with Medical Images
The review surveys deep learning applications across Alzheimer’s disease, Parkinson’s disease, autism spectrum disorder, and schizophrenia using neuroimaging and related clinical data. It describes disease-specific datasets, model families, and approaches for learning from spatial, temporal, multimodal, and demographic information.
- The review covers four disorders: Alzheimer’s disease, Parkinson’s disease, autism spectrum disorder, and schizophrenia.These include neurodegenerative, neurodevelopmental, and psychiatric conditions.
- Alzheimer’s disease: AD studies use generative and convolutional models, including DBN, DNM, SAE, AE, 2D-CNN, and 3D-CNN methods.
- Alzheimer’s disease: 3D-CNN methods preserve volumetric spatial information that 2D-CNN preprocessing can discard by decomposing brain images into slices.
- Alzheimer’s disease: AD prediction studies also combine synthetic PET with real sMRI, while GCN methods integrate neuroimaging information with demographic relationships for MCI prediction.
- Autism spectrum disorder: ASD methods include AE-, convolutional-, and RNN-based approaches, with 3D-CNN models used to capture spatial structure and temporal statistics in rs-fMRI.
- Schizophrenia: Schizophrenia studies use varied source datasets because no widely used neuroimaging dataset exists for its diagnosis.
4 Discussion and Future Direction
The paper identifies limited training data, missing modalities, multimodal fusion, cross-scale information, and multi-site heterogeneity as continuing challenges in neuroimaging-based brain-disorder analysis. It discusses augmentation, transfer learning, image imputation, intermediate representation fusion, and methods addressing site differences as possible directions.
- Neuroimaging deep learning often lacks the large training samples required for neural-network development.The paper highlights data augmentation and transfer learning as possible responses.
- Missing modalities from patient dropouts and poor data quality can reduce training subjects and degrade diagnosis performance.Recent studies directly impute missing neuroimages, but correspondence with non-imaging data remains unexplored.
- Multimodal fusion remains challenging, with deep networks offering intermediate fusion across learned representations at different abstraction levels.Further investigation is needed to determine which integration layer is optimal for a given problem.
- Different imaging modalities represent brain information at different temporal and spatial scales, complicating their use in early diagnosis and intervention.The paper contrasts minute-scale sMRI information with second-scale fMRI information.
- Multi-site studies face inter-site heterogeneity caused by differing scanning parameters and subject populations.
5 Conclusion
The review finds that deep learning models achieve state-of-the-art performance across four brain disorders using brain images, while identifying challenges and future research directions.
- Deep learning models achieve state-of-the-art performance across Alzheimer’s disease, Parkinson’s disease, Autism spectrum disorder, and Schizophrenia using brain images.
- The paper summarizes potential challenges in neuroimaging-based brain disorder analysis and discusses possible research directions.
- The authors foresee human-machine collaboration for diagnosis and treatment developing toward symbiosis as pathogenesis knowledge, deep learning techniques, and open-source datasets improve.