Source-linked AI summary
Deep learning for neuroimaging: a validation study
Sergey M. Plis, Devon R. Hjelm, Ruslan Salakhutdinov, Vince D. Calhoun
TL;DR
Neuroimaging needs methods that can learn physiologically meaningful representations and reveal latent structure, but deep learning’s flexibility makes transfer and parameter selection difficult. The paper evaluates deep learning on fMRI and sMRI, comparing RBMs with ICA, studying depth, and introducing constraint-based visualization. It finds physiologically meaningful representations, improved group separation with depth, and potential for exploratory neuroimaging analysis.
Problem
Deep learning’s flexibility makes porting it to neuroimaging a difficult parameter-optimization problem, despite the field’s need for feature learning and latent-structure discovery.
Method
The study applies RBMs and deeper belief networks to fMRI and sMRI, compares shallow RBM features with ICA, examines depth effects, and uses constraint-based nonlinear embedding for visualization.
Results
Deep learning learns physiologically meaningful representations, RBMs perform competitively with ICA, deeper networks increase group separation, and DBNs show potential for exploratory analysis.
Takeaways & Limitations
Deep learning has high potential for neuroimaging applications, including feature learning, classification, and exploratory analysis of latent structure.
Takeaways & Limitations
The constraint-based embedding may fail to satisfy its neighborhood constraint on general datasets and is guaranteed to converge only when a solution exists.
Abstract
from arXiv · showhide
Deep learning methods have recently made notable advances in the tasks of classification and representation learning. These tasks are important for brain imaging and neuroscience discovery, making the methods attractive for porting to a neuroimager's toolbox. Success of these methods is, in part, explained by the flexibility of deep learning models. However, this flexibility makes the process of porting to new areas a difficult parameter optimization problem. In this work we demonstrate our results (and feasible parameter ranges) in application of deep learning methods to structural and functional brain imaging data. We also describe a novel constraint-based approach to visualizing high dimensional data. We use it to analyze the effect of parameter choices on data transformations. Our results show that deep learning methods are able to learn physiologically important representations and detect latent relations in neuroimaging data.
1 Introduction
Deep learning is attractive for neuroimaging because it can learn features automatically, while model depth may capture indirect cross-modality relations. This study validates deep learning on fMRI and sMRI by examining RBM competitiveness, depth effects, and latent-structure discovery.
- Neuroimaging seeks descriptive features of brain structure and function from non-invasive measurements, motivating data-driven feature-learning methods.
- Deep learning automatically learns features from data, contributing substantially to classification improvements reported across speech, signal, image, video, and text tasks.The passage contrasts occasional improvements exceeding 30% with prior-decade gains of 1–2%.
- Deep models may better represent indirect cross-modality relations, such as relationships between genetics and phenotypes occurring at deeper conceptual levels.
- The study applies deep learning to functional and structural MRI to test RBM competitiveness with ICA, examine structural-MRI depth effects, and assess latent-structure discovery.It also addresses visualization of transformations because feature-learning performance measures and classification accuracy are incomplete for deeper models.
2 A shallow belief network for feature learning
The shallow RBM is evaluated as a neuroimaging feature-learning model, with comparisons to matrix-factorization methods and Infomax ICA on synthetic and fMRI data. It models voxel-level data, computes feature time courses, and performs competitively with ICA while producing sharper, more localized features.
- RBM applies hidden-unit receptive fields to the high-dimensional spatial dimension, whereas ICA applies its weight matrix to the shorter temporal dimension.
- 2.1 A restricted Boltzmann machine: An RBM models the data distribution with a Gibbs distribution over visible and hidden variables connected by a bipartite graph.For fMRI, visible variables represent approximately Gaussian voxel values.
- 2.2 Synthetic data: RBM and ICA showed the best overall performance across synthetic datasets, while sNMF estimated spatial maps well but performed worse for time courses.The comparison included Infomax ICA, PCA, sparse PCA, and sparse NMF.
- 2.2 Synthetic data: RBM and ICA had similar synthetic-data performance, with ICA slightly better for spatial-map estimation and RBM slightly better for time-course estimation.Their functional network connectivity estimates were also comparable.
- 2.3 An fMRI data application: For fMRI application, a 70969-unit Gaussian-visible, 64-unit hyperbolic-tangent-hidden RBM used ϵ=0.08 and λ=0.1 selected from searched ranges.The selected values reduced reconstruction error or receptive-field span; values outside the ranges caused unstable or slow learning or uninterpretable features.
- 2.3 An fMRI data application: The fMRI workflow normalized volumes, used feed-forward RBM activations to compute feature time series, and excluded features likely representing white matter, ventricles, or artifacts.The resulting features were compared with those obtained by Infomax ICA.
- 2.3 An fMRI data application: RBM performed competitively with ICA while providing sharper and more localized features, with feature-time-course correlations forming physiologically supported groups.The authors note that this sharper localization assessment is subjective and may relate to L1 regularization.
3 Validating the depth effect
The paper evaluates whether increasing DBN depth improves structural MRI representations and classification, while using a constraint-based embedding to inspect learned neighborhood relations. It combines unsupervised layerwise pre-training, supervised fine-tuning, and a simple local-neighborhood mapping to separate the effects of deep transformations from the embedding procedure.
- Motivation: DBNs are evaluated on sMRI because each subject or session is represented by one labeled brain volume, enabling depth-specific classification comparisons.The study asks whether the commonly reported accuracy benefits of depth also hold for structural neuroimaging data.
- Model and training: DBN training uses unsupervised RBM pre-training followed by supervised feed-forward fine-tuning with a soft-max layer.Each layer is trained as an RBM on the previous layer’s inputs before backpropagation-based fine-tuning.
- Constraint-based embedding: The embedding preserves each point’s k nearest neighbors in the higher-dimensional space while leaving most mapping decisions to the DBN transformation.This design is intended to assess how learned representations affect neighborhood organization rather than attribute map quality to a complex embedding method.
- Experimental design: The tested architecture uses 50 hidden units in the first two layers and 100 in the third, with depths 1, 2, and 3 fine-tuned for comparison.The models use all-to-all interlayer connectivity and larger networks produced similar results.
- Results: All models show only a slight accuracy increase from depth 1 to depth 2, followed by a significant improvement at greater depth.This supports the reported depth-related classification trend for sMRI data.
- Results: Deeper networks increasingly separate patients and controls in 2D maps, including 54 held-out subjects, indicating continued transformation beyond the depth-2 classification change.The maps use activations from networks trained on 335 subjects and show further separation with depth.
- Results: For Huntington disease data, depth-3 representation capacity recovers the F-score drop observed at shallower depths, but the analysis uses no testing data.The authors distinguish this representational-capacity result from the earlier generalization evaluation.
4 Conclusions
Deep learning shows promise for neuroimaging: shallow RBMs learn physiologically meaningful features, while DBN depth improves classification and group separation across varied sMRI datasets.
- Shallow RBMs produce physiologically meaningful, focal features whose time-course correlations connect them into functional groups.
- DBN depth improves classification and increases group separation across two sMRI datasets collected under varying conditions and preprocessing.
- DBNs show robustness across multiple sites, disease groups, acquisition conditions, and preprocessing pipelines.
- Combining a DBN with a new mapping method can reveal hidden relations in neuroimaging data.
- The paper provides a baseline intended to help researchers identify workable parameter regions for deep-learning neuroimaging analyses.
5 Supplementary material
RBM representations exhibit greater modularity than ICA representations in the supplementary fMRI analysis, although RBM also overestimates strong functional-network correlations.
- RBM modularity averaged 0.40 ± 0.060 across subjects, compared with 0.35 ± 0.056 for ICA.
- The modularity difference was significant, with t = 7.15 and p < 1e−6 in a paired t-test.
- RBM’s larger correlation scale highlights that it overestimated strong functional-network connectivity values.
- Figure S1 presents subject-averaged RBM and ICA correlation matrices with method-specific component ordering and different color scales.
- Figure S2 shows RBM–ICA sample-map pairs thresholded at 2 standard deviations, with pairings aided by spatial correlations, temporal properties, and visual inspection.