PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep convolutional neural networks for brain image analysis on magnetic resonance imaging: a review

Jose Bernal, Kaisar Kushibar, Daniel S. Asfaw, Sergi Valverde, Arnau Oliver, Robert Martí, Xavier Lladó

arXiv:1712.03747v3cs.CV

TL;DR

Brain MRI CNN research needs synthesis because performance depends on architecture and processing choices while data, computation, and acquisition variability remain challenging. This paper reviews published deep CNN methods, their strategies and evaluations, and identifies patterns and future directions. The review finds no universal architecture: strong results arise from several designs, while robustness across differing acquisition characteristics remains limited.

  • Problem

    Brain MRI CNN research spans varied architectures and processing strategies, while limited accurate data, computational demands, imbalance, and acquisition variability constrain development and evaluation.

  • Method

    The paper conducts a detailed literature review of deep CNN architectures, preprocessing, data preparation, post-processing, reported public-dataset results, advantages, disadvantages, and future trends.

  • Results

    The review reports no universal architecture; sufficiently deep 2D CNNs, cascades, shortcut connections, and parallel networks showed top performance in their respective applications.

  • Takeaways & Limitations

    CNN research in brain MRI should consider architecture and training choices together with computational cost, class imbalance, and robustness to acquisition differences.

  • Takeaways & Limitations

    CNNs can perform exceptionally on similarly acquired images but tend to fall behind traditional methods when acquisition characteristics vary slightly.

Abstract

from arXiv · show

In recent years, deep convolutional neural networks (CNNs) have shown record-shattering performance in a variety of computer vision problems, such as visual object recognition, detection and segmentation. These methods have also been utilised in medical image analysis domain for lesion segmentation, anatomical segmentation and classification. We present an extensive literature review of CNN techniques applied in brain magnetic resonance imaging (MRI) analysis, focusing on the architectures, pre-processing, data-preparation and post-processing strategies available in these works. The aim of this study is three-fold. Our primary goal is to report how different CNN architectures have evolved, discuss state-of-the-art strategies, condense their results obtained using public datasets and examine their pros and cons. Second, this paper is intended to be a detailed reference of the research activity in deep CNN for brain MRI analysis. Finally, we present a perspective on the future of CNNs in which we hint some of the research directions in subsequent years.

1. Introduction

CNNs have become important in medical imaging, but brain MRI analysis remains constrained by limited, subjective training annotations. This review surveys deep CNN work in brain MRI, its strategies, results, advantages, disadvantages, and future directions.

  • CNNs achieved strong performance in visual recognition, detection, and segmentation and were subsequently applied to medical image analysis.
  • Limited labelled data and subjective, error-prone expert annotations hinder medical image models from matching human-rater accuracy.Creating accurate, representative, preprocessed labels is time-consuming.
  • Brain MRI research covers lesion detection and segmentation, tissue segmentation, and brain parcellation across neonatal, infant, and adult subjects.Public datasets and MICCAI-organised evaluation frameworks support quantitative comparison.
  • The review is presented as the first detailed survey of deep CNN techniques applied to brain MRI analysis, covering publications from 2012 to June 2017.It is also intended as a reading basis for newcomers.
  • The paper introduces CNN concepts, surveys brain MRI methods and their pros and cons, compares reported numerical results, and discusses future trends.

2. Deep convolutional neural networks

Deep CNNs replace manually engineered feature descriptions with learned representations from array-like data. Their development addressed training difficulties such as vanishing gradients while exploiting hierarchical visual features.

  • Conventional supervised learning used prior-knowledge-based, low-dimensional feature vectors and a trainable classifier, making performance depend on heuristic design.
  • A generic CNN applies convolution, activation, and pooling operations, then flattens feature maps in fully connected layers before classification.
  • Representation learning instead discovers suitable detection and classification representations automatically from input data.
  • Early deep-learning solutions to vanishing gradients included greedy layer-wise pretraining followed by supervised fine-tuning and the ReLU activation function.
  • CNNs process array-like data by learning compositional hierarchies in which low-level features form patterns and higher-level structures.

2.1. Building blocks of CNN

CNNs build hierarchical image representations through learned convolutional features, nonlinear activations, and pooling-related operations. The reviewed activation families include ReLU variants, Maxout, and ELU variants, each offering different optimization or performance characteristics.

  • Convolution and pooling extract increasingly complex features while weight sharing handles high-dimensional 2D and 3D data.
  • Activation functions map inputs to sparse representations that can provide invariance to slight data variability and computational efficiency.
  • Rectified linear units (ReLUs) and variants: ReLU variants modify the negative-input slope: α = 0 yields ReLU, a small constant yields Leaky ReLU, and learned α yields PReLU.
  • Rectified linear units (ReLUs) and variants: SReLU learns parameters controlling piecewise behavior and reportedly improves established CNN architectures over ReLU, LReLU, and PReLU on MNIST and ImageNet.
  • Maxout and variants: Maxout computes maxima across K affine feature maps, but increases the number of weights learned in each layer by a factor of K.
  • Exponential Linear Units (ELU) and variants: ELU uses identity behavior for positive inputs and saturated negative outputs, while PELU learns controlling parameters and reportedly improves training speed and performance over ELU and ReLU.

2.1.3. Pooling and sub-sampling layer

Pooling summarizes local activation neighborhoods after convolution and nonlinearity. Common and specialized schemes trade off parameter reduction, input-size flexibility, regularization, deformation handling, and empirical performance.

  • Typical convolutional layers produce feature maps, apply nonlinear activations, and then pool outputs before the next convolutional layer.
  • Pooling summarizes non-overlapping neighborhoods to reduce parameters in later layers and control over-fitting.
  • Max pooling and average pooling differ by using neighborhood maxima versus averages, with max pooling identified as the most common type.
  • Stochastic Pooling: Stochastic pooling samples an activation from each region using probabilities obtained by normalizing the region's activations.
  • Spatial pyramid pooling: Spatial pyramid pooling generates fixed-length representations regardless of image size or scale when inserted before the fully connected layer.
  • Deformation pooling: Deformation pooling models deformation constraints or geometric relationships that max and average pooling cannot learn.
  • Combination of max and average pooling: Mixed max-average and gated max-average pooling generalize max and average pooling and adjust their combination during training.

2.1.4. Fully connected layer

Fully connected layers flatten convolutional feature maps into fixed one-dimensional representations, but their parameter cost and fixed-input, single-output structure limit segmentation use. CNN training also requires suitable losses, imbalance handling, regularization, and augmentation.

  • Fully connected layer: Fully connected layers connect every unit to the previous layer and transform two-dimensional feature maps into predefined one-dimensional feature vectors.
  • Fully connected layer: Fully connected layers require many parameters and produce a single output when used as the output layer, which is inconvenient for segmentation.
  • Fully connected layer: Replacing fully connected layers with 1 × 1 convolutional layers allows models to retain their features while addressing fixed-size input constraints.
  • Loss functions: Softmax loss combines softmax and cross-entropy, while contrastive losses measure similarity and can enforce intra-class similarity alongside inter-class separation.
  • Class imbalance: Class imbalance can bias classifiers toward majority classes, so training may use undersampling, class merging or subdivision, and misclassification penalties.
  • Regularisation: Large CNN parameter counts make regularization important when labelled training data are limited, with L1/L2 penalties, early stopping, and dropout among the available methods.
  • Data augmentation: Data augmentation artificially expands training data, but no theoretical basis identifies which transformation groups improve model generalization.

2.1.7. Optimisation

Deep CNN optimization uses gradient-based parameter updates, but non-convex objectives, learning-rate selection, initialization, and changing activation distributions complicate training. Normalization and adaptive methods are presented as strategies for faster or more stable convergence.

  • Gradient descent updates CNN parameters by back-propagation using stochastic, mini-batch, or batch estimates of the loss.
  • Non-convex loss functions create optimization difficulty through local minima and saddle points, motivating momentum, Nesterov acceleration, and saddle-point escape methods.
  • Adaptive gradient methods adjust each parameter's learning rate instead of relying on one globally shared rate.
  • Weight initialization affects symmetry during back-propagation, and Xavier initialization normalizes each neuron's output variance to one.
  • Normalisation: Contrast normalization mean-centers training data and improves SGD convergence speed, while batch normalization addresses internal covariate shift and permits higher learning rates.

3. CNN methods for brain image analysis

Deep learning-based brain MRI studies have focused primarily on automatic segmentation, especially for lesions, tumours, tissues, the whole brain, and sub-cortical structures. The review organizes these methods within a four-stage pipeline and discusses them in the context of growing MICCAI challenge participation.

  • Most deep learning studies of brain MRI have addressed lesion, tumour, tissue, whole-brain, and sub-cortical structure segmentation.
  • MICCAI challenges have helped drive increasing participation in brain MRI segmentation research using deep learning approaches.
  • The review examines brain MRI methods through preprocessing, data preparation, classification, and post-processing stages.

3.1. Pre-processing

MRI preprocessing prepares images for subsequent segmentation by improving input quality and consistency. The reviewed strategies include skull stripping, artefact and bias-field correction, intensity normalization, and modality registration.

  • Preprocessed and well-annotated training data substantially influences CNN segmentation performance.
  • Skull stripping removes non-brain tissues such as the eyes, fat, spinal cord, and skull from acquired MRI volumes.
  • MRI acquisition artefacts and magnetic-field inhomogeneity can reduce method performance, motivating bias-field correction before or within tissue segmentation.
  • Intensity normalization addresses non-uniform tissue intensities across scanner vendors and acquisition protocols.
  • Image registration transforms different MRI modalities into a common coordinate space when required by the analyzed dataset.

3.2. Data preparation

Data preparation comprises CNN-specific operations performed before network input, principally augmentation, patch extraction, and intensity normalization. These operations address limited training data, overfitting, class imbalance, and the spatial organization of MRI inputs.

  • Data preparation separates CNN-specific operations from generic preprocessing, including data augmentation and patch extraction.
  • Data augmentation increases training samples to mitigate overfitting when deep CNNs face limited labelled data.
  • Augmentation can also alleviate classifier bias caused by strong imbalance between majority and minority classes.
  • Patches may be extracted in 2D, 2.5D, or 3D from one or multiple imaging modalities.
  • After patch extraction, training data are normalized by subtracting mean intensity and dividing by standard deviation.

3.3. CNN architectures

The review classifies CNN architectures by interconnected modules, modalities, patch dimensions, prediction strategy, and contextual information. Across the literature, 2D models were most common, while 3D models captured volumetric context more directly but required greater computational resources.

  • 3.3. CNN architectures: CNN architectures are classified by operating modules, input modalities, patch dimension, predictions per step, and contextual information.
  • 3.3.1. Interconnected operating modules: Single-path networks use one information flow through convolutional, pooling, nonlinear, fully connected, and output layers.
  • 3.3.1. Interconnected operating modules: Parallel multi-path architectures combine complementary features from different network inputs, scales, planes, or dimensionalities.
  • 3.3.1. Interconnected operating modules: Cascaded networks pass one network’s output to another to refine segmentation, increasing overall depth and expressive power.
  • 3.3.3. Patch dimension: 3D architectures performed best in one reported experiment despite being the most computationally expensive.
  • 3.3.3. Patch dimension: 10 of 27 reviewed algorithms used 2D CNNs, which simplify model development but do not fully exploit MRI’s three-dimensional structure.
  • 3.3.5. Contextual information: Patch-based processing limits encoded spatial distribution, while larger patches increase contextual information and the number of trainable parameters.

3.4. Post-processing

Post-processing refines segmentation outputs by creating hard labels or removing false positives, using methods such as connected-component analysis, CRFs, and Grow cut. Contrast-driven refinement can shrink low-contrast sub-cortical and brainstem structures.

  • Post-processing refines segmentation results by creating hard labels or removing false positives.Connected-component analysis and Conditional Random Fields are among the methods used.
  • Connected-component analysis retains only the k largest segmented areas to remove spurious outputs.
  • Contrast-driven refinement may produce unsatisfactory smoothing, shrinking sub-cortical structures and parts of the brainstem in low-contrast regions.The paper illustrates this issue with the mesencephalon and several sub-cortical structures.
  • Grow cut can iteratively refine initial tumour segmentations until a stable segmentation is obtained.It is based on continuous-state cellular automata and user-labelled seed voxels.

4. Evaluation

The review evaluates CNN-based brain MRI segmentation across public datasets and clinical data using standardized frameworks, datasets, and metrics. Reported results span MS lesion, tumour, tissue, and sub-cortical structure segmentation, while limited public training data remains a practical constraint.

  • Public evaluation resources include MICCAI challenges, IBSR, and other datasets covering multiple MRI modalities and segmentation tasks.Dataset summaries report scans, modalities, scanners, tasks, reconstruction matrices, and pixel spacing.
  • The main challenge evaluation measures are DSC, specificity, sensitivity, positive predictive value, ASD, AVD, and MHD.
  • The reviewed CNN models address MS lesion, brain tumour, tissue, and brain structure segmentation tasks.
  • Public datasets often contain too few training samples, encouraging researchers to use private databases and limiting deep CNN performance.
  • 87.0% complete-region DSC, 77.0% core DSC, and 73.0% active-region DSC were reported by Urban et al. on BRATS 2013.
  • 90.6% mean DSC was reported by Wachinger et al. on MICCAI 2012 for sub-cortical structure segmentation.

5. Discussion and future directions

Deep CNNs benefit from patch-based computation and increasingly sophisticated architectures, but brain MRI performance remains constrained by computation, class imbalance, limited data, and acquisition variability. The review therefore emphasizes architecture selection, validation across datasets, domain adaptation, and more efficient training.

  • Patch-based processing lets researchers train deep CNNs on brain MRI while taking computational advantage of small 2D and 3D patches.
  • No universal architecture dominates; depth, cascade designs, shortcut connections, and parallel networks showed top performance in respective applications.
  • Deep CNN analysis requires substantial memory and processing time, while small lesions and structures create severe class imbalance.Proposed responses include two-phase training, careful patch selection, and specialized loss functions.
  • CNNs perform well on similar acquisition characteristics but can fall behind FAST and SPM when test images vary in resolution or intensity range.
  • Future work includes validating models across datasets, advancing domain adaptation, and improving convolution and optimization efficiency.The review mentions FFT methods, faster matrix multiplication, and parallelized or distributed SGD variants.
Loading 1712.03747v3…