Source-linked AI summary
Machine Learning Techniques for Biomedical Image Segmentation: An Overview of Technical Aspects and Introduction to State-of-Art Applications
Hyunseok Seo, Masoud Badiei Khuzani, Varun Vasudevan, Charles Huang, Hongyi Ren, Ruoxiu Xiao, Xiao Jia, Lei Xing
TL;DR
Medical image segmentation requires accurate delineation of structures, but available training data and computational resources constrain different learning paradigms. This review surveys classical and deep learning methods, their applications, training challenges, and heuristics. It reports that classical methods can be sample-efficient and competitive with deep networks in small-data settings, while deep models benefit substantially from more training data.
Problem
Biomedical image segmentation needs accurate and efficient methods, while medical datasets are often scarce and annotated-data curation remains a bottleneck for supervised deep learning.
Method
The paper reviews classical machine learning methods, deep learning architectures, medical-segmentation applications, training challenges, and heuristics for addressing those challenges.
Results
Classical methods can outperform deep networks on small training datasets, whereas increasing sample size significantly improves FCN performance; atlas-based segmentation reaches 98.0% versus 94.0% for deep learning on mandible segmentation.
Takeaways & Limitations
Kernel SVMs are sample-efficient and computationally simpler for small medical-imaging training sets, while deep architectures require sufficient data to avoid overfitting.
Takeaways & Limitations
Deep-learning segmentation depends on large labeled datasets, yet accessible high-quality medical data and annotated training samples are often scarce.
Abstract
from arXiv · showhide
In recent years, significant progress has been made in developing more accurate and efficient machine learning algorithms for segmentation of medical and natural images. In this review article, we highlight the imperative role of machine learning algorithms in enabling efficient and accurate segmentation in the field of medical imaging. We specifically focus on several key studies pertaining to the application of machine learning methods to biomedical image segmentation. We review classical machine learning algorithms such as Markov random fields, k-means clustering, random forest, etc. Although such classical learning models are often less accurate compared to the deep learning techniques, they are often more sample efficient and have a less complex structure. We also review different deep learning architectures, such as the artificial neural networks (ANNs), the convolutional neural networks (CNNs), and the recurrent neural networks (RNNs), and present the segmentation results attained by those learning models that were published in the past three years. We highlight the successes and limitations of each machine learning paradigm. In addition, we discuss several challenges related to the training of different machine learning models, and we present some heuristics to address those challenges.
1. Introduction
Biomedical image segmentation clusters images into coherent feature-based sub-regions and classifies them, supporting knowledge extraction and computer-aided diagnosis across imaging modalities. The review introduces supervised and unsupervised approaches, then organizes classical and deep learning methods for medical-image segmentation.
- Segmentation clusters images into coherent sub-regions using extracted features and classifies each region into predetermined classes.The paper also frames segmentation as a form of image compression.
- Biomedical segmentation supports knowledge inference and computer-aided diagnosis in modalities including MRI, CT, and colonoscopy.
- Unsupervised methods use image structure, intensity, or gradients to delineate target boundaries through thresholding, graph cuts, edge detection, or deformation.
- Supervised methods incorporate prior task knowledge through training samples, including atlas-based models, SVMs, random forests, and k-nearest-neighbor clustering.Atlas-based methods can capture organ shape and generate more accurate results than unsupervised models.
- The review covers classical machine learning, non-machine-learning approaches, and deep learning architectures applied to medical image segmentation.Its deep-learning coverage includes methods published in the past three years.
2. Classical machine learning methods
Classical machine-learning segmentation methods use hand-designed or statistically structured representations, offering sample efficiency and interpretability while requiring careful feature and parameter choices. In the reviewed colonoscopy and MRI examples, kernel SVMs performed well with limited data, whereas unsuitable bandwidths and high-capacity FCNs caused errors or overfitting.
- Kernel support vector machine (SVM): Kernel SVMs are sample-efficient and transparent because training mainly tunes classifier hyperparameters rather than learning representations from large datasets.Their representations are built from pre-specified filters, contrasting with deep learning’s data-learned representations.
- Kernel support vector machine (SVM): Kernel SVM segmentation combines pre-specified filter banks, kernel feature selection, random feature maps, and a downstream classifier.The filter bank generates representations, feature selection removes redundant or noisy features, and the kernel captures nonlinear relationships between representations and labels.
- Kernel support vector machine (SVM): Kernel methods avoid explicit nonlinear feature mappings through inner products, but their Gram matrices impose prohibitive computational and memory costs for large training sets.Random Fourier features are proposed to approximate low-dimensional embeddings of shift-invariant kernels explicitly.
- Kernel support vector machine (SVM): Unsuitable bandwidths of γ=0.1 and γ=1 significantly degrade classification accuracy by producing class mixtures that a downstream linear SVM cannot separate.The result makes bandwidth selection a critical design choice for kernel-based segmentation.
- Segmentation results of medical images from classical machine learning: MRF segmentation incorporates neighboring-pixel context, and simulated annealing or iterated conditional modes produced error rates below 10% in most simulated MR-image cases.The reviewed formulation uses local image features and prior contextual information to connect spatial continuity.
- Segmentation results of medical images from classical machine learning: Kernel SVM segmentation outperformed FCN on sampled GIANA colonoscopy images, including a bleeding region that FCN missed.With small training data, kernel SVM achieved higher mean IoU scores, while FCN was prone to overfitting; increasing sample size substantially improved FCN but changed kernel-SVM performance little.
3. Other related segmentation methods
This section reviews atlas-based segmentation, deformable models, and superpixels as specialized or preprocessing approaches for medical image segmentation. It also reports strong segmentation results for atlas-based and superpixel-based methods.
- Atlas-based segmentation: Atlas-based segmentation registers a manually generated atlas to a new image so corresponding coordinates provide segmentation labels.The atlas maps spatial coordinates to labels, while image registration determines the coordinate transform.
- Deformable model segmentation: Deformable models use elastic curves or surfaces that image-derived forces deform to delineate object shapes while minimizing a cost function.Internal forces preserve model smoothness, while external forces arise from the image.
- Superpixels: Superpixels group pixels into perceptually meaningful regions that become basic processing units for subsequent segmentation.They are commonly used as a preprocessing step and should improve both speed and segmentation quality.
- Reported results: 98.0% versus 94.0% segmentation performance was reported for atlas-based and deep-learning methods, respectively, on the mandible.The review notes that some atlas-based methods can outperform deep-learning methods.
- Reported results: 89.3% mean DSC was achieved by a superpixel-based 3D graph-cut method combined with a 3D active-contour model for prostate MR images.Superpixels served as graph-cut units, while the active contour addressed graph-cut smoothing drawbacks.
4. Deep learning methods
This section introduces deep-learning terminology and describes ANN components, including layered perceptrons, activation functions, convolution, pooling, and back-propagation. These concepts define how networks transform inputs and update weights during training.
- Terminology: A receptive field is the region that can influence an activation, while a selective window is a selected pixel region.These terms describe spatial regions used or represented within deep-learning architectures.
- Terminology: Overfitting occurs when results are too sensitive to particular datasets, whereas hyperparameters are parameters whose values are not set before learning.Atrous distance denotes the spacing between convolution-kernel elements.
- Terminology: Pooling reduces signal dimensionality within individual network layers, and stride specifies how far a convolution kernel shifts.These operations and parameters affect spatial processing in convolutional networks.
- ANN fundamentals: An ANN is a fully connected network that cascades perceptrons from input to output across multiple layers.Its nodes use activation functions to impose nonlinearity.
- Training: Back propagation propagates prediction loss through the network and uses gradient descent with the chain rule to update weights.The update reduces a predefined loss function during training.
- Architecture and operations: Figure 6 depicts perceptron mathematics, multilayer perceptrons, back-propagation, and convolution as connected examples of neural-network operations.The figure also shows shared kernel weights applied during convolution.
4.1. Overview of deep learning networks
The review introduces ANN, CNN, and RNN architectures for biomedical image segmentation, emphasizing their basic operations, training mechanisms, and practical constraints. Deep models learn representations from data but require substantial labeled datasets, while transfer learning can help address limited medical data.
- Artificial Neural Network (ANN): ANNs connect perceptrons from input to output through multiple fully connected layers and use activation functions to model nonlinear transformations.
- Artificial Neural Network (ANN): Back-propagation updates weights and biases by propagating prediction loss backward and applying gradient descent through the chain rule.
- Convolutional Neural Network (CNN): CNNs use learned convolutional filters, nonlinearities, and pooling to extract hierarchical local and global image features efficiently.
- Convolutional Neural Network (CNN): Pooling reduces feature dimensionality and improves robustness to deformations and translations; common operations include subsampling, average pooling, and max pooling.
- Training challenges: Reliable supervised deep-learning segmentation requires abundant labeled data, but medical datasets are generally scarce and annotation curation is a bottleneck.
- Recurrent Neural Network (RNN): RNNs feed output information back into later inputs, providing memory for sequential data; LSTM networks help retain previous information.
4.2. Segmentation results of medical images from deep learning
The review describes patch-based, fully convolutional, cascaded, and recurrent architectures for medical image segmentation. It reports strong modified U-Net results while noting trade-offs involving patch selection, volumetric memory, class imbalance, and overfitting.
- Patch-wised convolutional neural network: Patch-based networks classify small image patches, but overlapping patches duplicate computation and require careful selection of patch size and field of view.
- Patch-wised convolutional neural network: Markov random fields can post-process patch-based CNN outputs to merge voxel connectivity and organ morphology information.
- Fully convolutional network (FCN): FCNs use convolutional, pooling, and upsampling layers to produce full-size segmentation maps with fewer hyperparameters and lower computational complexity.
- Fully convolutional network (FCN): 3D architectures such as V-Net use three-dimensional convolutions to preserve correlations between adjacent slices in volumetric medical images.
- Fully convolutional network (FCN): FCN segmentation can suffer severe data imbalance, such as lesion voxels being typically 500 times fewer than normal voxels; a Tversky-index loss addresses precision–recall trade-offs.
- Fully convolutional network (FCN): 86.68% DSC, 24.93% VOE, and −0.53% RVD were obtained for liver-tumor segmentation using the modified U-Net.
- Cascaded and recurrent architectures: Cascaded networks separate ROI detection from final segmentation, while CNN–RNN combinations can model spatial and temporal information in dynamic imaging.
4.3. Implementation of deep learning
The review surveys open-source software frameworks and publicly available datasets used to develop and evaluate deep-learning segmentation methods. It identifies common tools and examples of modality- and organ-specific datasets.
- Software frameworks: The reviewed open-source deep-learning frameworks include Caffe, TensorFlow, Torch, and Keras.
- Software frameworks: These frameworks support training and deployment through languages and bindings including C++, Python, MATLAB, and multiple backend systems.
- Public datasets: Public segmentation datasets cover brain, lung, prostate, and other medical-imaging applications across modalities such as MRI and CT.
- Public datasets: Examples include BRATS, ISLES, MSSEG, NeoBrainS12, MRBrainS, and LIDC-IDRI datasets.
5. Outlook and Discussion
Deep learning achieves segmentation results comparable to or better than manual and classical methods, but its practical use is constrained by training-data, computational, and tuning demands. The review also discusses GAN-based data synthesis, 2.5D processing, and data-driven improvements to classical methods.
- 97.8% segmentation accuracy exceeded 92.0% for manual segmentation in cochlear imaging, while liver segmentation reached 97.31% versus 96.29% for active contouring and 96.74% for graph cut.
- Inadequate training datasets can cause expressive deep architectures to overfit, even with dropout, sparse regularization, and model averaging.Increasing training-dataset size was associated with improved CNN classification accuracy, while augmentation can expand datasets through translation, flipping, rotation, and deformation.
- GANs address limited training data by simultaneously training a generator to synthesize data from noise and a discriminator to distinguish synthesized from real data.The generator seeks a distribution similar to the original data.
- 2.5D methods provide partial volumetric information as a compromise between memory-intensive 3D convolutions and information-losing 2D slice processing.Examples include multiple slice images, orthogonal views, and maximum or minimum intensity projections.
- Data-driven feature extraction could improve classical machine learning algorithms that commonly rely on pre-specified filters.
- Deep learning networks remain sensitive to hyperparameter choices, with small changes potentially producing disproportionately large changes in output.Layer count, regularization coefficients, and dropout rates are often selected empirically.