PaperlayerView exact revision on arXiv

Source-linked AI summary

Deep learning and its application to medical image segmentation

Holger R. Roth, Chen Shen, Hirohisa Oda, Masahiro Oda, Yuichiro Hayashi, Kazunari Misawa, Kensaku Mori

arXiv:1803.08691v1cs.CV

TL;DR

Anatomical variation and low contrast make automatic medical-image segmentation challenging. The paper applies a 3D U-Net-like fully convolutional network to labeled CT volumes and reports state-of-the-art multi-organ segmentation performance, with about 90% average testing Dice score.

  • Problem

    Automated medical-image segmentation is challenging because anatomy varies in shape and size across patients, and low contrast can hinder segmentation.

    Source: §I · p. 1

  • Method

    The paper trains a 3D U-Net-like fully convolutional network to map CT volumes to segmentation labels using 3D convolutions, skip connections, and voxel-wise probability outputs.

    Source: §II · p. 2, A. 3D Fully convolutional networks (3D… · p. 2

  • Results

    ∼90% average Dice score in testing across all targeted organs, with state-of-the-art performance in automated multi-organ segmentation of abdominal CT.

    Source: §IV · p. 4

  • Takeaways & Limitations

    Deep 3D fully convolutional networks can be efficiently trained on modern GPUs for automated multi-organ CT segmentation.

    Source: §IV · p. 4

    The model does not constrain segmented anatomy shape and can produce disconnected or isolated regions.

    Source: §IV · p. 5

Abstract

from arXiv · show

One of the most common tasks in medical imaging is semantic segmentation. Achieving this segmentation automatically has been an active area of research, but the task has been proven very challenging due to the large variation of anatomy across different patients. However, recent advances in deep learning have made it possible to significantly improve the performance of image recognition and semantic segmentation methods in the field of computer vision. Due to the data driven approaches of hierarchical feature learning in deep learning frameworks, these advances can be translated to medical images without much difficulty. Several variations of deep convolutional neural networks have been successfully applied to medical images. Especially fully convolutional architectures have been proven efficient for segmentation of 3D medical images. In this article, we describe how to build a 3D fully convolutional network (FCN) that can process 3D images in order to produce automatic semantic segmentations. The model is trained and evaluated on a clinical computed tomography (CT) dataset and shows state-of-the-art performance in multi-organ segmentation.

I. INTRODUCTION

Anatomical variation and low contrast make automated medical-image segmentation difficult, while deep learning and fully convolutional networks offer data-driven approaches. The section introduces CNNs as hierarchical feature learners and outlines their use in medical imaging.

I. INTRODUCTION

  • Deep learning learns features directly from imaging data, while anatomical variation and low contrast make automated medical-image segmentation challenging [1].Source: §I · p. 1
  • Fully convolutional networks have improved medical-image semantic segmentation, and the paper describes using them to segment volumetric images with high accuracy and robustness.Source: §I · p. 1

A. Convolutional neural networks (CNN)

B. Fully convolutional networks (FCN)

Fully convolutional networks address the loss of spatial information in conventional CNNs, enabling semantic segmentation while recovering the input image’s spatial dimensions.

B. Fully convolutional networks (FCN)

C. Related work

Prior CT segmentation approaches include registration-based and atlas-fusion methods, while deep-learning approaches use FCNs to learn segmentation features directly from imaging data.

  • Classical multi-organ CT segmentation includes statistical shape models, registration, and multi-atlas label fusion, as well as combinations of registration and machine learning –.Source: C. Related work · p. 1
  • The figures illustrate CNN-based whole-body CT anatomy classification [10] and FCN applications to medical semantic segmentation [17]–[19], [25].Source: Figure 1 · p. 2, Figure 2 · p. 2
  • Registration-based methods have struggled with complex patient-to-patient shape variation, particularly for very non-rigid abdominal organs.Source: C. Related work · p. 2
  • Many newer medical-image segmentation methods use 2D or 3D FCNs to learn useful features directly from imaging data, avoiding hand-crafted organ-specific features [1],,,, –.Source: C. Related work · p. 2

II. METHODS

The paper applies a 3D U-Net-like FCN to map labeled CT volumes to organ-segmentation probability maps, training on cropped subvolumes and predicting over larger volumes using overlapping tiles.

II. METHODS

  • 3D FCNs make end-to-end pixel-wise semantic segmentation feasible, and the paper applies a 3D U-Net architecture to multi-organ CT segmentation.Source: §II · p. 2

A. 3D Fully convolutional networks (3D U-Net)

B. Data augmentation

The training data are augmented with smooth B-spline deformations, rotations, and translations to generate plausible variations and encourage more robust solutions.

B. Data augmentation

  • Random smooth B-spline deformations use a maximum displacement of 4 and a grid spacing of 24 voxels.The same deformation fields are applied to images and labels.Source: B. Data augmentation · p. 3
  • At each iteration, random rotations range from -20° to +20° and translations range from -20 to +20 voxels per dimension.These augmentations are described as generating plausible deformations and encouraging convergence to more robust solutions while reducing overfitting.Source: B. Data augmentation · p. 3

C. Loss function

The model minimizes a differentiable Dice-based loss to optimize the Dice similarity coefficient, combining class losses with adjustable weights. Training and testing curves are shown for optimization with Dice loss.

C. Loss function

  • The differentiable Dice loss is minimized for each class to optimize the Dice similarity coefficient on the training data.The passage identifies the loss as a differentiable version proposed by Milletari et al..Source: C. Loss function · p. 3
  • The total loss combines losses across foreground classes and background, with each class contribution controlled by a weight factor.In this example, all class weights are kept at 1; alternative weighting schemes are cited as,.Source: C. Loss function · p. 3
  • The figure presents training and testing curves for the 3D FCN using Dice loss for optimization.Source: Figure 4 · p. 3

D. Implementation

The models use Keras with a TensorFlow backend and Adam optimization, and inference takes less than one minute per case.

D. Implementation

  • Inference takes less than 1 minute per case.Source: D. Implementation · p. 3
  • Training runs for 50,000 iterations and takes about one week on an NVIDIA Quadro P6000 GPU with 24 GB.Source: D. Implementation · p. 3
  • The model is implemented in Keras with a TensorFlow backend and optimized using Adam at an initial learning rate of 1×10−2.Automatic differentiation is used to compute optimization gradients.Source: D. Implementation · p. 3

III. EXPERIMENTS & RESULTS

The study uses contrast-enhanced abdominal CT scans from gastric cancer patients, downsampled for computational efficiency. It presents a training-results table and visual examples of multi-organ segmentations.

III. EXPERIMENTS & RESULTS

  • The dataset contains 377 portal-venous-phase contrast-enhanced abdominal CT scans from gastric cancer patients.Each volume has 460–1177 slices of 512×512 pixels, with voxel dimensions [0.59-0.98, 0.59-0.98, 0.5-1.0] mm.Source: A. Data · p. 3
  • The study includes a table of quantitative training results for the 3D FCN and visualizes segmentations alongside ground truth in axial and 3D views.The predictions are up-sampled to half the original CT resolution for the visualization.Source: Figure 5 · p. 4, Table 1 · p. 4
  • All images are downsampled by a factor of 4 for GPU memory constraints and computational efficiency.The resulting axial image size is 128 × 128×(number of slices)/4.Source: A. Data · p. 4

B. Evaluation

On a random split, the model achieved similar average Dice scores on training and testing patients. Its results were comparable to or better than state-of-the-art CT segmentation architectures, although direct comparison across studies is difficult.

B. Evaluation

  • The authors interpret the similar training and testing scores as evidence that the dataset size and data augmentation were sufficient to avoid overfitting.Source: B. Evaluation · p. 4
  • 89.3 ± 6.5% average Dice in testing (range [63.1, 95.6]) compared with 89.4 ± 6.4% in training (range [42.2, 95.9]).The testing results cover 37 patients.Source: Table 2 · p. 4, B. Evaluation · p. 4
  • The model's results were comparable to or better than other state-of-the-art deep learning architectures for single- and multi-organ CT segmentation.Direct comparison is difficult because studies use different datasets and validation schemes.Source: B. Evaluation · p. 4

IV. CONCLUSIONS

The example 3D FCN achieves state-of-the-art abdominal CT multi-organ segmentation with approximately 90% average Dice in testing and can be trained efficiently on modern GPUs. The current model lacks anatomical shape constraints, while the authors identify greater GPU memory, larger datasets, and possible topological constraints as future directions.

IV. CONCLUSIONS

  • ∼90% average Dice in testing across all targeted organs is reported for state-of-the-art automated multi-organ segmentation of abdominal CT.Source: §IV · p. 4
  • Deep 3D FCNs can be efficiently trained on modern GPUs.Source: §IV · p. 4
  • Without shape constraints, the model can produce disconnected or isolated regions.Source: §IV · p. 5
  • More GPU memory would allow whole CT volumes to be processed at higher resolution.Source: §IV · p. 5
  • The authors expect larger datasets to likely further improve automated multi-organ segmentation performance.Source: §IV · p. 5
  • Anatomical constraints could be included in future work to guarantee topologically correct segmentation results.Source: §IV · p. 5
Loading 1803.08691v1…