Source-linked AI summary
VoxResNet: Deep Voxelwise Residual Networks for Volumetric Brain Segmentation
Hao Chen, Qi Dou, Lequan Yu, Pheng-Ann Heng
TL;DR
The paper addresses how to use contextual representations for volumetric medical-image segmentation, where much medical imaging data is volumetric. It extends residual learning to 3D with VoxResNet and an auto-context refinement stage, and reports superior performance on a brain-segmentation benchmark.
Problem
Volumetric medical-image segmentation remains challenging, and how to fully use contextual representations for volumetric recognition has not been well studied.
Source: §1 · p. 1Method
VoxResNet extends residual learning to 3D, while Auto-context VoxResNet combines initial probability maps with original volumes for segmentation refinement.
Source: §2.2 · p. 3, §2.3 · p. 4Results
VoxResNet significantly outperformed other state-of-the-art methods on a challenging brain-segmentation benchmark using volumetric MR images.
Source: §1 · p. 2Takeaways & Limitations
The experiments support the efficacy of the method for volumetric data, and the authors state it can be applied to other volumetric image-segmentation problems.
Source: §4 · pp. 6–7
Abstract
from arXiv · showhide
Recently deep residual learning with residual units for training very deep neural networks advanced the state-of-the-art performance on 2D image recognition tasks, e.g., object detection and segmentation. However, how to fully leverage contextual representations for recognition tasks from volumetric data has not been well studied, especially in the field of medical image computing, where a majority of image modalities are in volumetric format. In this paper we explore the deep residual learning on the task of volumetric brain segmentation. There are at least two main contributions in our work. First, we propose a deep voxelwise residual network, referred as VoxResNet, which borrows the spirit of deep residual learning in 2D image recognition tasks, and is extended into a 3D variant for handling volumetric data. Second, an auto-context version of VoxResNet is proposed by seamlessly integrating the low-level image appearance features, implicit shape information and high-level context together for further improving the volumetric segmentation performance. Extensive experiments on the challenging benchmark of brain segmentation from magnetic resonance (MR) images corroborated the efficacy of our proposed method in dealing with volumetric data. We believe this work unravels the potential of 3D deep learning to advance the recognition performance on volumetric image segmentation.
1 Introduction
Volumetric medical-image segmentation is important but challenging, and existing approaches leave gaps in contextual representation and network depth. The paper proposes VoxResNet and reports superior brain-segmentation performance on a benchmark.
1 Introduction
- Volumetric segmentation supports computer-aided diagnosis through quantitative measurements and can aid surgical treatment, but high dimensionality and complexity make it challenging.Source: §1 · p. 1
- Prior volumetric CNNs either aggregate neighboring slices or orthogonal planes, limiting their use of contextual information, or use 3D networks with shallow representations or optimization degradation as depth increases.Source: §1 · pp. 1–2
- VoxResNet significantly outperformed other state-of-the-art methods on a challenging brain-segmentation benchmark using volumetric MR images.The introduction reports the comparison without giving metric values.Source: §1 · p. 2
- The paper proposes VoxResNet, a deep voxelwise residual network that adapts residual learning to object segmentation in volumetric data.Source: §1 · p. 2
2 Method
The method extends residual learning to 3D volumetric segmentation, then combines multi-modality appearance with auto-context information in a refinement network.
2.1 Deep Residual Learning
- Deep residual learning represents a deeper feature as a shallow feature plus accumulated residual functions, allowing information to propagate through the network smoothly.A residual unit learns an additive function relative to its input feature.Source: §2.1 · p. 2
2.2 VoxResNet for Volumetric Image Segmentation
- VoxResNet extends residual networks to 3D, using 25 volumetric convolutional layers, four deconvolutional layers, and skip-connected VoxRes modules.Three stride-2 convolutions reduce input resolution by a factor of eight, increasing receptive field; four auxiliary classifiers provide multi-level context through deep supervision.Source: §2.2 · p. 3
- The network uses 3 × 3 × 3 kernels and 3D operations throughout to learn volumetric features, with intermediate batch normalization and ReLU activations.Source: §2.2 · p. 3
2.3 Multi-modality and Auto-context Information Fusion
- Multi-modality images are concatenated as input so their complementary information is jointly fused during network training.The paper reports consistent improvement over using any single modality.Source: §2.3 · p. 4
- Auto-context VoxResNet combines original volumes with probability maps from an initial VoxResNet to refine segmentation and remove outliers.Further iterative refinement gave only marginal improvements, so the auto-context output is used as the final result.Source: §2.3 · p. 4
3 Experiments
On MRBrainS, multi-modality VoxResNet outperformed other deep-learning methods, and adding auto-context further improved Dice performance; the methods placed at the top of a 37-team leaderboard. The experiments used multi-sequence MRI, leave-one-out validation, and a held-out challenge test set.
3.1 Dataset and Pre-processing
- MRBrainS provided five manually segmented training subjects and 15 held-out test subjects, with T1-weighted, T1-IR, and T2-FLAIR scans.Preprocessing produced six input volumes from original and processed images, with per-slice intensity normalization.Source: §3.1 · pp. 4–5
3.2 Evaluation and Comparison
- In leave-one-out validation, combining modalities improved segmentation over a single modality, especially on DC, and auto-context further improved DC.The authors also report visually more accurate results with both multi-modality and auto-context than with multi-modality alone.Source: §3.2 · p. 5, Table 1 · p. 6
- On MRBrainS, multi-modality VoxResNet outperformed other deep-learning methods, auto-context further improved DC, and the methods placed at the top of a 37-team leaderboard.Table 2 reports challenge results for DC, HD, and AVD; only its top 10 teams are shown.Source: §3.2 · p. 6, Table 2 · p. 6
3.3 Implementation Details
- Training took about one day, while processing each test volume took less than two minutes on a workstation with one NVIDIA TITAN X GPU.GPU memory limits led to on-the-fly sampling of 80 × 80 × 80 input regions and overlap-tiling at test time.Source: §3.3 · p. 6
4 Conclusions
VoxResNet extends deep residual learning to 3D volumetric segmentation, while its auto-context version integrates image appearance, implicit shape, and high-level context. The authors report efficacy on brain segmentation and describe broader volumetric segmentation as an application, with other object tasks left for future investigation.
4 Conclusions
- VoxResNet extends deep residual learning to volumetric data in a 3D variant for segmentation.The authors describe the method as having potential to advance biomedical volumetric image segmentation.Source: §4 · p. 6
- The auto-context version integrates low-level appearance, implicit shape, and high-level context to further improve volumetric segmentation performance.The authors report that experiments on the brain-segmentation benchmark corroborated the method's efficacy.Source: §4 · p. 7
- The authors state that VoxResNet can apply to other volumetric image-segmentation problems, and identify more volumetric object tasks as future work.Source: §4 · p. 7