Source-linked AI summary
Projection-Based 2.5D U-net Architecture for Fast Volumetric Segmentation
Christoph Angermann, Markus Haltmeier, Ruth Steiger, Sergiy Pereverzyev Jr, Elke Gizewski
TL;DR
3D convolutions make volumetric segmentation costly in memory and training time, while 2D slices lack full-volume information. The paper addresses this with a projection-based 2.5D U-net that applies 2D convolutions to maximum-intensity projections and reconstructs volumetric masks. For the tested binary segmentation task, it outperforms the 3D U-net while using less memory and training almost 15× faster.
Problem
Volumetric segmentation with 3D convolutions requires substantial memory and long training times, while 2D slice images do not contain full-volume information.
Source: §1 · p. 2Method
The projection-based 2.5D U-net converts volumes into maximum-intensity projection images, processes them with 2D convolutions, and reconstructs volumetric segmentation masks.
Source: §1 · p. 2, §2.2 · p. 5, §3.1 · p. 8Results
The proposed 2.5D U-net outperforms the 3D U-net on evaluation metrics, uses less memory, and can be trained almost 15× faster for the targeted application.
Source: §1 · p. 2, §3.1 · p. 9, §4 · p. 11Takeaways & Limitations
The 2.5D U-net enables storage-efficient volumetric segmentation of very large biomedical scans without 3D convolutional layers.
Source: §4 · pp. 10–11The 3D U-net remains constrained by long training times and limited choices of convolutional layers and filter counts for very large inputs.
Source: §2.3 · p. 6
Abstract
from arXiv · showhide
Convolutional neural networks are state-of-the-art for various segmentation tasks. While for 2D images these networks are also computationally efficient, 3D convolutions have huge storage requirements and require long training time. To overcome this issue, we introduce a network structure for volumetric data without 3D convolutional layers. The main idea is to include maximum intensity projections from different directions to transform the volumetric data to a sequence of images, where each image contains information of the full data. We then apply 2D convolutions to these projection images and lift them again to volumetric data using a trainable reconstruction algorithm.The proposed network architecture has less storage requirements than network structures using 3D convolutions. For a tested binary segmentation task, it even shows better performance than the 3D U-net and can be trained much faster.
1 Introduction
Volumetric segmentation is difficult because 3D convolutions require substantial memory and training time, while 2D slices lack full-volume context. The paper introduces a projection-based network that uses 2D images retaining information from the complete volume and reports faster, more accurate segmentation than 3D-convolutional networks.
1 Introduction
- 3D U-net extensions face huge memory requirements and long training times, whereas 2D slices omit information from the full 3D volume.These limitations make volumetric segmentation challenging despite the success of U-net architectures for 2D biomedical segmentation.Source: §1 · p. 2
- Maximum intensity projections from different directions transform a volume into 2D images containing information from the full 3D image.The projections are used as the basis for the proposed network structure.Source: §1 · p. 2
- The proposed network targets accurate volumetric segmentation of very large 3D volumes, demonstrated on arteries and veins in magnetic resonance angiography scans.Figure 1.1 shows vessels marked in multiple planes and the corresponding 3D segmentation mask.Source: §1 · p. 2, Figure 1.1 · p. 2
- The proposed network trains 15× faster, uses an order of magnitude less memory, and still produces more accurate results than networks with 3D convolutions.The comparison is stated for the tested segmentation task.Source: §1 · p. 2
2 Background
Volumetric blood-vessel segmentation is challenging because full 3D context conflicts with the memory and training demands of 3D convolutions. The paper develops a 2D U-net workflow on MIP images and compares it with a 3D U-net baseline.
2.1 Volumetric segmentation of blood vessels
- Manual volumetric masks support automated blood-vessel segmentation for large MRA cohorts, but the task involves large volumes and severe class imbalance.The volumes contain 96 × 288 × 224 voxels, with 99.76% of voxels labeled background on average.Source: §2.1 · p. 3
2.2 Segmentation of MIP images
- The 2D workflow projects each 3D scan and its ground truth into ten MIP image pairs using a 36° rotation around the vertical axis.The resulting dataset contains 1190 pairs of 2D images and segmentation masks from 119 patients.Source: §2.2 · pp. 3–4
- The MIP images are segmented with a 2D U-net whose contracting and upsampling paths use convolutional blocks, pooling, transposed convolutions, and skip concatenations.The U-net outputs per-pixel foreground probabilities, which are thresholded to form the final mask.Source: §2.2 · p. 4
- The 2D U-net uses Dice loss and training safeguards including batch normalization, dropout, learning-rate scheduling, early stopping, and validation-based weight restoration.Training uses Adam with learning rate 0.001 and a 70/15/15 training, validation, and evaluation split.Source: §2.2 · pp. 5–6
2.3 Segmentation with the 3D U-net
- The 3D U-net replaces the 2D layers with 3D convolutions and pooling but requires more than 8 GB memory and averages 969 minutes of training.Its reported mean IU is 80.5% and Dice-coefficient is 74.8%.Source: §2.3 · p. 6
- Although the 3D U-net achieves high precision in this application, its memory and training-time demands restrict layer and filter choices and make larger scans difficult without cropping or sliding windows.The limitation is explicitly tied to the huge input volumes and constrained model capacity.Source: §2.3 · p. 6
3 Projection-Based 2.5D U-net
The proposed 2.5D U-net segments volumetric scans by processing MIP images with 2D convolutions and reconstructing voxel probabilities. It addresses slice-wise information loss and 3D-convolution resource costs while outperforming the 3D U-net in the reported MRA application.
3 Projection-Based 2.5D U-net
- Slice-wise 2D segmentation loses connections between slices and achieved a Dice-coefficient of 15.1%, motivating the projection-based alternative.The slice-wise baseline also reported Dice-loss 0.849, mean accuracy 54.5%, and mean IU 54.3%.Source: §3 · pp. 6–7
3.1 Proposed 2.5D U-net architecture
- The 2.5D U-net transforms a 3D scan into multiple MIP images, applies a shared 2D U-net, and reconstructs voxel probabilities with learned filtration and backprojection.The reconstruction operator combines projection-direction outputs, while fine-tuning uses average pooling, a learnable shift, and sigmoid activation.Source: §3.1 · pp. 7–8
- Filtration before backprojection uses direction-specific 1 × 3 filters and produces a more satisfying output than unfiltered reconstruction.The unfiltered reconstruction creates a shroud, whereas the learned filters reduce this problem.Source: §3.1 · p. 8, Figure 3.2a · p. 8, Figure 3.2b · p. 8
- Twelve equidistant projection directions were selected as a good choice, with five randomized runs yielding Dice-loss 0.201, mean accuracy 91.6%, mean IU 86.1%, and Dice-coefficient 83.7%.The projection-direction count was evaluated by comparing performance across different values of p.Source: §3.1 · pp. 8–9, Figure 3.3 · p. 9
- The proposed 2.5D U-net clearly outperformed the 3D U-net on evaluation metrics for volumetric MRA segmentation.The reported visual comparisons include ground truth, 3D U-net, and 2.5D U-net segmentations.Source: §3.1 · p. 9, Figure 3.4 · p. 9, Figure 3.5 · p. 10
- The 2.5D model required 3.7 gigabytes during training and 3914 seconds to adjust its weights, while constructing MIP images increased application time to 11.37 seconds.Table 2 summarizes time and storage observations for the slice-wise 2D, 3D, and proposed 2.5D approaches.Source: §3.1 · p. 10, Table 2 · p. 10
4 Conclusion
The paper proposes a projection-based 2.5D U-net for fast volumetric segmentation, addressing the training-time and memory demands of 3D U-nets. Using 12 deterministic projection directions, it reports reliable, storage-efficient 3D segmentation that is more accurate than other 3D-convolution approaches for the targeted application.
4 Conclusion
- The proposed projection-based 2.5D U-net is presented as a fast alternative to 3D U-nets for volumetric segmentation.The conclusion contrasts the satisfying results of 3D U-nets with their long training time and substantial memory requirements.Source: §4 · pp. 10–11
- The projection-based 2.5D U-net enables reliable 3D segmentation of very large biomedical scans while avoiding the memory concerns of 3D convolutions.The architecture uses 12 deterministic projection directions.Source: §4 · p. 11
- For the targeted application, the 2.5D U-net produces more accurate segmentations than other approaches using 3D convolutions and trains almost 15× faster.The paper states that numerical evaluation metrics appear in Table 1, while training time, application time, and storage requirements are summarized in Table 2.Source: §4 · p. 11
- Future work will investigate random projection directions and whether the method improves accuracy over 3D convolutions in other applications.The current implementation uses only deterministic projection directions, and comparative studies are identified as future work.Source: §4 · p. 11