Source-linked AI summary
Deep Attentive Features for Prostate Segmentation in 3D Transrectal Ultrasound
Yi Wang, Haoran Dou, Xiaowei Hu, Lei Zhu, Xin Yang, Ming Xu, Jing Qin, Pheng-Ann Heng, Tianfu Wang, Dong Ni
TL;DR
Automatic prostate segmentation in TRUS is important but difficult because prostate boundaries may be missing or ambiguous, intensities are inhomogeneous, and shapes vary. The paper develops a 3D attention-guided network that selectively refines multi-level features at each layer. On challenging TRUS volumes, it reports strong segmentation performance across nearly all evaluated metrics, while evaluation on only forty volumes limits generalizability.
Problem
Automatic TRUS prostate segmentation is important for image-guided intervention and treatment planning but is challenged by ambiguous or missing boundaries, inhomogeneous intensities, and variable prostate shapes.
Method
A 3D attention-guided feature-pyramid network learns weights over complementary multi-level features to refine representations at each layer.
Results
The method consistently outperformed compared methods on almost all metrics, achieving mean Dice 0.90, Jaccard 0.82, and Precision 0.90.
Takeaways & Limitations
The attention module provides a feature-refinement strategy for combining low-level details and high-level semantics, with potential use in other segmentation or detection networks.
Takeaways & Limitations
Experiments used four-fold cross-validation on only forty TRUS volumes, which may have caused overfitting to training samples and limits demonstrated generalizability.
Abstract
from arXiv · showhide
Automatic prostate segmentation in transrectal ultrasound (TRUS) images is of essential importance for image-guided prostate interventions and treatment planning. However, developing such automatic solutions remains very challenging due to the missing/ambiguous boundary and inhomogeneous intensity distribution of the prostate in TRUS, as well as the large variability in prostate shapes. This paper develops a novel 3D deep neural network equipped with attention modules for better prostate segmentation in TRUS by fully exploiting the complementary information encoded in different layers of the convolutional neural network (CNN). Our attention module utilizes the attention mechanism to selectively leverage the multilevel features integrated from different layers to refine the features at each individual layer, suppressing the non-prostate noise at shallow layers of the CNN and increasing more prostate details into features at deep layers. Experimental results on challenging 3D TRUS volumes show that our method attains satisfactory segmentation performance. The proposed attention mechanism is a general strategy to aggregate multi-level deep features and has the potential to be used for other medical image segmentation tasks. The code is publicly available at https://github.com/wulalago/DAF3D.
I. INTRODUCTION
TRUS prostate segmentation matters for image-guided interventions but remains difficult because boundaries can be ambiguous or missing, intensities are inhomogeneous, and prostate shapes vary substantially.
- Example images depict large shape variations, inhomogeneous prostate intensities, and missing or ambiguous boundaries.
- TRUS boundary delineation supports treatment planning, biopsy needle placement, brachytherapy, cryotherapy, and registration with preoperative MR images.
- Automatic segmentation is challenged by poor contrast, acoustic shadowing, other structures, inhomogeneous tissue intensity, and large prostate shape variation.
- Visual comparisons contrast conventional multi-level features with proposed attentive features across shallow-to-deep layers and predicted segmentations.
A. Relevant Work
Earlier prostate-segmentation methods used shape statistics, hand-crafted classification features, or CNNs, while this work addresses the limitations of undifferentiated multi-level features with a 3D attentive feature-pyramid network.
- Shape-statistics methods used initialization, dynamic contours, and edge-guided boundary delineation for 2D or 3D TRUS segmentation.
- Hand-crafted texture and patch features with classifiers tend to fail on ambiguous or missing boundaries because they capture limited high-level semantic knowledge.
- CNNs learn multi-level features for segmentation, and prior work applied U-Net-like networks to prostate segmentation in 2D TRUS slices.
- Directly combining multi-level convolutional features can localize boundaries poorly because shallow layers add non-prostate regions while deep layers lose boundary details.
- The proposed 3D feature pyramid network uses attention modules to learn complementary low-level detail and high-level semantic information for deep attentive features.
B. Contributions
The paper proposes attention-guided refinement of multi-level features for 3D TRUS segmentation, combining feature hierarchies and multi-scale context to better represent prostate tissue.
- B. Contributions: The attention module learns weights for multi-level features to refine features at each individual layer.
- B. Contributions: The network combines deep and shallow spatial contexts through a 3D attention-guided feature-pyramid architecture for TRUS prostate segmentation.
- II. DEEP ATTENTIVE FEATURES FOR 3D SEGMENTATION: The task requires combining multi-scale context, region semantics, and boundary details because prostate TRUS images contain ambiguous or missing boundaries and inhomogeneous intensities.
- A. Network Architecture: The network extracts feature maps at multiple resolutions, using shallow layers for detail and deep layers for semantic information.
- A. Network Architecture: A feature pyramid network merges multi-level maps, but its combined features can retain shallow-layer noise and lose subtle prostate parts from coarse deep features.
- A. Network Architecture: Attention modules take single-layer and multi-layer features as inputs to produce refined attentive feature maps.
- A. Network Architecture: ASPP resamples attentive features at different scales before the network combines them and predicts segmentation with deep supervision.
B. Deep Attentive Features
The deep attentive module uses layer-wise attention to combine multi-level features, refining each single-layer feature with complementary detail and semantic information. It generates attentive maps from single-layer and multi-layer features, weights the multi-layer features, and merges them into refined representations.
- B. Deep Attentive Features: Layer-wise attention selectively leverages complementary features across all scales to refine features at individual layers.The module is designed to combine low-level detail with high-level semantic information.
- B. Deep Attentive Features: The module concatenates single-layer and multi-layer features to produce unnormalized attention weights for each layer.The resulting attentive map indicates the importance of multi-layer features for that specific layer.
- B. Deep Attentive Features: The attentive map weights multi-layer features element by element before convolutional fusion with the corresponding single-layer features.This fusion produces the final attentive feature for each layer.
- B. Deep Attentive Features: For shallow layers, multi-layer features suppress details outside semantic saliency regions while retaining details within those regions.This addresses shallow-layer detail that lacks sufficient prostate semantics.
- B. Deep Attentive Features: For deep layers, multi-layer features enhance boundary details that may be missing from representations focused on the whole prostate.The attention module therefore refines shallow and deep features in different ways.
C. Implementation Details
The framework uses a 3D ResNeXt backbone, combines Dice and binary cross-entropy losses, and trains end-to-end with supervised signals across network outputs. Training converges after 14 epochs, while a 20-epoch run takes about 54 hours.
- C. Implementation Details: The proposed framework is implemented in PyTorch with 3D ResNeXt as its backbone network.
- C. Implementation Details: Dice loss and binary cross-entropy loss are combined to consider local boundary detail and global shape similarity.Binary cross-entropy preserves boundary details but can be affected by class imbalance, while Dice loss emphasizes compact global shape similarity.
- C. Implementation Details: Each supervised signal is defined as the sum of Dice loss and binary cross-entropy loss.
- C. Implementation Details: The total loss sums losses across all supervised signals, using empirically specified weights for intermediate, refined, and final outputs.The listed weights are (0.4, 0.5, 0.7, 0.8, 0.4, 0.5, 0.7, 0.8, 1).
- C. Implementation Details: Training uses Adam with learning rate 0.001 and mini-batch size 1 on a single TITAN Xp GPU.The learning curve indicates convergence after 14 epochs; training for 20 epochs takes about 54 hours.
A. Materials
The study evaluates prostate segmentation using one reconstructed 3D TRUS volume from each of 40 patients. Expert clinicians produced and verified the ground-truth boundaries, and training used rotation and flipping augmentation.
- A. Materials: The dataset contains TRUS volumes from 40 patients recruited at the First Affiliate Hospital of Sun Yat-Sen University.The study received institutional ethics approval and informed consent from all patients.
- A. Materials: One reconstructed 3D TRUS volume was acquired per patient using a Mindray DC-8 ultrasound system with an integrated 3D probe.Each volume contains 170 × 132 × 80 voxels with voxel size 0.5 × 0.5 × 0.5 mm^3.
- A. Materials: Two experienced urological clinicians delineated and verified the prostate boundaries using neighboring-slice 3D information.One clinician manually delineated each slice, and another refined or confirmed the boundaries.
- A. Materials: Training data were augmented through rotation and flipping.
B. Experimental Methods
The evaluation compares the attention guided network with three segmentation networks and with an attention-removed customized FPN ablation. Performance is assessed by multiple 3D overlap, boundary-distance, and voxel-classification metrics under four-fold cross-validation.
- B. Experimental Methods: The attention guided network is compared with 3D FCN, 3D U-Net, and BCRNN using retrained implementations and tuned training parameters.The U-Net and BCRNN methods were developed specifically for TRUS segmentation.
- B. Experimental Methods: The ablation model, 3D customized FPN, removes the attention modules and sends multi-layer features directly through ASPP for final prediction.Four-fold cross-validation evaluates the segmentation performance of the different models.
- B. Experimental Methods: The reported metric results are summarized as mean ± standard deviation, with best results highlighted in bold.
- B. Experimental Methods: The statistical analysis includes ANOVA and Wilcoxon rank-sum tests to assess differences between methods across metrics.The ANOVA F-statistics reported for all metrics exceed the stated critical value of 2.42.
- B. Experimental Methods: Evaluation uses Dice, Jaccard, conformity coefficient, average distance of boundaries, 95% Hausdorff distance, precision, and recall, all computed in 3D.Better segmentation has smaller ADB and 95HD, and larger values for the other metrics.
C. Segmentation Performance
The attention-guided network improves 3D TRUS prostate segmentation across quantitative metrics, statistical tests, and qualitative boundary and surface comparisons.
- The attention module selects complementary multi-level information to refine each shallow-level feature using learned attentive maps.
- 0.90 mean Dice, 0.82 Jaccard, 0.78 CC, 3.32-voxel ADB, 8.37-voxel 95HD, and 0.90 Precision were the best values among compared methods.
- 2.57% to 45.74% improvements over the 3D cFPN ablation were reported across Dice, Jaccard, CC, ADB, 95HD, and Precision.
- ANOVA and Wilcoxon tests indicated significant differences, with the proposed method significantly better on Dice, Jaccard, CC, ADB, 95HD, and Precision.
- The method inferred missing or ambiguous boundaries and produced the most similar 2D contours and 3D surfaces to clinician-derived ground truths.
- Whole-prostate segmentation required 0.30 seconds for the proposed method on 170 × 132 × 80-voxel inputs, faster than 3D FCN, 3D U-Net, and BCRNN.
IV. DISCUSSION
The discussion positions attention-based multi-level feature refinement as the paper’s central contribution while acknowledging limited evaluation data and overfitting risk.
- The study presents a 3D CNN with attention mechanisms to refine multi-level features for challenging TRUS segmentation.
- Directly applying multi-level convolutional features can distract localization, motivating selective refinement of low-level details and high-level semantics.
- Visual comparisons show the proposed method more closely matches ground truths and robustly covers the whole prostate region.
- The hybrid loss combines binary cross-entropy for boundary details with Dice loss for global shape similarity.
- The evaluation used four-fold cross-validation on only forty TRUS volumes, which may have caused overfitting to training samples.
- Future work will evaluate generalizability on larger datasets with mutually exclusive training, validation, and test subsets.
V. CONCLUSION
The paper concludes that deep attentive features improve 3D TRUS prostate segmentation by selectively aggregating complementary multi-level information.
- The proposed attention module learns feature-importance weights for each layer to refine multi-level features in 3D TRUS segmentation.
- Experiments on challenging TRUS volumes show satisfactory segmentation performance, while the attention mechanism may generalize to other medical image tasks.