Source-linked AI summary
Attention Enriched Deep Learning Model for Breast Tumor Segmentation in Ultrasound Images
Aleksandar Vakanski, Min Xian, Phoebe Freer
TL;DR
Breast-tumor segmentation lacks robust, generalizable domain priors because tumor appearance varies substantially across cases. The paper adds external visual-saliency maps and attention blocks to a U-Net, emphasizing salient regions during segmentation. Validation on 510 images showed improved segmentation performance, while the method remains dependent on saliency-map quality.
Problem
Breast tumors vary in shape, boundaries, and appearance, making generalized domain priors difficult to incorporate into deep-learning segmentation.
Method
The model integrates multi-scaled external visual-saliency maps into encoder attention blocks of a U-Net to emphasize high-saliency regions.
Results
The approach was validated on 510 images and demonstrated improved segmentation performance over the basic U-Net, with the single-highest-saliency-contour variant improving further.
Takeaways & Limitations
The architecture provides a way to incorporate radiologists’ visual attention and task-specific prior information into breast-tumor segmentation.
Takeaways & Limitations
Performance depends on saliency-map quality, since low-quality maps may fail to improve or may degrade segmentation results.
Abstract
from arXiv · showhide
Incorporating human domain knowledge for breast tumor diagnosis is challenging, since shape, boundary, curvature, intensity, or other common medical priors vary significantly across patients and cannot be employed. This work proposes a new approach for integrating visual saliency into a deep learning model for breast tumor segmentation in ultrasound images. Visual saliency refers to image maps containing regions that are more likely to attract radiologists visual attention. The proposed approach introduces attention blocks into a U-Net architecture, and learns feature representations that prioritize spatial regions with high saliency levels. The validation results demonstrate increased accuracy for tumor segmentation relative to models without salient attention layers. The approach achieved a Dice similarity coefficient of 90.5 percent on a dataset of 510 images. The salient attention model has potential to enhance accuracy and robustness in processing medical images of other organs, by providing a means to incorporate task-specific knowledge into deep learning architectures.
Introduction
The paper addresses inconsistent breast-ultrasound tumor analysis and the difficulty of encoding variable tumor priors. It proposes attention-enriched U-Net models that use visual saliency and auxiliary anatomical information to focus segmentation on salient regions.
- Breast-ultrasound analysis remains vulnerable to inconsistent performance across imaging equipment because intensity, contrast, and density vary between images.
- Breast tumors’ variable shapes and boundaries make conventional shape, curvature, texture, intensity, and region-count priors difficult to generalize.
- The approach integrates topological information through visual-saliency maps and anatomical information by decomposing breast tissue into skin, fat, mammary, and muscle layers.
- Attention blocks are integrated into a U-Net encoder to learn representations that emphasize target regions with high saliency values.
- The model uses external visual-saliency maps as auxiliary inputs rather than merging internal feature representations from different network layers.
- The paper contributes an attention-enriched model for tumor-saliency priors and a confidence-level calculation for visual-saliency maps.
Related Works
Prior knowledge can improve medical-image segmentation, but breast tumors resist generalized geometric and appearance priors. Related attention methods motivate the paper’s use of saliency and anatomical information as alternative task-specific guidance.
- Priors in medical image segmentation: Existing breast-tumor methods use ultrasound and mammography images, while prior approaches have incorporated topology or redesigned architectures with increased computational expense.
- Priors in medical image segmentation: Task-specific priors can improve medical-image segmentation and are especially important when annotated datasets are small.
- Priors in medical image segmentation: Breast tumors’ geometric and appearance variability makes generalized shape, boundary, curvature, moment, intensity, and region-count priors difficult to extract.
- Priors in medical image segmentation: The paper introduces topology information through region connectivity and visual saliency, combined with anatomical tissue-layer information.
- Attention mechanism in deep learning: Attention mechanisms emphasize discriminative features while suppressing less relevant features for task-specific representations.
- Attention mechanism in deep learning: Soft attention is differentiable and trained by backpropagation, whereas hard attention is nondifferentiable and relies on sampling.
- Attention mechanism in deep learning: Existing U-Net attention gates and AttentionNet demonstrate related uses of attention for medical segmentation but differ from the proposed saliency-guided approach.
Materials and Methods
The method combines breast ultrasound images with precomputed visual saliency maps in a U-Net-based segmentation model. Saliency-guided attention weights feature maps toward high-saliency regions, while low-confidence saliency maps are identified and eliminated before evaluation.
- Dataset: 510 breast ultrasound images from three hospitals were used to validate the proposed approach.The images were acquired with different types of ultrasound devices.
- Network architecture: The model uses a U-Net encoder–decoder with skip connections to recover spatially accurate segmentation masks.The proposed architecture reduces the number of convolutional filters to account for the relatively small dataset.
- Salient attention: Precomputed saliency maps are supplied as an auxiliary input and introduced at multiple encoder layers through an image pyramid.The same saliency maps are used during training and inference.
- Salient attention: Attention blocks combine layer feature maps with salient maps and output down-sampled weighted maps at four encoder levels.The blocks assign greater weights to feature-map areas associated with higher saliency values.
- Saliency maps: Visual saliency maps assign values to image pixels indicating their estimated probability of belonging to a tumor, including background regions.The maps are generated without using ground-truth masks and rely on anatomical and other saliency constraints.
- Saliency-map quality control: Low-confidence saliency maps are eliminated using contour area, cumulative intensity, and mean intensity criteria.The confidence algorithm uses only saliency maps and is intended to identify ambiguity about tumor regions; poor maps can degrade model performance.
- Evaluation: Performance is evaluated with DSC, JI, TPR, FPR, ACC, AUC-ROC, HD, and MD.For FPR, HD, and MD, lower values are preferred; the paper defines FPR relative to positive tumor pixels rather than background pixels.
Results
Comparative validation evaluated U-Net, U-Net-SA, and U-Net-SA-C using segmentation examples, five-fold cross-validation, and statistical testing. Salient-attention models generally improved segmentation and reduced false positives, while performance declined on the full 562-image dataset.
- Qualitative results: Salient-attention models reduced false-positive tumor predictions compared with U-Net, especially in Figure 5 rows B, E, and G with high-quality saliency maps.The models used information from salient maps to improve predictions in non-tumor regions.
- Qualitative results: Salient attention improved true-positive predictions in rows H and I and helped locate small tumors in cases C and I.Rows J and K also illustrate more accurate predictions associated with the geometry of salient regions.
- Comparative evaluation: U-Net-SA and U-Net-SA-C outperformed basic U-Net across all reported performance metrics in five-fold cross-validation.U-Net-SA-C, which uses a single highest-saliency contour, further improved segmentation performance over U-Net-SA.
- Computational cost: U-Net-SA and U-Net-SA-C required 12 and 13 milliseconds per image, respectively, compared with 7.58 minutes per fold for basic U-Net training.Average training times were 8.54 minutes for U-Net-SA and 8.08 minutes for U-Net-SA-C, versus 7.58 minutes for U-Net.
- Statistical analysis: Almost all tested metrics showed statistically significant median differences between the proposed models and U-Net, except TPR and HD between U-Net and U-Net-SA.Wilcoxon signed rank testing used P-value < 0.05 as the significance threshold.
- Dataset analysis: On the original 562-image dataset, performance decreased for both attention models and basic U-Net, indicating that the removed 52 images were more challenging for segmentation.The confidence-level algorithm was reported to improve performance on the reduced 510-image dataset by preventing poor saliency data from inhibiting predictions.
Discussion
Attention blocks improved segmentation over basic U-Net, with single-contour saliency further reducing false positives under a one-tumor assumption. Performance and robustness depend on saliency-map quality and diverse imaging-system validation, while future work targets richer prior integration.
- Results: Attention-block models outperformed the basic U-Net, and the single-contour U-Net-SA-C variant improved performance further.The single-contour design increased spatial attention to one salient region and reduced false positives, assuming each image contains only one tumor.
- Method: Additional 3×3 and 1×1 convolutional refinement layers balance inaccurate salient-region boundaries and improve tumor segmentation.The refinement layers lessen attention-coefficient values compared with the unrefined attention design.
- Validation: Validation images collected across imaging systems make the dataset suitable for evaluating robustness to source-related image variation.
- Limitation: Low-quality saliency maps may fail to improve or may degrade segmentation, motivating confidence scoring to remove low-confidence maps.The paper identifies saliency estimation quality as a constraint, although improving saliency estimation is outside this work’s main focus.
- Future work: Future work includes custom prior-encoding loss functions and combining annotated breast-tissue layers with salient maps in unified segmentation.
Conclusion
The paper proposes a U-Net variant that incorporates radiologists’ visual attention through encoder attention blocks and multi-scaled saliency maps. Validated on 510 images, the approach improved breast-tumor segmentation while addressing the difficulty of applying conventional anatomical priors.
- Architecture: The architecture integrates attention blocks along the U-Net encoder’s contracting path to incorporate radiologists’ visual attention.
- Attention mechanism: Multi-scaled saliency maps suppress low-saliency regions and focus learned representations on high-saliency regions.
- Validation: Validation on 510 images demonstrated improved segmentation performance for breast tumors in ultrasound images.