Source-linked AI summary
Enhanced U-Net: A Feature Enhancement Network for Polyp Segmentation
Krushi Patel, Andres M. Bur, Guanghui Wang
TL;DR
Polyp segmentation in colonoscopy is difficult because of variable polyp appearance, low contrast, and examination-related factors. The paper proposes Enhanced U-Net with SFEM and AGCM to refine semantic and fine-grained features, and reports improved performance across five datasets against established models.
Problem
Polyp segmentation is challenging because polyps vary in shape, size, texture, and appearance, while colonoscopy is affected by illumination and human factors.
Method
Enhanced U-Net combines SFEM for multi-scale semantic enhancement with AGCM for selective fine-grained feature aggregation using decoder information.
Results
The model improves Dice and IoU over U-Net on EndoScene and Kvasir and improves mean-Dice and mean-IoU over U-Net on ColonDB, ETIS, and CVC-300.
Takeaways & Limitations
The integrated modules achieve state-of-the-art performance on five publicly available benchmark datasets and show strong learning and generalization ability.
Abstract
from arXiv · showhide
Colonoscopy is a procedure to detect colorectal polyps which are the primary cause for developing colorectal cancer. However, polyp segmentation is a challenging task due to the diverse shape, size, color, and texture of polyps, shuttle difference between polyp and its background, as well as low contrast of the colonoscopic images. To address these challenges, we propose a feature enhancement network for accurate polyp segmentation in colonoscopy images. Specifically, the proposed network enhances the semantic information using the novel Semantic Feature Enhance Module (SFEM). Furthermore, instead of directly adding encoder features to the respective decoder layer, we introduce an Adaptive Global Context Module (AGCM), which focuses only on the encoder's significant and hard fine-grained features. The integration of these two modules improves the quality of features layer by layer, which in turn enhances the final feature representation. The proposed approach is evaluated on five colonoscopy datasets and demonstrates superior performance compared to other state-of-the-art models.
I. INTRODUCTION
The paper targets automatic polyp segmentation because colonoscopy is affected by polyp variability, low contrast, and human factors. Enhanced U-Net combines SFEM and AGCM to improve feature quality and reports state-of-the-art performance on five benchmark datasets.
- Motivation: Colonoscopy-based polyp segmentation is difficult because polyps vary in shape, size, texture, and appearance, while illumination and operator factors can mislead examination.Automatic segmentation is presented as assistance for physicians and improved polyp detection.
- Motivation: Existing U-Net variants do not use decoder features to calculate encoder attention and rely on pooling that may lose spatial information.The paper identifies both limitations as motivations for feature enhancement.
- Proposed approach: SFEM enhances deeper semantic features with multi-scale patch-wise non-local attention, while AGCM selectively aggregates significant fine-grained encoder features using previous decoder information.SFEM fuses outputs from different patch scales; AGCM suppresses insignificant and noisy features.
- Proposed approach: The enhanced U-Net sends encoder features through AGCM, distributes SFEM-enhanced features to decoder layers, and applies auxiliary losses at decoder outputs.Decoder features are combined with outputs from AGCM and SFEM.
- Outcome: The integrated modules improve feature quality layer by layer and achieve state-of-the-art performance on five publicly available benchmark datasets.The source code is publicly available according to the contribution statement.
II. METHOD
Enhanced U-Net consists of an encoder, decoder, SFEM, and AGCM arranged to refine representations across decoding stages. Its decoder uses convolutional blocks and auxiliary losses while combining features from the proposed modules.
- Architecture: The architecture contains a ResNet-34 encoder, five-layer decoder, SFEM attached to the deepest encoding layer, and AGCM connections between encoder and decoder stages.Each decoder layer uses two convolution layers followed by batch normalization and ReLU activation.
- Feature flow: SFEM, AGCM, and decoder feature maps are concatenated and passed to subsequent decoding and AGCM operations.An additional convolution reduces channels before the deepest features enter SFEM.
- Training: Auxiliary losses are attached to every decoding layer through deep supervision.
A. Semantic Feature Enhancement Module
SFEM enhances deep semantic features with parallel patch-wise non-local attention at multiple scales. Unlike pyramid pooling, its branch outputs retain the same size, avoiding spatial-information loss from upsampling.
- Design: SFEM is motivated by the semantic importance of deeper CNN features and uses three parallel patch-wise non-local branches.
- Multi-scale branches: The first branch processes four W/2 × H/2 patches, while the second processes sixteen W/4 × H/4 patches with separate non-local spatial attention.
- Feature fusion: The branch outputs are concatenated and followed by a squeeze-and-excitation block.The figure description also includes a whole-image non-local-attention branch.
- Advantage: SFEM enhances semantic information without losing spatial information because each branch output preserves the same size.Pyramid pooling instead produces reduced-size outputs that require upsampling, which can lose spatial information.
B. Adaptive Global Context Module
AGCM converts coarse SFEM-related representations into finer features through spatial cross-layer attention. It uses current encoder features as queries and prior decoder, AGCM, and SFEM features as keys and values for selective aggregation.
- Purpose: AGCM addresses noise in coarse SFEM features by refining them layer by layer with spatial cross-layer attention.
- Cross-layer attention: The current encoder feature map serves as the query, while fused features from the previous decoder, AGCM, and SFEM serve as key and value.The resulting attention features match the encoder feature-map size and require no resizing before aggregation.
- Cross-layer attention: AGCM establishes relationships between the current encoder layer and fused features from preceding network components rather than using only same-layer encoder features.
- Feature aggregation: The module selectively aggregates fine-grained encoder features into the decoder instead of directly combining them through addition or concatenation.
C. Loss Function
The loss function combines IoU, focal, and Dice losses to capture complementary pixel-level, hard-region, and foreground-focused objectives.
- The composite loss combines pixel-based IoU, focal, and Dice losses.Focal loss emphasizes difficult pixels, weighted IoU loss emphasizes hard neighborhoods, and Dice loss focuses on the foreground object.
III. EXPERIMENTS
The experiments evaluate Enhanced U-Net across five polyp-segmentation datasets using two dataset-splitting modes and multiple established metrics. Training uses standardized resizing, cropping, augmentation, and optimization settings.
- Enhanced U-Net is evaluated on five benchmark datasets: ETIS, CVC-ClinicDB, CVC-ColonDB, EndoScene, and Kvasir.The study compares it with U-Net, U-Net++, ResUNet++, ACS, and PraNet.
- Set-1 splits Kvasir-SEG and CVC-ColonDB individually, whereas Set-2 combines them for training and evaluates on a different dataset.
- Training resizes images, randomly crops 256 X 256 regions, and applies flips, rotation, and zoom augmentation.
- Performance is assessed with recall, precision, specificity, Dice-score, IoU, and accuracy using the same metric procedure as ACM and PraNet.
C. Results
Enhanced U-Net outperforms the compared methods on same-dataset and cross-dataset evaluations, with substantial gains over U-Net and strong generalization to unseen datasets.
- 12.09% Dice and 16.06% IoU gains on EndoScene, plus 9.73% Dice and 13.61% IoU gains on Kvasir, are reported versus U-Net.
- 22.79% mean-Dice and 21.48% mean-IOU gains on ColonDB are reported versus U-Net.
- 25.25% mean-Dice and 24.7% mean-IOU gains on ETIS are reported versus U-Net.
- 17.62% mean-Dice and 18.6% mean-IOU gains on CVC-300 are reported versus U-Net.
- The generated segmentation outputs are described as visually similar to the ground-truth masks and difficult to differentiate.
D. Ablation Study
The ablation study tests SFEM, AGCM, and loss-function variants on two dataset modes, while qualitative and tabular materials document segmentation outputs and evaluation settings.
- D. Ablation Study: The ablation study trains a U-Net baseline with SFEM, AGCM, the proposed loss, or module combinations, then tests on EndoScene and CVC-300.
- D. Ablation Study: Table I organizes original images, ground-truth masks, and model-generated output masks in three columns.
- D. Ablation Study: Tables II, III, and IV report learning or generalization results for EndoScene, Kvasir-SEG, ColonDB, ETIS, and CVC-300 under their stated train-test settings.
- D. Ablation Study: Adding Dice loss alongside focal loss and weighted IoU loss further increases performance.
2) Effect of SFEM:
SFEM improves the baseline U-Net’s performance on both test sets, with especially large gains on set-2.
- 2) Effect of SFEM:: SFEM increases mean Dice, mean IoU, and Accuracy on set-1 by 2.62%, 1.65%, and 0.74%, respectively.These gains indicate improved learning ability compared with the baseline U-Net.
- 2) Effect of SFEM:: SFEM increases mean Dice, mean IoU, and Accuracy on set-2 by 5.04%, 6.56%, and 0.71%, respectively.The larger set-2 gains indicate SFEM’s generalization capability.
3) Effect of AGCM::
The proposed architecture improves feature quality layer by layer and consistently outperforms several U-Net baselines and recent polyp-segmentation models. Experiments also demonstrate strong learning and generalization ability, with potential application to other medical image segmentation tasks.
- 3) Effect of AGCM::: The architecture improves feature quality layer by layer for automatic polyp segmentation from colonoscopy images.This feature enhancement strategy is the central architectural contribution.
- 3) Effect of AGCM::: The model consistently outperforms U-Net, U-Net++, ResUNet, ACSNet, and PraNet by substantial margins across different datasets.The conclusion reports broad comparisons rather than a result from a single benchmark.
- 3) Effect of AGCM::: Experiments demonstrate strong learning capability and generalization ability for the proposed model.These findings support the model’s performance beyond the baseline setting.
- 3) Effect of AGCM::: The proposed model could be directly applied to other medical image segmentation tasks.The paper states this as a supported extension beyond polyp segmentation.