Source-linked AI summary
ResUNet++: An Advanced Architecture for Medical Image Segmentation
Debesh Jha, Pia H. Smedsrud, Michael A. Riegler, Dag Johansen, Thomas de Lange, Pal Halvorsen, Havard D. Johansen
TL;DR
Colonoscopy polyp segmentation is needed to support accurate detection of abnormal tissue, but polyp appearances and available labeled data make the task challenging. The paper proposes ResUNet++, an improved ResUNet combining residual, squeeze-and-excitation, ASPP, and attention units, and evaluates it on public datasets against U-Net and ResUNet. ResUNet++ achieves higher segmentation performance than these baselines, while its larger parameter count increases training time and image resizing may discard useful information.
Problem
Accurate automated pixel-wise polyp segmentation remains challenging because polyps vary widely in appearance and labeled medical datasets are costly to collect.
Method
ResUNet++ extends ResUNet with residual, squeeze-and-excitation, ASPP, and attention units for medical image segmentation.
Results
ResUNet++ outperforms U-Net and ResUNet on Kvasir-SEG and CVC-612 in reported segmentation evaluations.
Takeaways & Limitations
The proposed architecture produces semantically accurate polyp predictions across the evaluated publicly available datasets.
Takeaways & Limitations
ResUNet++ uses more parameters, increasing training time, while image resizing may lose useful information and experiments were run on one Nvidia-DGX-2 machine.
Abstract
from arXiv · showhide
Accurate computer-aided polyp detection and segmentation during colonoscopy examinations can help endoscopists resect abnormal tissue and thereby decrease chances of polyps growing into cancer. Towards developing a fully automated model for pixel-wise polyp segmentation, we propose ResUNet++, which is an improved ResUNet architecture for colonoscopic image segmentation. Our experimental evaluations show that the suggested architecture produces good segmentation results on publicly available datasets. Furthermore, ResUNet++ significantly outperforms U-Net and ResUNet, two key state-of-the-art deep learning architectures, by achieving high evaluation scores with a dice coefficient of 81.33%, and a mean Intersection over Union (mIoU) of 79.27% for the Kvasir-SEG dataset and a dice coefficient of 79.55%, and a mIoU of 79.62% with CVC-612 dataset.
I. INTRODUCTION
The paper targets automated pixel-wise polyp segmentation to support colonoscopy, addressing difficult appearances and limited labeled data. It proposes ResUNet++ and evaluates it against U-Net and ResUNet.
- Motivation: Colorectal polyps can precede cancer, but adenomatous polyps are frequently missed during endoscopic examinations.A real-time CAD system could act as a second observer by highlighting polyps in the endoscopic video stream.
- Challenges: Polyp segmentation is difficult because datasets are costly to label and polyps exhibit varied shapes, sizes, colors, and appearances.High interclass similarity, intra-class variation, and resemblance to stool or background mucosa further complicate segmentation.
- Approach: Semantic-segmentation methods motivate the adaptation of ResUNet as a basis for automatic polyp segmentation and detection.The paper positions ResUNet as a U-Net variant with prior state-of-the-art performance for road-image extraction.
- Contribution: ResUNet++ is proposed as a medical image segmentation architecture and evaluated on two publicly available datasets.The paper reports a performance boost compared with U-Net and ResUNet.
- Contributions: The architecture combines residual, squeeze-and-excitation, ASPP, and attention blocks, while the authors also provide the expert-annotated Kvasir-SEG dataset.The dataset is released to support development and reproducible research.
II. RELATED WORK
Prior work established end-to-end semantic segmentation and U-Net-based extensions, but polyp studies often rely on narrow or non-public evaluation settings. These limitations motivate more comparable and reproducible approaches.
- Existing research: Automatic gastrointestinal disease detection and classification in colonoscopic videos has been studied for two decades, with machine-learning performance approaching expert endoscopists.Polyp detection has received particular attention.
- Semantic segmentation: FCN introduced end-to-end pixel-to-pixel segmentation without additional post-processing, while U-Net extended this approach for biomedical images.Many later architectures modify or extend U-Net for natural and biomedical image segmentation.
- Evaluation limitations: Published polyp-detection studies often perform well only on specific datasets and use small training and validation sets.Such settings limit generalizability and robustness according to the paper.
- Evaluation limitations: Some studies focus on specific polyp types or use non-public datasets, making comparisons and result reproduction difficult.The paper identifies these practices as limitations of existing work.
III. RESUNET++
ResUNet++ extends ResUNet with residual, squeeze-and-excitation, ASPP, and attention components for medical image segmentation. Its encoder–bridge–decoder design produces a final pixel-wise segmentation map.
- Architecture: ResUNet++ builds on ResUNet and combines residual blocks, squeeze-and-excitation blocks, ASPP, and attention blocks.ResUNet itself combines deep residual learning with the U-Net architecture.
- Residual blocks: Residual blocks support deeper encoders by propagating information across layers while reducing computational cost.The paper also associates them with improved channel inter-dependencies.
- Architecture layout: The architecture contains one stem block, three encoder blocks, an ASPP bridge, and three decoder blocks.Residual units combine batch normalization, ReLU activation, and convolutional layers.
- Feature processing: Encoder outputs pass through squeeze-and-excitation blocks, while ASPP enlarges filter field-of-view to capture broader context.The encoder blocks use successive 3 × 3 convolutions, identity mappings, and strided convolution for spatial reduction.
- Output: A final 1 × 1 convolution with sigmoid activation produces the segmentation map.The decoder output is passed through ASPP before this final projection.
A. Residual Units
The paper motivates residual units as a way to train deeper networks while addressing degradation and preserving information through skip connections.
- Residual Units: Increasing network depth can improve accuracy but may hamper training and cause degradation.Deep residual learning is presented as a framework for facilitating training and addressing this problem.
- Residual Units: ResUNet uses full pre-activation residual units, whose skip connections propagate information without degradation.The paper links this design to improved network construction.
B. Squeeze and Excitation Units
ResUNet++ incorporates squeeze-and-excitation processing and ASPP to recalibrate features and capture contextual information at multiple scales.
- Squeeze and Excitation Units: Squeeze-and-excitation blocks recalibrate channel responses to emphasize relevant features and suppress unnecessary ones.The process begins by squeezing each channel into a global information embedding.
- Squeeze and Excitation Units: ASPP captures multi-scale contextual information by fusing parallel atrous convolutions with different rates.In ResUNet++, ASPP serves as a bridge between the encoder and decoder.
D. Attention Units
The architecture uses attention in the decoder to focus on essential feature-map regions, and its evaluation compares ResUNet++ with U-Net and ResUNet on two public datasets.
- Attention Units: Attention mechanisms identify which parts of the network require greater focus during semantic segmentation.They also reduce the computational cost of encoding each polyp image into a fixed-dimensional vector.
- Attention Units: ResUNet++ replaces direct encoder–decoder feature concatenation with decoder attention blocks that emphasize essential feature-map areas.This design differs from the direct concatenation used in U-Net and ResUNet.
- Attention Units: The evaluation trains, validates, and tests ResUNet++ alongside U-Net and ResUNet using two publicly available datasets.The datasets include Kvasir-SEG and CVC-ClinicDB, with the latter containing 612 images from 31 colonoscopy sequences.
B. Implementation details
The experiments use GPU-based implementations, standardized image preprocessing and augmentation, and comparisons across several segmentation architectures.
- Implementation details: All architectures were implemented with Keras and TensorFlow and evaluated on a Volta 100 GPU system.The experiments used a single GPU on an Nvidia DGX-2 system.
- Implementation details: Images were cropped with a 320×320 margin, resized to 256 × 256 pixels, and augmented using geometric and photometric transformations.Augmentations included flips, scaling, rotation, cutout, and brightness changes.
- Implementation details: ResUNet++ was compared with U-Net and ResUNet on Kvasir-SEG and CVC-612, while the original ResUNet implementation used MSE and performed unsatisfactorily.A modified ResUNet configuration produced a performance boost on both datasets.
- Implementation details: Table I reports evaluation results for all models on the Kvasir-SEG dataset.The table is the designated comparison of model performance for that dataset.
A. Results on the Kvasir-SEG dataset
On the reported experiments, ResUNet++ achieves the strongest segmentation performance across the evaluated comparisons, with qualitative masks that better capture polyp shape information.
- A. Results on the Kvasir-SEG dataset: The proposed model achieves the highest Dice coefficient, mIoU, and recall on Kvasir-SEG, with competitive precision.U-Net has the highest precision, but its Dice coefficient and mIoU are not competitive.
- A. Results on the Kvasir-SEG dataset: ResUNet++ obtains the highest Dice coefficient, mIoU, and recall on CVC-612, while maintaining competitive precision.These results extend the reported advantage beyond the Kvasir-SEG evaluation.
- A. Results on the Kvasir-SEG dataset: Across Tables I–II and Figure 3, ResUNet++ outperforms the baseline architectures in Dice coefficient, mIoU, and recall.The authors report both quantitative and qualitative superiority over the baselines.
- A. Results on the Kvasir-SEG dataset: Figure 3 compares input images, ground-truth masks, and predictions from U-Net, ResUNet, ResUNet-mod, and ResUNet++.The qualitative comparison concerns segmentation-mask quality on Kvasir-SEG.
VI. DISCUSSION
ResUNet++ produced satisfactory segmentation on both datasets and generally improved performance, while results remained sensitive to training choices and could potentially improve with more data, augmentation, and post-processing.
- ResUNet++ produced satisfactory results on both Kvasir-SEG and CVC-612, with segmentation maps that captured shape information more accurately than other architectures on Kvasir-SEG.ResUNet-mod and U-Net also produced competitive segmentation masks.
- Dice coefficient values were higher across the evaluated loss functions, while mIoU was significantly lower for alternatives to dice coefficient loss.The authors selected dice coefficient loss based on their empirical evaluation.
- Model results were influenced by the number of filters, batch size, optimizer, and loss function.
- Performance could potentially improve with a larger dataset, additional augmentation techniques, and post-processing steps.
- ResUNet++ uses more parameters, increasing training time, and image resizing may have caused loss of useful information.The authors also note that further code optimization may influence architecture results.
VII. CONCLUSION
ResUNet++ is proposed for more accurate colorectal polyp segmentation in colonoscopy images, combining several architectural components and outperforming U-Net and ResUNet in semantically accurate predictions. The architecture is positioned as a baseline for further work toward clinically useful segmentation, with post-processing as a potential improvement.
- ResUNet++ addresses the need for more accurate segmentation of colorectal polyps found during colonoscopy examinations.
- ResUNet++ combines residual units, squeeze and excitation units, Atrous Spatial Pyramidal Pooling, and attention units.
- ResUNet++ outperformed U-Net and ResUNet in producing semantically accurate predictions across evaluations using different available datasets.
- The architecture can serve as a strong baseline for further investigation toward developing a clinically useful method.
- Post-processing techniques could potentially improve the model's segmentation results.