Source-linked AI summary
Automatic Polyp Segmentation via Multi-scale Subtraction Network
Xiaoqi Zhao, Lihe Zhang, Huchuan Lu
TL;DR
Colorectal polyp segmentation matters for early colorectal cancer detection, but existing feature-fusion operations can introduce redundancy and weaken localization and edge quality. MSNet uses multi-scale subtraction units and LossNet supervision to capture complementary, detailed, and structural features; experiments show strong performance across five datasets with approximately 70fps inference.
Problem
Manual colonoscopy is labor-intensive and costly, with a high misdiagnosis rate, motivating accurate automatic polyp segmentation for colorectal cancer prevention.
Method
MSNet pyramidally combines subtraction units for multi-scale cross-level complementary features and uses a training-free LossNet to supervise segmentation from details to structure.
Results
MSNet outperforms state-of-the-art methods under different evaluation metrics on five challenging datasets and runs at approximately 70fps.
Takeaways & Limitations
Subtraction reduces redundancy among multi-level decoder inputs, while LossNet jointly targets lesion geometry from contour to body to improve segmentation accuracy.
Abstract
from arXiv · showhide
More than 90\% of colorectal cancer is gradually transformed from colorectal polyps. In clinical practice, precise polyp segmentation provides important information in the early detection of colorectal cancer. Therefore, automatic polyp segmentation techniques are of great importance for both patients and doctors. Most existing methods are based on U-shape structure and use element-wise addition or concatenation to fuse different level features progressively in decoder. However, both the two operations easily generate plenty of redundant information, which will weaken the complementarity between different level features, resulting in inaccurate localization and blurred edges of polyps. To address this challenge, we propose a multi-scale subtraction network (MSNet) to segment polyp from colonoscopy image. Specifically, we first design a subtraction unit (SU) to produce the difference features between adjacent levels in encoder. Then, we pyramidally equip the SUs at different levels with varying receptive fields, thereby obtaining rich multi-scale difference information. In addition, we build a training-free network "LossNet" to comprehensively supervise the polyp-aware features from bottom layer to top layer, which drives the MSNet to capture the detailed and structural cues simultaneously. Extensive experiments on five benchmark datasets demonstrate that our MSNet performs favorably against most state-of-the-art methods under different evaluation metrics. Furthermore, MSNet runs at a real-time speed of $\sim$70fps when processing a $352 \times 352$ image. The source code will be publicly available at \url{https://github.com/Xiaoqi-Zhao-DLUT/MSNet}. \keywords{Colorectal Cancer \and Automatic Polyp Segmentation \and Subtraction \and LossNet.}
1 Introduction
Colorectal polyps can precede colorectal cancer, making accurate automatic segmentation clinically significant. MSNet addresses redundancy in conventional multi-level feature fusion with subtraction-based multi-scale features and LossNet supervision, achieving strong benchmark performance at real-time speed.
- Motivation: Colorectal polyps can become cancerous, so accurate automatic segmentation supports early detection and reduces reliance on costly, error-prone manual colonoscopy interpretation.Colonoscopy is commonly used but involves manual labor, expense, and a high misdiagnosis rate.
- Problem: Existing U-shaped polyp-segmentation methods commonly fuse encoder features through addition or concatenation, which can introduce redundant information.These operations can weaken level-specific complementarity and hinder simultaneous localization and boundary refinement.
- Method: MSNet uses subtraction units between adjacent encoder levels and pyramidally combines them to capture complementary cross-level information at multiple scales.The design targets scale diversity while aggregating level-specific and differential features for decoder prediction.
- Method: LossNet provides training-free supervision across feature levels, optimizing segmentation from details to structure.This complements supervision based on the prediction itself and supports polyp-aware feature learning.
- Results: MSNet outperforms state-of-the-art methods across five challenging datasets and processes 352 × 352 images at approximately 70fps.The reported experiments evaluate performance under different metrics, while the inference speed is described as real time.
2 Method
MSNet combines multi-scale subtraction of encoder features with LossNet supervision to enhance complementary information and guide segmentation from detail to structure.
- MSNet architecture: MSNet uses five encoder blocks, a multi-scale subtraction module, and four decoder blocks to produce the final polyp segmentation.Res2Net-50 extracts five feature levels, whose channels are reduced to 64 before subtraction processing.
- Multi-scale subtraction module: The subtraction unit computes absolute element-wise differences between adjacent activated feature maps, followed by convolution.This operation is designed to capture complementary information and highlight differences for the decoder.
- Multi-scale subtraction module: Multiple subtraction units are concatenated across orders and receptive fields to capture higher-order complementary information across feature levels.Level-specific and cross-level differential features are aggregated into complementarity-enhanced features before decoding.
- LossNet: LossNet uses an ImageNet-pretrained classification network to compare multi-scale prediction and ground-truth features during training.The resulting feature differences provide additional supervision beyond weighted IoU and binary cross-entropy losses.
- LossNet: LossNet supervises feature levels from detail to structure because low-level features encode boundaries while high-level features encode location.This produces comprehensive supervision across feature levels using pixel-level Euclidean-distance losses.
3 Experiments
Experiments evaluate MSNet on five benchmark datasets using six metrics, comparisons with established methods, qualitative visualizations, and component ablations. MSNet outperforms competing approaches and operates at approximately 70fps, while subtraction units and LossNet improve ablation performance.
- Experimental Setup: MSNet is evaluated on five benchmark datasets using mean Dice, mean IoU, weighted F-measure, MAE, S-measure, and E-measure.Higher values are better for all metrics except MAE.
- Experimental Setup: The comparison includes U-Net, U-Net++, SFA, and PraNet, using predictions supplied by authors or generated from released code.
- Quantitative Evaluation: Across all datasets and six metrics, MSNet outperforms the other approaches, with especially strong performance on CVC-ColonDB and ETIS.Against PraNet on ETIS, the reported gains are 14.1% in mDice, 15.3% in mIoU, 13.0% in Fw, 6.2% in Sα, 4.8% in Eφ, and 35.5% in MAE.
- Efficiency: ∼70fps is MSNet's reported real-time inference speed and the fastest among the compared state-of-the-art methods.The model processes 352 × 352 inputs in the reported experimental setting.
- Qualitative Evaluation: Visual comparisons report detection of small, medium, and large polyps, as well as multiple polyps with more captured details.
- Ablation Study: 7.8%, 7.4%, 6.7%, and 4.4% are the reported CVC-ColonDB gains from adding subtraction units, measured by mDice, mIoU, Fw, and MAE, respectively.The ablation compares the subtraction-unit model with the common FPN baseline.
- Ablation Study: 11.8%, 14.1%, 13.0%, and 5.5% are the reported ETIS gains from adding LossNet, measured by mDice, mIoU, Fw, and MAE, respectively.The comparison is between the model with LossNet and the model using the multi-scale subtraction module without LossNet.
4 Discussion
The discussion attributes MSNet's behavior to reduced redundancy from subtraction and geometric supervision from LossNet. Together, these components target cross-level complementarity and lesion shape in binary segmentation.
- Multi-scale Subtraction Module: Subtraction produces decoder inputs with less redundancy across levels and enhances their scale-specific properties.The discussion identifies this as a potential mechanism for other segmentation tasks.
- LossNet: LossNet supervises binary segmentation using lesion geometry, jointly targeting contours and the lesion body to improve overall segmentation accuracy.
5 Conclusion
The conclusion presents MSNet as a multi-scale subtraction network for colonoscopy polyp segmentation, combining cross-level complementary information with level-specific features and training-free multi-level supervision. The reported model outperforms state-of-the-art methods and runs at approximately 70fps.
- Conclusion: MSNet combines pyramidally concatenated subtraction units with level-specific features to enhance multi-scale polyp representations.The subtraction units extract lower-order and higher-order cross-level complementary information.
- Conclusion: A training-free loss network supervises predictions from different feature levels to optimize segmentation of both structure and details.
- Conclusion: ∼70fps is the reported fastest speed among existing polyp segmentation methods, while MSNet notably outperforms state-of-the-art methods across evaluation metrics.