Source-linked AI summary
Stepwise Feature Fusion: Local Guides Global
Jinfeng Wang, Qiming Huang, Feilong Tang, Jia Meng, Jionglong Su, Sifan Song
TL;DR
Polyp segmentation is challenged by variable morphology, indistinct mucosa boundaries, and limited generalization of existing models to unseen data. SSFormer addresses this with a pyramid Transformer encoder and Progressive Locality Decoder, and reports SOTA or superior performance across learning, generalization, and additional medical-image benchmarks.
Problem
Variable polyp morphology and indistinct mucosa boundaries make segmentation difficult, while existing deep-learning models may overfit current datasets and generalize poorly to unseen colonoscopy data.
Method
SSFormer combines a pyramid Transformer encoder with a Progressive Locality Decoder that emphasizes local features and progressively aggregates multi-stage features.
Results
SSFormer improves SOTA performance by about 3% on ETIS, 1.8% on CVC-ClinicDB, and 1% on Kvasir, while also achieving SOTA or superior performance in additional benchmarks.
Takeaways & Limitations
The results support SSFormer's reported robust learning and generalization ability for polyp segmentation and its potential use in other medical image segmentation tasks.
Abstract
from arXiv · showhide
Colonoscopy, currently the most efficient and recognized colon polyp detection technology, is necessary for early screening and prevention of colorectal cancer. However, due to the varying size and complex morphological features of colonic polyps as well as the indistinct boundary between polyps and mucosa, accurate segmentation of polyps is still challenging. Deep learning has become popular for accurate polyp segmentation tasks with excellent results. However, due to the structure of polyps image and the varying shapes of polyps, it easy for existing deep learning models to overfitting the current dataset. As a result, the model may not process unseen colonoscopy data. To address this, we propose a new State-Of-The-Art model for medical image segmentation, the SSFormer, which uses a pyramid Transformer encoder to improve the generalization ability of models. Specifically, our proposed Progressive Locality Decoder can be adapted to the pyramid Transformer backbone to emphasize local features and restrict attention dispersion. The SSFormer achieves statet-of-the-art performance in both learning and generalization assessment.
1 Introduction
Polyp segmentation remains difficult because polyps vary in size and morphology and have indistinct boundaries with mucosa, while existing models may generalize poorly beyond their training data. SSFormer addresses these challenges with a pyramid Transformer encoder and a Progressive Locality Decoder, reporting stronger benchmark performance across learning and generalization assessments.
- Variable polyp size, shape, and indistinct mucosa boundaries make accurate segmentation challenging.
- Existing CNN-based segmentation models show strong benchmark learning ability but may lack generalization across datasets.
- SSFormer uses a pyramid Transformer encoder to support generalization and multi-scale feature processing in polyp segmentation.
- SSFormer introduces a Progressive Locality Decoder that aggregates multi-stage features to emphasize local details and reduce attention dispersion.
- 3%, 1.8%, and 1% are the reported improvements over SOTA on ETIS, CVC-ClinicDB, and Kvasir, respectively.
2 Methodology
SSFormer combines a pyramid Transformer encoder with a Progressive Locality Decoder that first emphasizes local features and then fuses feature levels progressively. This design is intended to reduce attention dispersion, preserve critical details, and support accurate, generalizable segmentation.
- 2 Methodology: Pyramid Transformer encoders replace CNN encoders to provide multi-scale feature processing and stronger generalization.
- 2.2 Aggregate local and global features stepwise (PLD): The PLD combines Local Emphasis and Stepwise Feature Aggregation to process local and global features across Transformer feature pyramids.
- Local Emphasis: The Local Emphasis module uses convolutional receptive fields to increase neighboring-patch weights and refocus attention on local features.
- Local Emphasis: Local Emphasis cleans cluttered noise and re-condenses attention around contours and boundaries after feature processing.
- Stepwise Feature Aggregation (SFA): Stepwise Feature Aggregation progressively fuses shallow local features into deeper global features, reducing the information gap between feature levels.
- Stepwise Feature Aggregation (SFA): Concatenation and addition are both supported in the fusion unit and perform equally well; concatenation is SSFormer's default.
- 2.3 Stepwise Segmentation Transformer: SSFormer-S and SSFormer-L use standard and large encoder scales and achieve SOTA or competitive performance across several segmentation benchmarks.
3 Experiments
The experiments assess SSFormer’s learning ability on matched benchmark splits, generalization on unseen datasets, and the contributions of its encoder-decoder design. SSFormer improves reported state-of-the-art results across several benchmarks and performs best with PLD paired with MiT.
- Experimental setup: 1450 images from Kvasir and CVC-ClinicDB formed a shared training set, with testing on CVC-ColonDB and ETIS for unseen-data generalization.ETIS is identified as the most challenging benchmark because of its varied polyp types and sizes.
- Learning ability: SSFormer improves state-of-the-art performance by about 1.8% on CVC-ClinicDB and about 1% on Kvasir using 80%/10%/10% train-evaluation-test splits.These results are reported as evidence of accurate prediction and learning ability.
- Additional benchmarks: SSFormer achieves state-of-the-art or excellent performance on the 2018 Data Science Bowl and ISIC-2018 medical segmentation benchmarks.These were additional experiments beyond the polyp segmentation benchmarks.
- Generalization ability: On unseen datasets, SSFormer significantly improves state-of-the-art performance by 3% on ETIS and achieves superior performance on CVC-ColonDB.The reported generalization tests include cross-dataset evaluation and describe robust generalization and accurate prediction abilities.
- Ablation study: PLD performs best with the MiT encoder under the reported encoder-decoder comparison, while concatenation and addition perform equally well in feature fusion.Concatenation is the default feature-fusion operation in SSFormer.
4 Conclusions
SSFormer is presented as a deep learning model with robust generalization and learning ability for polyp segmentation. Additional experiments report learning ability on ISIC-2018 and 2018 Data Science Bowl benchmarks, while its local feature emphasis module constrains Transformer attention dispersion.
- SSFormer is proposed with robust generalization and learning ability for polyp segmentation.
- Additional experiments report SSFormer learning ability on ISIC-2018 and 2018 Data Science Bowl benchmarks.
- The local feature emphasis module effectively constrains Transformer attention dispersion.