PaperlayerView exact revision on arXiv

Source-linked AI summary

MEGANet: Multi-Scale Edge-Guided Attention Network for Weak Boundary Polyp Segmentation

Nhat-Tan Bui, Dinh-Hieu Hoang, Quang-Thuc Nguyen, Minh-Triet Tran, Ngan Le

arXiv:2309.03329v3cs.CV

TL;DR

Polyp segmentation is difficult because colonoscopy images contain complex backgrounds, variable polyp structures, and indistinct boundaries. MEGANet combines a U-Net-style encoder–decoder with multi-scale Edge-Guided Attention using Laplacian-derived high-frequency features, and experiments report advantages across evaluation metrics and comparisons with existing methods.

  • Problem

    Polyp segmentation must handle complex backgrounds, varying polyp sizes and shapes, and indistinct foreground–background boundaries that complicate accurate segmentation.

  • Method

    MEGANet integrates a multi-scale Edge-Guided Attention module into a U-Net decoder and uses Laplacian-based high-frequency features to preserve edge and boundary information.

  • Results

    MEGANet outperforms existing state-of-the-art methods across six evaluation metrics on five benchmark datasets.

  • Takeaways & Limitations

    The reported results support MEGANet as an effective polyp-segmentation approach for preserving weak-boundary details across varied colonoscopy-image challenges.

  • Takeaways & Limitations

    The Laplacian-based edge-extraction choice is presented as a proof of concept without exhaustive practical evaluation, and further technique search could improve performance.

Abstract

from arXiv · show

Efficient polyp segmentation in healthcare plays a critical role in enabling early diagnosis of colorectal cancer. However, the segmentation of polyps presents numerous challenges, including the intricate distribution of backgrounds, variations in polyp sizes and shapes, and indistinct boundaries. Defining the boundary between the foreground (i.e. polyp itself) and the background (surrounding tissue) is difficult. To mitigate these challenges, we propose Multi-Scale Edge-Guided Attention Network (MEGANet) tailored specifically for polyp segmentation within colonoscopy images. This network draws inspiration from the fusion of a classical edge detection technique with an attention mechanism. By combining these techniques, MEGANet effectively preserves high-frequency information, notably edges and boundaries, which tend to erode as neural networks deepen. MEGANet is designed as an end-to-end framework, encompassing three key modules: an encoder, which is responsible for capturing and abstracting the features from the input image, a decoder, which focuses on salient features, and the Edge-Guided Attention module (EGA) that employs the Laplacian Operator to accentuate polyp boundaries. Extensive experiments, both qualitative and quantitative, on five benchmark datasets, demonstrate that our MEGANet outperforms other existing SOTA methods under six evaluation metrics. Our code is available at https://github.com/UARK-AICV/MEGANet.

1. Introduction

MEGANet targets weak-boundary polyp segmentation, where complex backgrounds, varying polyp geometry, and indistinct boundaries complicate accurate localization. It integrates multi-scale edge-guided attention into a U-Net-style encoder–decoder to preserve boundary information during decoding.

  • Motivation: Colorectal cancer’s high prevalence and mortality make early detection important, while manual polyp localization requires skilled experts.Computer-aided segmentation is presented as vital for clinical evaluation.
  • Challenge: Encoder–decoder segmentation accuracy depends heavily on combining multi-scale encoder features with semantically enriched decoder features.The introduction identifies this feature fusion as central to existing polyp-segmentation architectures.
  • Approach: EGA uses multi-scale edge-related features to enhance decoder predictions while addressing weak boundaries without succumbing to noise or the semantic gap.The module applies classical edge extraction methods across decoder levels.
  • Approach: MEGANet integrates an Edge-Guided Attention module into the U-Net decoding process to preserve crucial edge and boundary information.The architecture contains an encoder for visual representation, a salient-feature decoder, and EGA for edge preservation.

2. Related Work

Prior polyp-segmentation methods improve feature learning, boundary modeling, context, or positional information, but MEGANet explicitly targets weak boundaries with high-frequency edge information. It uses the Laplacian operator to retain edge details that CNN-based features may not accurately reconstruct.

  • Existing architectures: U-Net variants such as U-Net++ improve medical-image segmentation through encoder–decoder learning and nested skip connections that address semantic gaps.These approaches primarily enhance feature learning rather than specifically addressing medical-image boundary challenges.
  • Boundary and context methods: Boundary-focused methods add boundary decoders, boundary-sensitive losses, reverse attention, active contours, offset losses, or dilated attention for polyp segmentation.Other methods address scale variation, complementary information, or positional information through specialized modules and losses.
  • MEGANet’s distinction: MEGANet differs by incorporating high-frequency edge information from classical image processing to address weak boundaries explicitly.The paper identifies CNN-based features as not optimized for accurately reconstructing input-image edges.
  • MEGANet’s distinction: The Laplacian operator is selected because its second-order derivatives produce meaningful edge structures without adding computational complexity.The paper contrasts it with hand-crafted first-order operators such as Sobel and Prewitt.

3. Proposed MEGANet

MEGANet combines a U-Net-style encoder–decoder with multi-scale Edge-Guided Attention modules to preserve high-frequency edge information and address weak polyp boundaries. The EGA modules connect contracting and expanding paths by integrating encoded features, higher-level predictions, and Laplacian-derived features.

  • Architecture: MEGANet comprises an encoder, decoder, and EGA module integrated into the U-Net decoding process.The encoder captures visual representations, while the decoder extracts salient features and restores decoding maps toward the input resolution.
  • Architecture: The EGA module operates between contracting and expanding paths at every resolution level to preserve critical information lost during downsampling.Its output feeds the expanding path, establishing a linkage between encoder and decoder features.
  • EGA inputs: Each EGA module receives encoded visual features, higher-level decoder predictions, and high-frequency features obtained through classical edge detection.These inputs are processed to generate a refined decoding feature map.
  • High-frequency feature extraction: The Laplacian pyramid supplies high-frequency information because its levels retain edges, contours, and other low-level details across scales.Each Laplacian level is formed by subtracting an upsampled smaller Gaussian-pyramid level from the current level; MEGANet uses f^l = L1(I) and downsamples it for later layers.
  • High-frequency feature extraction: MEGANet selects the level-1 Laplacian feature because repeated Gaussian filtering and downsampling degrade finer edge details at lower resolutions.The paper describes the Laplacian-based choice as a proof of concept and notes that exhaustive edge-detector selection was not performed.
  • EGA procedure: The EGA procedure decomposes higher-level predictions into reverse and boundary attention maps before combining them with encoder features and an attention mask.The mask guides attention toward vital regions while suppressing background noise and redundant information.

4. Experiments

MEGANet is evaluated on five benchmark datasets using quantitative, qualitative, efficiency, and ablation analyses. The experiments report strong comparative performance, efficient variants, and component-specific effects, including sensitivity to high-frequency feature noise.

  • Experimental setup: MEGANet is evaluated on five standard datasets using cross-dataset testing with varied resolutions and image-acquisition processes.Training uses Kvasir and CVC-ClinicDB, while testing includes Kvasir, CVC-ClinicDB, ColonDB, ETIS, and CVC-300.
  • Experimental setup: The study compares MEGANet with eight SOTA methods using ResNet-34 and Res2Net-50 backbones.The comparison includes U-Net, U-Net++, SFA, PraNet, SANet, MSNet, PEFNet, and M2UNet.
  • Quantitative evaluation: MEGANet (Res2Net-50) excels on numerous metrics, while MEGANet (ResNet-34) leads by notable gaps on most ETIS metrics.The authors associate the ResNet-34 advantage on ETIS with smaller polyps and reduced overfitting from its bounded capacity.
  • Efficiency comparison: 2.9% mDice and 3.2% mIOU improvements over M2UNet are achieved by MEGANet (ResNet-34) with a comparable parameter count.MEGANet (Res2Net-50) achieves the best performance across all ClinicDB metrics while using relatively few parameters.
  • Qualitative evaluation: Qualitative comparisons cover complex backgrounds, varying polyp sizes and configurations, and indistinct boundaries across multiple datasets.The authors particularly report low false-positive behavior in examples involving ETIS and indistinct-boundary cases.
  • Ablation study: Ablation results indicate that EGA components improve predictive performance, but removing the high-frequency feature performs best on Kvasir because mucous-background texture introduces noise.The study also compares the proposed high-frequency computation with Laplacian-pyramid features.

5. Conclusion

MEGANet is presented as a polyp-segmentation approach centered on retaining high-frequency boundary details through its EGA module. The paper reports evaluation using localization and accuracy metrics that demonstrates the method’s advantages.

  • 5. Conclusion: MEGANet integrates an Edge-Guided Attention module to retain high-frequency details, including edges, for weak-boundary polyp segmentation.The EGA module combines encoder features, high-frequency components, and decoder features across levels.
  • 5. Conclusion: The EGA module combines information from encoder features, high-frequency components, and decoder features at each level.The high-frequency component is derived from the base level rather than Gaussian filtering at every layer.
  • 5. Conclusion: The evaluation uses localization metrics mDice and mIoU together with accuracy metrics F w ϕ and MAE.The supplied conclusion states that these metrics demonstrate advantages for MEGANet.

(Supplementary Material)

The supplementary material identifies the paper’s authors, their institutional affiliations, and the associated arXiv version.

  • (Supplementary Material): The paper lists Nhat-Tan Bui, Dinh-Hieu Hoang, Quang-Thuc Nguyen, Minh-Triet Tran, and Ngan Le as authors.
  • (Supplementary Material): The authors are affiliated with the University of Arkansas and Vietnamese institutions including VNU-HCM and Vietnam National University.
  • (Supplementary Material): The document is identified as arXiv:2309.03329v3, dated 5 November 2023.

1. Visualization of the level-1 Laplacian pyramid

The level-1 Laplacian-pyramid feature highlights image boundaries and texture details, helping guide attention toward polyp-relevant edges. On Kvasir images, however, intricate mucosal backgrounds can introduce noise into this high-frequency feature.

  • 2. Visualization of the level-1 Laplacian pyramid: The EGA module uses f l as a high-frequency feature derived from the first-level Laplacian pyramid of the input image.Here, f l corresponds in most cases to L1(I).
  • 2. Visualization of the level-1 Laplacian pyramid: The Laplacian feature highlights boundaries and texture nuances, guiding attention toward edges relevant to polyps during decoding.
  • 2. Visualization of the level-1 Laplacian pyramid: On Kvasir images, complex mucosal backgrounds make polyp boundaries difficult to distinguish and can cause f l to contain noise.The paper reports that removing f l achieves the highest score on the Kvasir dataset.

2. Visualization of the decoded predicted feature

During decoding, EGA derives boundary and reverse-attention maps from higher-layer predictions and Laplacian features. These maps focus lower decoder layers on polyp boundaries while refining imprecise predictions and distinguishing polyps from mucosa.

  • 2. Visualization of the decoded predicted feature: At decoding layer i, EGA uses a higher-layer predicted feature and decomposes it into reverse-attention and boundary-attention maps.
  • 2. Visualization of the decoded predicted feature: The Laplacian-derived boundary-attention map concentrates on the polyp–mucosa boundary and directs lower decoder layers toward it.
  • 2. Visualization of the decoded predicted feature: The reverse-attention map removes and refines imprecise higher-layer predictions to separate polyps from mucous membranes.
  • 2. Visualization of the decoded predicted feature: Combining boundary and reverse attention helps recognize whole object regions and distinguish polyps from background using more accurate boundaries.
Loading 2309.03329v3…