PaperlayerView exact revision on arXiv

Source-linked AI summary

Duplex Contextual Relation Network for Polyp Segmentation

Zijin Yin, Kongming Liang, Zhanyu Ma, Jun Guo

arXiv:2103.06725v3eess.IVcs.CV

TL;DR

Polyp segmentation is difficult because polyps vary in appearance and their boundaries can be blurred, while prior methods mainly model context within individual images. DCRNet combines within-image and cross-image contextual relations using two attention modules and a memory queue. The paper reports state-of-the-art performance across three polyp-segmentation datasets.

  • Problem

    Polyp appearance varies and boundaries can be blurred, while previous methods mainly model within-image dependencies and ignore cross-image contextual relations.

    Source: §1 · p. 1

  • Method

    DCRNet combines interior and exterior contextual-relation modules with a queue-based memory for region embeddings from previous images.

    Source: §1 · pp. 1–2

  • Results

    DCRNet is reported to outperform state-of-the-art methods on EndoScene, Kvasir-SEG, and PICCOLO.

    Source: §1 · p. 2, §4 · p. 4, §3.3 · p. 4

  • Takeaways & Limitations

    The results support modeling contextual relations across images as well as within individual images for polyp segmentation.

    Source: §1 · p. 1, §4 · p. 4

Abstract

from arXiv · show

Polyp segmentation is of great importance in the early diagnosis and treatment of colorectal cancer. Since polyps vary in their shape, size, color, and texture, accurate polyp segmentation is very challenging. One promising way to mitigate the diversity of polyps is to model the contextual relation for each pixel such as using attention mechanism. However, previous methods only focus on learning the dependencies between the position within an individual image and ignore the contextual relation across different images. In this paper, we propose Duplex Contextual Relation Network (DCRNet) to capture both within-image and cross-image contextual relations. Specifically, we first design Interior Contextual-Relation Module to estimate the similarity between each position and all the positions within the same image. Then Exterior Contextual-Relation Module is incorporated to estimate the similarity between each position and the positions across different images. Based on the above two types of similarity, the feature at one position can be further enhanced by the contextual region embedding within and across images. To store the characteristic region embedding from all the images, a memory bank is designed and operates as a queue. Therefore, the proposed method can relate similar features even though they come from different images. We evaluate the proposed method on the EndoScene, Kvasir-SEG and the recently released large-scale PICCOLO dataset. Experimental results show that the proposed DCRNet outperforms the state-of-the-art methods in terms of the widely-used evaluation metrics.

1. INTRODUCTION

Polyp segmentation is challenging because polyps vary in appearance and often have blurred boundaries. DCRNet addresses the gap in prior within-image context modeling by combining within-image and cross-image relations, with a memory queue for embeddings from previous images.

1. INTRODUCTION

  • Polyps vary in size, color, and texture, while blurred boundaries make accurate segmentation challenging.Automatic segmentation can locate polyps and reduce manual labor and misdiagnosis rates.Source: §1 · p. 1
  • Prior methods focus on within-image context, leaving cross-image semantic similarity largely unmodeled in segmentation.The introduction notes that co-occurring visual patterns can persist across images despite color differences.Source: §1 · p. 1
  • DCRNet combines interior and exterior contextual-relation modules to model relations within an image and across images.The parallel attention-based modules can be incorporated into encoder-decoder architectures; features are aggregated according to estimated similarities.Source: §1 · pp. 1–2
  • A Region Cross-Batch Memory queue stores characteristic region embeddings from previous images, allowing similar features from different samples to be related.The exterior module uses across-image similarity to enhance features with contextual region embeddings from other images.Source: §1 · p. 2
  • DCRNet is reported to outperform state-of-the-art methods on EndoScene, Kvasir-SEG, and PICCOLO.The introduction identifies the three-dataset comparison as an experimental contribution.Source: §1 · p. 2

2. METHOD

DCRNet processes encoder features through parallel interior and exterior relation modules, then fuses their representations for pixel-level prediction. Its exterior module relates current features to buffered region embeddings, while a first-in-first-out memory queue limits how long past embeddings remain available.

2.1. Overview

  • Encoder features pass through parallel ICR and ECR modules, with ROM storing up-to-date embeddings from previous mini-batches before the two module outputs are fused for pixel-level prediction.The overview identifies ResNet-34 as the encoder backbone.Source: §2.1 · p. 2

2.2. Interior Contextual-Relation Module

  • The interior module uses position attention to make each pixel representation a weighted sum of all pixels in the same image.This adaptively strengthens contextual relations with same-class surroundings and weakens those with different-class areas.Source: §2.2 · p. 2

2.3. Exterior Contextual-Relation Module

  • The exterior module compares current mini-batch features with buffered region embeddings to form a contextual attention map and augment the representations.Its contextual region embedding begins with a coarse map whose entries indicate the degree that each pixel belongs to the polyp region.Source: §2.3 · pp. 2–3

2.4. Region Cross-Batch Memory

  • Region Cross-Batch Memory uses a first-in-first-out queue, discarding the earliest mini-batch as new embeddings are enqueued.The bank size should be moderate: too small limits contextual relations, while too large retains out-of-date data; memory is removed at inference.Source: §2.4 · p. 3

2.5. Loss Functions

  • Training combines weighted binary cross-entropy and Dice losses, with deep supervision on three intermediate decoder maps and the coarse segmentation map.The maps are up-sampled to label size; the stated strategy considers pixel-level and region-level measurements.Source: §2.5 · p. 3

3. EXPERIMENTS

Across EndoScene, Kvasir-SEG, and PICCOLO, DCRNet is reported to outperform state-of-the-art methods; the experiments also assess its efficiency and component contributions.

3.1. Datasets

  • EndoScene has 912 images, Kvasir-SEG has 1,000, and PICCOLO has 3,433, including White-Light and Narrow-Band images.The experiments use the datasets’ default splits for EndoScene and PICCOLO and a 60/20/20 split for Kvasir-SEG; images are resized to 224 × 224.Source: §3.1 · p. 3

3.2. Implementation Details and Evaluation Metrics

  • Training uses Adam at 1e-4 for 150 epochs, batch size 4, augmentation, and dataset-specific memory sizes of 20 or 40.The memory size is 20 for Kvasir-SEG and 40 for EndoScene and PICCOLO; evaluation includes MAE, Dice, IoU, boundary F-measure, and Sα.Source: §3.2 · p. 4

3.3. Results and Analysis

  • DCRNet achieves superior performance on all metrics across three benchmarks, with substantial IoU, Dice, and F-measure margins indicating stronger region and boundary accuracy.The comparison includes medical image segmentation methods and polyp segmentation methods.Source: §3.3 · p. 4
  • On PICCOLO, DCRNet runs faster than the compared methods and has the fewest parameters under the reported inference analysis.The inference comparison uses batch size 4 on a 1080Ti GPU.Source: §3.3 · p. 4, Table 2 · p. 4

3.4. Ablation Study

  • Progressively adding ICR, ECR, and ROM improves Dice by 0.68%, 1.11%, and 2.31%, respectively.The improvement from Backbone + ECR + ROM is more pronounced on PICCOLO than on EndoScene.Source: §3.4 · p. 4, Table 3 · p. 4

4. CONCLUSION

DCRNet captures polyp contextual relations within and across images using two parallel modules and a queue-based memory. It achieves state-of-the-art performance on three datasets, with ablations supporting the effectiveness of its components.

  • DCRNet explores co-occurring polyp visual patterns across images and captures contextual relations within and across images using two parallel modules.A queue-based memory stores characteristic region embeddings from previous images.Source: §4 · p. 4
  • Experiments report state-of-the-art performance on three datasets across widely used evaluation metrics, and ablations demonstrate the effectiveness of the proposed components.Source: §4 · p. 4

Related papers

Loading 2103.06725v3…