PaperlayerView exact revision on arXiv

Source-linked AI summary

Feature Aggregation and Propagation Network for Camouflaged Object Detection

Tao Zhou, Yi Zhou, Chen Gong, Jian Yang, Yu Zhang

arXiv:2212.00990v1cs.CV

TL;DR

Camouflaged object detection remains difficult because foreground objects resemble their surroundings. FAP-Net combines boundary modeling, multi-scale aggregation, and gated cross-level fusion and propagation, and the authors report superior COD performance with extension to polyp segmentation.

  • Problem

    Camouflaged object detection remains challenging because foreground objects resemble their backgrounds, while scale variation, unequal encoder-feature contributions, and indistinct boundaries remain concerns.

  • Method

    FAP-Net uses boundary-enhanced representations, multi-scale aggregation from each layer, and cross-level fusion with gated propagation from encoder to decoder.

  • Results

    The authors report that FAP-Net outperforms other state-of-the-art COD methods on three benchmark datasets and is effective when extended to polyp segmentation.

  • Takeaways & Limitations

    The framework provides a unified end-to-end approach for fusing cross-level features and propagating context in camouflaged-object and polyp segmentation.

Abstract

from arXiv · show

Camouflaged object detection (COD) aims to detect/segment camouflaged objects embedded in the environment, which has attracted increasing attention over the past decades. Although several COD methods have been developed, they still suffer from unsatisfactory performance due to the intrinsic similarities between the foreground objects and background surroundings. In this paper, we propose a novel Feature Aggregation and Propagation Network (FAP-Net) for camouflaged object detection. Specifically, we propose a Boundary Guidance Module (BGM) to explicitly model the boundary characteristic, which can provide boundary-enhanced features to boost the COD performance. To capture the scale variations of the camouflaged objects, we propose a Multi-scale Feature Aggregation Module (MFAM) to characterize the multi-scale information from each layer and obtain the aggregated feature representations. Furthermore, we propose a Cross-level Fusion and Propagation Module (CFPM). In the CFPM, the feature fusion part can effectively integrate the features from adjacent layers to exploit the cross-level correlations, and the feature propagation part can transmit valuable context information from the encoder to the decoder network via a gate unit. Finally, we formulate a unified and end-to-end trainable framework where cross-level features can be effectively fused and propagated for capturing rich context information. Extensive experiments on three benchmark camouflaged datasets demonstrate that our FAP-Net outperforms other state-of-the-art COD models. Moreover, our model can be extended to the polyp segmentation task, and the comparison results further validate the effectiveness of the proposed model in segmenting polyps. The source code and results will be released at https://github.com/taozh2017/FAPNet.

I. INTRODUCTION

Camouflaged object detection is difficult because objects resemble their surroundings and vary in size, appearance, and structure. FAP-Net addresses these challenges with boundary guidance, multi-scale aggregation, and cross-level feature fusion and propagation.

  • I. INTRODUCTION: COD is challenging because camouflaged objects have high intrinsic similarity to their backgrounds and diverse size, color, shape, and texture.These properties make accurate detection difficult even as deep learning methods have improved performance.
  • I. INTRODUCTION: Existing methods leave three issues open: characterizing multi-scale information, weighting encoder features by contribution, and locating indistinct object boundaries.The paper identifies scale variation, unbalanced multi-level feature use, and unclear boundaries as remaining challenges.
  • I. INTRODUCTION: FAP-Net introduces a Boundary Guidance Module that learns boundary-enhanced representations and incorporates them into the decoder layer by layer.The module preserves local characteristics and boundary information from the original images.
  • I. INTRODUCTION: Its Multi-scale Feature Aggregation Module extracts multi-scale information from each convolutional level to handle camouflaged-object scale variations.The module learns aggregated features using multi-scale information from individual layers.
  • I. INTRODUCTION: Its Cross-level Fusion and Propagation Module fuses cross-level features and propagates useful encoder information to the decoder while adaptively balancing encoder-block contributions.The proposed framework is trained end to end and was evaluated on COD and polyp segmentation tasks.

B. Multi-scale/level Feature Learning

The paper reviews multi-scale and multi-level feature learning before positioning FAP-Net as an architecture that aggregates, fuses, and propagates encoder features for COD.

  • B. Multi-scale/level Feature Learning: Multi-scale feature learning constructs representations at different granularities to support detection and segmentation across object scales.Feature Pyramid Network is cited as a representative approach that builds multi-scale feature maps.
  • B. Multi-scale/level Feature Learning: Multi-level feature integration merges representations from selected network levels to improve final features in segmentation and visual recognition.Prior approaches use shortcut connections or merge deep features from selected layers.
  • B. Multi-scale/level Feature Learning: FAP-Net contains boundary guidance, multi-scale feature aggregation, and cross-level fusion and propagation as its three key components.The architecture overview identifies these components as the core modules of the proposed network.
  • B. Multi-scale/level Feature Learning: A gated mechanism adaptively controls information flow, providing a basis for FAP-Net’s feature propagation design.The related work describes gated mechanisms as tools for adaptively integrating modalities or multi-level features.
  • B. Multi-scale/level Feature Learning: The encoder extracts multi-level features, MFAM aggregates multi-scale information, and CFPM integrates and propagates features toward the decoder.Boundary-enhanced representations are also combined with decoder features, followed by multiple side-output supervision strategies.

B. Boundary Guidance Module

The paper uses boundary information and multi-scale feature aggregation to improve camouflaged object representations. BGM emphasizes object boundaries, while MFAM aggregates features across scales to address scale variation.

  • Boundary Guidance Module: Camouflaged objects have indistinct boundaries from their surroundings, making boundary localization critical for COD.The paper motivates explicit boundary modeling because camouflage weakens the boundary between object and background.
  • Boundary Guidance Module: BGM learns boundary-enhanced representations and injects them into the decoder through layer-wise guidance.The module also supervises early convolutional layers with boundary detection loss and cascades the enhanced representation to multiple supervisions.
  • Multi-scale Feature Aggregation Module: MFAM aggregates image features at different scales within a convolutional layer to characterize camouflaged objects with varying sizes.It addresses the limitation that an individual convolutional layer handles only a particular scale.
  • Multi-scale Feature Aggregation Module: MFAM reduces channels with independent 1×1 convolutions and uses two streams with different kernels to capture complementary scale information.The two streams use 3×3 and 5 × 5 convolutions, exchange information, and are fused with residual connections.
  • Multi-scale Feature Aggregation Module: Residual fusion preserves the original feature information while combining the enhanced multi-scale representations.The resulting aggregated features are intended to improve handling of object scale variations.

D. Cross-level Fusion and Propagation Module

CFPM combines adjacent-level features and propagates encoder context to the decoder. Its gated fusion adaptively weights encoder and decoder contributions while preserving feature information through residual connections.

  • Motivation: Existing methods may pass encoder features directly to the decoder while ignoring the differing contributions of features from different levels.The paper identifies this as a reason valuable context is not adequately propagated into the decoder.
  • Cross-level Feature Fusion: CFPM first fuses two cross-level aggregated features to exploit their correlations.One feature is upsampled to match the other’s resolution before fusion.
  • Cross-level Feature Fusion: The fused feature generates sigmoid-normalized attention maps that adaptively enhance the cross-level feature representations.A residual connection combines the enhanced features with their originals to preserve original information.
  • Decoder Feature Propagation: The propagation component combines fused encoder features with the output of the preceding CFPM to transmit context toward the decoder.This propagation step is described as combining the previous CFPM output with fused encoder features.
  • Decoder Feature Propagation: Spatial-wise gates assign position-dependent weights to the two feature maps after convolutional processing, concatenation, and softmax normalization.The gates control the relative contributions of the encoder and decoder streams.
  • Decoder Feature Propagation: When fusing f5 and f4, CFPM uses feature fusion without the feature propagation component.This is an explicit exception in the module’s application across levels.

E. Overall Loss Function

The overall loss combines weighted IoU and binary cross-entropy objectives with multiple side-output supervisions. This design addresses both global structure and local pixel-level restrictions.

  • Detection Loss: The detection loss combines weighted IoU loss with binary cross-entropy loss.Weighted IoU provides a global restriction, while BCE supplies a local restriction.
  • Detection Loss: Weighted IoU increases the weights of hard pixels to highlight their importance during optimization.The loss therefore emphasizes difficult regions rather than treating all pixels equally.
  • Multiple Supervisions: Multiple supervisions are applied to four side-output maps and the ground-truth map after resizing each map to the ground-truth resolution.The overall loss is formulated over these supervised outputs.

IV. EXPERIMENTS

The experiments evaluate FAP-Net against state-of-the-art methods, test its components through ablation studies, and extend it to polyp segmentation.

  • Experimental Settings: The experimental section specifies datasets, evaluation metrics, and implementation details before presenting comparisons and ablation studies.These experiments assess the model against other state-of-the-art methods and the effectiveness of its key components.
  • Task Extension: The paper extends the proposed model to the polyp segmentation task.The extension is included as a final experimental investigation.
  • Evaluation: The experiments are organized to compare FAP-Net with other methods and validate each key component.The section combines benchmark evaluation, ablation analysis, and cross-task application.

A. Experimental Setup

Experiments use three public COD datasets and five evaluation metrics to compare models comprehensively. The training setup uses a pretrained Res2Net-50 backbone, Adam optimization, scheduled learning-rate decay, and augmentation.

  • Datasets: Experiments use CHAMELEON, CAMO, and COD10K, with 76, 1,250, and 5,066 images, respectively.CHAMELEON is testing-only; CAMO and COD10K provide separate training and testing splits.
  • Evaluation Metrics: Five metrics evaluate COD performance: Sα, mean Fβ, mean Eφ, M, and precision-recall analysis.The metrics assess structure, precision-recall balance, enhanced alignment, pixel error, and threshold-dependent performance.
  • Evaluation Metrics: Precision and recall are computed from thresholded saliency maps and combined across thresholds to form PR curves.Different thresholds from 0 to 255 produce recall-precision pairs describing performance at different operating points.
  • Evaluation Metrics: The enhanced-alignment measure averages threshold-specific Eφ values, while M measures average pixel-level relative error.Ground truth and normalized predictions are compared at the pixel level for M.
  • Evaluation Metrics: Fβ uses a weighted harmonic mean of precision and recall, with β² set to 0.3 to emphasize precision.The maximum Fβ across fixed thresholds is reported.
  • Implementation Details: The model is implemented in PyTorch with a pretrained Res2Net-50 backbone and trained using Adam, learning-rate decay, and data augmentation.Augmentations include random flipping, cropping, and rotation at scaling ratios of 0.75, 1, and 1.25.

B. Comparison with State-of-the-art Methods

The comparison evaluates FAP-Net against 20 state-of-the-art COD methods using established benchmark protocols. Three compared models were retrained from released code, while results for the others were collected from their publications.

  • Compared Methods: FAP-Net is compared with 20 state-of-the-art methods spanning detection, segmentation, pyramid, attention, and COD-specific architectures.The comparison includes FPN, MaskRCNN, UNet++, BASNet, CPD, EGNet, PraNet, SINet, PFNet, and other published methods.
  • Evaluation Protocol: GateNet, MINet, and DNTDF were retrained using their released code, whereas results for the other methods were collected from existing sources.

2) Quantitative Comparison:

FAP-Net achieves the strongest reported quantitative and qualitative COD results across the benchmark comparisons. It improves substantially over leading baselines while using fewer parameters than SINet and PFNet.

  • Quantitative Results: FAP-Net outperforms all compared methods on CHAMELEON across every reported evaluation metric.The paper specifically notes stronger localization than methods using auxiliary edge or boundary information.
  • Quantitative Results: 4.2%, 4.0%, and 2.7% improvements over PFNet are reported for Sα, Fβ, and Eφ, respectively.
  • Quantitative Results: 6.0%, 9.3%, and 4.9% improvements over PraNet are reported for Sα, Fβ, and Eφ, respectively.
  • Quantitative Results: FAP-Net achieves the best PR and F-measure curves compared with other COD methods.These curves are reported on the three camouflaged object datasets.
  • Model Complexity: FAP-Net has minimal parameters compared with the representative COD methods SINet and PFNet.
  • Qualitative Results: Qualitative comparisons show more accurate and complete segmentation across scale variation, similar textures, weak boundaries, multiple objects, and embedded appearances.The visual comparison includes EGNet, CPD, PraNet, SINet, and PFNet.

4) Super-class Performance Comparison:

FAP-Net maintains strong performance across COD10K super-classes, and ablations attribute gains to its baseline framework, MFAM, CFPM, and BGM components. Cross-dataset generalization is also evaluated by training on one dataset and testing on others.

  • Super-class Performance Comparison: FAP-Net obtains the best performance across the Flying, Terrestrial, Aquatic, and Amphibian COD10K super-classes using four evaluation metrics.
  • Ablation Study: The basic FAP-Net framework performs better than FPN and Attention U-Net in the ablation study.
  • Ablation Study: MFAM improves COD performance by extracting aggregated features with multi-scale convolutional kernels to address object scale variation.MFAM also outperforms the Inception module on most evaluation metrics.
  • Ablation Study: CFPM gains arise from both adjacent-level feature fusion and subsequent encoder-to-decoder feature propagation.The proposed CFPM also performs better than a simple Attention Gate unit.
  • Ablation Study: BGM further improves COD performance by integrating boundary-enhanced features into the decoder to provide low-level structure information.
  • Cross-dataset Generalization: Cross-dataset analysis trains models on CAMO and COD1K and tests them on other datasets to assess generalizability.The study compares FAP-Net with MINet, PraNet, SINet, and PFNet.

E. Extension Application

FAP-Net is extended to polyp segmentation, where shape and size variation and non-sharp polyp boundaries pose challenges. Across four datasets, the model outperforms compared methods and handles challenging polyp appearances.

  • Automatic polyp segmentation supports locating polyp regions for further diagnosis or treatment in screening systems.
  • The evaluation uses four polyp datasets and compares FAP-Net with UNet, UNet++, SFA, and PraNet.
  • 2.9%, 3.3%, 1.2%, and 1.6% improvements over PraNet are reported on CVC-ColonDB for mDice, mIoU, Sα, and weighted Fβ, respectively.
  • 14.2%, 13.4%, 5.9%, and 9.5% improvements over PraNet are reported on ETIS for mDice, mIoU, Sα, and weighted Fβ, respectively.
  • Visual comparisons indicate that FAP-Net can accurately locate and segment polyps despite varied sizes and non-sharp boundaries.
  • The paper concludes that FAP-Net outperforms other polyp segmentation methods after being applied to the polyp segmentation task.
Loading 2212.00990v1…