PaperlayerView exact revision on arXiv

Source-linked AI summary

Camouflaged Object Detection via Context-aware Cross-level Fusion

Geng Chen, Si-Jie Liu, Yu-Jia Sun, Ge-Peng Ji, Ya-Feng Wu, Tao Zhou

arXiv:2207.13362v1cs.CV

TL;DR

COD remains challenging because camouflage creates low boundary contrast and substantial variation in object appearance. C2F-Net addresses this with attention-guided cross-level fusion, global context refinement, and coarse-guided low-level refinement, and it outperforms reported state-of-the-art COD methods while showing promise for polyp segmentation.

  • Problem

    Low boundary contrast and variation in object size and shape make accurate camouflaged object detection difficult, particularly in occluded and multiple-object scenes.

  • Method

    C2F-Net combines ACFM-based attention-guided cross-level feature fusion, DGCM-based global context refinement, and CIM-based coarse-guided low-level feature refinement.

  • Results

    C2F-Net outperforms state-of-the-art COD methods on three public datasets, while evaluation on four polyp segmentation datasets demonstrates promising downstream potential.

  • Takeaways & Limitations

    The results support C2F-Net as an effective COD model with promising applicability to polyp segmentation.

Abstract

from arXiv · show

Camouflaged object detection (COD) aims to identify the objects that conceal themselves in natural scenes. Accurate COD suffers from a number of challenges associated with low boundary contrast and the large variation of object appearances, e.g., object size and shape. To address these challenges, we propose a novel Context-aware Cross-level Fusion Network (C2F-Net), which fuses context-aware cross-level features for accurately identifying camouflaged objects. Specifically, we compute informative attention coefficients from multi-level features with our Attention-induced Cross-level Fusion Module (ACFM), which further integrates the features under the guidance of attention coefficients. We then propose a Dual-branch Global Context Module (DGCM) to refine the fused features for informative feature representations by exploiting rich global context information. Multiple ACFMs and DGCMs are integrated in a cascaded manner for generating a coarse prediction from high-level features. The coarse prediction acts as an attention map to refine the low-level features before passing them to our Camouflage Inference Module (CIM) to generate the final prediction. We perform extensive experiments on three widely used benchmark datasets and compare C2F-Net with state-of-the-art (SOTA) models. The results show that C2F-Net is an effective COD model and outperforms SOTA models remarkably. Further, an evaluation on polyp segmentation datasets demonstrates the promising potentials of our C2F-Net in COD downstream applications. Our code is publicly available at: https://github.com/Ben57882/C2FNet-TSCVT.

I. INTRODUCTION

COD is difficult because camouflage weakens object boundaries and produces wide appearance variation, especially in occluded or multi-object scenes. C2F-Net addresses these challenges by combining context-aware cross-level fusion, global context refinement, and coarse-guided low-level refinement.

  • Motivation: Low boundary contrast and variation in object size and shape make camouflaged objects difficult to identify and segment.These difficulties are illustrated by butterflies blending into leaves and occluded tortoises on the ground.
  • Motivation: Existing methods degrade on occluded, multiple-object, and indefinable-boundary scenes, motivating richer context and cross-level feature fusion.The paper identifies a significantly large receptive field and effective cross-level fusion as useful for these cases.
  • Proposed Method: C2F-Net uses ACFM to compute attention coefficients from multi-level features and integrate cross-level features under MSCA guidance.ACFM performs attention computation, feature refinement, and feature integration.
  • Proposed Method: DGCM exploits global context from fused features, while cascaded ACFMs and DGCMs generate a coarse segmentation map from high-level features.The coarse map subsequently guides refinement of low-level features before final prediction by CIM.
  • Proposed Method: CIM predicts the final result from low-level features refined with the coarse prediction map.The authors identify low-level feature refinement and CIM as an improvement over the conference version.
  • Results: Experiments on three benchmark datasets report that C2F-Net outperforms 19 state-of-the-art models across five COD evaluation metrics.Additional polyp segmentation experiments indicate promising downstream performance.

II. RELATED WORK

Prior COD work progressed from hand-crafted visual features to end-to-end deep learning, while related research enriched contextual representations. The paper also situates COD within downstream applications such as polyp segmentation.

  • A. Camouflaged Object Detection: Hand-crafted COD methods combine visual cues but tend to fail in real-world applications because they work mainly in relatively simple scenarios.The cited cues include texture, color, gradient, and motion.
  • A. Camouflaged Object Detection: Deep learning models were adopted for COD because they automatically learn features and train in an end-to-end manner.Related methods use image flipping, motion, texture refinement, edge priors, adversarial learning, or positioning and focus.
  • B. Context-aware Deep Learning: Contextual information is used to enrich feature representations through multi-scale, pyramid, global-context, and adaptive coarse-to-fine designs.The related methods include PSPNet, ASPP, PoolNet, global context flow, and local-global context integration.
  • C. Downstream Applications of COD: COD supports downstream applications including polyp segmentation, COVID-19 lung infection segmentation, surface defect detection, and transparent object detection.The paper presents polyp segmentation as especially attractive because of its medical application value and close relationship to COD.
  • Paper Scope: The paper presents the proposed C2F-Net framework, its components, loss function, experiments, ablations, and polyp-segmentation application.These topics organize the subsequent method and evaluation sections.

A. Overall Architecture

C2F-Net extracts multi-scale backbone features, fuses high-level features with attention and global context, and uses the resulting representation to refine features for COD.

  • Feature extraction: Res2Net-50 extracts multi-scale features that are divided into low-level Ql = {f1, f2, f3} and high-level Qh = {f3, f4, f5}.
  • Attention-induced cross-level fusion: ACFM uses MSCA-derived attention cues to integrate cross-level features and alleviate variation in camouflaged-object scale.
  • Attention-induced cross-level fusion: MSCA combines global and local contexts through two branches, emphasizing large objects while preventing small objects from being ignored.
  • Dual-branch global context: DGCM enhances ACFM outputs through convolutional and average-pooling branches, MSCA-based attention, feature addition, and a residual connection.
  • Dual-branch global context: DGCM adaptively extracts multi-scale information from a specific feature level to enhance the fused ACFM features.

D. Camouflage Inference Module

The Camouflage Inference Module exploits refined low-level features through multi-scale processing to generate the final detection prediction.

  • Feature refinement: CIM uses low-level features refined by the second DGCM output and a modified RFB to expand receptive fields and reduce computation.
  • Multi-scale residual processing: CIM introduces a Multi-scale Residual Block with two streams to detect local and multi-scale features.
  • Multi-scale residual processing: The streams use different convolutional kernel sizes, are concatenated, and pass through a 3 × 3 layer to form a fused multi-scale representation.
  • Prediction generation: Residual learning preserves input information, while three 1 × 1 convolutions and two MRBs generate the final prediction P.

E. Loss Function

The experiments evaluate C2F-Net with weighted pixel and overlap losses for joint optimization of initial and final predictions across benchmark COD settings.

  • Loss design: The loss function assigns different pixel weights by comparing each center pixel with its surroundings, emphasizing hard pixels.
  • Joint optimization: The total loss jointly optimizes the initial prediction map fD and final prediction map P against ground truth G.
  • Experimental setting: Experiments compare C2F-Net with 19 SOTA COD methods using five evaluation metrics on three benchmark datasets.

B. Datasets

C2F-Net is evaluated on three COD datasets using five widely used metrics, with qualitative comparisons against established COD methods.

  • Datasets: The evaluation uses CHAMELEON, CAMO, and COD10K as three benchmark COD datasets.CHAMELEON contains 76 testing images; CAMO contains 1,250 images; COD10K contains 5,066 pixel-level annotated images.
  • Evaluation metrics: The five metrics are MAE, S-measure, F-measure, weighted F-measure, and E-measure.
  • Evaluation metrics: MAE measures the average difference between normalized prediction and ground truth, while S-measure evaluates object-aware and region-aware structural similarity.
  • Evaluation metrics: F-measure evaluates binary prediction maps, and its weighted version considers weighted precision and weighted recall.
  • Qualitative comparison: Figure 3 compares C2F-Net qualitatively with seven SOTA COD methods across object categories and challenge factors including multiple objects and occlusion.
  • Evaluation metrics: Adaptive F-measure and E-measure are used in the evaluation, with lower MAE and higher remaining metrics indicating better performance.

D. Comparison with State-of-the-art COD Methods

C2F-Net is compared with 19 SOTA COD methods across three benchmark datasets and achieves stronger performance across the reported metrics.

  • C2F-Net is compared with 19 state-of-the-art COD methods using results reported for three benchmark datasets.
  • Performance on CAMO: On CAMO, C2F-Net outperforms 19 SOTA models across all metrics, exceeding LSR in Sα, Fβ, and Eφ by 1.6%, 4.9%, and 1.6%, respectively.
  • Performance on CHAMELEON: On CHAMELEON, C2F-Net performs best across all metrics, exceeding LSR in Eφ and Fβ by 1.2% and 2.7%, respectively.
  • Performance on COD10K: On COD10K, C2F-Net achieves new SOTA performance across all five metrics on the challenging 2,026-image test set.

2) Per-subclass Performance:

C2F-Net shows strong COD10K performance across object super-classes and qualitative examples, including cases involving multiple objects and occlusion.

  • Aquatic and Flying: C2F-Net outperforms other SOTA methods on Aquatic and Flying across all four reported metrics.
  • Terrestrial: On Terrestrial, C2F-Net achieves the best performance in Fβ, Eφ, and Sα.
  • Amphibian: On Amphibian, C2F-Net outperforms other SOTA models except for Eφ and MAE.
  • Qualitative Evaluation: Qualitative comparisons cover five COD10K super-classes plus multiple-object and occlusion cases, where C2F-Net improves detailed branches and boundary recognition.

4) Model Size and Inference Time:

The full C2F-Net improves ablated variants and the prior conference version, while using few parameters but incurring a reduced inference speed after adding CIM.

  • Model Size and Inference Time: CIM reduces inference speed from 39.07 FPS in the IJCAI version to 27.41 FPS in C2F-Net, motivating more efficient future designs.
  • Model Size and Inference Time: C2F-Net has the minimal parameter count among representative models, with memory cost much lower than ERRNet and PFNet.
  • Effectiveness of ACFM: Adding ACFM increases CAMO Fβ by 2.15% over the basic network and improves detection of the main camouflaged-object part.
  • Effectiveness of DGCM and CIM: Adding DGCM increases CHAMELEON Eφ by 1.81% over the basic network, while CIM improves performance across all three benchmark datasets.
  • Module Combinations: The ACFM-and-DGCM combination generally outperforms the first four ablation settings, and MSCA significantly improves results, especially on CAMO-Test.
  • Ablation Study: The full C2F-Net is better than all ablated configurations and is more conducive to identifying and locating camouflaged objects.
  • Comparison with the Previous Version: Compared with the IJCAI 2021 conference version, the full model performs better on all three datasets because CIM exploits low-level features guided by coarse prediction.

F. Application to Polyp Segmentation

The paper evaluates C2F-Net as a COD downstream model for polyp segmentation on four public datasets using five evaluation metrics.

  • C2F-Net is evaluated for polyp segmentation on ETIS, CVC-ClinicCB, CVC-ColonDB, and CVC-300.
  • The comparison includes UNet, UNet++, SFA, and PraNet, with results assessed using five metrics.

1) Qualitative Evaluation:

C2F-Net shows strong qualitative and quantitative performance across polyp segmentation datasets, including challenging small-object, lighting, and low-contrast conditions. Its design integrates cross-level features and global context to support these results.

  • CVC-300: C2F-Net outperforms four competing methods on CVC-300, with Eφ improving 3.30% over ParNet.
  • CVC-ClinicDB: On CVC-ClinicDB, the performance of Fβ increases by 4.01%.The dataset contains 612 open-access images from 31 colonoscopy clips.
  • CVC-ColonDB: On CVC-ColonDB, C2F-Net performs better than competing methods except on MAE, with Eφ improving 4.01% over ParNet.CVC-ColonDB contains 380 images from 15 short colonoscopy sequences.
  • ETIS: On ETIS, Eφ and Fβ improve by 7.15% and 7.24%, respectively, compared with ParNet.ETIS contains 196 polyp images for early colorectal-cancer diagnosis.
  • Visual comparison: Visual evaluations show that C2F-Net detects small-polyp boundaries and performs well under different lighting and low-contrast conditions.UNet and UNet++ fail on the small-polyp examples shown.
  • Model design: C2F-Net combines DGCM-based global-context extraction, ACFM-guided cross-level fusion, and CIM-based prediction from low-level features refined by coarse maps.These components form the paper’s proposed architecture for COD and its downstream evaluation.
Loading 2207.13362v1…