Source-linked AI summary
Mutual Graph Learning for Camouflaged Object Detection
Qiang Zhai, Xin Li, Fan Yang, Chenglizhao Chen, Hong Cheng, Deng-Ping Fan
TL;DR
Camouflaged object detection remains difficult because foreground and background features are intrinsically similar. MGL jointly learns COD and edge extraction by recurrently reasoning over high-order graph relations with typed interaction functions, and experiments report superior performance to existing state-of-the-art methods.
Problem
Camouflaged object detection is challenging because intrinsic similarities between foreground objects and their surroundings make deep features indistinguishable.
Method
MGL jointly learns COD and camouflaged object-aware edge extraction through recurrent high-order graph reasoning with typed functions for complementary relations.
Results
MGL achieves superior performance to existing state-of-the-art methods across challenging camouflaged object detection benchmarks.
Takeaways & Limitations
Explicitly mining true-edge information and mutual guidance helps address camouflage difficulties including occlusions and indefinable boundaries.
Abstract
from arXiv · showhide
Automatically detecting/segmenting object(s) that blend in with their surroundings is difficult for current models. A major challenge is that the intrinsic similarities between such foreground objects and background surroundings make the features extracted by deep model indistinguishable. To overcome this challenge, an ideal model should be able to seek valuable, extra clues from the given scene and incorporate them into a joint learning framework for representation co-enhancement. With this inspiration, we design a novel Mutual Graph Learning (MGL) model, which generalizes the idea of conventional mutual learning from regular grids to the graph domain. Specifically, MGL decouples an image into two task-specific feature maps -- one for roughly locating the target and the other for accurately capturing its boundary details -- and fully exploits the mutual benefits by recurrently reasoning their high-order relations through graphs. Importantly, in contrast to most mutual learning approaches that use a shared function to model all between-task interactions, MGL is equipped with typed functions for handling different complementary relations to maximize information interactions. Experiments on challenging datasets, including CHAMELEON, CAMO and COD10K, demonstrate the effectiveness of our MGL with superior performance to existing state-of-the-art methods.
1. Introduction
Camouflaged object detection is difficult because foreground and background features are intrinsically similar, while existing approaches incompletely exploit auxiliary information and cross-task dependencies. MGL addresses these gaps by jointly and mutually learning COD and edge information through typed graph reasoning, with experiments reporting superiority over existing methods.
- Motivation: Camouflaged objects blend with their surroundings, making them difficult for both humans and current deep models to identify.COD supports applications including image retrieval, species discovery, traffic risk management, and medical image analysis.
- Open issues: Existing approaches often use auxiliary tasks only to assist COD, overlook mutual influence, and model dependencies in coordinate space rather than through higher-order relations.These limitations are especially problematic under heavy occlusions and indefinable boundaries.
- Proposed approach: MGL jointly learns COD and camouflaged object-aware edge extraction to exploit complementary information for representation co-enhancement.The model treats edge extraction as an auxiliary task because edge disruption is identified as a key camouflage factor.
- Proposed approach: MGL uses typed graph-based interaction modules to transfer semantic guidance and true-edge information between COD and COEE.RIGR reasons about high-level dependencies for COEE, while ECGR improves true edge visibility for COD.
- Evidence: The authors report extensive benchmark experiments demonstrating MGL’s superiority in mining mutual guidance information for camouflaged object detection.Evaluations include strong baselines and current state-of-the-art methods across multiple benchmarks.
2. Related Work
Related work covers camouflaged object detection, mutual graph reasoning, and the proposed graph-based interaction framework. Existing COD methods use search, segmentation, or auxiliary classification information, while graph methods model long-range or cross-modal relations in related vision tasks.
- Camouflaged Object Detection: COD extends generic and salient object detection to objects concealed by blending into their surroundings.SINet searches for camouflaged objects before segmentation, while ANet incorporates classification information into representation learning.
- Mutual Graph Learning: Figure 2 summarizes S-MGL, whose cross-graph interaction and edge supportive graph convolution operations enable information exchange.The flowchart marks the main components from (a) to (e).
- Graph-based Vision Methods: Related vision methods use graph convolution or bidirectional graphs to model long-range context, semantic relations, and object-background cooccurrence.Other approaches apply cascade graphs to multi-scale cross-modality information or adaptive GCNs to co-saliency detection.
3. Our Approach
MGL jointly learns camouflage detection and edge representations by repeatedly reasoning over task-specific graphs. Its RIGR and ECGR modules exchange complementary information to improve object localization and boundary representation.
- Overview: MGL formulates COD and COEE as mutually supportive tasks, learning camouflage maps and edge maps from a shared image.The model decouples the input into task-specific representations for roughly locating targets and capturing true edges.
- Mutual Learning: RIGR can be stacked with ECGR for recurrent mutual learning, allowing graph representations to be updated through repeated task interaction.CGI transfers COD information to COEE, while ECGR uses COEE information to guide COD representation learning.
- Multi-Task Feature Extraction: MTFE produces separate COD and COEE feature maps while preserving spatial and high-level semantic information.The feature maps have h×w spatial resolution and c channels.
- Region-Induced Graph Reasoning: RIGR projects task features into semantic graphs, exchanges information across graphs through CGI, performs intra-graph convolution, and reprojects enhanced representations.Graph projection uses soft assignments to learn node embeddings and adjacency matrices from feature affinities.
- Edge-Constricted Graph Reasoning: ECGR predicts an edge map, fuses edge and COD features, and applies edge-supportive graph convolution to improve object localization.Its supportive graph links COD features with edge-based node embeddings generated from the predicted edge map.
4. Experiments
Experiments evaluate MGL on established COD benchmarks using standard segmentation metrics and comparisons with strong baselines. Results show consistent gains from mutual graph reasoning, recurrent refinement, edge-aware auxiliary learning, and typed high-order interactions.
- Datasets and evaluation: The experiments use CHAMELEON, CAMO, and COD10K, with CAMO and COD10K training sets combined for training.COD10K contains 10,000 images, while CHAMELEON contains 76 and CAMO contains 2,500.
- Datasets and evaluation: Evaluation uses MAE, E-measure, S-measure, weighted F-measure, and F-measure for the auxiliary COEE task.The models are compared against 16 recent methods using the same training set and recommended settings.
- Comparison with state-of-the-art methods: S-MGL outperforms compared methods across all CHAMELEON metrics, lowering SINet’s MAE by 27.3% and improving its weighted F-measure by 8.5%.R-MGL further improves performance and establishes a new record.
- Comparison with state-of-the-art methods: On COD10K, S-MGL reaches Sα 81.1%, Eφ 85.1%, F wβ 65.5%, and MAE 0.037, while R-MGL further boosts performance.The authors report new records across all metrics on the largest COD test set.
- Auxiliary task: S-MGL and R-MGL outperform HED, DSS, and MUL on COEE, improving both the main COD task and the auxiliary edge-extraction task.The results support mutual benefits between the two tasks when trained with the same extracted edge labels.
- Ablation study: RIGR improves a ResNet50-FCN baseline across datasets, ECGR adds further accuracy, and recurrent learning further refines performance.Two recurrent steps provide promising performance according to the parameter study.
- Ablation study: Graph-based MGL outperforms non-local operations and conventional mutual learning by capturing high-order relations between COD and COEE.The comparisons indicate that explicitly mining auxiliary edge information improves result reliability in ambiguous scenes.
5. Conclusion
The paper presents MGL as a graph-based joint framework for camouflaged object detection and true-edge extraction. Its RIGR and ECGR modules, organized recurrently, improve edge visibility and exploit complementary information to address camouflage challenges.
- Conclusion: MGL jointly learns camouflaged object detection and camouflaged object-aware edge extraction through graph-based mutual learning.The framework is designed to mine complementary information between the two tasks.
- Conclusion: RIGR and ECGR work together to mine complementary information and improve the visibility of true object edges.These modules provide the principal graph-reasoning components of MGL.
- Conclusion: MGL can be formulated as recurrent graph reasoning to further exploit useful information and address occlusions and indefinable boundaries.The authors suggest that the framework may also benefit tasks such as panoptic segmentation requiring multi-source representation enhancement.