PaperlayerView exact revision on arXiv

Source-linked AI summary

Concealed Object Detection

Deng-Ping Fan, Ge-Peng Ji, Ming-Ming Cheng, Ling Shao

arXiv:2102.10274v2cs.CV

TL;DR

Concealed object detection seeks to identify objects that closely resemble their backgrounds, a setting more difficult than traditional object detection and segmentation. The paper builds the COD10K dataset, proposes the SINet search-and-identification framework, and benchmarks existing methods. SINet achieves state-of-the-art performance across the tested COD datasets, while the study identifies annotation-driven research directions and practical scope boundaries.

  • Problem

    Concealed object detection lacks a sufficiently large dataset and standard benchmark, despite the high similarity between targets and backgrounds.

  • Method

    The paper constructs COD10K and develops SINet, an end-to-end search-and-identification framework for concealed object detection.

  • Results

    SINet achieves the new state-of-the-art across existing COD datasets and outperforms 12 cutting-edge baselines on the tested datasets.

  • Takeaways & Limitations

    COD10K and the benchmark provide a basis for developing models and pursuing directions such as concealed instance segmentation, edge detection, proposal, and ranking.

  • Takeaways & Limitations

    Current deep-learning COD algorithms remain limited for real-world challenges, and existing methods rely on costly pixel-level annotations.

Abstract

from arXiv · show

We present the first systematic study on concealed object detection (COD), which aims to identify objects that are "perfectly" embedded in their background. The high intrinsic similarities between the concealed objects and their background make COD far more challenging than traditional object detection/segmentation. To better understand this task, we collect a large-scale dataset, called COD10K, which consists of 10,000 images covering concealed objects in diverse real-world scenarios from 78 object categories. Further, we provide rich annotations including object categories, object boundaries, challenging attributes, object-level labels, and instance-level annotations. Our COD10K is the largest COD dataset to date, with the richest annotations, which enables comprehensive concealed object understanding and can even be used to help progress several other vision tasks, such as detection, segmentation, classification, etc. Motivated by how animals hunt in the wild, we also design a simple but strong baseline for COD, termed the Search Identification Network (SINet). Without any bells and whistles, SINet outperforms 12 cutting-edge baselines on all datasets tested, making them robust, general architectures that could serve as catalysts for future research in COD. Finally, we provide some interesting findings and highlight several potential applications and future directions. To spark research in this new field, our code, dataset, and online demo are available on our project page: http://mmcheng.net/cod.

1 INTRODUCTION

Concealed object detection targets objects whose appearance blends closely with their surroundings, making it substantially harder than traditional detection and segmentation. The paper introduces COD10K, the SINet framework, and a broad benchmark to establish this research area.

  • High intrinsic similarity between targets and non-targets makes COD more challenging than traditional object segmentation and detection.
  • COD identifies objects that share patterns such as texture, color, or direction with their natural or man-made environment.
  • COD10K Dataset: COD10K contains 10,000 images spanning 78 object categories with hierarchical labels, challenging attributes, and matting-level annotations.The annotations include category, bounding-box, object-level, and instance-level labels.
  • COD Framework: SINet is a simple, efficient search-and-identification framework that achieves state-of-the-art performance across existing COD datasets.Its reported overall training time is 4 hours.
  • COD Benchmark: The study evaluates 12 state-of-the-art baselines across COD10K and previous datasets to provide a broad COD benchmark.
  • Paper Extension: The paper extends its conference version with deeper COD10K analysis, NCD and GRA modules, extensive experiments, ablations, and additional benchmarking.It also discusses future directions including concealed object ranking, proposal, and instance segmentation.

2 RELATED WORK

Related work distinguishes generic, salient, and concealed object detection and situates COD among datasets and camouflage types. Existing COD datasets are smaller or less densely annotated, motivating COD10K's broader coverage and annotation design.

  • Object detection is broadly divided into generic, salient, and concealed object detection.
  • Generic Object Segmentation: Generic object segmentation includes semantic and panoptic segmentation, and concealed objects can be viewed as difficult generic-object cases.
  • Salient Object Detection: Salient object detection segments attention-grabbing objects, while salient images can also provide negative samples for COD.
  • COD Datasets: CHAMELEON contains 76 images, while CAMO contains 2.5K images across eight categories; COD10K is designed to be more challenging and densely annotated.
  • Types of Camouflage: Concealed images include natural camouflage used by animals and artificial camouflage used in art, gaming, manufacturing defects, and other settings.
  • COD Formulation: COD is class-agnostic pixel-level labeling, with 0 for background pixels and 1 for pixels assigned to concealed objects.The paper focuses on object-level COD and leaves concealed instance detection for future work.

3 COD10K DATASET

COD10K is designed as a large, diverse, hierarchically annotated benchmark for concealed-object research. Its statistics emphasize challenging object appearance, varied categories, reduced center bias, and high-quality annotations.

  • Image collection: COD10K combines concealed, salient, and non-concealed images collected from multiple photography and public-domain websites to enrich training and negative samples.Most concealed images come from Flickr, while additional images come from Visual Hunt, Pixabay, Unsplash, Free-images, and other sources.
  • Dataset composition: COD10K contains 10,000 images spanning 10 super-classes and 78 sub-classes, including 69 concealed and 9 non-concealed categories.The dataset includes 5,066 concealed, 3,000 background, and 1,934 non-concealed images.
  • Annotation system: The dataset uses hierarchical annotations covering categories, bounding boxes, challenging attributes, objects, and instances.Instance-level annotation yields 5,069 objects and 5,930 instances, while attributes include occlusions and indefinable boundaries.
  • Dataset features: Object sizes range from 0.01% to 80.74% of image area, with an average of 8.94%, indicating a broader size range than CAMO-COCO and CHAMELEON.The reported distribution is normalized by image area.
  • Dataset statistics and quality: Compared with existing datasets, COD10K contains more difficult objects and suffers from less center bias, while its annotations undergo 10-fold cross-validation.Three viewers participate in quality control, and matting-level annotation costs approximately 60 minutes per image.
  • Dataset features: COD10K contains diverse concealed-object types and many Full HD images, broadening the visual conditions represented in the benchmark.Examples include concealed body paintings and objects concealed in daily life; the dataset also includes a large number of 1080p images.

4 COD FRAMEWORK

SINet decouples concealed object detection into a search stage and an identification stage, following a predator-inspired coarse-to-fine process. Its TEM, NCD, and GRA modules capture texture, locate candidates, and progressively refine predictions.

  • Network Overview: SINet separates concealed object detection into search and identification stages, with the latter precisely detecting objects in a cascaded manner.The search phase finds concealed objects, while identification refines the result.
  • Search Phase: TEM captures fine-grained textures by enlarging contextual cues through parallel branches with different dilation rates and asymmetric convolutions.Its branches use dilation rates d ∈ {1,3,5,7}; asymmetric convolutions improve inference efficiency without decreasing representation capabilities.
  • Search Phase: NCD aggregates only the top-three highest-level features to improve efficiency while addressing semantic consistency within layers and context bridging across layers.The selected features are f_k for k ∈ {3,4,5}; NCD produces refined features for locating concealed-object candidates.
  • Identification Phase: Reverse guidance mines discriminative concealed regions by erasing estimated object regions, addressing the rough structural and textural detail captured by the global location map.The global map C6 is derived from the three highest layers and provides only a relatively rough object location.
  • Identification Phase: GRA combines reverse guidance with group guidance in cascaded residual blocks to progressively refine concealed-object predictions.The group operation separates guidance priors from candidate features before subsequent processing, avoiding direct multiplication that can accumulate inaccurate guidance.

5 COD BENCHMARK

The benchmark evaluates COD metrics, datasets, baselines, SINet, subclass behavior, qualitative outputs, and component ablations. SINet achieves strong performance across datasets while its design choices affect generalization and accuracy.

  • Evaluation: S-measure, E-measure, and MAE provide complementary evaluation of concealed-object predictions, including pixel-level, image-level, and localized accuracy.MAE measures pixel-level error, whereas E-measure evaluates pixel-level matching with image-level statistics; the paper also considers Fβ^w as an alternative to traditional Fβ.
  • Benchmark setup: 12 deep learning baselines were selected using classical architecture, recent publication, and field-leading performance criteria, then evaluated on CHAMELEON, CAMO, and COD10K.The evaluation uses the whole CHAMELEON dataset and the test sets of CAMO and COD10K under shared baseline training settings.
  • Overall results: SINet achieves new state-of-the-art performance across all metrics on CHAMELEON and remains best on the more challenging CAMO and COD10K datasets.On COD10K, the test set contains 2,026 images; the authors attribute SINet’s performance to search and identification modules that learn diversified coarse-to-fine features.
  • Qualitative and subclass analysis: SINet handles lighting changes, appearance variation, and indefinable boundaries with finer details than the compared qualitative baselines.PFANet often localizes inaccurately, while PraNet can miss fine object details; SINet is reported to infer concealed objects more completely in these cases.
  • Efficiency: SINet reduces training time to 4 hours versus 48 hours for EGNet while achieving state-of-the-art performance across the evaluated datasets.The authors note that fully comparing against existing state-of-the-art salient-object-detection models is beyond the paper’s scope.
  • Generalization: COD10K shows stronger cross-dataset generalization than CAMO, with a reported last-column performance drop of -6.0%, while also containing more challenging scenes.The cross-dataset settings differ from Table 2, so these performances are not directly comparable with the main benchmark results.
  • Ablation studies: Ablations show that NCD, progressive group sizing, and limited reverse guidance improve performance, whereas repeated reverse guidance can confuse learning.NCD raises CAMO mean Eφ from 0.869 to 0.882; replacing the partial decoder increases CHAMELEON Fβ^w by 1.7%, and progressive group sizing improves generalizability.

6 DOWNSTREAM APPLICATIONS

COD systems are proposed as core components for downstream tasks where targets resemble their backgrounds. The paper illustrates medical applications including polyp and lung-infection segmentation.

  • Applications: COD models are suitable for downstream applications in medicine, art, and agriculture because their targets share visual appearance with surrounding backgrounds.The paper presents these applications as potential uses and toy examples intended to stimulate future ideas.
  • Medical applications: Polyp segmentation shares COD challenges such as appearance variation and blurred boundaries, making SINet a potential component for more robust results.PraNet performs strongly in both polyp and concealed-object segmentation, while the paper proposes retraining SINet for this application.
  • Medical applications: COVID-19 lung-infection segmentation is presented as another potential application, where an AI system could support early screening after retraining SINet on relevant datasets.The paper points to existing segmentation and survey work for further application details.

6.2 Application II: Manufacturing

Manufacturing defects can resemble product backgrounds through low contrast and ambiguous boundaries. COD-style systems are presented as a possible basis for automated recognition across several materials.

  • Manufacturing: Surface defects in wood, textile, and magnetic tile products are difficult to identify because of low contrast and ambiguous boundaries.Traditional systems mainly rely on human inspection, which the paper characterizes as subjective and time-consuming.
  • Related visual inspection: For pest-style visual monitoring, the system can generate bounding boxes for local images or provide whole-environment pest counts.The figure specifically illustrates locust-plague density monitoring as an example.
  • Related visual inspection: AI-based fruit maturity monitoring is contrasted with manual inspection of Persea Americana and Myrica Rubra as a way to improve production efficiency.These fruits are used as examples of objects whose maturity can be difficult to assess visually.

6.3 Application III: Agriculture

Agricultural monitoring is presented as a potential COD application for detecting visually concealed or difficult-to-monitor targets. The examples concern locust outbreaks and fruit maturity.

  • Agriculture: AI-based monitoring could support regulation and containment of desert locust plagues that threaten crops, food supplies, and finances.The paper notes that collecting relevant insect data for COD models requires rich biological knowledge.
  • Agriculture: Persea Americana and Myrica Rubra can resemble green leaves early in ripening, making COD-style identification potentially useful for monitoring efficiency.The paper presents these fruits as sharing characteristics with concealed objects.

6.4 Application IV: Art

The paper presents concealed-object detection as a basis for applications that transform, detect, and retrieve visually hidden content. These applications span art, transparent-object awareness, and image search.

  • 6.4 Application IV: Art: Background warping can convert concealed objects into salient objects, potentially supplying training data for data-hungry deep learning models.The paper also connects this application to studying visual feature-search mechanisms.
  • 6.4 Application IV: Art: Transparent-object detection could help robots identify windows, glass doors, and glass walls to avoid unseen obstacles.Transparent objects inherit the appearance of their backgrounds and are treated as a concealed-object sub-task.
  • 6.4 Application IV: Art: Adding concealed detection to an image search engine enables it to identify a concealed butterfly and return butterfly images rather than only similar backgrounds.The example contrasts search without concealed detection against search with a concealed detection system.
  • 6.4 Application IV: Art: Concealed and salient object detection are opposite tasks that can support multitask learning and user-controlled adjustment of saliency.A scroll bar could let users customize the degree of saliency produced from concealed objects.

6.5 Application V: Daily Life

The paper identifies transparent-object detection as a daily-life application of concealed-object detection and illustrates how concealed detection can improve image-search results.

  • 6.5 Application V: Daily Life: Transparent objects such as glass doors and walls inherit their backgrounds’ appearance, making them difficult to notice in daily life.Transparent-object detection and tracking are presented as promising sub-tasks of concealed object detection.
  • 6.5 Application V: Daily Life: A search engine without concealed detection misses a concealed butterfly and returns images with similar backgrounds.The example is shown as the search-engine baseline condition.
  • 6.5 Application V: Daily Life: With concealed detection, the search engine identifies the concealed butterfly and returns several butterfly images.The paper describes this result as being obtained by simply changing the keyword in the illustrated example.

7 POTENTIAL RESEARCH DIRECTIONS

The paper outlines research directions spanning weaker supervision, additional modalities, richer task formulations, and automated model design. These directions respond to annotation costs, limited current capabilities, and the rich labels in COD10K.

  • 7 POTENTIAL RESEARCH DIRECTIONS: The paper states that leading deep-learning algorithms remain limited relative to generic object detection and do not yet effectively solve real-world COD10K challenges.It highlights long-standing challenges including few-shot, weakly supervised, unsupervised, self-supervised, limited-data, and unseen-class settings, as well as multimodal detection.
  • 7 POTENTIAL RESEARCH DIRECTIONS: Existing deep methods rely on fully supervised pixel-level annotations, motivating weakly or semi-supervised detection to reduce intensive annotation costs.The paper notes that pixel-level masks are manually marked with professional tools.
  • 7 POTENTIAL RESEARCH DIRECTIONS: Self-supervised learning is proposed as a benchmark direction for learning concealed-object representations.The motivation is recent progress in self-supervised representation learning for image, audio, and video.
  • 7 POTENTIAL RESEARCH DIRECTIONS: Concealed-object detection in audio, thermal, group-image, and depth modalities could address challenges in pest monitoring, robotics, and artist design.The paper states that existing concealed data is based only on static images or dynamic videos.
  • 7 POTENTIAL RESEARCH DIRECTIONS: COD10K’s class and subclass labels could support a large-scale, fine-grained concealed-object classification task.The paper frames concealed-object classification as a likely future area of attention.
  • 7 POTENTIAL RESEARCH DIRECTIONS: Future work includes concealed-object proposal and tracking, instance segmentation, ranking, and a universal network for localization, segmentation, and ranking.These directions extend beyond the paper’s segmentation-centered formulation and use the relationships among visual tasks.
  • 7 POTENTIAL RESEARCH DIRECTIONS: Neural architecture search could replace manually designed features and architectures that may not be optimal.The paper presents automated machine learning as a potential route for this search.
  • 7 POTENTIAL RESEARCH DIRECTIONS: Transferring salient objects to concealed objects and adversarial interaction between SOD and COD are proposed to increase training data and feature-extraction ability.These ideas are listed among problems needing further study.

8 CONCLUSION

The paper concludes with a comprehensive concealed-vision study built around COD10K, a benchmark, and SINet. It reports competitive results and identifies dataset expansion and model improvement as future plans.

  • 8 CONCLUSION: The study contributes the densely annotated COD10K dataset, a large-scale benchmark, the end-to-end SINet framework, and several potential applications.The conclusion presents these as the paper’s specific contributions.
  • 8 CONCLUSION: SINet is competitive with cutting-edge baselines and produces more visually favorable results.The conclusion does not provide a specific numerical comparison in this passage.
  • 8 CONCLUSION: The authors plan to extend COD10K with multi-view images, textual descriptions, and video, while improving performance through receptive-field search and feature representations.These are stated as future dataset and model-development plans.
Loading 2102.10274v2…