PaperlayerView exact revision on arXiv

Source-linked AI summary

Anabranch Network for Camouflaged Object Segmentation

Trung-Nghia Le, Tam V. Nguyen, Zhongliang Nie, Minh-Triet Tran, Akihiro Sugimoto

arXiv:2105.09451v1cs.CV

TL;DR

Camouflaged object segmentation is difficult and underexplored because objects blend into their backgrounds. The paper addresses this gap with the CAMO dataset and an Anabranch Network that combines classification and segmentation, reporting effective performance across fully convolutional networks. Its scope is bounded by a focus on regions, while broader potential ideas remain future work.

  • Problem

    Camouflaged object segmentation is difficult even for humans and lacks a standard public dataset for training and evaluation.

  • Method

    The Anabranch Network combines classification and segmentation streams, using image-level object-presence awareness to guide segmentation.

  • Results

    Experiments on the newly built dataset demonstrate the effectiveness and superiority of the proposed network across fully convolutional networks.

  • Takeaways & Limitations

    The CAMO dataset provides a benchmark intended to promote advances in camouflaged object segmentation.

  • Takeaways & Limitations

    The paper focused only on regions, leaving the broader potential idea as future work.

Abstract

from arXiv · show

Camouflaged objects attempt to conceal their texture into the background and discriminating them from the background is hard even for human beings. The main objective of this paper is to explore the camouflaged object segmentation problem, namely, segmenting the camouflaged object(s) for a given image. This problem has not been well studied in spite of a wide range of potential applications including the preservation of wild animals and the discovery of new species, surveillance systems, search-and-rescue missions in the event of natural disasters such as earthquakes, floods or hurricanes. This paper addresses a new challenging problem of camouflaged object segmentation. To address this problem, we provide a new image dataset of camouflaged objects for benchmarking purposes. In addition, we propose a general end-to-end network, called the Anabranch Network, that leverages both classification and segmentation tasks. Different from existing networks for segmentation, our proposed network possesses the second branch for classification to predict the probability of containing camouflaged object(s) in an image, which is then fused into the main branch for segmentation to boost up the segmentation accuracy. Extensive experiments conducted on the newly built dataset demonstrate the effectiveness of our network using various fully convolutional networks. \url{https://sites.google.com/view/ltnghia/research/camo}

1. Introduction

Camouflaged object segmentation is difficult because objects blend into their surroundings and remains underexplored, despite applications in surveillance, search-and-rescue, and wildlife preservation. The paper introduces the CAMO dataset and Anabranch Network, which combines classification awareness with segmentation.

  • Problem: Camouflaged objects conceal their texture in the background, making them difficult to identify even for humans.The challenge applies to naturally and artificially camouflaged objects, including animals, insects, soldiers, and weapons.
  • Motivation: Camouflaged object segmentation is technically difficult but supports surveillance, search-and-rescue, wildlife preservation, and species discovery.The paper specifically connects detection and segmentation to these practical scenarios.
  • Research gap: Camouflaged object segmentation has been well explored neither in the literature nor through a public training and testing dataset.This limits both the training of deep models and evaluation of proposed networks.
  • Contributions: The CAMO dataset contains 1250 images with manually annotated pixel-wise ground truths and challenging variations including clutter, occlusion, multiple objects, and distraction.The paper presents it as a dataset for promoting methods and benchmarking camouflage segmentation.
  • Contributions: The Anabranch Network combines classification and segmentation streams, using classification as awareness of whether camouflaged objects exist before segmentation.The design aims to segment objects only when they are present and otherwise produce no segmentation.
  • Evaluation: Experiments on the newly built dataset verify ANet's effectiveness across various fully convolutional networks.The paper reports extensive experiments supporting the proposed network and its general use with FCNs.

2. Related Work

Prior work covered camouflaged-object detection with handcrafted low-level features but rarely addressed segmentation, partly because suitable datasets were unavailable. Salient-object segmentation offers related techniques, yet camouflage remains harder because objects blend into cluttered backgrounds and may be absent from an image.

  • Camouflaged Object Segmentation: Earlier camouflage methods used handcrafted low-level features and worked mainly for simple, non-uniform backgrounds.Their performance was unsatisfactory when foreground and background were strongly similar.
  • Camouflaged Object Segmentation: The field lacked a standard dataset with enough training and testing data to develop and evaluate deep segmentation networks.The paper presents this shortage as a major reason for limited prior work.
  • Salient Object Segmentation: Salient-object methods evolved from color and patch comparisons to FCNs producing pixel-wise saliency maps with multi-scale features.The reviewed approaches include superpixel-CNN pipelines, skip-layer FCNs, and refinement or pooling mechanisms.
  • Comparison: Camouflaged segmentation is harder than salient segmentation because camouflage reduces distinctiveness, obscures boundaries, and becomes especially difficult in cluttered backgrounds.The same object can also be salient or camouflaged depending on its relationship with the background.
  • Comparison: Unlike salient-object methods, camouflage segmentation must handle images where no camouflaged object is guaranteed to exist.This motivates an approach that determines object presence rather than segmenting every image indiscriminately.
  • Two-Stream Networks: ANet differs from earlier two-stream networks by assigning different architectures to its classification and segmentation streams.The classification output is fused with the segmentation stream for the camouflage task.

3. Camouflaged Object Dataset

The CAMO dataset was introduced to provide pixel-wise training and testing data for camouflaged-object segmentation. It combines natural and artificial camouflage with diverse categories and difficult visual attributes, including multiple objects and clutter.

  • Motivation: CAMO was created because no adequate public dataset supported camouflaged-object segmentation and evaluation.The dataset includes pixel-wise ground-truth annotations and separate training and testing roles.
  • Dataset Composition: CAMO includes diverse animal categories and environmental settings, with category ratios reported in Fig. 3.The animal categories include amphibians, birds, insects, mammals, and reptiles.
  • Dataset Composition: The dataset contains both naturally camouflaged animals and artificially camouflaged humans.These categories cover environments such as forests, deserts, underwater scenes, battlefields, and body-painting contexts.
  • Dataset Challenges: About 8% of images contain multiple objects, including separate, connected, or overlapping instances.This composition increases the segmentation challenge.
  • Dataset Challenges: The dataset targets object appearance ambiguity, background clutter, shape complexity, small objects, occlusion, and distraction.Examples include similar object and background colors, thin boundaries or holes, object area below 0.1 of the image, and disconnected visible parts.

4. Anabranch Network

ANet combines an image-level classification stream with a pixel-level segmentation stream for camouflaged-object segmentation. The classification probability is fused with the segmentation map, and the framework can use different fully convolutional segmentation networks.

  • Motivation: ANet addresses the difficulty of unknown object presence by classifying whether an image contains camouflaged objects before segmentation.This avoids systematically segmenting every input image.
  • Network Architecture: The classification stream predicts image-level camouflage presence, while the segmentation stream produces pixel-wise semantic information.Their outputs are fused into a pixel-wise camouflage map.
  • Network Architecture: The classification output is a scalar probability multiplied with each pixel of the segmentation stream’s 2D map.This operation produces the final camouflage map.
  • Network Architecture: ANet is general because its segmentation stream can employ any end-to-end fully convolutional network.The segmentation architecture can therefore be switched while retaining the two-stream design.
  • Results: Experiments report robust segmentation accuracy both when camouflaged objects are present and when they are absent, together with computational efficiency.The classification stream is intended to compensate for limitations of salient-object segmentation features.
  • Network Architecture: The two streams use different architectures and separate task roles rather than duplicating the same network structure.The classification stream is built on convolutional feature-extraction layers, whereas the segmentation stream remains an FCN.

5. Experiments

The experiments establish evaluation procedures for camouflaged object segmentation and present ANet results as initial baselines. They also examine domain transfer, classification assistance, efficiency, and remaining dataset challenges.

  • Experimental scope: Experiments evaluate ANet and other baselines on camouflaged object segmentation datasets.The section introduces datasets, evaluation criteria, implementation details, and comparisons with FCN models.
  • Experimental scope: Domain transfer from salient object segmentation to camouflaged object segmentation is examined experimentally.
  • Experimental findings: Classifying camouflaged and non-camouflaged images can boost segmentation performance.
  • Experimental findings: ANet demonstrates short network training and fast running time, providing initial baselines for camouflaged object segmentation.
  • Open challenges: The CAMO dataset remains challenging and leaves room for further research.

5.1. Datasets and Experimentation Setup

The experiments use CAMO and an expanded CAMO-COCO5 dataset to represent both images containing camouflaged objects and realistic images where their presence is not guaranteed.

  • Datasets: CAMO images contain at least one camouflaged object, providing prior information that an object is present.
  • Datasets: CAMO-COCO5 combines CAMO camouflaged images with 1250 additional MS-COCO non-camouflaged images.
  • Dataset construction: CAMO-COCO includes zero-mask ground-truth labels for non-camouflaged images.All pixels in these labels have zero values.
  • Dataset splits: For each camouflaged and non-camouflaged image set, 80% was used for training and 20% for testing.The split corresponds to 1000 training images and 250 test images.
  • Dataset splits: CAMO-COCO is used as the experimental dataset described in Table 2.

5.2. Evaluation Criteria

The evaluation uses F-measure, IOU, and MAE, with binarization procedures differing between settings where camouflaged-object presence is guaranteed or uncertain.

  • Metrics: F-measure, IOU, and MAE are used to evaluate the obtained segmentation results.F-measure and IOU are higher-is-better metrics, while MAE is smaller-is-better.
  • Metrics: F-measure balances precision and recall, with β² = 0.3 emphasizing precision.
  • Metrics: MAE is computed as the average pixel-wise absolute difference between predicted camouflage and ground-truth maps.
  • Evaluation settings: MAE uses the raw grayscale camouflage map, whereas the other metrics use binarized results.
  • Evaluation settings: When object presence is assumed, an adaptive threshold θ = µ + η is used, where µ and η are the map mean and standard deviation.
  • Evaluation settings: When object presence is not guaranteed, evaluation uses a separate real-world-oriented binarization context.

5.3. Implementation Details

ANet combines a segmentation stream with a classification stream built on recent FCN models, trained sequentially and evaluated against comparable FCN baselines.

  • Network design: The segmentation stream employs recent salient object segmentation FCN models, including DHS, DSS, SRM, and WSS.
  • Training procedure: Each model trains two streams sequentially: segmentation is fine-tuned on CAMO, followed by classification training on CAMO-COCO.
  • Baselines: The baseline model keeps fixed parameters in the segmentation stream.
  • Optimization: Training uses SGD with momentum β = 0.9, weight decay 0.0005, and mini-batches of size 2.
  • Training procedure: The segmentation stream is trained for 10 epochs and the classification stream for 3 epochs.The corresponding learning rates are 10^-4 and 10^-6, respectively.
  • Baselines: Comparison FCN models are trained on CAMO or CAMO-COCO for 10 epochs with matched parameter settings.

5.4. Experimental Results

Experiments evaluate ANet with multiple fully convolutional segmentation streams on CAMO and CAMO-COCO. ANet generally improves over its original FCN and maintains good accuracy across both datasets, while classification-stream performance reaches around 90%.

  • ANet versus FCN: ANet yields better results than its employed original FCN across different FCN choices, including non-camouflaged images.The authors attribute this improvement to fusion with the classification stream.
  • Transferability: The results indicate potential transfer from salient-object segmentation to camouflaged-object segmentation.Fine-tuned saliency models show moderate performance, and the authors use salient-object FCNs as ANet segmentation streams.
  • FCN baselines: Fine-tuned FCNs achieve around 60+% accuracy in Fβ and IOU, with around 0.1 MAE.Models fine-tuned on the dataset used for testing perform better than pre-trained models or models fine-tuned on the other dataset.
  • Cross-dataset performance: ANet maintains comparable accuracy on CAMO and CAMO-COCO, whereas each FCN performs well primarily on one dataset.The classification stream is especially important for CAMO-COCO images that may not contain camouflaged objects.
  • Statistical evaluation: ANet significantly outperforms or matches the best methods on all CAMO-COCO metrics at the reported 90% significance level.On CAMO, ANet is significantly better than FCNs fine-tuned on CAMO-COCO but worse than FCNs fine-tuned on CAMO.
  • Classification and efficiency: The classification stream achieves around 90% accuracy and outperforms SVM-BoW, while ANet baselines significantly outperform AlexNet and VGG-16.Adding the classification stream does not incur much additional processing time.

5.5. Discussion

The discussion identifies domain-transfer weaknesses and examines joint training. Segmentation accuracy remains constrained by the transferred FCN, while joint training helps some streams but hurts others.

  • Failure cases: Camouflaged-object segmentation accuracy is far below salient-object segmentation accuracy, which can reach around 90% in Fβ.The authors attribute this mainly to insufficient segmentation ability when transferring from salient-object to camouflaged-object segmentation.
  • Failure cases: Failure cases involve challenging CAMO scenarios such as object appearance and background clutter.Figure 9 presents input, ground-truth, and FCN-result triplets.
  • Stream fusion: The classification stream is expected to boost segmentation, but simple multiplication remains insufficient for further effective fusion.The discussion identifies fusion design as an unresolved issue.
  • Joint training: Joint training does not improve every method: DHS and WSS lose accuracy, whereas DSS and SRM improve slightly.The authors relate this variation to the distinct tasks and training-data requirements of the two streams.
  • Joint training: Joint training can interfere with the two streams because the available training data cannot simultaneously suit both tasks.The authors state that the two-step training approach is sufficient for ANet on CAMO-COCO and leave further investigation for future work.

6. Conclusion

The paper introduces a new benchmark dataset and the Anabranch Network for camouflaged-object segmentation. The network combines classification and segmentation streams, and experiments report effective, computationally efficient performance.

  • Conclusion: The paper addresses camouflaged-object segmentation by providing a new image dataset with pixel-wise ground-truth annotations.The authors intend the dataset to promote advances in this area.
  • Conclusion: The Anabranch Network is a simple, flexible, end-to-end model for camouflaged-object segmentation.It uses classification and segmentation streams that are combined to support segmentation.
  • Conclusion: Applying the framework to different FCNs and extensive experiments on the new dataset demonstrate the proposed network’s effectiveness.The conclusion describes the combined streams as showing baseline performance and the framework as superior in the reported experiments.
  • Conclusion: The method is computationally efficient, according to the reported conclusion.The paper also suggests that classification can potentially be combined with segmentation for suitable instance-segmentation applications.
  • Limitations and future work: The study focuses on regions rather than videos or instance segmentation, leaving those extensions for future work.The authors specifically mention video segmentation and classifying individual instances as future directions.
Loading 2105.09451v1…