PaperlayerView exact revision on arXiv

Source-linked AI summary

CAD-Net: A Context-Aware Detection Network for Objects in Remote Sensing Imagery

Gongjie Zhang, Shijian Lu, Wei Zhang

arXiv:1903.00857v1cs.CV

TL;DR

Object detectors developed for ground-level imagery can lose performance on remote-sensing images because of sparse texture, low contrast, arbitrary orientations, and scale variation. CAD-Net addresses this with global and local context modules plus spatial-and-scale-aware attention, achieving superior results on two public datasets.

  • Problem

    Ground-level object detectors experience sharp performance drops on remote-sensing images because objects have sparse texture, low contrast, arbitrary orientations, and large scale variations.

  • Method

    CAD-Net combines global scene context, local neighboring-object context, and spatial-and-scale-aware attention within an end-to-end detection network.

  • Results

    CAD-Net achieves superior detection performance on two public remote-sensing datasets, including a 15.8% mAP improvement over the Faster RCNN baseline.

  • Takeaways & Limitations

    Global and local context together with attention provide a supported approach for improving object detection in degraded remote-sensing imagery.

Abstract

from arXiv · show

Accurate and robust detection of multi-class objects in optical remote sensing images is essential to many real-world applications such as urban planning, traffic control, searching and rescuing, etc. However, state-of-the-art object detection techniques designed for images captured using ground-level sensors usually experience a sharp performance drop when directly applied to remote sensing images, largely due to the object appearance differences in remote sensing images in term of sparse texture, low contrast, arbitrary orientations, large scale variations, etc. This paper presents a novel object detection network (CAD-Net) that exploits attention-modulated features as well as global and local contexts to address the new challenges in detecting objects from remote sensing images. The proposed CAD-Net learns global and local contexts of objects by capturing their correlations with the global scene (at scene-level) and the local neighboring objects or features (at object-level), respectively. In addition, it designs a spatial-and-scale-aware attention module that guides the network to focus on more informative regions and features as well as more appropriate feature scales. Experiments over two publicly available object detection datasets for remote sensing images demonstrate that the proposed CAD-Net achieves superior detection performance. The implementation codes will be made publicly available for facilitating future researches.

I. INTRODUCTION

Remote-sensing object detection is important but difficult because objects often lose contrast and texture, appear densely, arbitrarily oriented, and vary greatly in scale. CAD-Net addresses these challenges by incorporating global and local context with spatial-and-scale-aware attention.

  • Remote-sensing imagery supports multi-class detection for applications including urban planning, searching, rescuing, and environmental monitoring.
  • Objects in optical remote-sensing images often lack contrast and texture, while density, arbitrary orientations, scale variation, and noise further challenge detection.
  • CAD-Net is designed to learn correlations between objects and global scenes, while also using local contextual information for remote-sensing detection.
  • Its spatial-and-scale-aware attention guides processing toward informative regions and appropriate feature scales.
  • The paper reports state-of-the-art performance on two challenging remote-sensing object-detection datasets using an end-to-end trainable network.

II. RELATED WORK

Generic object detectors have advanced substantially, but remote-sensing detection requires adaptations for degraded appearance, arbitrary orientation, and other domain-specific characteristics. CAD-Net extends Faster RCNN to detect multiple object types in remote-sensing imagery.

  • A. Generic Object Detection: Deep neural networks have substantially improved object detection, with two-stage and single-stage detector families emerging for standard benchmarks.
  • A. Generic Object Detection: Faster RCNN and FPN provide proposal generation and multi-scale feature learning within the two-stage detection framework.
  • B. Object Detection in Optical Remote Sensing Images: CAD-Net builds on Faster RCNN and introduces components targeting remote-sensing characteristics such as weak contrast, sparse texture, and arbitrary orientations.
  • B. Object Detection in Optical Remote Sensing Images: Earlier remote-sensing methods commonly used handcrafted features, targeted single object types, or struggled with cluttered backgrounds.
  • B. Object Detection in Optical Remote Sensing Images: CAD-Net performs multi-class detection of different object types within the same image and is evaluated on large-scale accessible datasets.

C. Contexts and Attention Mechanism

Context and attention mechanisms are used to compensate for weak object-level visual evidence in remote-sensing imagery. CAD-Net combines global and local context modules with attention that selects informative regions and scales.

  • Contextual information can improve object detection, particularly for small objects, while visual attention directs processing toward informative regions.
  • CAD-Net combines GCNet for scene-level context, PLCNet for object-level context, and spatial-and-scale-aware attention for robust remote-sensing detection.
  • The framework extends Faster RCNN with FPN and adds an oriented bounding-box branch to better align detections with arbitrary object orientations.
  • Region-based detection commonly uses RoIPooling to focus on a proposal and ignore the rest of the image.
  • Because remote-sensing edges and texture details can be degraded, global and local contexts are incorporated to compensate for lost discriminative features.

1) Global Context Network:

The Global Context Network learns scene-level semantics and their correlations with objects, using global context to supplement object features for remote-sensing detection.

  • PLCNet extracts proposal-region features at each feature scale and learns their correlations as supplementary detection information.
  • Scene semantics correlate with objects such as ships and seas or airplanes and airports, providing useful global detection priors.
  • The GCNet learns correlations between the whole scene and objects to compensate for lost discriminative object features.
  • GCNet global features are extracted from the ResNet-101 C5 feature maps and pooled into a vector.

2) Pyramid Local Context Network:

The Pyramid Local Context Network learns cross-scale correlations between proposals and neighboring objects or features, then combines local, region, and global context for detection.

  • PLCNet exploits local contexts because neighboring objects and features can compensate for information lost in remote-sensing imagery.
  • For each proposal, a Context Pyramid extracts and concatenates pooled features from local contexts at different scales before convolutional fusion.
  • The fused local-context features are concatenated with region and global-context features for proposal classification and bounding-box regression.
  • Cross-scale context can clarify ambiguous proposals, such as identifying a ship from surrounding ship clusters and harbors.

B. Spatial-and-Scale-Aware Attention Module

The spatial-and-scale-aware attention module adaptively emphasizes informative regions at suitable feature scales, addressing sparse texture, low contrast, noise, and scale variation.

  • Spatial-aware attention targets prominent regions, while scale-aware attention selects relevant feature-map scales.
  • The module operates on FPN feature maps P2−P5 and generates separate scale-specific attention maps through convolutional layers.
  • The module suppresses irrelevant and noisy areas while focusing on informative regions at appropriate feature scales.
  • Its attention responses focus on small ships at lower layers and large harbors at deeper layers.
  • Attention also highlights useful texture details degraded by noise, including harbor skeletons and a court middle line.

IV. EXPERIMENTS

CAD-Net is evaluated on two public remote-sensing object-detection datasets using mAP, with DOTA providing diverse quadrilateral annotations and NWPU-VHR10 providing horizontal bounding-box annotations.

  • The experiments evaluate CAD-Net on the DOTA and NWPU-VHR10 remote-sensing object-detection datasets.
  • Datasets: DOTA contains 2,806 aerial images, over 188,000 annotated object instances, and 15 object categories across training, validation, and test subsets.
  • Datasets: NWPU-VHR10 contains 800 very-high-resolution images, including 650 positive and 150 negative images, with 10 annotated object types.
  • Evaluation Metrics: Mean Average Precision (mAP) is used throughout the experiments to evaluate multi-class object detection.Its definition follows the metric used in the PASCAL VOC 2012 object-detection challenge.

B. Implementation Details

The implementation adapts CAD-Net to dataset-specific bounding-box formats and large remote-sensing images, using patch-based processing and a ResNet-101 backbone.

  • CAD-Net uses horizontal bounding boxes and oriented bounding boxes to accommodate different annotation settings.DOTA annotations support oriented boxes, while NWPU-VHR10 provides horizontal boxes only.
  • For DOTA, CAD-Net produces both horizontal and oriented bounding-box results, whereas NWPU-VHR10 receives horizontal-box results only.
  • Data Pre-processing: Training images are cropped into 1,600 × 1,600 patches with 800-pixel overlap, while inference uses 4,096 × 4,096 patches with 1,024-pixel overlap.Zero padding is applied when images are smaller than the inference patch size, alongside global contrast normalization.
  • Network Setup: The network uses an ImageNet-pretrained ResNet-101 backbone that is fine-tuned on the two remote-sensing datasets.The model is designed to output HBB and OBB simultaneously when oriented-box ground truth is available.
  • Optimization: Training uses momentum SGD on one 16GB Nvidia Tesla P100 GPU with batch size 1.Training runs use 130,000 iterations for DOTA and 30,000 for NWPU-VHR10.

C. Experimental Results

CAD-Net outperforms the Faster R-CNN baseline and state-of-the-art methods on the reported datasets, while qualitative results show improved detection in adverse conditions and several remaining failure modes.

  • DOTA Results: +15.8% mAP over the Faster R-CNN baseline demonstrates CAD-Net’s effectiveness on the DOTA test set.CAD-Net also exceeds state-of-the-art performance by up to 2% under two training setups.
  • NWPU-VHR10 Results: CAD-Net achieves superior object-detection performance on NWPU-VHR10 across three random dataset separations.The dataset split uses 75% of positive images for training and the remainder for testing, excluding negative images from training.
  • Qualitative Results: Compared with Faster R-CNN with FPN, CAD-Net correctly detects objects in examples involving misclassification, occlusion, low texture, and low contrast.
  • Qualitative Results: The authors attribute the superior detection performance to global and local contexts, spatial-and-scale-aware attention, balanced semantics, and rotation-angle regression.
  • Failure Cases: CAD-Net remains prone to failures involving strong light interference, small vehicles, long thin objects, and heavily overlapping objects.The cited explanations include limited relevant training images, inaccurate annotations, limited anchor aspect ratios, and object overlap.

D. Ablation Study

Ablation results show that global context, local context, and spatial-scale-aware attention each improve detection and work complementarily when combined.

  • The DOTA validation-set ablation compares a Faster R-CNN with FPN baseline against models adding GCNet, PLCNet, attention, and paired components.
  • Context Modules: ∼2.5% mAP improvement comes from adding GCNet, while PLCNet provides ∼1.5% mAP improvement over the baseline.These results support contributions from global scene-level and local neighboring-object or feature context.
  • Attention Module: ∼2.5% mAP improvement results from adding Spatial-Scale-Aware Attention to generate spatial-and-scale-aware feature maps.
  • Full Model: 64.8% mAP is achieved by the full CAD-Net, a 5% improvement over the baseline model.The paired-component experiments indicate that PLCNet, GCNet, and the attention module are complementary.
  • Evaluation Scope: The ablation study uses the DOTA validation set because test annotations are inaccessible and test-set submissions are limited.

V. CONCLUSIONS

CAD-Net combines scene-level and object-level context with spatial-and-scale-aware attention for remote sensing detection, achieving superior performance while remaining vulnerable to ultra-long and heavily overlapped objects.

  • CAD-Net uses GCNet and PLCNet to extract scene-level and object-level contextual information correlated with target objects.These modules provide extra guidance for object detection in remote sensing images.
  • Its spatial-and-scale-aware attention module emphasizes degraded texture details and selects scale-adaptive features across feature-map levels.
  • CAD-Net achieves superior object detection performance compared with state-of-the-art techniques on two public datasets.
  • The network still tends to fail for ultra-long or heavily overlapped objects.The authors identify improved use of contextual information as a direction for future work.
Loading 1903.00857v1…