PaperlayerView exact revision on arXiv

Source-linked AI summary

Asymmetric Contextual Modulation for Infrared Small Target Detection

Yimian Dai, Yiquan Wu, Fei Zhou, Kobus Barnard

arXiv:2009.14530v1cs.CV

TL;DR

Single-frame infrared small target detection is challenged by scarce intrinsic target information and the lack of a public, high-quality dataset. The paper introduces SIRST and an asymmetric contextual modulation module combining top-down global feedback with bottom-up point-wise channel attention. Its networks achieve better performance than state-of-the-art approaches, with fewer parameters and layers.

  • Problem

    Infrared small-target detection lacks public high-quality data, while targets have minimal intrinsic information and can be submerged by complex backgrounds.

  • Method

    The paper contributes SIRST and an ACM module that exchanges high-level semantics and low-level fine details through asymmetric top-down and bottom-up modulation.

  • Results

    The proposed networks achieve better performance than other state-of-the-art approaches with fewer parameters and layers.

  • Takeaways & Limitations

    SIRST and ACM provide a public benchmark and an infrared-specific feature-fusion design for advancing single-frame small-target detection.

  • Takeaways & Limitations

    Top-down modulation can lose small targets because deeper features may submerge them in background and global average pooling can weaken their features.

Abstract

from arXiv · show

Single-frame infrared small target detection remains a challenge not only due to the scarcity of intrinsic target characteristics but also because of lacking a public dataset. In this paper, we first contribute an open dataset with high-quality annotations to advance the research in this field. We also propose an asymmetric contextual modulation module specially designed for detecting infrared small targets. To better highlight small targets, besides a top-down global contextual feedback, we supplement a bottom-up modulation pathway based on point-wise channel attention for exchanging high-level semantics and subtle low-level details. We report ablation studies and comparisons to state-of-the-art methods, where we find that our approach performs significantly better. Our dataset and code are available online.

1. Introduction

Single-frame infrared small target detection is difficult because targets have minimal intrinsic information, scenes violate traditional assumptions, and public high-quality data are scarce. The paper addresses these gaps with the SIRST dataset and asymmetric contextual modulation that exchanges semantic and fine-detail information across feature levels.

  • 1. Introduction: Traditional model-driven methods rely on target-background contrast and thresholded saliency maps but are vulnerable to low-contrast targets, distractors, and scene-sensitive hyperparameters.These methods are learning-free and computationally friendly, yet their assumptions are not robust in variable real-world scenes.
  • 1. Introduction: Single-frame detection lacks a public high-quality dataset, while small targets provide minimal texture or shape information and require both global semantics and fine-resolution predictions.The paper describes this resolution-semantics tension as an endogenous contradiction of deep networks.
  • 1. Introduction: SIRST is an open dataset with manually labeled representative frames and five annotation forms for training diverse machine-learning approaches.It is described as the first public dataset of this kind and 4× larger than other private datasets.
  • 1. Introduction: The proposed asymmetric contextual modulation supplements top-down semantic feedback with a reverse bottom-up pathway that encodes smaller-scale visual details into deeper layers.This design targets the scale mismatch between infrared small targets and objects in generic datasets.
  • 1. Introduction: Top-down modulation uses global channel attention, whereas bottom-up modulation uses point-wise channel attention to preserve and highlight small-target features.The paper motivates point-wise attention as suitable for varying spatial pooling scales and small target size.
  • 1. Introduction: Networks built with ACM perform significantly better than their original host networks with only a modest number of additional parameters, and achieve the best detection performance against state-of-the-art methods on SIRST.Ablation studies are reported as demonstrating the effectiveness of the proposed architecture.

2. Related Work

Prior infrared small-target work is constrained by heuristic assumptions, scarce private data, and limited deep-learning approaches. The paper differs by introducing a bidirectional ACM fusion module and an open, richly annotated single-frame benchmark.

  • 2. Related Work: Existing methods commonly formulate detection as outlier detection under assumptions such as saliency, sparsity in low-rank backgrounds, or pop-out structure in smooth backgrounds.They derive outlierness maps through saliency detection, matrix or tensor decomposition, or local contrast measurements.
  • 2. Related Work: Deep learning approaches remain few, while the paper contributes an infrared-specific ACM cross-layer fusion module and a benchmark toolkit with datasets, metrics, implementations, and augmentation tricks.The benchmark is intended to support more consistent evaluation of competing approaches.
  • 2. Related Work: Cross-layer fusion commonly combines lower-level and higher-level features through summation or concatenation, which does not dynamically select relevant lower-layer information.Recent modulation methods use high-level features to guide low-level feature processing.
  • 2. Related Work: ACM exchanges high-level semantics and fine details through bidirectional pathways, adding point-wise channel attention to preserve subtle infrared target details.This distinguishes it from one-directional top-down modulation.
  • 2. Related Work: Infrared datasets are often private and small, making performance comparisons potentially unfair or inaccurate.Sequence datasets may also be used for training and testing, unlike the paper’s explicit single-frame dataset construction.
  • 2. Related Work: SIRST selects one representative image from each sequence and provides five annotation forms, making it an open single-frame dataset for multiple task formulations.It is described as the first public and largest dataset compared with other private datasets.

3. SIRST: From Model-Driven to Data-Driven

SIRST bridges model-driven and data-driven infrared small-target detection by providing a public annotated dataset, tailored evaluation support, and dataset statistics exposing the limits of saliency assumptions.

  • SIRST: From Model-Driven to Data-Driven: SIRST is presented as both a dataset and toolkit intended to bridge data-driven deep learning with model-driven infrared small-target detection.The toolkit includes implementations of state-of-the-art methods, customized evaluation metrics, and data augmentation utilities.
  • SIRST: From Model-Driven to Data-Driven: The dataset contains 427 images and 480 target instances, split roughly into 50% training, 20% validation, and 30% testing.One representative image is selected from each infrared sequence to avoid overlap across splits.
  • SIRST: From Model-Driven to Data-Driven: SIRST supplies five annotation forms supporting image classification, instance segmentation, bounding-box regression, semantic segmentation, and instance spotting.All targets are treated as one category because infrared small-target detection is framed as a binary outlier-detection problem.
  • SIRST: From Model-Driven to Data-Driven: About 90% of images contain one target, but roughly 10% contain additional targets that global-uniqueness assumptions would ignore.This distribution explains why sparse or salient-target formulations can be attractive while remaining incomplete.
  • SIRST: From Model-Driven to Data-Driven: About 55% of targets occupy only 0.02% of image area, corresponding to approximately 3 × 3 pixels in a 300 × 300 image.The paper emphasizes preserving and highlighting target features in deep layers because smaller targets require more contextual information.
  • SIRST: From Model-Driven to Data-Driven: Only 35% of targets are brightest in their images; selecting brightest pixels yields detection rate 0.35 and false alarm rate 65%.The proposed method is reported at detection rate 0.84 and false alarm rate 0.0065%, challenging brightness-based saliency assumptions.
  • SIRST: From Model-Driven to Data-Driven: Normalized IoU is proposed to better balance model-driven and data-driven methods, while ROC remains necessary for evaluating performance across sliding thresholds.IoU and nIoU measure segmentation under a fixed threshold, whereas ROC reflects overall threshold-varying performance.

4. Asymmetric Contextual Modulation

The ACM module addresses the risk that deep global context can suppress infrared small-target details by combining top-down semantic modulation with bottom-up point-wise detail modulation. It is designed for cross-layer fusion in ACM-FPN and ACM-U-Net.

  • ACM targets deep models that detect infrared small targets while encoding high-level context without overwhelming finer target details.
  • Top-Down Attentional Modulation: Top-down global modulation uses high-level features to modulate low-level features, but global pooling can weaken already-submerged small-target responses.The approach assumes high-level features provide accurate target semantics and global channel context is suitable, assumptions that may fail as networks deepen.
  • Bottom-Up Point-wise Attentional Modulation: Bottom-up point-wise channel attention aggregates channel context independently at each spatial position to enrich high-level features with low-level spatial details.Its modulation weights have the same shape as the high-level feature, enabling element-wise highlighting of infrared small targets.
  • Asymmetric Contextual Modulation Module: Asymmetric contextual modulation combines top-down global attention and bottom-up local attention to exchange semantic information and spatial details across feature levels.The fused design is illustrated as the proposed asymmetric contextual modulation and used to construct ACM-FPN and ACM-U-Net.

5. Experiments

Experiments evaluate down-sampling, modulation direction, attention asymmetry, and comparisons against state-of-the-art methods using FPN and U-Net hosts on SIRST. ACM-based networks consistently perform best, while ablations support preserving small-target features and exchanging low- and high-level information.

  • Impact of Down-Sampling Scheme: ACM-based networks outperform Regular-ACM networks, showing that customized down-sampling is necessary to preserve small-target features in deeper layers.The regular scheme downsamples feature maps four times more.
  • Impact of Bi-directional Attentional Modulation: BiLocal consistently outperforms TopDownLocal under the same point-wise channel-attention parameter budget, supporting bi-directional modulation.The comparison uses the shared parameter budget C2.
  • Impact of Asymmetric Attentional Modulation: The proposed ACM performs best across all settings against BiLocal and BiGlobal, supporting global top-down and point-wise local bottom-up attention.This asymmetric design assigns global channel attention to high-level semantic feedback and point-wise channel attention to low-level detail modulation.
  • Comparison to State-of-the-Art Methods: ACM-FPN and ACM-U-Net achieve the best results across network depths, including with fewer layers, whereas top-down global modulation loses advantage as depth increases.The experiments use IoU and nIoU comparisons across FPN and U-Net host networks.
  • Comparison to State-of-the-Art Methods: The proposed networks achieve the best IoU and nIoU among 19 methods, while data-driven methods generally outperform model-driven methods.The authors also report that nIoU is better than IoU for evaluating infrared small-target detection.
  • Comparison to State-of-the-Art Methods: ACM-FPN and ACM-U-Net achieve the best ROC performance among seven selected methods, although RIPT ranks differently under ROC than under IoU and nIoU.IoU and nIoU use a fixed threshold, whereas ROC evaluates performance over a sliding threshold.

6. Conclusion

The paper contributes the SIRST dataset and an asymmetric contextual modulation module for infrared small-target detection. The module combines bottom-up fine-detail exchange with point-wise channel attention, achieving better performance with fewer parameters and layers.

  • The paper contributes an open dataset for detecting and segmenting infrared small targets in single-frame scenarios.
  • The asymmetric contextual modulation module supplements top-down semantic feedback with bottom-up modulation to exchange low-level details and high-level features.
  • Point-wise channel attention highlights infrared small-target features instead of allowing background neighborhoods to overwhelm them.
  • Compared with other state-of-the-art approaches, the proposed networks achieve better performance with fewer parameters and layers.
Loading 2009.14530v1…