PaperlayerView exact revision on arXiv

Source-linked AI summary

Dense Nested Attention Network for Infrared Small Target Detection

Boyang Li, Chao Xiao, Longguang Wang, Yingqian Wang, Zaiping Lin, Miao Li, Wei An, Yulan Guo

arXiv:2106.00487v3cs.CV

TL;DR

SIRST detection is challenged by the loss of tiny targets in deep CNN layers and requires evaluation across difficult scenes. The paper introduces DNANet with nested feature interaction and attention, alongside NUDT-SIRST and revised metrics; experiments report better Pd, Fa, and IoU than state-of-the-art methods.

  • Problem

    Existing CNN-based SIRST methods can lose small targets in deep layers, while target context and clutter conditions complicate detection.

  • Method

    DNANet uses DNIM for repeated multi-level feature fusion, CSAM for adaptive enhancement, and a new dataset and evaluation metrics for SIRST assessment.

  • Results

    Experiments on public and self-developed datasets report superior performance over state-of-the-art methods in Pd, Fa, and IoU.

  • Takeaways & Limitations

    Repeated fusion and enhancement maintain small-target information and incorporate contextual information for SIRST detection.

Abstract

from arXiv · show

Single-frame infrared small target (SIRST) detection aims at separating small targets from clutter backgrounds. With the advances of deep learning, CNN-based methods have yielded promising results in generic object detection due to their powerful modeling capability. However, existing CNN-based methods cannot be directly applied for infrared small targets since pooling layers in their networks could lead to the loss of targets in deep layers. To handle this problem, we propose a dense nested attention network (DNANet) in this paper. Specifically, we design a dense nested interactive module (DNIM) to achieve progressive interaction among high-level and low-level features. With the repeated interaction in DNIM, infrared small targets in deep layers can be maintained. Based on DNIM, we further propose a cascaded channel and spatial attention module (CSAM) to adaptively enhance multi-level features. With our DNANet, contextual information of small targets can be well incorporated and fully exploited by repeated fusion and enhancement. Moreover, we develop an infrared small target dataset (namely, NUDT-SIRST) and propose a set of evaluation metrics to conduct comprehensive performance evaluation. Experiments on both public and our self-developed datasets demonstrate the effectiveness of our method. Compared to other state-of-the-art methods, our method achieves better performance in terms of probability of detection (Pd), false-alarm rate (Fa), and intersection of union (IoU).

I. INTRODUCTION

SIRST detection must separate tiny, dim, and shapeless infrared targets from clutter, while existing approaches struggle with changing target and background conditions. The paper proposes DNA-Net modules, a dataset, and evaluation evidence for more robust detection.

  • Infrared targets range from one pixel to tens of pixels and often have low signal-to-clutter ratios in noisy backgrounds.
  • Handcrafted traditional methods struggle with dramatic variations in target size, shape, signal-to-clutter ratio, and clutter background.
  • CNN-based methods motivate DNA-Net, but pooling can cause small targets to disappear in deep layers of conventional U-shape networks.
  • DNIM enables progressive multi-level feature interaction, while CSAM adaptively enhances features through channel and spatial attention.
  • NUDT-SIRST provides numerous target shapes, varied sizes, diverse clutter backgrounds, and ground-truth annotations.
  • Experiments on public and NUDT datasets report superior performance and greater robustness to clutter, target size, and target shape variations.

II. RELATED WORK

Prior SIRST work spans handcrafted and CNN-based detection, but small-target loss in deep layers remains a robustness problem and available datasets remain constrained. DNA-Net is presented as a pipeline combining nested feature extraction, pyramid fusion, and target-region clustering.

  • II. RELATED WORK: Traditional SIRST methods use filtering, local-contrast, or low-rank formulations based on handcrafted features and fixed hyperparameters.
  • II. RELATED WORK: CNN-based SIRST methods include generic detection frameworks and segmentation networks that produce pixel-level classification and localization outputs.
  • II. RELATED WORK: Small-target loss in deep layers remains and limits robustness to clutter, signal-to-clutter ratio, shape, and size changes.
  • II. RELATED WORK: Existing open datasets have limited capacity, variety, or annotation quality, motivating NUDT-SIRST with varied targets, sizes, clutter backgrounds, and accurate annotations.
  • II. RELATED WORK: DNA-Net sequentially performs feature extraction, feature-pyramid fusion, and eight-connected neighborhood clustering to generate detection results.
  • II. RELATED WORK: Its DNIM repeatedly fuses multi-layer features at middle skip-connection nodes before decoder processing.

B. The Feature Extraction Module

The feature extraction module uses densely nested U-shaped subnetworks and repeated multi-directional feature interaction to preserve small-target representations while extracting high-level information. CSAM then adaptively enhances fused multi-level features to address their semantic gap.

  • Traditional U-shaped networks use encoders, decoders, and skip connections to combine low-level spatial features with high-level semantic information.
  • Increasing network depth enlarges receptive fields but can lose point targets after repeated pooling operations.
  • Dense nested structure stacks U-shaped subnetworks with different depths, matching their receptive fields to targets of different sizes.
  • For j > 0, each DNIM node receives outputs from three directions through dense plain and nested bidirectional interactive skip connections.
  • CSAM adaptively enhances features after each DNIM fusion using cascaded channel and spatial attention maps.

3) Channel and Spatial Attention Module:

CSAM sequentially applies channel and spatial attention to multi-layer features, enhancing representations during DNIM fusion. The supplied dataset passages provide comparative context for NUDT-SIRST but do not define this module.

  • Channel attention maps are stretched to the feature-map dimensions before element-wise multiplication.
  • Spatial attention uses a 7×7 convolution and stretches its attention maps to the feature-map dimensions before multiplication.

C. The Feature Pyramid Fusion Module

The feature pyramid fusion module aligns multi-layer features spatially, concatenates shallow spatial information with deep semantic information, and clusters connected target pixels to compute centroids.

  • Multi-layer features are upscaled to a common spatial size before fusion.
  • Shallow features contribute spatial and profile information, while deep features contribute semantic information to global robust feature maps.
  • An eight-connected neighborhood module clusters pixels belonging to the same target and calculates each target’s centroid.

IV. THE NUDT-SIRST DATASET

NUDT-SIRST addresses limited scene diversity and dataset capacity through large-scale infrared small-target data synthesis. Its construction adapts target properties to backgrounds and produces realistic intensity, boundaries, target types, and annotations.

  • Motivation: Existing SIRST datasets lack sufficient scenes or data capacity, while accurate pixel-level annotation is costly to collect.
  • Dataset synthesis: The synthesis process identifies background types, selects target sizes and types, and combines candidate targets with real infrared backgrounds.
  • Dataset synthesis: Adaptive intensity adjustment fixes target SCR at empirical values C of 3, 4, 5, or 6, and Gaussian blur smooths synthesized boundaries.
  • Dataset comparison: About 37% of NUDT-SIRST images contain at least two targets, 27% of targets occupy no more than 0.01% of the image, and 96% satisfy SPIE’s small-target definition.
  • Dataset comparison: NUDT-SIRST comparisons evaluate target count, target size, and target brightness across public SIRST datasets.

A. Evaluation Metrics

The paper evaluates localization with target-level Pd and Fa, while using IoU to assess pixel-level shape description. These metrics reflect distinct aspects of SIRST detection, where a single pixel can substantially affect small-target evaluation.

  • For a 3 × 3 target, one falsely predicted pixel causes an 11.1% decrease in Pd, motivating target-level localization metrics.The paper therefore uses Pd and Fa for localization and IoU for shape description.
  • IoU measures profile description ability as the intersection-to-union area ratio between predictions and labels.The metric evaluates pixel-level overlap rather than target-level localization.
  • Pd measures the ratio of correctly predicted targets to all targets using a centroid-deviation criterion.Targets are counted as correctly predicted when centroid deviation is below Dthresh, set to 3 in this paper.
  • Fa measures the ratio of falsely predicted pixels to all image pixels using the same centroid-deviation threshold.Pixels associated with targets whose centroid deviation exceeds the threshold are considered falsely predicted.

4) Receiver Operation Characteristics:

The experiments compare DNA-Net with traditional and CNN-based SIRST methods on public and NUDT-SIRST data under standardized evaluation procedures. Results report strong quantitative performance and robustness to challenging scene variations.

  • Comparison setup: DNA-Net is compared with eight traditional and three CNN-based methods on the NUAA-SIRST and NUDT-SIRST datasets.CNN-based competitors were retrained on the same training datasets for fair comparison.
  • Comparison setup: The main comparison excludes NUST-SIRST because only about 30% of its targets meet the SPIE definition of small targets.The authors provide updated NUST-SIRST results and trained models separately for broader comparison.
  • Evaluation procedure: Traditional-method predictions undergo adaptive thresholding, while CNN-based methods retain fixed thresholds from their original papers.The adaptive threshold uses the output maximum, standard deviation, and average; CNN thresholds are 0, 0, and 0.5 for ACM, ALCNet, and MDvsFA-cGAN.
  • Quantitative results: DNA-Net achieves significant improvements over traditional methods across datasets containing varied SCR, clutter backgrounds, target shapes, and target sizes.The paper attributes this robustness to learned discriminative features, whereas handcrafted parameters limit traditional-method generalization.
  • Quantitative results: DNA-Net shows obvious improvements over MDvsFA-cGAN, ACM, and ALCNet in the Table II comparison.The paper relates these gains to a backbone tailored for SIRST detection and progressive feature fusion in deep layers.
  • Quantitative results: DNA-Net is superior to other deep-learning methods under different predefined deviation thresholds.Table IV reports Pd and Fa under varying Dthresh settings.

2) Qualitative Results:

Qualitative evaluations show that DNA-Net produces precise localization and shape segmentation with low false-alarm rates. Its accuracy remains stable as NUDT-SIRST includes more spot and extended targets, unlike traditional methods.

  • DNA-Net produces precise target localization and shape segmentation under a very low false-alarm rate on NUDT-SIRST and NUAA-SIRST.The qualitative comparison includes enlarged target regions and visualizations across six test images.
  • Traditional methods perform well mainly on point targets but generate many false-alarm regions in local highlight areas.Their performance decreases as spot and extended target proportions increase because handcrafted features cannot adapt to target-size variation.
  • CNN-based methods outperform traditional methods, but MDvsFA-cGAN still produces false alarms and missed detections in complicated NUDT-SIRST scenes.DNA-Net is reported as more robust to scene changes and produces better shape segmentation than ALCNet.
  • DNA-Net-ResNet10-Light achieves the highest IoU and Pd, the lowest Fa, and comparable training and inference time among the compared methods.The lightweight variant reduces half of DNA-Net-ResNet10’s channels while retaining reported computational efficiency.

D. Ablation Study

Ablation studies show that DNIM and CSAM materially improve DNA-Net by preserving multi-scale target information and enhancing informative features. Removing or simplifying these modules lowers IoU and Pd while increasing Fa.

  • DNIM: DNIM interacts with features at different scales to enlarge receptive fields while maintaining fine-grained features at the finest scale.Its dense nested skip connections progressively aggregate multi-scale features.
  • DNIM: Removing DNIM decreases IoU by 2.08% and Pd by 2.23%, while increasing Fa by 4.298 ×10^-6 on NUDT-SIRST.The paper attributes these changes to losing progressive multi-scale aggregation and target information at the finest scale.
  • DNIM: DNA-Net-left-to-right decreases IoU by 1.20% and Pd by 1.44%, while increasing Fa by 0.426 ×10^-6 versus DNA-Net on NUDT-SIRST.This variant lacks full interaction among shallow, own, and deep layers, leaving shallow localization and profile information underused.
  • DNIM: DNA-Net-top-to-bottom decreases IoU by 1.34% and Pd by 1.77%, while increasing Fa by 3.459 ×10^-6 on NUDT-SIRST.Its tri-directional interaction is limited to the core, so high-level information is not fully exploited at shallow layers.
  • CSAM: Removing CSAM decreases IoU by 1.19%, Pd by 2.11%, and increases Fa by 1.515 ×10^-6 on NUDT-SIRST.Replacing CSAM with element-wise summation produces larger changes: IoU decreases 1.84%, Pd 2.11%, and Fa increases 2.487 ×10^-6.
  • CSAM: Removing channel attention decreases IoU by 0.82% and Pd by 1.77%, while increasing Fa by 0.658 ×10^-6 on NUDT-SIRST.The channel attention unit is reported to enhance feature representation by exploiting informative channels.
  • CSAM: Removing spatial attention decreases IoU by 0.95% and Pd by 2.00%, while increasing Fa by 0.095 ×10^-6 on NUDT-SIRST.Spatial attention directs the network toward local informative areas amid cloud and noise.

3) The Feature Pyramid Fusion Module (FPFM):

The feature pyramid fusion module combines shallow spatial features with deep semantic features, and its ablations show that removing fusion or deeper outputs degrades detection performance.

  • The Feature Pyramid Fusion Module (FPFM):: FPFM fuses shallow features with rich spatial information and deep features with rich semantic information to produce robust output feature maps.The module extracts and combines representations from both shallow and deep layers.
  • The Feature Pyramid Fusion Module (FPFM):: 0.83% lower IoU, 1.89% lower Pd, and 1.81 ×10−6 higher Fa result when FPFM is removed on NUDT-SIRST.These changes are reported for DNA-Net w/o FPFM relative to DNA-Net.
  • The Feature Pyramid Fusion Module (FPFM):: Removing the deepest-layer output lowers IoU by 0.23% and Pd by 0.84%, while increasing Fa by 3.01×10−6.Similar degradation is observed when outputs from progressively deeper layers are removed.

E. Benefits of The Synthesized Dataset

The synthesized dataset supports real infrared small-target detection by supplying precise labels and improving segmentation shape relative to manually labeled masks.

  • E. Benefits of The Synthesized Dataset: Mixed real and synthesized SIRST images are used to train DNA-Net and ACM, with evaluation performed on real NUAA-SIRST test images.The experiments vary the ratio of real training images while testing on real images.
  • E. Benefits of The Synthesized Dataset: Outputs from the network trained on the mixed dataset have more reasonable segmentation shapes than manually labeled NUAA-SIRST ground-truth masks.Figure 14 presents input images, public masks, and DNA-Net outputs for this comparison.
  • E. Benefits of The Synthesized Dataset: The paper attributes this improvement to synthesized SIRST images having absolutely precise labels that help the network learn infrared small-target structure.The stated effect is improvement on real SIRST images.
  • E. Benefits of The Synthesized Dataset: Experiments on the developed and public datasets report superiority over state-of-the-art methods.The conclusion summarizes the reported outcome across both dataset sources.
Loading 2106.00487v3…