Source-linked AI summary
Infrared Small Target Detection with Scale and Location Sensitivity
Qiankun Liu, Rui Liu, Bolun Zheng, Hongkui Wang, Ying Fu
TL;DR
IRSTD loss functions often lack sensitivity to target scale and location, while model design has emphasized complex structures. The paper proposes SLS loss and a simple multi-scale head for U-Net, forming MSHNet. MSHNet achieves state-of-the-art performance, and SLS also improves existing detectors, though its multi-scale training is based on a stated hypothesis.
Problem
IoU and Dice losses lack sensitivity to target scales and locations, while deep-learning IRSTD methods primarily emphasize complex model structures and under-explore losses.
Method
The paper combines scale-weighted IoU and a center-point location penalty in SLS loss, then applies it across predictions from a simple multi-scale head added to plain U-Net.
Results
MSHNet outperforms existing state-of-the-art methods, and applying SLS to existing detectors further improves IoU performance, demonstrating effectiveness and generalization.
Takeaways & Limitations
A more effective loss paired with a simpler model structure can achieve state-of-the-art IRSTD performance while balancing detection performance, FLOPs, and inference time.
Takeaways & Limitations
Applying SLS to different MSHNet prediction scales is motivated by a hypothesis that scale-dependent attention will improve overall detection performance.
Abstract
from arXiv · showhide
Recently, infrared small target detection (IRSTD) has been dominated by deep-learning-based methods. However, these methods mainly focus on the design of complex model structures to extract discriminative features, leaving the loss functions for IRSTD under-explored. For example, the widely used Intersection over Union (IoU) and Dice losses lack sensitivity to the scales and locations of targets, limiting the detection performance of detectors. In this paper, we focus on boosting detection performance with a more effective loss but a simpler model structure. Specifically, we first propose a novel Scale and Location Sensitive (SLS) loss to handle the limitations of existing losses: 1) for scale sensitivity, we compute a weight for the IoU loss based on target scales to help the detector distinguish targets with different scales: 2) for location sensitivity, we introduce a penalty term based on the center points of targets to help the detector localize targets more precisely. Then, we design a simple Multi-Scale Head to the plain U-Net (MSHNet). By applying SLS loss to each scale of the predictions, our MSHNet outperforms existing state-of-the-art methods by a large margin. In addition, the detection performance of existing detectors can be further improved when trained with our SLS loss, demonstrating the effectiveness and generalization of our SLS loss. The code is available at https://github.com/ying-fu/MSHNet.
1. Introduction
IRSTD is difficult because infrared targets are small, dim, and affected by noise and clutter. The paper addresses under-explored loss functions by proposing SLS loss and a simple MSHNet detector.
- IRSTD supports applications including maritime surveillance and traffic management but remains difficult because targets are small and dim under noise and clutter.
- Traditional IRSTD methods rely on manually designed features, whereas recent deep-learning methods automatically learn features through gradient descent.
- Existing deep-learning IRSTD research emphasizes complex model structures while leaving loss functions under-explored.
- IoU and Dice losses can assign identical values to targets with different scales or locations, limiting detectors’ ability to distinguish and localize them.
- SLS loss weights IoU by scale discrepancy and adds a center-point location penalty, while MSHNet adds a simple multi-scale head to plain U-Net.
- Applying SLS across prediction scales enables MSHNet to outperform existing state-of-the-art methods, while applying SLS to existing detectors further boosts detection performance.
2. Related Work
Prior IRSTD work spans hand-crafted traditional methods and deep-learning methods focused on feature extraction. This paper instead emphasizes a general scale- and location-sensitive loss with a simple multi-scale head.
- Existing IRSTD Methods: Traditional IRSTD methods include filtering-based, local-contrast-based, and low-rank-based approaches that depend on hand-crafted priors and manual hyperparameter tuning.
- Loss Functions for IRSTD: IoU and Dice losses are commonly used but are insensitive to target scales and locations, hindering accurate distinction between different targets.
- Loss Functions for IRSTD: Some specialized losses target adversarial training, target edges, or target-background likelihoods, but their architecture-specific designs limit broader utility.
- Loss Functions for IRSTD: GIoU and CIoU losses have been adopted for box-level IRSTD but still lack scale and location sensitivity.
- Loss Functions for IRSTD: The proposed general loss distinguishes targets by scale and location, enabling different detectors to achieve better detection performance.
- Model Structures for IRSTD: MSHNet introduces a simple multi-scale head to plain U-Net instead of complex structures, achieving state-of-the-art performance with less time consumption.
3. Scale and Location Sensitive Loss
SLS loss combines scale-sensitive weighting of IoU with a location-sensitive center-point penalty. Its formulation emphasizes scale discrepancies and differentiates location errors more finely.
- SLS loss combines a scale-sensitive loss, LS, and a location-sensitive loss, LL.
- Scale Sensitive Loss: The scale-sensitive component applies a weight to IoU loss to address its insensitivity to target scales.
- Scale Sensitive Loss: The weight decreases as the gap between predicted and ground-truth pixel counts increases, producing a larger scale-sensitive loss when scales differ.
- Location Sensitive Loss: The location-sensitive component computes predicted and ground-truth center points by averaging target-pixel coordinates, then represents them in polar coordinates.
- MSHNet Detector: MSHNet feeds decoder feature maps at different scales into dedicated heads, upsamples predictions when needed, concatenates them, and applies SLS to each prediction.
- Location Sensitive Loss: The location loss distinguishes most different center-point errors, making the detector sensitive to error types and supporting more accurate target localization.
4. MSHNet Dectector
MSHNet adds a simple multi-scale head to U-Net, producing predictions at multiple decoder scales and combining them into a final prediction. SLS loss is applied to every prediction, with the final loss combining losses across scales.
- 4.1. Multi-Scale Head: MSHNet feeds decoder feature maps at different scales into separate prediction heads to produce multi-scale predictions.Each prediction head uses a convolution layer followed by sigmoid activation.
- 4.1. Multi-Scale Head: Each prediction head has dedicated parameters, and the multi-scale predictions are spatially upsampled and concatenated to form the final prediction.The final prediction has spatial dimensions H × W × 1.
- 4.2. Training MSHNet with SLS Loss: SLS loss is applied to all MSHNet predictions so targets at different scales can receive different loss values and attention.This design uses the scale sensitivity of SLS loss across predictions with different scales.
- 4.2. Training MSHNet with SLS Loss: The final MSHNet loss combines SLS losses computed between predictions at each scale and correspondingly downsampled ground-truth labels.Ground-truth labels are spatially downsampled with max-pooling for scale-specific supervision.
5. Experiments
Experiments evaluate MSHNet on two infrared small-target datasets using pixel- and target-level metrics, comparisons with existing methods, visualizations, and ablations. MSHNet achieves strong overall performance, while analyses show benefits and trade-offs from scale/location-sensitive loss and multi-scale prediction.
- Experimental Setup: Experiments use IRSTD-1k and NUDT-SIRST, with IoU, false alarm rate (Fa), and probability of detection (Pd) as evaluation metrics.IRSTD-1k contains 1,001 images and NUDT-SIRST contains 1,327 images.
- Comparison to Existing Methods: MSHNet performs best on all reported metrics and datasets, according to the quantitative comparison.The comparison includes traditional and deep-learning-based detectors.
- Comparison to Existing Methods: Compared with DNANet, MSHNet gains 1.45% and 0.57% in IoU and 2.04% and 1.06% in Pd on IRSTD-1k and NUAA-SIRST, respectively.False alarm rates are also reduced by a large margin on both datasets.
- Qualitative Results: MSHNet detects challenging small targets more effectively, with higher-confidence predictions and peak counts closer to ground truth in the visual analyses.The paper attributes this behavior to the scale and location sensitivity of SLS loss and the multi-scale head.
- Detection Results for Different Scales of Targets: MSHNet outperforms DNANet across all target-scale levels and performs much better than ISNet for targets larger than 40 pixels.Targets are grouped into (0,10], (10,40], and (40,∞] pixel ranges.
- Ablation Studies: SLS consistently improves detectors’ IoU, while location sensitivity can increase false alarms; increasing multi-scale-head scales improves IoU from 63.10% to 67.16%.The default configuration uses four scales.
6. Conclusion
The paper combines SLS loss with a simple MSHNet to improve infrared small target detection while balancing inference time and computational cost. Applying SLS across prediction scales improves overall performance and can also boost existing detectors, although false alarms may increase.
- The scale-count ablation removes the remaining smallest scale when reducing MSHNet's number of adopted scales and evaluates IoU, Pd, and Fa.
- SLS loss emphasizes targets with large predicted-versus-ground-truth scale gaps and distinguishes location errors between target center points.
- MSHNet adds a simple Multi-Scale Head to a plain U-Net and produces multi-scale predictions for each input.
- Applying SLS loss across prediction scales gives different target scales different attention, resulting in better overall detection performance.
- MSHNet achieves state-of-the-art detection performance with a better balance between inference time and floating-point operation count.
- SLS loss boosts the overall detection performance of existing detectors, but may introduce more false alarms because its location-sensitive component can treat some noise as targets.