Pinwheel-shaped Convolution and Scale-based Dynamic Loss for Infrared Small Target Detection

Jiangnan Yang; Shuangli Liu; Jingjun Wu; Xinyu Su; Nan Hai; Xueli Huang

Source-linked AI summary

Pinwheel-shaped Convolution and Scale-based Dynamic Loss for Infrared Small Target Detection

Jiangnan Yang, Shuangli Liu, Jingjun Wu, Xinyu Su, Nan Hai, Xueli Huang

arXiv:2412.16986v1Dec 22, 2024cs.CV

TL;DR

Infrared small-target detection is challenged by dim, low-SNR targets, complex backgrounds, and losses whose scale and location sensitivity varies with target size. The paper introduces Gaussian-aligned PConv, scale-based dynamic loss, and the SIRST-UAVB benchmark, reporting consistent improvements across multiple models and datasets. A stated limitation is that mask-based SDM loss is unsuitable for overly large batch sizes because batch-averaged target sizes can misassign loss coefficients.

Problem
Infrared small-target methods overlook target pixel-distribution characteristics, while existing losses overlook IoU fluctuations and scale-dependent sensitivity.
Method
The paper replaces lower-layer standard convolutions with Gaussian-aligned PConv, dynamically adjusts scale and location loss coefficients by target size, and constructs SIRST-UAVB.
Results
PConv and SD Loss consistently improved performance across multiple detection and segmentation models and datasets, with PConv plus SDB loss achieving the highest mAP50 across the evaluated detection models.
Takeaways & Limitations
The results support PConv and SD Loss as generalizable additions for improving infrared small-target detection accuracy, stability, and robustness across models.
Takeaways & Limitations
Mask-based SDM loss may misassign scale and location coefficients at overly large batch sizes because target size and mask loss are batch-averaged.

Abstract

from arXiv · show

These recent years have witnessed that convolutional neural network (CNN)-based methods for detecting infrared small targets have achieved outstanding performance. However, these methods typically employ standard convolutions, neglecting to consider the spatial characteristics of the pixel distribution of infrared small targets. Therefore, we propose a novel pinwheel-shaped convolution (PConv) as a replacement for standard convolutions in the lower layers of the backbone network. PConv better aligns with the pixel Gaussian spatial distribution of dim small targets, enhances feature extraction, significantly increases the receptive field, and introduces only a minimal increase in parameters. Additionally, while recent loss functions combine scale and location losses, they do not adequately account for the varying sensitivity of these losses across different target scales, limiting detection performance on dim-small targets. To overcome this, we propose a scale-based dynamic (SD) Loss that dynamically adjusts the influence of scale and location losses based on target size, improving the network's ability to detect targets of varying scales. We construct a new benchmark, SIRST-UAVB, which is the largest and most challenging dataset to date for real-shot single-frame infrared small target detection. Lastly, by integrating PConv and SD Loss into the latest small target detection algorithms, we achieved significant performance improvements on IRSTD-1K and our SIRST-UAVB dataset, validating the effectiveness and generalizability of our approach. Code -- https://github.com/JN-Yang/PConv-SDloss-Data

Introduction

Infrared small targets are difficult to detect because they are dim, low-contrast, texture-poor, scale-varying, and obscured by complex backgrounds. The paper addresses these challenges with PConv, SD Loss, and the SIRST-UAVB benchmark.

Motivation: Infrared small targets often have low SNR and SCR, little texture, varying size and shape, and complex-background interference.These conditions arise especially in mid-to-long-range observation.
PConv: Standard convolution overlooks the Gaussian spatial distribution observed in infrared small targets.PConv is proposed as a plug-and-play alternative for lower backbone layers.
PConv: PConv aligns convolution with infrared target characteristics, enhancing bottom-layer feature extraction and expanding the receptive field.The module is designed to improve feature extraction while adding only a minimal parameter increase.
SD Loss: Existing losses overlook IoU fluctuations and scale-dependent sensitivity in scale and location terms.The proposed mechanism dynamically adjusts their coefficients according to target size.
Dataset: SIRST-UAVB was developed as a large, challenging real-shot single-frame dataset containing UAV and bird targets.It addresses limitations of existing datasets, including low small-target proportions, simple backgrounds, and limited scale.
Contributions: Integrating PConv and SD Loss into infrared detection methods produced significant and consistent improvements across public and newly constructed datasets.The methods were applied to both bounding-box and mask label formats.

Related Work

Deep learning has become mainstream for infrared small-target detection, but existing architectures, losses, and datasets leave important challenges in feature extraction, regression stability, and real-world evaluation. The paper responds with Gaussian-aligned PConv, scale-based dynamic loss, and SIRST-UAVB.

Deep Learning Methods: Deep learning methods became mainstream because they automatically learn multilayer target features from growing infrared small-target datasets.The related work includes CNN-based detection and segmentation approaches.
Convolutional Modules: The paper focuses on improving bottom-layer feature extraction through a convolutional module aligned with the Gaussian distribution of infrared targets.This shifts emphasis from network architecture design to the convolutional operation itself.
Loss Functions: Existing IoU-based losses do not fully address fluctuation errors or the sensitivity of weak, small-scale targets.The cited alternatives combine overlap, positional, or shape information but retain these limitations in the stated context.
Loss Functions: Scale-based Dynamic Loss varies the influence of scale and location losses with target size to reduce IoU fluctuation and improve stability.The approach is presented as a simple mechanism for stable performance improvements.
Datasets: Existing benchmarks are limited by small scale, low diversity, simple challenges, or simulated targets, motivating the SIRST-UAVB dataset.The paper contrasts SIRST, IRSTD-1K, and NUDT-SIRST as examples of these limitations.

Methodology

PConv replaces standard lower-layer convolutions with asymmetric, orientation-aware operations aligned to infrared small targets’ Gaussian spatial distribution. SD Loss dynamically adjusts scale and location-loss influence according to target size, while SIRST-UAVB provides a challenging UAV and bird benchmark.

Pinwheel-shaped Convolution: PConv uses asymmetric padding and parallel horizontal and vertical kernels to process different image regions.Batch normalization and SiLU follow each convolution for training stability and speed.
Pinwheel-shaped Convolution: The first PConv layer concatenates four interleaved convolution outputs before producing the module output.Output dimensions depend on input height, width, channel count, and stride.
Pinwheel-shaped Convolution: PConv expands the receptive field by 178% with a 111% parameter increase for k=3, and by 444% with a 122% increase for k=4.Grouped convolution supports receptive-field expansion while limiting parameter growth relative to the reported alternatives.
Pinwheel-shaped Convolution: PConv outputs enhance contrast between infrared targets and backgrounds while suppressing clutter-like signals.Figure 4 visualizes the comparison between PConv and standard Conv outputs.
Scale-based Dynamic Loss: SD Loss adjusts BBox and mask scale-location loss weights according to target area and feature-map scale.For BBox targets larger than 81, LSDB degenerates into CIoU loss; δ controls the coefficient range.
Benchmark: SIRST-UAVB contains 3,000 infrared images of UAVs and birds collected across seasons, weather conditions, and complex backgrounds.Targets vary in orientation, scale, and occlusion, with many nearly invisible to the naked eye and manually checked annotations.

Experimental Settings

Experiments evaluate detection and segmentation using complementary metrics on IRSTD-1K and SIRST-UAVB, with controlled model-training settings and module comparisons.

Datasets: IRST-1K contains 1,000 real infrared images, while SIRST-UAVB contains smaller targets; both datasets use a 4:1 train-test split.IRST-1K images have 512 × 512 resolution and larger average targets than SIRST-UAVB.
Detection Metrics: Detection evaluation uses precision, recall, and mAP50, measuring classification correctness and target-location accuracy.Precision is the proportion of predicted positives that are true, recall is the proportion of true samples detected, and mAP50 assesses location accuracy.
Experimental Design: Table 1 compares convolution modules in YOLOv8n-p2 detection and MSHNet segmentation after replacing their first two standard layers.Detection reports P, R, and mAP50 alongside parameter counts; segmentation reports IoU, Pd, and Fa.
Segmentation Metrics: Segmentation evaluation uses IoU for pixel-level shape description and false-alarm and probability-of-detection rates for localization.False-alarm rate uses false-positive pixels relative to total pixels, while probability of detection uses correctly predicted targets relative to total targets.
Training Settings: Ablation experiments use PyTorch on RTX3090 GPUs with separate input sizes, batch sizes, epoch counts, patience, and learning rates for detection and segmentation.Detection uses 640-pixel inputs, batch size 64, 700 epochs, patience 70, and learning rate 0.01; segmentation uses 256-pixel inputs, batch size 4, 400 epochs, and learning rate 0.05.

Comparison with Other Methods

PConv is compared with alternative receptive-field and parameter-efficient convolution modules, while SDB and SDM losses are evaluated across datasets and label formats.

Convolution Module Comparison: PConv(4,4) performs best overall on IRSTD-1K, while PConv(4,3) provides the most balanced gains and the best enhancement on SIRST-UAVB.Larger kernels benefit larger targets in IRSTD-1K, whereas increasing kernel length adds no gain for smaller SIRST-UAVB targets.
Convolution Module Comparison: PConv significantly outperforms other convolution modules in MSHNet segmentation, with kernel length 4 preferred initially and length 3 sufficient in later layers.This configuration preserves performance while reducing computational overhead after downsampling.
Convolution Module Comparison: PConv aligns with Gaussian IRST gray distributions, expands the receptive field, and improves bottom-layer feature extraction with only a negligible parameter increase.Alternative modules target parameter reduction or receptive-field expansion, but MixConv uses more parameters without surpassing PConv.
Loss Comparison: SDB delivers consistent, balanced detection improvements across IRSTD-1K and SIRST-UAVB, unlike SAFit, whose performance drops substantially on IRSTD-1K.NWD and SAFit also use exponential operations, whereas SDB is simpler and more efficient.
Loss Comparison: SDM(δ = 0.5) achieves the best overall segmentation performance while maintaining balance across datasets.For detection, smaller δ works better on IRSTD-1K and larger δ on SIRST-UAVB; segmentation consistently favors δ = 0.5.

Ablation Experiments on Multiple Models

Ablations across detection and segmentation architectures show that PConv and scale-based dynamic losses improve performance broadly, though the best configuration can depend on the model.

Detection Models: PConv and SDB consistently improve performance across EFLNet, YOLOv5n, YOLOv8n-p2, and DNANet detection models.Their combination achieves the highest mAP50 scores across all evaluated detection models.
Detection Models: The combined approach produces significant precision and recall gains, particularly in EFLNet, supporting improvements in detection accuracy, stability, and generalization.The authors report these gains as evidence that the method addresses limitations of traditional convolutional layers and loss functions.
Segmentation Models: PConv combined with SDM consistently improves segmentation, especially in DNANet, with additional gains in ISNet and MSHNet.MSHNet improves over baseline with the combination but does not exceed PConv with SDM loss alone, indicating architecture-specific configuration effects.
Qualitative Analysis: Qualitative results indicate that PConv reduces missed detections, SD loss enhances weak-signal detection, and their combination reduces false alarms.The reported effects are associated with improved robustness in the visual analyses.

Conclusion

The paper combines PConv, SD Loss, and the SIRST-UAVB benchmark to improve infrared small target detection and segmentation. Experiments across multiple models and datasets show effectiveness, generalization, accuracy, and robustness improvements.

PConv uses infrared small targets’ Gaussian distribution to provide an efficient, larger receptive field with minimal parameters.The module is plug-and-play and improves lower-layer feature extraction.
SD Loss addresses IoU fluctuation issues by dynamically adjusting scale and location loss contributions.The loss is designed to improve regression for targets with varying scales.
Applying PConv and SD Loss across multiple models demonstrates effectiveness and strong generalization capabilities.The paper reports significant and consistent performance improvements across public datasets and SIRST-UAVB.
SIRST-UAVB is introduced as a large-scale, challenging benchmark with detailed annotations for real infrared small target detection and segmentation.The dataset includes comprehensive spatial-domain challenges.

Supplementary Experiment

The supplementary experiment evaluates PConv and SDB loss across target scales using YOLOv8n-p2. Both components improve detection, especially for targets with box areas below 16, while kernel settings of 4 and 3 provide stable gains.

The ablation study evaluates PConv and SDB loss under YOLOv8n-p2 after dividing SIRST-UAVB by target scale.The experiment examines detection performance across different target-size groups.
PConv and SDB loss enhance detection across target scales, particularly for small targets with box areas less than 16.The improvements affect multiple evaluation metrics.
Using PConv kernel sizes of 4 and 3 for two layers provides stable and significant performance improvements.
Larger combinations of PConv kernel sizes produce better detection performance in the reported experiment.

Limitation

The mask-based SDM loss has a batch-size limitation because its scale and location coefficients are computed from batch-averaged target sizes. Large target-size variation within a batch can therefore weaken scale-specific weighting.

Mask-based SDM loss may not support excessively large batch sizes because SLS loss is averaged across the batch.The paper notes that future work will assign influence coefficients according to each target’s size.
When target sizes vary greatly within a batch, the averaged target size cannot assign Sloss and Lloss coefficients to each target accurately.The paper states that increasing batch size aggravates this negative effect.