Source-linked AI summary
HCF-Net: Hierarchical Context Fusion Network for Infrared Small Object Detection
Shibiao Xu, ShuChen Zheng, Wenhao Xu, Rongtao Xu, Changwei Wang, Jiguang Zhang, Xiaoqiang Teng, Ao Li, Li Guo
TL;DR
Infrared small-object detection is challenged by tiny objects and complex backgrounds. HCF-Net addresses this with PPA, DASI, and MDCR, and achieves strong SIRST performance while surpassing other traditional and deep learning models.
Problem
Infrared small-object detection is difficult because tiny objects can lose crucial information and blend into complex backgrounds.
Method
HCF-Net models infrared small-object detection as semantic segmentation and combines PPA, DASI, and MDCR within an upgraded U-Net architecture.
Results
80.09% IoU and 78.31% nIoU are achieved on SIRST, significantly surpassing other methods.
Takeaways & Limitations
HCF-Net demonstrates strong infrared small-object detection performance against traditional segmentation and deep learning models.
Abstract
from arXiv · showhide
Infrared small object detection is an important computer vision task involving the recognition and localization of tiny objects in infrared images, which usually contain only a few pixels. However, it encounters difficulties due to the diminutive size of the objects and the generally complex backgrounds in infrared images. In this paper, we propose a deep learning method, HCF-Net, that significantly improves infrared small object detection performance through multiple practical modules. Specifically, it includes the parallelized patch-aware attention (PPA) module, dimension-aware selective integration (DASI) module, and multi-dilated channel refiner (MDCR) module. The PPA module uses a multi-branch feature extraction strategy to capture feature information at different scales and levels. The DASI module enables adaptive channel selection and fusion. The MDCR module captures spatial features of different receptive field ranges through multiple depth-separable convolutional layers. Extensive experimental results on the SIRST infrared single-frame image dataset show that the proposed HCF-Net performs well, surpassing other traditional and deep learning models. Code is available at https://github.com/zhengshuchen/HCFNet.
I. INTRODUCTION
Infrared small object detection is difficult because tiny objects can lose weak, unclear features during downsampling and blend into low-contrast, complex backgrounds. HCF-Net addresses these challenges by framing detection as semantic segmentation and combining three specialized modules.
- Infrared small objects have weak thermal signals and unclear contours, creating a significant risk of information loss during repeated downsampling.
- Lower contrast and limited physical information make infrared objects容易 to submerge in complex backgrounds.
- HCF-Net frames infrared small object detection as semantic segmentation to improve object-shape and boundary depiction for localization and segmentation.
- PPA preserves and enhances small-object representations through hierarchical feature fusion and attention across multiple downsampling steps.
- DASI adaptively selects and fuses high- and low-dimensional features in U-Net skip connections, while MDCR captures multi-scale receptive-field features for object-background modeling.
- On the SRIST single-frame infrared image dataset, HCF-Net demonstrates a significant advantage over several state-of-the-art detection methods.
II. RELATED WORK
Infrared small object detection has progressed from scenario-limited traditional methods to more robust deep learning approaches. The paper situates HCF-Net within this shift toward neural-network-based detection.
- Traditional approaches include filter-based, human-visual-system, and low-rank methods, each with limitations in scenarios, real-time performance, or interference resistance.
- Deep learning methods generally provide higher recognition accuracy, greater robustness across scenes and devices, and lower costs than traditional approaches.
- Prior deep learning work includes ImageNet-pretrained detection, deep convolutional recognition, attention-guided context modeling, asymmetric context modulation, and nested U-Net representations.
III. METHOD
HCF-Net upgrades U-Net with PPA, DASI, and MDCR to address small-object information loss and weak background distinctiveness. The method emphasizes multi-branch, multi-scale feature extraction and selective feature processing.
- HCF-Net is an upgraded U-Net architecture built around PPA, DASI, and MDCR for infrared small-object detection.
- PPA: PPA replaces conventional encoder-decoder convolutions with parallel local, global, and serial convolution branches.
- PPA: The three PPA branch outputs are separately computed and summed to form a fused feature representation.
- PPA: Patch size p controls local and global branches through non-overlapping patch aggregation, displacement, and attention-based interaction.
- PPA: PPA partitions features into spatial patches, applies channel averaging and feed-forward processing, then adjusts spatial weights for feature extraction.
- PPA: Feature selection reweights each token using cosine similarity to a task embedding and applies a learned linear transformation for channel selection.
2) Feature fusion and attention:
The attention module sequentially applies channel attention and spatial attention to extracted features, selecting informative channels and spatial locations.
- Channel attention first reweights the extracted features across channels, followed by spatial attention that reweights locations across the feature map.The channel map is one-dimensional, while the spatial map is two-dimensional.
- The attention process is designed to enhance feature representations after multi-branch feature extraction.
- The module uses element-wise multiplication to produce channel-selected and then spatially selected features.The resulting features are denoted Fc and Fs before subsequent normalization and activation.
B. Dimension-Aware Selective Integration Module
DASI adaptively fuses high-dimensional, low-dimensional, and current-layer features by aligning them, partitioning channels, and selectively aggregating corresponding partitions.
- DASI addresses the loss of small-object information in high-dimensional features and insufficient context in low-dimensional features during downsampling.
- High-dimensional and low-dimensional inputs are aligned with current-layer features through convolution and interpolation before channel-wise partitioning.
- The aligned features are divided into four equal channel segments, with hi, li, and ui representing corresponding high-dimensional, low-dimensional, and current-layer partitions.
- DASI applies activation and selective aggregation to partitioned features, then merges the aggregated partitions into the output Fu.
- The weighting parameter α controls the feature emphasis: α > 0.5 prioritizes fine-grained features, whereas α < 0.5 emphasizes context features.
C. Multi-Dilated Channel Refiner Module
MDCR refines channel representations with parallel depth-wise separable dilated convolutions, multi-scale channel recombination, and pointwise fusion.
- MDCR uses depth-wise separable convolutions with different dilation rates to capture spatial features across multiple receptive-field sizes.This design supports more detailed modeling of object-background differences for small-object discrimination.
- The module partitions input features into four heads along the channel dimension before applying separate dilated convolutions to each head.
- Each head uses a distinct dilation rate, denoted d1, d2, d3, and d4, to provide varied spatial context.
- MDCR splits processed head features into individual channels and interleaves them across heads to increase multi-scale feature diversity.
- Pointwise convolution performs intergroup and cross-group information fusion, producing the output Fo with lightweight aggregation.
- The module uses ReLU and batch normalization alongside its channel-refinement operations.
D. Loss design
HCF-Net uses deep supervision with a multi-scale loss combining binary cross-entropy and Intersection over Union losses. The experiments also report ablation and comparative evaluations using IoU and nIoU on SIRST.
- D. Loss design: Deep supervision applies a loss at each scale to address small-object loss during downsampling.Each scale combines binary cross-entropy and Intersection over Union losses.
- D. Loss design: The SIRST evaluation includes an ablation study with U-Net as the baseline and a comparative evaluation reporting IoU and nIoU.
- D. Loss design: The total loss weights multiple scales with [λ0, λ1, λ2, λ3, λ4] = [1, 0.5, 0.25, 0.125, 0.0625].
A. Datasets and Evaluation Metrics
The study evaluates HCF-Net on the SIRST single-frame infrared image dataset using IoU and nIoU as standard metrics. The dataset is divided into training and test sets at an 8:2 ratio.
- A. Datasets and Evaluation Metrics: HCF-Net is assessed on SIRST using Intersection over Union and normalized Intersection over Union.These are the two standard metrics used in the experiments.
- A. Datasets and Evaluation Metrics: SIRST is partitioned into training and test sets in an 8:2 ratio.
B. Implementation Details.
The implementation trains HCF-Net with Adam for 300 epochs using batch size 4 on an NVIDIA GeForce GTX 3090 GPU. On 512×512 three-channel inputs, the model requires 93.16 GMac and 15.29 million parameters, while experiments report strong SIRST performance.
- B. Implementation Details.: HCF-Net uses the Adam optimizer, batch size 4, and 300 training epochs on an NVIDIA GeForce GTX 3090 GPU.
- B. Implementation Details.: For 512×512 three-channel inputs, HCF-Net has a computational cost of 93.16 GMac and 15.29 million parameters.
- B. Implementation Details.: HCF-Net achieves IoU 80.09% and nIoU 78.31% on SIRST, surpassing other methods.Visual results also describe more detections, few false positives, precise localization in complex backgrounds, and detailed shape and texture descriptions.
- B. Implementation Details.: The paper addresses small-object loss and background clutter with HCF-Net modules that enhance small-object detection performance.