PaperlayerView exact revision on arXiv

Source-linked AI summary

One-Stage Cascade Refinement Networks for Infrared Small Target Detection

Yimian Dai, Xiang Li, Fei Zhou, Yulei Qian, Yaohong Chen, Jian Yang

arXiv:2212.08472v2cs.CV

TL;DR

Infrared small target detection is challenged by tiny targets being missed during label assignment, sensitive localization, and scarce real-world data. The paper addresses these issues with ASPB, OSCAR’s top-down cascade refinement, and a benchmark combining SIRST-V2, normalized contrast, and DeepInfrared. Its ablations report improved detection performance, including a rise from 75.6% to 79.2% when combining TDSM and TDCR, while shallower prediction layers trade better performance for 2–3 times higher computation.

  • Problem

    Infrared small target detection is limited by tiny targets being mislabeled as background, sensitive bounding-box localization, and scarce real-world training images.

  • Method

    The paper combines ASPB all-scale pseudo-box label assignment with OSCAR top-down cascade refinement and a benchmark comprising SIRST-V2, normalized contrast, and DeepInfrared.

  • Results

    79.2%: combining TDSM and TDCR improves the baseline from 75.6% to 79.2%, while shallower prediction layers improve performance at 2–3 times higher computational effort.

  • Takeaways & Limitations

    The proposed cascade framework improves infrared small target detection while maintaining the efficiency of a one-stage design and provides a benchmark for further research.

Abstract

from arXiv · show

Single-frame InfraRed Small Target (SIRST) detection has been a challenging task due to a lack of inherent characteristics, imprecise bounding box regression, a scarcity of real-world datasets, and sensitive localization evaluation. In this paper, we propose a comprehensive solution to these challenges. First, we find that the existing anchor-free label assignment method is prone to mislabeling small targets as background, leading to their omission by detectors. To overcome this issue, we propose an all-scale pseudo-box-based label assignment scheme that relaxes the constraints on scale and decouples the spatial assignment from the size of the ground-truth target. Second, motivated by the structured prior of feature pyramids, we introduce the one-stage cascade refinement network (OSCAR), which uses the high-level head as soft proposals for the low-level refinement head. This allows OSCAR to process the same target in a cascade coarse-to-fine manner. Finally, we present a new research benchmark for infrared small target detection, consisting of the SIRST-V2 dataset of real-world, high-resolution single-frame targets, the normalized contrast evaluation metric, and the DeepInfrared toolkit for detection. We conduct extensive ablation studies to evaluate the components of OSCAR and compare its performance to state-of-the-art model-driven and data-driven methods on the SIRST-V2 benchmark. Our results demonstrate that a top-down cascade refinement framework can improve the accuracy of infrared small target detection without sacrificing efficiency. The DeepInfrared toolkit, dataset, and trained models are available at https://github.com/YimianDai/open-deepinfrared to advance further research in this field.

I. INTRODUCTION

Infrared small target detection is challenged by weak target characteristics, localization sensitivity, label-assignment failures, and limited real-world data. The paper addresses these issues with a benchmark, ASPB label assignment, and OSCAR’s top-down cascade refinement.

  • Challenges: Infrared small targets are difficult to detect because tiny targets can be mislabeled as background, bounding-box perturbations strongly affect IoU, and real-world training data are scarce.These factors create challenges in target assignment, localization, and dataset construction.
  • Benchmark: The paper introduces SIRST-V2, normalized contrast, and DeepInfrared as a benchmark comprising real-world high-resolution data, localization evaluation, and an open-source toolkit.The benchmark is intended to support development of infrared small target detection algorithms.
  • Label assignment: ASPB decouples spatial assignment from target size with scale-adaptive pseudo-boxes and treats target boxes at all scales as positive samples.This addresses cases where tiny targets cover no feature points on a stride-8 feature map.
  • OSCAR: OSCAR uses feature-pyramid predictions as soft region proposals in a top-down, coarse-to-fine cascade for bounding-box refinement while retaining one-stage efficiency.The refinement head also uses cross-head modulation and a normalized contrast branch to suppress low-quality predictions.
  • Outcome: The proposed framework combines benchmark resources and cascade refinement to improve infrared small target detection without sacrificing the efficiency of one-stage methods.The paper presents OSCAR as a comprehensive solution to the task’s data, evaluation, assignment, and localization challenges.

A. Infrared Small Target Detection

Infrared small target detection has been approached through model-driven enhancement or suppression methods and supervised learning models. OSCAR reframes the task as bounding-box regression and applies top-down cascade refinement rather than conventional scale-based divide-and-conquer processing.

  • Traditional Methods: Traditional methods model infrared detection as target enhancement or background suppression using local contrast or sparse-plus-low-rank assumptions.These approaches exploit outlier behavior or background self-correlation to separate targets from backgrounds.
  • Traditional Methods: Traditional methods can produce false positives and miss low-contrast targets because distractors resemble targets and contrast values are not semantically normalized.Their reliance on grayscale values or handcrafted low-level features limits discrimination in complicated scenarios.
  • Supervised Methods: Supervised CNN methods improve performance but semantic-segmentation and image-translation formulations require expensive high-resolution feature maps to recover pixel-level details.The paper instead formulates detection as bounding-box regression, which it describes as less affected by labeling ambiguity.
  • Two-Stage Methods: Two-stage detectors generate proposals before classification, whereas OSCAR performs soft region proposals on dense predictions and remains a one-stage detector.OSCAR down-weights easy background samples rather than passing a sparse set of candidate boxes to a separate stage.
  • Single-Stage Methods: OSCAR treats multi-scale predictions as progressive target approximations instead of assigning high-level maps to large objects and low-level maps to small objects.The paper presents top-down cascade bounding-box refinement as its distinguishing design choice.

A. SIRST-V2: Pay More Attention to Urban Scenarios

SIRST-V2 broadens real-world infrared small-target evaluation with high-resolution, cluttered scenes, while normalized contrast and mNoCoAP address IoU’s sensitivity to ambiguous target boundaries and centroid localization.

  • SIRST-V2 dataset: SIRST-V2 contains 1024 real-world images, mostly 1280 × 1024, forming a challenging high-resolution benchmark.The dataset is presented as the largest and highest-resolution single-frame infrared small-target dataset known to the authors.
  • Urban scenarios: Urban scenes add cranes, streetlights, and similar background distractors that require high-level semantic understanding to distinguish targets.These cases exclude many simple backgrounds and challenge methods based mainly on saliency or low-rank and sparse decomposition.
  • Localization evaluation: IoU is unreliable for tiny targets because small predicted- or ground-truth-box perturbations can sharply change the metric despite similar target centroids.Long-range imaging also makes target boundaries ambiguous, allowing multiple plausible ground-truth boxes.
  • Normalized contrast: Normalized contrast combines local contrast, Gaussian center preference, and coarse-label normalization into a representation less sensitive to bounding-box disturbances.The resulting map is intended to reflect target location while accommodating target shape, appearance, and label ambiguity.
  • mNoCoAP: mNoCoAP replaces IoU in the true-positive criterion with predicted NoCo and averages nine centroid-precision levels from δ = 0.1 to 0.9.The metric is designed to evaluate localization quality across detection paradigms using centroid-focused precision.

C. DeepInfrared Toolkit: An Open-Source Benchmark

DeepInfrared provides an open-source platform for reproducible infrared small-target experiments, while ASPB addresses missed positives caused by feature strides smaller than typical target sizes.

  • DeepInfrared toolkit: DeepInfrared supplies trained models, training scripts, logs, flexible architectures, dataset loaders, augmentation pipelines, and task-specific evaluation metrics.Its stated purpose is to support reproducible experiments and comparisons for infrared small-target detection.
  • Label-assignment problem: Infrared targets commonly span 2 × 2 to 12 × 12 pixels, so a stride-8 feature map may contain no feature point inside a target box.Such targets can be mislabeled as background, leaving a substantial portion of objects at a given scale untrained.
  • All-Scale Pseudo-Box: ASPB decouples target size from spatial assignment by treating targets as positive at all scales and replacing undersized boxes with scale-adaptive pseudo-boxes.For targets with h × w < s^2, the pseudo-box uses the same center and a p × p size with p ≥ s.
  • Pseudo-box trade-off: When p = s, every tiny target receives at least one positive feature point; larger p values add neighboring positives but increase low-quality predictions.The pseudo-box therefore trades improved assignment coverage against prediction quality and class-imbalance effects.

B. Top-Down Cascade Refinement Head

OSCAR combines high-level soft proposals with low-level refinement to address sample imbalance and localization errors in one-stage infrared small-target detection. Top-down score modulation transfers coarse-layer confidence to fine-layer predictions in a single forward pass.

  • Design motivation: High-level feature maps reduce negative samples, while low-level maps provide the spatial resolution needed for precise bounding-box regression.These requirements conflict across pyramid scales, making a simple separation of classification and localization insufficient.
  • Head structure: The OSCAR Head uses a high-level soft proposal head and a low-level refinement head.The proposal head predicts coarse classification scores and boxes that initialize refinement at the lower level.
  • Score modulation: Top-down score modulation uses more accurate coarse-layer scores to modulate less accurate fine-layer scores.The mechanism is implemented through feature transformation, up-sampling, and element-wise modulation before fine-layer prediction.
  • Efficiency: Unlike serialized region proposals with feature cropping and ROI Pooling, OSCAR performs soft proposal processing in one forward pass.This design supports parallel processing while retaining proposal-like modulation.

2) Top-Down Cascaded Regression:

OSCAR performs cascaded coarse-to-fine regression by using high-level predictions as learnable anchors for low-level refinement. Its feature-pyramid predictions approximate cascaded soft region proposals for the same infrared small target.

  • Cascaded regression: A high-level layer first predicts coarse target locations and sizes, and the fine-level layer further refines them.The coarse prediction serves as a learnable anchor for improving regression accuracy on fine feature maps.
  • Feature fusion: Coarse boxes are up-sampled and combined with fine-level features before precise object locations and sizes are predicted.The refinement head receives both coarse information and fine-resolution features.
  • Architecture: OSCAR reformulates multi-level feature-pyramid predictions as coarse-to-fine approximations toward the same infrared small target.The architecture is illustrated using feature-map down-sampling ratios s = 4, 8, 16, 32 relative to the input image.
  • Dynamic anchors: The coarse boxes can be viewed as dynamic anchors because concatenated up-sampled features support subsequent refinement.The concatenation operation combines up-sampled and low-level features for the refined prediction.

C. Normalized Contrast Prediction

The normalized contrast branch suppresses low-quality boxes introduced by broad pseudo-box-based assignment. During inference, normalized contrast is combined with classification scores so boxes distant from target centers receive lower final scores.

  • Motivation: ASPB improves class imbalance but can generate low-quality boxes from positive feature points far from target centers.The normalized contrast branch is added to predict localization quality for detected boxes.
  • Localization quality: Unlike centerness, normalized contrast is computed from the original input image and remains applicable when targets are smaller than the feature-map stride.It acts as a softened semantic segmentation mask for localization quality.
  • Training objective: The overall loss combines focal classification loss, IoU regression loss, and quality focal loss.The supplied loss description also defines Npos and uses λ = 1 as the regression-loss balance weight.
  • Inference: At inference, normalized contrast is combined with two-level classification scores, lowering scores for boxes far from target centers.This makes non-maximum suppression more likely to remove low-quality detections.

V. EXPERIMENTS

The experiments evaluate OSCAR against model-driven and data-driven infrared small-target detectors and analyze pseudo-box assignment. The ablation evidence shows that pseudo-box assignment improves AP, while the chosen pseudo-box size balances positive-sample coverage against low-quality detections.

  • Experimental settings: The evaluation compares OSCAR with model-driven methods including FKRW, LCM, MPCM, WLDM, IPI, NIPPS, RIPT, and SMSL.Table I provides the detailed hyper-parameter settings used for the model-driven comparisons.
  • Experimental settings: The broader comparison includes ISTDU, ACMNet, ALCNet, FCOS, Faster R-CNN, QueryDet, and RFLA under the DeepInfrared toolkit.Default hyper-parameters and a standard 1x learning schedule were used for these comparisons.
  • Experimental questions: The ablation investigates how center-based label assignment mislabels tiny targets and how pseudo-box size affects detector performance.These experiments address both the impact of target mislabeling and the choice of pseudo-box size.
  • Pseudo-box assignment: 75.6% AP is achieved with pseudo-box spatial assignment versus 71.9% AP using the original center-based assignment.The ablation uses a simple anchor-free detector baseline and attributes the gap to reduced target mislabeling.
  • Pseudo-box assignment: Detection performance rises as pseudo-box size increases from p = s, but overly large pseudo-boxes introduce low-quality detections.The study selects p = 1.5s as a suitable size for subsequent experiments.

2) Reasonableness of Top-Down Refinement:

The experiments separate classification from localization to show why top-down refinement is beneficial. Classification improves with lower-resolution maps, whereas bounding-box regression requires higher-resolution features.

  • Resolution effects: Oracle-localized SimpleGrid improves as feature-map resolution decreases, while the baseline worsens as down-sampling increases.SimpleGrid isolates classification; the baseline performs both classification and bounding-box regression.
  • Ablation design: The ablation architecture uses separate high-level classification and low-level regression branches to inspect these roles.The SimpleGrid setup adds oracle localization to evaluate classification independently.
  • Resolution effects: Bounding-box regression, not classification, requires high-resolution feature maps.Classification accuracy drops with larger feature maps because of severe class imbalance.
  • Top-down refinement: High-level layers classify feature points more accurately and can modulate lower-level features for refinement.Low-resolution maps are sufficient when only target presence or absence must be determined.

3) Necessity of Joint Classification and Regression:

The ablations show that classification and regression should remain jointly coordinated rather than simply assigned to separate layers. OSCAR combines top-down score modulation and cascaded regression, with both improving the baseline together.

  • Joint prediction: Decoupling classification and localization reduces final detection performance from 75.6% to 55.7%.The authors attribute the decline to redundant low-level points producing low-quality boxes that rank highly.
  • Cascade refinement: Combining TDCR and TDSM improves the baseline from 75.6% to 79.2%.TDSM suppresses target-like background interference, while TDCR improves final box accuracy through multi-stage regression.
  • Normalized contrast: The normalized contrast branch suppresses pseudo-box-induced false alarms and improves performance from 79.2% to 80.3%.It better matches the metric’s target-centroid annotation than centerness.
  • Efficiency: A shallower prediction layer improves detection but requires 2–3 times more computation.Reducing head channels can preserve efficiency while improving performance: 83.52% versus 82.77% at 25.36 versus 25.55 GFLOPS.
  • Efficiency: ResNet-18 is sufficient because increasing network depth decreases detection performance under the same parameter settings.The experiments therefore emphasize accurate regression over a more powerful backbone.

C. Comparison with State-of-the-Arts

On SIRST-V2, data-driven methods outperform traditional methods in detection performance and speed, while bounding-box regression outperforms semantic segmentation. OSCAR further exceeds competing deep detectors, including FCOS.

  • Overall comparison: Data-driven methods outperform model-driven methods in both detection performance and detection speed.Traditional methods require iterative prediction, whereas deep networks use a single forward pass.
  • Overall comparison: Data-driven methods maintain linear computational growth with image size, unlike traditional methods’ exponential growth.This difference becomes increasingly relevant for high-resolution infrared images.
  • Deep-learning methods: Semantic segmentation approaches are outperformed by all bounding-box regression methods.The paper attributes this to pixel-label uncertainty caused by long-distance imaging and the task’s centroid-localization objective.
  • Deep-learning methods: OSCAR outperforms FCOS by 7.2%.The paper links this gain to regressing target coordinates in a two-step cascade.
  • Deep-learning methods: OSCAR also outperforms RFLA and QueryDet, whose assumed object scales exceed the infrared targets studied here.OSCAR targets objects smaller than 12×12, mostly 3×3 to 5×5, whereas RFLA is designed for 16×16 to 32×32 objects.

D. Visual Analysis

Visual analyses show that traditional methods can detect real targets but often produce false alarms, especially in urban scenes with target-like distractors. OSCAR is presented as accurate and efficient on SIRST-V2.

  • Visualization scheme: The visualizations encode true positives with red circles, false positives with green circles, and false negatives with yellow circles.Better performance corresponds to more red circles and fewer green and yellow circles.
  • Real-target images: Traditional methods detect real targets but may generate false alarms, with low-rank methods typically outperforming local-contrast methods.IPI avoids false alarms in the shown examples and performs well among traditional methods quantitatively.
  • Background-only and urban scenes: Urban SIRST-V2 scenes contain cranes, street lights, and other background distractions similar in scale to infrared targets.The corresponding experiment evaluates suppression of false alarms through semantic understanding.
  • Overall assessment: The OSCAR network achieves superior SIRST-V2 performance while remaining efficient in detection and inference speed.The authors position it as a reliable baseline for future infrared small-target research.
Loading 2212.08472v2…