PaperlayerView exact revision on arXiv

Source-linked AI summary

Attentional Local Contrast Networks for Infrared Small Target Detection

Yimian Dai, Yiquan Wu, Fei Zhou, Kobus Barnard

arXiv:2012.08573v1cs.CV

TL;DR

Infrared small-target detection is difficult because targets have minimal intrinsic features, while limited data and background distractors challenge existing approaches. ALCNet combines learned discriminative features with local-contrast domain knowledge, accelerated by cyclic shifts and complemented by bottom-up attentional modulation. On SIRST, the paper reports superior performance to purely model-driven and purely data-driven competitors, while noting evaluation sensitivity to boundary errors and target representation.

  • Problem

    Minimal target features, limited training data, distractors, and assumptions in existing methods make single-frame infrared small-target detection challenging.

  • Method

    ALCNet integrates learned deep features with a cyclic-shift local-contrast refinement layer and bottom-up attentional modulation for single-frame detection.

  • Results

    ALCNet significantly outperforms the compared purely model-driven methods and purely data-driven networks on the open SIRST dataset.

  • Takeaways & Limitations

    Combining deep networks with local-contrast domain knowledge and target-preserving cross-layer fusion is reported as promising for infrared small-target detection.

  • Takeaways & Limitations

    Small-target IoU and nIoU evaluation is highly sensitive to one- or two-pixel boundary shifts, and an appropriate target representation remains open.

Abstract

from arXiv · show

To mitigate the issue of minimal intrinsic features for pure data-driven methods, in this paper, we propose a novel model-driven deep network for infrared small target detection, which combines discriminative networks and conventional model-driven methods to make use of both labeled data and the domain knowledge. By designing a feature map cyclic shift scheme, we modularize a conventional local contrast measure method as a depth-wise parameterless nonlinear feature refinement layer in an end-to-end network, which encodes relatively long-range contextual interactions with clear physical interpretability. To highlight and preserve the small target features, we also exploit a bottom-up attentional modulation integrating the smaller scale subtle details of low-level features into high-level features of deeper layers. We conduct detailed ablation studies with varying network depths to empirically verify the effectiveness and efficiency of the design of each component in our network architecture. We also compare the performance of our network against other model-driven methods and deep networks on the open SIRST dataset as well. The results suggest that our network yields a performance boost over its competitors. Our code, trained models, and results are available online.

I. INTRODUCTION

Infrared small targets provide few intrinsic features, while distractors, limited data, and strong prior assumptions challenge both conventional model-driven and purely data-driven methods. ALCNet combines learned discriminative features with local-contrast domain knowledge, attentional cross-layer modulation, and empirical comparisons on SIRST.

  • Motivation: Long imaging distances leave infrared targets only a few pixels wide, with little texture or shape information.
  • Motivation: Model-driven methods rely on assumptions about target sparsity or saliency and background smoothness or correlation, limiting robustness to dim targets and distractors.
  • Motivation: Limited infrared training data and scarce target characteristics make accurate detection challenging for off-the-shelf deep networks, even with the SIRST dataset.
  • Proposed approach: ALCNet combines deep feature learning with model-driven local-contrast mechanisms so labeled data and domain knowledge jointly guide single-frame detection.
  • Proposed approach: The network uses a cyclic-shift local-contrast refinement layer and bottom-up attentional modulation to preserve subtle low-level target details in deeper features.
  • Evaluation: Ablation studies and SIRST comparisons report that ALCNet achieves the best performance against state-of-the-art model-driven and data-driven methods.
  • Relation to prior work: Unlike prior implicit context encoding and fixed raw-feature approaches, the design learns target features while integrating cross-layer information and local contrast.

III. MODULARIZING THE LOCAL CONTRAST PRIOR

The paper converts patch-based local contrast into a network-compatible dilated measure by replacing fixed patches with dilation-controlled neighborhoods. This decouples feature scale from contrast-measure scale and supports multi-scale processing on feature maps.

  • Design goals: The module is designed as a plug-in nonlinear feature-refinement layer that removes the patch-based constraint and measures local contrast efficiently.
  • Patch-based formulation: Traditional methods tie patch size to both feature-extraction scale and contrast-measure scale, enforcing an explicit equality constraint.
  • Dilated formulation: Dilation rate replaces the patch concept as the scale-control parameter, decoupling feature scale from local-contrast scale.
  • Dilated formulation: The dilated formulation uses overlapping receptive fields rather than strictly non-overlapped patches.
  • Local contrast computation: Directional local contrast is computed from an intermediate feature map at a specified position and dilation rate across neighborhood directions.

B. Cyclic Shift Accelerating Scheme

The cyclic-shift scheme accelerates depth-wise local-contrast computation by generating shifted neighborhood maps and evaluating tensor operations jointly. It reduces computation substantially while retaining physically interpretable contrast interactions over larger contexts.

  • Computational bottleneck: Direct depth-wise filtering would require 8(3d)^2HW multiplications and additions per feature map, causing cost to grow rapidly with dilation rate.
  • Acceleration assumption: The acceleration assumes feature-map margins are smooth and similar, supported by correlations in infrared images and suppression of background components by earlier convolutions.
  • Tensor formulation: The method formulates directional contrast in tensor form and computes it jointly across the feature map.
  • Cyclic shift scheme: Cyclic shifts generate neighborhood maps in eight directions, reducing local-difference computation to 8HW subtractions per feature map.
  • Efficiency: The cyclic-shift trick makes MPCM approximately 15% faster, increasing speed from 2.67 FPS to 3.07 FPS.
  • Interpretation: With large dilation rates, the local-contrast operation breaks the limited effective receptive field and encodes relatively long-range contextual interactions.

C. Multi-Scale Local Contrast Measure

The method measures multi-scale local contrast within feature maps and uses bottom-up attentional modulation to fuse fine details into coarse representations. Together, these modules embed local contrast priors while preserving subtle small-target information.

  • Multi-Scale Local Contrast Measure: MLC applies dilated local contrast modules with multiple dilation rates to an intermediate feature map, producing same-layer multi-scale contrast features.The resulting representation has dimensions C×H×W and addresses variation in target scale.
  • Multi-Scale Local Contrast Measure: Scale max-pooling selects the strongest contrast across scales, while end-to-end learning adjusts feature representations to reduce false alarms from background distractors.The network learns this adjustment from labeled data and the loss function rather than relying on a fixed representation.
  • Bottom-Up Local Attentional Modulation: The attentional weight map has the same shape as the input feature maps, enabling element-wise emphasis across spatial locations and channels.This allows BLAM to remain dynamically aware of subtle infrared small-target details.
  • Bottom-Up Local Attentional Modulation: BLAM dynamically weights high-level features using low-level features, embedding smaller-scale details into high-level coarse feature maps.The fused feature is obtained through cross-layer attentional modulation.
  • Multi-Scale Local Contrast Measure: Replacing the BLAM inputs with MLC features produces fused multiscale local-contrast feature maps across layers.The cross-layer fusion combines MLC(X) and MLC(Y) through element-wise modulation.

B. Network Architecture

ALCNet uses a modified ResNet-20-style backbone with limited downsampling and integrates MLC and BLAM modules into a feature pyramid architecture. The design restores full spatial resolution by iteratively fusing coarse and fine feature maps.

  • Network Architecture: The backbone downsamples the input only twice, preserving higher spatial resolution for detecting small infrared targets.Downsampling occurs at stage2_1 and stage3_1 with stride 2, rather than the conventional 32-fold reduction.
  • Network Architecture: The backbone is scaled by changing the block number b in each stage to study the relationship between network depth and performance.When b = 3, the architecture is the standard ResNet-20 backbone.
  • Network Architecture: BLAM iteratively fuses coarse high-layer feature maps with fine low-layer feature maps to recover full spatial resolution at the output.This fusion supports high-resolution prediction for small targets.
  • Network Architecture: ALCNet incorporates MLC and BLAM modules into a feature pyramid network, with blue lines denoting channel transformation and red lines denoting upsampling.The architecture combines same-layer multiscale contrast processing with cross-layer bottom-up modulation.

C. Problem Formulation and Optimization

The optimization formulation uses Soft-IoU loss to address the severe class imbalance between infrared small targets and background. The prediction map is produced by the proposed network and trained against labeled masks.

  • Problem Formulation and Optimization: Soft-IoU loss is adopted for the highly unbalanced infrared small-target segmentation task.It is selected specifically to handle the class imbalance between target and background pixels.
  • Problem Formulation and Optimization: The prediction score map is p = σ(M2LC(f, Θ)) ∈ R^H×W, while y ∈ R^H×W denotes the labeled mask for infrared image f.Θ denotes the weights of ALCNet.
  • Problem Formulation and Optimization: Training learns Θ by minimizing the total loss over N training samples.The formulation defines the network output and labels before specifying optimization over the dataset.
  • Problem Formulation and Optimization: Optimization uses AdaGrad with learning rate 0.1 for 400 epochs, weight decay 10^-4, and batch size 10.Weights are initialized using the strategy described by He et al.

V. EXPERIMENTS

Experiments evaluate ALCNet on SIRST through baseline comparisons and ablations covering local-contrast encoding, module design, network depth, and computational budgets. The dataset contains dim targets and substantial background clutter, making saliency-only methods insufficient.

  • Ablation Studies: Ablation studies examine local-contrast encoding, multiscale contrast measurement, bottom-up attentional modulation, and behavior under different parameter and computational budgets.The experiments are organized around questions about discriminative feature learning, dynamic multiscale contrast, and network design.
  • Dataset and Experimental Design: SIRST contains 427 images and 480 target instances, split approximately into 50% training, 20% validation, and 30% testing.The images come from hundreds of real-world videos and cover different scenarios.
  • Dataset and Experimental Design: Only 35% of SIRST targets contain the brightest pixel in the image, so raw-image thresholding and purely saliency-based assumptions are unreliable.Many targets are extremely dim and embedded in complex, heavily cluttered backgrounds.
  • Baseline Comparisons: The evaluation compares ALCNet with data-driven networks and non-learning model-driven methods on the public SIRST dataset.Data-driven baselines include FPN, SK-FPN, GAU-FPN, and TBC-Net; model-driven comparisons include SMSL, FKRW, and MPC-based methods.

2) Implementation Details:

The evaluation uses IoU, ROC curves, and nIoU on SIRST, with ablation architectures comparing local-contrast extraction and cross-layer fusion schemes. The DLC module adds no parameters, while DLC-FPN consistently outperforms FPN across network depths.

  • Evaluation metrics: IoU, ROC curves, and nIoU are used to evaluate ALCNet, while background-suppression metrics are unsuitable for its binary outputs.nIoU is designed for SIRST to balance model-driven and data-driven methods.
  • Ablation architectures: The ablation architectures vary same-layer multi-scale local contrast extraction and cross-layer feature fusion schemes.Plain denotes architectures without a local contrast module; BGAM and TLAM provide alternative attentional modulation designs.
  • Local contrast prior: DLC-FPN consistently and significantly outperforms FPN across network depths, despite using no additional parameters for its DLC module.DLC-FPN with b = 3 performs approximately like FPN with b = 4.
  • Experimental design: The experiments compare architectures by progressively varying network depth and assess their performance using IoU and nIoU.The reported ablations include FPN, DLC-FPN, MLC-FPN, PlainFCN, and ALCNet.
  • Local contrast prior: The local contrast module acts as a physically interpretable spatial-attention mechanism that encodes relatively long-range contextual interactions in a depth-wise manner.It shifts recognition from appearance-based cues toward local contrast while helping suppress background clutter.

2) Impact of Multi-scale Local Contrast Integration (Q2):

Multi-scale local contrast improves robustness to target-scale variation, while cross-layer fusion is vital for infrared small targets. ALCNet’s BLAM fusion achieves similar IoU and better nIoU than MLC-FPN with about half the parameters.

  • Multi-scale local contrast: A poorly chosen dilation rate can make DLC-FPN perform worse than FPN, especially on nIoU.This sensitivity motivates covering multiple dilation rates rather than relying on a single-scale measure.
  • Multi-scale local contrast: Multi-scale MLC-FPN consistently outperforms single-scale DLC-FPN by aggregating multiple dilation rates on the same feature layer.The reported MLC-FPN dilation rates are 13 and 17, improving robustness against target-scale variation.
  • Cross-layer feature fusion: Cross-layer feature integration is vital for infrared small targets, as PlainFCN without skip-connection fusion performs significantly worse.Low-level features provide localization detail, while high-level features contribute semantic information for ambiguous cases.
  • Cross-layer feature fusion: ALCNet’s BLAM fusion performs similarly to MLC-FPN on IoU and better on nIoU with only around 50% of its parameters.The comparison supports more sophisticated cross-layer fusion over simple element-wise addition.
  • Sensitivity analysis: Figure 8 evaluates DLC-FPN across dilation rates and network depths using IoU and nIoU, exposing sensitivity to the dilation-rate hyper-parameter.The comparison with FPN shows that DLC-FPN is generally better but not uniformly so under poor dilation choices.

3) Impact of Cross-layer Fusion Manners (Q3):

ALCNet’s bottom-up local modulation outperforms alternative fusion directions and competing networks, while comparisons across methods show stronger detection with efficient inference. Remaining errors mainly reflect boundary ambiguity and a small number of missed detections.

  • Fusion comparisons: ALCNet’s bottom-up local modulation outperforms both global-context aggregation and top-down semantic modulation under comparable budgets.Fine local information guides high-level features rather than the reverse direction.
  • Network comparisons: ALCNet achieves significantly better IoU and nIoU than FPN, SK-FPN, and GAU-FPN across tested network depths.With b = 2, it also outperforms baseline networks with b = 4.
  • State-of-the-art comparisons: ALCNet ranks best among learning and non-learning methods, with higher detection and lower false-alarm rates than GAU-FPN.Deep networks also have faster post-training inference than conventional saliency or low-rank plus sparse decomposition methods.
  • Error diagnosis: The reported failure cases include only two missed detections, while many segmentation errors arise from target-boundary discrepancies.For a 2 × 2 target, a one-pixel shift can induce about 50% error in IoU and nIoU.
  • Limitations: The segmentation formulation remains a limitation because boundary-sensitive IoU metrics may not reflect true detection performance for tiny targets.The paper identifies target representation and evaluation as an open question.

VI. CONCLUSION

The paper concludes that ALCNet benefits from integrating local-contrast domain knowledge with target-preserving cross-layer fusion. Its reported comparisons on SIRST show gains over purely model-driven and purely data-driven alternatives.

  • VI. CONCLUSION: ALCNet extracts and fuses local-contrast feature maps within layers and across layers to transplant domain knowledge into the network.The approach uses two stages of feature integration.
  • VI. CONCLUSION: Bottom-up local attentional modulation embeds subtle low-level details into high-level layers to preserve small targets in coarse features.This is presented as the target-preserving cross-layer fusion scheme.
  • VI. CONCLUSION: ALCNet significantly outperforms compared purely model-driven methods and purely data-driven networks on the open SIRST dataset.The conclusion identifies combining deep networks with domain knowledge as worthy of further research.
Loading 2012.08573v1…