PaperlayerView exact revision on arXiv

Source-linked AI summary

SSPNet: Scale Selection Pyramid Network for Tiny Person Detection from UAV Images

Mingbo Hong, Shuiwang Li, Yuchao Yang, Feiyu Zhu, Qijun Zhao, Li Lu

arXiv:2107.01548v1cs.CV

TL;DR

Detecting people in UAV images is challenging because they are often tiny and FPN’s feature fusion can create inconsistent gradients across layers. SSPNet combines scale-aware attention, selective feature sharing, and weighted negative sampling; it outperforms SOTA detectors on TinyPerson.

  • Problem

    Tiny people in UAV imagery are difficult to detect, and FPN’s direct feature addition can mix deep-layer negative features with shallow-layer positive features, causing inconsistent gradients.

    Source: §I · p. 1

  • Method

    SSPNet uses CAM heatmaps to identify scale-specific positive regions, SEM to emphasize those objects, SSM to guide adjacent-layer feature sharing, and WNS to prioritize representative samples.

    Source: §I · pp. 1–2

  • Results

    SSPNet outperforms SOTA detectors on the TinyPerson benchmark.

    Source: §I · p. 2

  • Takeaways & Limitations

    WNS guides the detector to focus on representative samples rather than overlooking them among thousands of easy samples.

    Source: §I · p. 2

Abstract

from arXiv · show

With the increasing demand for search and rescue, it is highly demanded to detect objects of interest in large-scale images captured by Unmanned Aerial Vehicles (UAVs), which is quite challenging due to extremely small scales of objects. Most existing methods employed Feature Pyramid Network (FPN) to enrich shallow layers' features by combing deep layers' contextual features. However, under the limitation of the inconsistency in gradient computation across different layers, the shallow layers in FPN are not fully exploited to detect tiny objects. In this paper, we propose a Scale Selection Pyramid network (SSPNet) for tiny person detection, which consists of three components: Context Attention Module (CAM), Scale Enhancement Module (SEM), and Scale Selection Module (SSM). CAM takes account of context information to produce hierarchical attention heatmaps. SEM highlights features of specific scales at different layers, leading the detector to focus on objects of specific scales instead of vast backgrounds. SSM exploits adjacent layers' relationships to fulfill suitable feature sharing between deep layers and shallow layers, thereby avoiding the inconsistency in gradient computation across different layers. Besides, we propose a Weighted Negative Sampling (WNS) strategy to guide the detector to select more representative samples. Experiments on the TinyPerson benchmark show that our method outperforms other state-of-the-art (SOTA) detectors.

I. INTRODUCTION

Tiny people in UAV images are difficult to detect, and FPN’s direct feature addition can mix deep-layer negative supervision with shallow-layer positive features. SSPNet uses scale-aware attention and selective feature sharing to address this gradient inconsistency, and it outperforms SOTA detectors on TinyPerson.

I. INTRODUCTION

  • UAV search images pose a detection challenge because people are often tiny, have low signal-to-noise ratios, and are easily contaminated by backgrounds.Source: §I · p. 1
  • Tiny-person anchors often fail to match at deep layers, so direct FPN addition mixes their negative features with shallow-layer positive features and creates inconsistent gradients.This can weaken deep-layer representations and burden shallow layers.Source: §I · p. 1
  • SSPNet’s CAM generates hierarchical attention heatmaps, while SEM emphasizes objects of specific scales and SSM uses adjacent heatmap intersections to guide feature sharing.The paper also proposes WNS to prioritize representative samples and reduce false alarms.Source: §I · pp. 1–2
  • SSPNet outperforms SOTA detectors on the TinyPerson benchmark.Source: §I · p. 2

II. METHODOLOGY

SSPNet includes a Context Attention Module (CAM), which builds hierarchical attention heatmaps from multi-scale context features. Its supervised heatmaps identify object-scale ranges that can be assigned as positives at different layers.

II. METHODOLOGY

  • SSPNet’s framework includes CAM, SEM, and SSM.Source: §II · p. 2

A. Context Attention Module

  • CAM upsamples backbone features to a common shape, concatenates them, applies ASPP to gather multi-scale context, and uses an activation gate to generate hierarchical attention heatmaps.The gate uses multiple 3×3 convolutions with different strides and sigmoid activation.Source: A. Context Attention Module · p. 2
  • Supervised attention heatmaps associate with objects matched by anchors at different layers, indicating which object scales can be positive at each layer.The heatmaps show different scale ranges across layers; objects unmatched by a layer’s anchors are treated as background.Source: A. Context Attention Module · p. 2

B. Scale Enhancement Module

SEM enhances scale-specific object cues using layerwise attention preferences, while SSM routes suitable features between adjacent layers to limit gradient inconsistency.

B. Scale Enhancement Module

  • SEM uses layer-specific attention heatmaps to enhance cues for objects at the scales preferred by each layer.Its residual connection preserves features around objects, where context may aid detection.Source: B. Scale Enhancement Module · p. 2

C. Scale Selection Module

  • SSM uses intersections of adjacent-layer attention maps to select deep-layer features suitable for delivery to shallow layers.The selected features are optimized toward the same class; other features are weakened to reduce gradient inconsistency.Source: C. Scale Selection Module · p. 2
  • The merged map at layer k and the preceding residual block's output contribute to the feature map routed at layer k−1.The equation description identifies nearest upsampling and attention-map intersection as part of this routing.Source: C. Scale Selection Module · p. 2

D. Weighted Negative Sampling

WNS combines confidence and IoF to guide negative-sample selection, while the loss design addresses background imbalance and attention-map learning.

D. Weighted Negative Sampling

  • WNS targets noisy UAV backgrounds and partial occlusion, which can leave person parts annotated as complete individuals.The authors motivate representative-sample selection as a way to enhance generalization.Source: D. Weighted Negative Sampling · p. 2
  • WNS fuses detection confidence and maximum IoF to adjust each sample's selection probability.The score is intended to favor representative samples in settings with noisy backgrounds and partially occluded, incompletely annotated people.Source: D. Weighted Negative Sampling · pp. 2–3

E. Loss Function

  • Both proposal and head losses use smooth L1 for box regression; proposal classification uses BCE, whereas head classification uses cross-entropy.The classification and regression losses are normalized by minibatch size and number of box locations, respectively.Source: E. Loss Function · p. 3
  • Dice loss prioritizes foreground attention, BCE supplies gradients when attention maps do not intersect, and OHEM uses a 1:3 positive-to-negative ratio.BCE learns poorly classified negatives, while dice loss learns class distribution to address imbalance.Source: E. Loss Function · p. 3

A. Experimental Setup and Evaluation Metric

Experiments use TinyPerson and evaluate detection with AP and MR, including AP across three tiny-object size intervals. The setup also specifies the training and inference image counts and the patch-based training procedure.

A. Experimental Setup and Evaluation Metric

B. Comparison to State-of-the-arts

On TinyPerson, SSPNet improves three detector architectures over their implemented baselines. The ablations report gains from SEM, SSM, and WNS, while attention loss improves on single BCE loss.

B. Comparison to State-of-the-arts

  • RetinaNet, Faster R-CNN, and Cascade R-CNN each improve with SSPNet over stronger implemented baselines using augmentation and multi-scale training.The comparison reports performance in AP and MR.Source: B. Comparison to State-of-the-arts · p. 4

C. Ablation Study

  • SEM improves AP tiny50 by 1.07% over the baseline while focusing on objects of specific scales rather than vast backgrounds.Source: C. Ablation Study · p. 4
  • SSM improves the baseline’s AP tiny50 by 1.2% by delivering suitable features from deep layers to shallow layers.Source: C. Ablation Study · p. 5
  • WNS improves baseline AP tiny50 by 0.35% without adding training parameters, guiding attention toward human parts and backgrounds easily mistaken for individuals.Source: C. Ablation Study · p. 5
  • Attention loss improves performance over single BCE loss by addressing its focus on pixel-level classification error.Source: C. Ablation Study · p. 5

D. Influence of Balanced Coefficient

The balanced coefficient performs best at 0.6, while relying on confidence or IoF alone performs worse. The attention-loss heatmap comparison shows clearer foreground boundaries than single BCE loss, while retaining context in non-object areas.

D. Influence of Balanced Coefficient

E. Effectiveness of Attention loss

  • Compared with single BCE loss, attention-loss heatmaps show clearer tiny-object boundaries, while retaining some non-object areas as context rather than becoming binary maps.The authors attribute better foreground extraction to dice loss.Source: E. Effectiveness of Attention loss · p. 5, Figure 5 · p. 6

F. Visual Analysis

Visual examples show SSPNet detecting people in varied sea and land scenes, though some people may be missed under occlusion.

F. Visual Analysis

  • In sea scenes, SSPNet recognizes and locates people on surfboards despite varied postures and cases where only heads are visible in backlight.Source: F. Visual Analysis · p. 6
  • In land scenes with scale variation, crowds, and cluttered backgrounds, most people are recognized and located.Source: F. Visual Analysis · p. 6
  • Some people may be missed due to occlusion; the authors suggest more extensive training data containing crowded scenarios could improve these problems.Source: F. Visual Analysis · p. 6

G. Mathematical Explanation

The derivation illustrates how cross-layer label assignments can create conflicting gradients, then explains how SSM uses attention weighting to address that inconsistency.

G. Mathematical Explanation

  • At location (i, j), a fifth-layer position positive in layer 2 but background elsewhere receives conflicting optimization directions.The layer-2 feature is optimized toward a positive sample, while the fifth-layer feature is optimized toward a negative sample.Source: G. Mathematical Explanation · p. 6
  • SSM adjusts gradient inconsistency with attention coefficient ψ, using resized merged maps between layers.The merged map is resized from layer m to layer l at position (i, j).Source: G. Mathematical Explanation · p. 6
  • The derivation treats the interpolation gradient as usually fixed and approximately 1, simplifying Equation (9).Source: G. Mathematical Explanation · p. 6

IV. CONCLUSION

SSPNet addresses inconsistent gradient computation in FPN-based tiny-person detection with attention-guided feature handling and representative negative sampling. The authors identify extension to a bidirectional feature pyramid network as future work.

IV. CONCLUSION

  • SSPNet targets inconsistent gradients in FPN-based tiny-person detection, which weaken shallow-layer representation.Source: §IV · p. 6
  • CAM produces layer-specific attention heatmaps with specific scale ranges under attention-loss supervision.Source: §IV · p. 6
  • SEM strengthens cues for objects at specific scales, while SSM uses intersection heatmaps to control deep-to-shallow feature sharing.Source: §IV · p. 6
  • Weighted Negative Sampling selects representative samples to support training on small datasets.Source: §IV · p. 6
  • Future work will extend SSPNet to a bidirectional feature pyramid network.Source: §IV · p. 6
Loading 2107.01548v1…