PaperlayerView exact revision on arXiv

Source-linked AI summary

Scale Match for Tiny Person Detection

Xuehui Yu, Yuqi Gong, Nan Jiang, Qixiang Ye, Zhenjun Han

arXiv:1912.10664v1cs.CV

TL;DR

Tiny-person detection in large-scale, long-distance images is underexplored because extremely small objects challenge feature representation and complex backgrounds increase false alarms. The paper introduces the TinyPerson benchmark and Scale Match, which aligns object scales across pre-training and detector-training datasets. Scale Match improves performance over FPN by 5%, while TinyPerson exposes challenging real-world detection conditions.

  • Problem

    Existing benchmarks and detectors are not well suited to tiny persons below 20 pixels in large-scale, long-distance images with massive backgrounds.

  • Method

    The paper introduces TinyPerson and Scale Match, which aligns object scales between network pre-training and detector-training datasets.

  • Results

    5%: Scale Match improves detection performance over the state-of-the-art FPN detector.

  • Takeaways & Limitations

    TinyPerson provides a benchmark for long-distance person detection with massive backgrounds, and Scale Match is presented as a plug-and-play block for object-scale processing.

  • Takeaways & Limitations

    The experiments use MS COCO as the extra dataset for Scale Match's scale transformation T.

Abstract

from arXiv · show

Visual object detection has achieved unprecedented ad-vance with the rise of deep convolutional neural networks.However, detecting tiny objects (for example tiny per-sons less than 20 pixels) in large-scale images remainsnot well investigated. The extremely small objects raisea grand challenge about feature representation while themassive and complex backgrounds aggregate the risk offalse alarms. In this paper, we introduce a new benchmark,referred to as TinyPerson, opening up a promising directionfor tiny object detection in a long distance and with mas-sive backgrounds. We experimentally find that the scale mis-match between the dataset for network pre-training and thedataset for detector learning could deteriorate the featurerepresentation and the detectors. Accordingly, we proposea simple yet effective Scale Match approach to align theobject scales between the two datasets for favorable tiny-object representation. Experiments show the significantperformance gain of our proposed approach over state-of-the-art detectors, and the challenging aspects of TinyPersonrelated to real-world scenarios. The TinyPerson benchmarkand the code for our approach will be publicly available(https://github.com/ucas-vg/TinyBenchmark).(Attention: evaluation rules of AP have updated in benchmark after this paper accepted, So this paper use old rules. we will keep old rules of AP in benchmark, but we recommand the new and we will use the new in latter research.)

1. Introduction

Tiny-person detection remains underexplored because existing benchmarks and detectors are poorly suited to very small people in large, complex scenes. The paper introduces TinyPerson and Scale Match to address this benchmark and representation gap.

  • Tiny-person detection is important for applications including surveillance, driving assistance, mobile robotics, and maritime quick rescue, yet remains far from well explored.
  • Existing pedestrian benchmarks mainly depict near- or middle-distance people and do not apply well to long-distance scenes across large areas.The paper gives marine search and rescue from a helicopter platform as an example.
  • Tiny objects are difficult because their small size and low signal-to-noise ratio blur them into complex backgrounds and deteriorate feature representation.
  • Scale Match aligns object scales between network pre-training and detector-training datasets to improve tiny-object feature representation.The approach is presented as simple and effective, using scale alignment to better utilize tiny-scale information.
  • TinyPerson is introduced as a benchmark for person detection at long distance with massive backgrounds, including maritime quick rescue scenarios.The benchmark includes publicly released train/validation annotations and an online evaluation benchmark.
  • 5%: Scale Match improves detection performance over the state-of-the-art FPN detector with a significant margin.

2. Related Work

Related work covers conventional pedestrian datasets, small-target benchmarks, CNN-based detection architectures, and specialized tiny-object detectors. TinyPerson differs through its very low-resolution people and maritime and beach scenes.

  • Traditional pedestrian datasets pursue robust algorithms using larger-capacity, richer-scene, and better-annotated benchmarks, but generally contain relatively high-resolution pedestrians.
  • TinyPerson represents people mainly below 20 pixels in maritime and beach scenes, supporting generalization to long-distance human-target detection and rescue scenarios.
  • WiderFace has a similar absolute-size distribution to TinyPerson but focuses on faces with higher resolution and larger relative sizes, while TinyNet is not publicly available.
  • CNN-based detection developed from region-proposal architectures such as R-CNN toward more efficient single-scale and single-stage approaches.
  • Single-stage detectors accelerate processing but show a performance drop, especially for tiny objects, motivating frameworks specialized for tiny-object detection.

3. Tiny Person Benchmark

TinyPerson is a benchmark for long-distance person detection with tiny objects, diverse annotations, dense scenes, and maritime backgrounds. Its experiments show that both absolute and relative object scale create distinct detection challenges.

  • Benchmark definition: TinyPerson defines object size as the square root of bounding-box area, using absolute size by default in subsequent discussion.Bounding boxes are represented by coordinates, width, and height; image dimensions support relative-size calculation.
  • Dataset construction: TinyPerson collects Internet videos sampled every 50 frames, removes repetitive images, and manually annotates 72,651 objects.The dataset focuses on seaside persons, includes varied poses and viewpoints, and contains images with more than 200 persons.
  • Dataset properties: TinyPerson differs from conventional pedestrian datasets through much smaller relative object sizes, large aspect-ratio variation, dense scenes, and long-distance seaside imagery.The benchmark is intended for person detection and can also support tasks such as counting.
  • Annotations and evaluation: The benchmark represents sea, earth, uncertain, and ignored regions with distinct annotation categories and uses IOD for ignored regions during evaluation.Sea and earth persons are combined into one detection class, while uncertain instances are treated as ignored during training and testing.
  • Absolute-size challenge: 40% lower MRtiny50 for tiny CityPersons than CityPersons demonstrates the severe impact of reducing absolute object size.Tiny CityPersons is created by down-sampling CityPersons by 4×4 so its mean absolute object size matches TinyPerson.
  • Relative-size challenge: Upsampling tiny CityPersons improves MRtiny50 by 29.95 and 16.31 points, whereas the same strategy yields limited gains for TinyPerson.TinyPerson’s smaller relative size introduces more false positives and stronger positive/negative imbalance from massive, complex backgrounds.

4. Tiny Person Detection

Scale Match addresses limited transfer from extra datasets by aligning object-size distributions with the detector-training dataset. It estimates the target distribution, rectifies long-tailed histograms, and resizes extra-dataset images before training.

  • Scale Match: Scale Match aligns object scales between pre-training data and the task-specific detector-training dataset.The approach transforms the extra dataset so its object-size distribution approaches that of the target training set.
  • Scale Match: Applying Scale Match to many extra-dataset targets makes the transformed size distribution approach the detector-training distribution.The framework supports training policies such as joint training or pre-training.
  • Scale Match: The method estimates the target size distribution from the detector-training set using a histogram of object sizes.A randomly sampled training set is used to approximate the full target distribution.
  • Scale Match: For each extra-dataset image, Scale Match samples a target size from the rectified histogram and uniformly within the selected bin.The image and its ground-truth boxes are resized using c = ŝ/s, preserving the image structure while targeting the sampled mean object size.
  • Monotone Scale Match: Monotone Scale Match is introduced because ordinary Scale Match can reverse relative object sizes when sampling target scales.It maps source sizes through a monotone function while preserving size ordering.

5. Experiments

Experiments establish TinyPerson as a demanding benchmark for tiny-person detection and show that matching pre-training object scales to detector-learning scales improves performance. Baseline analyses also reveal strong sensitivity to object size, localization precision, detector architecture, and pre-training data.

  • 5.2. Baseline for TinyPerson Detection: TinyPerson experiments compare representative one-stage, anchor-free, and two-stage detectors, using Faster RCNN-FPN as the baseline.RetinaNet, FCOS, and Faster RCNN-FPN are evaluated, with Adaptive RetinaNet and FreeAnchor also considered.
  • 5.2. Baseline for TinyPerson Detection: Adaptive RetinaNet achieves 10.43% improvement of AP tiny50 over RetinaNet by using P2–P6 FPN levels instead of P3–P7.The adjustment emphasizes lower-level spatial information for tiny objects.
  • 5.2. Baseline for TinyPerson Detection: Two-stage detection shows advantages over one-stage detection on TinyPerson, although FCOS outperforms RetinaNet and Faster RCNN-FPN on larger-scale settings.The comparison indicates that detector rankings change when object sizes become tiny.
  • 5.3. Analysis of Scale Match: Pre-training on MS COCO provides limited improvement because its object-size distribution differs from TinyPerson, motivating COCO100, SM COCO, and MSM COCO comparisons.COCO100 matches TinyPerson’s mean object size, while SM COCO and MSM COCO apply Scale Match transformations during pre-training.
  • 5.3. Analysis of Scale Match: 47.29% AP tiny50 is achieved with MSM COCO pre-training, while Scale Match also produces similar performance gains on Tiny Citypersons.MSM COCO uses Monotone Scale Match, and Tiny Citypersons provides an additional validation dataset.

6. Conclusion

The conclusion presents TinyPerson as a benchmark for detecting persons generally smaller than 20 pixels and identifies scale mismatch as a factor that deteriorates feature representation and detector performance. It proposes Scale Match as an efficient, plug-and-play approach for processing object scales and reusing annotated data.

  • 6. Conclusion: TinyPerson targets tiny-person detection, particularly persons less than 20 pixels in large-scale images, posing a major challenge for existing detectors.The dataset is introduced as a benchmark for this setting.
  • 6. Conclusion: The paper finds that scale mismatch between pre-training and detector-learning data can deteriorate feature representation and detector performance.This observation motivates the proposed approach.
  • 6. Conclusion: Scale Match is presented as an easy, efficient, plug-and-play block that aligns object scales and better utilizes existing annotated data.The paper describes it as applicable to general object detection tasks.
Loading 1912.10664v1…