PaperlayerView exact revision on arXiv

Source-linked AI summary

Vision Meets Drones: A Challenge

Pengfei Zhu, Longyin Wen, Xiao Bian, Haibin Ling, Qinghua Hu

arXiv:1804.07437v2cs.CV

TL;DR

VisDrone2018 addresses the lack of large, diverse public benchmarks for drone-based visual analysis. It constructs a richly annotated dataset spanning four detection and tracking tasks, providing broad real-world coverage for evaluation. Its scope includes 263 video clips, 179,264 frames, 10,209 images, and over 2.5 million annotated instances, while object labeling includes operational category distinctions and visibility-related attributes.

  • Problem

    Drone vision research is constrained by the limited size and scenario coverage of publicly available datasets, leaving thorough evaluation of algorithms open.

  • Method

    The paper constructs VisDrone2018 from diverse drone imagery and video, with rich annotations and four object detection and tracking tasks.

  • Results

    VisDrone2018 provides 263 video clips, 179,264 frames, 10,209 static images, and over 2.5 million annotated object instances.

  • Takeaways & Limitations

    The benchmark supports extensive evaluation and investigation of visual analysis algorithms on drone platforms across diverse real-world scenarios.

  • Takeaways & Limitations

    Human pose determines whether an instance is labeled pedestrian or person in the image-detection task.

Abstract

from arXiv · show

In this paper we present a large-scale visual object detection and tracking benchmark, named VisDrone2018, aiming at advancing visual understanding tasks on the drone platform. The images and video sequences in the benchmark were captured over various urban/suburban areas of 14 different cities across China from north to south. Specifically, VisDrone2018 consists of 263 video clips and 10,209 images (no overlap with video clips) with rich annotations, including object bounding boxes, object categories, occlusion, truncation ratios, etc. With intensive amount of effort, our benchmark has more than 2.5 million annotated instances in 179,264 images/video frames. Being the largest such dataset ever published, the benchmark enables extensive evaluation and investigation of visual analysis algorithms on the drone platform. In particular, we design four popular tasks with the benchmark, including object detection in images, object detection in videos, single object tracking, and multi-object tracking. All these tasks are extremely challenging in the proposed dataset due to factors such as occlusion, large scale and pose variation, and fast motion. We hope the benchmark largely boost the research and development in visual analysis on drone platforms.

1 INTRODUCTION

VisDrone2018 addresses the limited scale and scenario coverage of existing drone datasets with a comprehensive benchmark spanning annotated images, videos, and four detection or tracking tasks.

  • Drone-captured visual algorithms remain difficult to evaluate because publicly available large-scale benchmarks are limited in size and scenarios.
  • VisDrone2018 contains 263 video clips, 179,264 frames, 10,209 static images, and over 2.5 million annotated object instances.
  • The benchmark covers diverse locations, environments, object types, and scene densities, with data collected across 14 cities in China.
  • Its four tasks are object detection in images, object detection in videos, single object tracking, and multi-object tracking.
  • The benchmark provides annotations including object categories, bounding boxes, visibility-related attributes, occlusion, and truncation information.

2 RELATED WORK

Related benchmarks support progress in object detection and tracking, but drone-focused datasets remain comparatively scarce and constrained in scale or scenario coverage.

  • Computer vision benchmarks span generic detection, pedestrian detection, single-object tracking, multi-object tracking, 3D reconstruction, and optical flow.
  • Existing drone datasets include a 1,448-image car-counting dataset from parking lots and campus video sequences covering several object types.
  • Established detection datasets use vehicle-mounted or autonomous-driving imagery, including Caltech, KITTI-D, and other road-scene collections.
  • Single-object tracking datasets provide video sequences with initial target states, while multi-object tracking datasets cover surveillance, autonomous-driving, and multi-camera settings.
  • Table 1 compares current benchmarks and datasets, including the maximum resolution of their included videos or images.

3.1 Dataset Collection

VisDrone2018 was systematically collected as a large, diverse drone image/video dataset, with broad geographic, platform, environmental, and annotation coverage supporting four benchmark tasks.

  • The dataset was assembled to provide a thorough basis for evaluating visual algorithms on drone data.
  • Example figures show static images and video screenshots associated with the benchmark tasks.
  • The collection comprises 263 video clips with 179,264 frames and 10,209 additional static images.
  • Data were captured using multiple drone platforms across 14 Chinese cities, covering varied weather, lighting, and everyday scenarios.
  • The benchmark organizes evaluation for image detection, video detection, single-object tracking, and multi-object tracking through its project website.

3.2 Task 1: Object Detection in Images

Task 1 evaluates detection of predefined object categories in drone images using dense annotations for object identity, occlusion, truncation, and multiple detection metrics.

  • Task 1 uses 10,209 images split into 6,471 training, 548 validation, and 3,190 testing images.
  • The benchmark defines ten evaluated categories, including pedestrians, vehicles, bicycles, and several vehicle subtypes.
  • Annotations include category-specific bounding boxes, occlusion ratios, and truncation ratios for object instances.
  • Objects with truncation ratios larger than 50% are skipped during evaluation.
  • Detection is evaluated with APIoU=0.50:0.05:0.95, APIoU=0.50, APIoU=0.75, and ARmax metrics under detection-count limits.

3.3 Task 2: Object Detection in Videos

Task 2 evaluates object detection in drone videos using annotated bounding boxes, object attributes, and confidence-scored detections across challenging clips. Its primary ranking metric is APIoU=0.50:0.05:0.95.

  • Task 2 provides 96 challenging video clips, including 56 training clips containing 24,201 frames in total.
  • Detectors must produce bounding boxes and real-valued confidences for each object instance in every video frame.The benchmark uses the same object categories as Task 1 and supplies manually annotated bounding boxes, occlusion, and truncation ratios.
  • The benchmark also reports object counts by occlusion degree and category across training, validation, and testing sets.
  • APIoU=0.50:0.05:0.95 is the primary ranking metric, alongside APIoU=0.50, APIoU=0.75, and ARmax values from 1 to 500.

3.4 Task 3: Single Object Tracking

Task 3 is generic, model-free single-object tracking: given an initial target box, an algorithm locates that target through subsequent video frames. The benchmark supplies annotated sequences and ranks methods primarily by success score.

  • Task 3 requires online estimation of a target’s bounding boxes across frames from its initial bounding box in the first frame.
  • The benchmark provides 167 annotated video sequences split into training, validation, and testing sets.The splits contain 86, 11, and 70 sequences, respectively.
  • Tracking targets include pedestrians, cars, buses, and animals, with annotated examples and target statistics provided.
  • Performance uses success and precision scores, with success score serving as the primary ranking metric.Success is measured by the area under the overlap-threshold curve, while precision is the percentage of centers within 20 pixels of ground truth.

3.5 Task 4: Multi-Object Tracking

Task 4 recovers object trajectories in drone videos and separates tracking without prior detections from tracking with prior detections. The two sub-tasks use different evaluation protocols.

  • Task 4 uses the Task 2 video data and defines Task 4A without prior detections and Task 4B with prior detections.
  • The benchmark illustrates tracking data with annotated video frames and reports object counts and occlusion distributions across dataset splits.
  • Task 4A: Task 4A evaluates identity-linked tracklets at IoU thresholds 0.25, 0.50, and 0.75 using mean average precision averaged across object classes and thresholds.
  • Task 4B: Task 4B compares algorithms using the average rank of 10 metrics, including MOTA, IDF1, false alarms, misses, identity switches, fragmentation, and processing speed.

4 CONCLUSION

VisDrone2018 is a large-scale benchmark for drone-based object detection and tracking, built from diverse real-world imagery and video across 14 Chinese cities. It contains over 2.5 million annotated object instances and is intended to support visual-analysis research.

  • VisDrone2018 introduces a large-scale benchmark for object detection and tracking on drone platforms.
  • The dataset spans 14 Chinese cities, diverse drone platforms, and varied real-world environments and object densities.
  • More than 2.5 million object instances were annotated with important attributes after over 6,000 worker hours of effort.
Loading 1804.07437v2…