PaperlayerView exact revision on arXiv

Source-linked AI summary

Vision-based Anti-UAV Detection and Tracking

Jie Zhao, Jingshu Zhang, Dongdong Li, Dong Wang

arXiv:2205.10851v1cs.CV

TL;DR

The paper addresses the lack of sufficient visible-light data and reliable computer-vision methods for anti-UAV detection and tracking. It introduces the DUT Anti-UAV dataset, evaluates detectors and trackers, and fuses detection with tracking; experiments show significant tracking improvements after fusion.

  • Problem

    Existing anti-UAV datasets are insufficient for training high-performance models, while generic vision methods perform poorly on UAV detection and tracking.

  • Method

    The paper constructs the visible-light DUT Anti-UAV dataset and evaluates retrained detectors and trackers, including a simple algorithm that fuses detection with tracking.

  • Results

    23.4% Success improvement is reported for SiamFC+Faster-RCNN(VGG16) compared with baseline SiamFC, while fusion significantly improves tracking performance for all trackers overall.

  • Takeaways & Limitations

    DUT Anti-UAV provides detection and tracking benchmarks, and detection fusion offers a new approach to improving UAV tracking performance.

Abstract

from arXiv · show

Unmanned aerial vehicles (UAV) have been widely used in various fields, and their invasion of security and privacy has aroused social concern. Several detection and tracking systems for UAVs have been introduced in recent years, but most of them are based on radio frequency, radar, and other media. We assume that the field of computer vision is mature enough to detect and track invading UAVs. Thus we propose a visible light mode dataset called Dalian University of Technology Anti-UAV dataset, DUT Anti-UAV for short. It contains a detection dataset with a total of 10,000 images and a tracking dataset with 20 videos that include short-term and long-term sequences. All frames and images are manually annotated precisely. We use this dataset to train several existing detection algorithms and evaluate the algorithms' performance. Several tracking methods are also tested on our tracking dataset. Furthermore, we propose a clear and simple tracking algorithm combined with detection that inherits the detector's high precision. Extensive experiments show that the tracking performance is improved considerably after fusing detection, thus providing a new attempt at UAV tracking using our dataset.The datasets and results are publicly available at: https://github.com/wangdongdut/DUT-Anti-UAV

I. INTRODUCTION

UAVs create safety and privacy concerns, while existing anti-UAV systems and generic vision models remain insufficient for reliable detection and tracking. The paper addresses this gap with a visible-light dataset, evaluations, and a detection–tracking fusion algorithm.

  • UAVs are widely used, but invading or unauthorized UAVs threaten public safety, personal safety, and privacy.
  • Generic object detection and tracking methods do not perform well when directly applied to UAV detection and tracking.UAVs can blend into noisy complex backgrounds, and occlusion creates tracking challenges.
  • Existing anti-UAV datasets are insufficient for training high-performance models, motivating a visible-light dataset with detection and tracking subsets.
  • DUT Anti-UAV contains detection and tracking subsets, including 5,200 training images, 2,600 validation images, 2,200 testing images, and 20 tracking sequences.
  • The study evaluates 14 retrained detectors and 8 trackers on DUT Anti-UAV.
  • A simple fusion algorithm integrates detection into tracking and significantly improves tracking performance for most tracker–detector combinations.

II. RELATED WORK

Related work spans non-vision anti-UAV systems and computer-vision methods for UAV detection and tracking. These approaches establish the field's capabilities while highlighting hardware demands and challenges from aerial viewpoints.

  • UAVs are used in logistics, transportation, monitoring, and other fields, creating safety concerns that motivate anti-UAV research.
  • Existing anti-UAV systems combine sensors such as acoustic, video, and radio-frequency modalities for drone detection and localization.
  • Dynamic coordinate tracing uses a dual-axis rotary mechanism, cameras, and sensing modules to track UAV flight coordinates.
  • The dynamic coordinate tracing device is inexpensive and practical, but requires substantial hardware facilities.

C. UAV dataset

Prior UAV datasets and benchmarks support detection and tracking research but differ in modality, object distribution, environmental coverage, and task emphasis. DUT Anti-UAV is positioned as an additional dataset for both tasks.

  • Computer-vision UAV tracking has grown alongside datasets intended to support robust detection and tracking models.
  • MAV-VID contains 64 videos and 40,323 images, with concentrated UAV locations and small detected objects.The average object size is 0.66% of the entire picture.
  • The Drone-vs-Bird Detection Challenge requires detectors to distinguish UAVs from visually similar birds.
  • Anti-UAV provides visible and infrared data from 318 fully labeled videos totaling 186,494 images across day and night environments.
  • Anti-UAV motion is broad but concentrated mainly in the central image area, with smaller positional variance than DUT Anti-UAV.
  • A prior benchmark combined three UAV datasets and evaluated four detectors and three trackers, whereas this work evaluates 14 detector versions and 8 trackers.

III. DUT ANTI-UAV BENCHMARK

DUT Anti-UAV is a manually annotated benchmark containing detection and tracking subsets designed to test algorithms on varied UAV data. Its construction includes diverse UAV types and dataset attributes.

  • DUT Anti-UAV contains detection and tracking subsets, with three detection splits and 20 tracking sequences involving various UAVs.
  • The dataset includes more than 35 UAV types and outdoor scenes spanning skies, buildings, jungles, farmland, and playgrounds.
  • The detection subset contains 10,000 images and 10,109 annotated objects, while tracking sequences include short-term and long-term sequences.
  • All images and frames are manually and precisely annotated.
  • DUT Anti-UAV includes varied resolutions, lighting conditions, and weather conditions to diversify its data.

B. Dataset characteristics

DUT Anti-UAV emphasizes small UAVs, varied scales and aspect ratios, diverse outdoor scenes, and broad imaging conditions to support detection and tracking research.

  • The dataset contains many small UAV objects against complicated outdoor backgrounds, making detection and tracking more difficult than general benchmarks.The proposed dataset is characterized by a larger proportion of small objects and complex outdoor backgrounds.
  • Image resolution: Detection images range from 160×240 to 3744×5616 pixels, while tracking frames use 1080×1920 or 720 × 1280 resolutions.This resolution diversity is intended to support adaptation to different image sizes and reduce overfitting.
  • Object and background: The dataset includes more than 35 UAV types and scenes spanning skies, buildings, vegetation, farmland, and playgrounds.These settings include diverse outdoor backgrounds and object appearances.
  • Object and background: Lighting and weather vary across day, night, dawn, dusk, sunny, cloudy, and snowy conditions.The authors identify these changes as important for training robust UAV detectors.
  • Object aspect ratio: UAV aspect ratios range from 1.0 to 6.67, with most objects between 1.0 and 3.0 and substantial within-sequence changes.In video10, the ratio changes between 1.0 and 4.33.
  • Object position: Objects are mostly centered, while horizontal and vertical motion ranges vary; tracking sequences maintain continuous bounding-box annotations.The position distribution is described using relative center locations and scatter plots.

C. Dataset challenges

The dataset exposes challenges for vision-based UAV detection and tracking, including small targets, confusing backgrounds, changing illumination, motion, blur, camera movement, and out-of-view events.

  • Key challenges include small objects, complex or similar backgrounds, obvious light changes, blur, fast motion, camera motion, and targets leaving the field of view.Figures 4 and 5 provide examples reflecting these detection and tracking difficulties.

A. Detection on DUT Anti-UAV dataset

The study evaluates multiple detector families and backbones on DUT Anti-UAV, using precision–recall analysis and mAP/FPS measurements. Cascade-RCNN with ResNet50 achieves the best reported detection performance, while YOLOX with ResNet18 is fastest.

  • The evaluation covers two-stage Faster-RCNN, Cascade-RCNN, and ATSS detectors alongside one-stage YOLOX and SSD methods.The selected methods represent both two-stage and one-stage detection approaches.
  • Two-stage models typically provide higher accuracy, whereas one-stage models generally offer better speed.This is presented as a general distinction between the evaluated detector families.
  • The study uses ResNet18, ResNet50, and VGG16 backbones to create 14 detector versions, all retrained on the DUT Anti-UAV training subset.Performance is evaluated with mean average precision (mAP) and frames per second (FPS).
  • Cascade-RCNN with ResNet50 performs best, while YOLOX with ResNet18 is fastest.These rankings are reported from the mAP and FPS results in Table II.
  • Precision–recall curves are compared at different IoU thresholds to reflect comprehensive detector performance.Precision is plotted against recall, with recall as the abscissa and precision as the ordinate.
  • Faster-RCNN and Cascade-RCNN produce accurate bounding boxes and confidence scores, whereas YOLO often mistakes background for the target.This qualitative comparison is reported from visualized detection results.

B. Tracking on DUT Anti-UAV dataset

The paper evaluates eight existing trackers on the DUT Anti-UAV tracking dataset using Success, Precision, and Norm Pre metrics. LTMU performs best, while SiamFC performs worst among the reported trackers.

  • B. Tracking on DUT Anti-UAV dataset: Success measures overlap accuracy, Precision measures center-location error, and Norm Pre addresses Precision’s sensitivity to target size and image resolution.Norm Pre ranks trackers using area under the normalized precision curve.
  • B. Tracking on DUT Anti-UAV dataset: Eight existing tracking algorithms are evaluated on the DUT Anti-UAV tracking dataset.The evaluated set includes SiamFC, ECO, SPLT, ATOM, SiamRPN++, DiMP, TransT, and LTMU.
  • B. Tracking on DUT Anti-UAV dataset: LTMU achieves the best performance, with Success 0.608 and Norm Pre 0.783.The result is reported for the tracking dataset before detection fusion.
  • B. Tracking on DUT Anti-UAV dataset: TransT, DiMP, and ATOM obtain Success values of 0.586, 0.578, and 0.574, respectively.These trackers are reported as performing well relative to the evaluated methods.
  • B. Tracking on DUT Anti-UAV dataset: SiamFC performs worst, with Success 0.381 and Precision 0.623.The comparison uses the noDET tracking results.

C. Tracking with detection

The proposed fusion strategy invokes a detector when the tracker is unreliable and selects a high-confidence detection when it exceeds both thresholds and the tracker score. Across tracker–detector combinations, fusion improves tracking performance, with results remaining stable across parameter changes.

  • C. Tracking with detection: The fusion method invokes detection when the tracker confidence score falls below τt.The detector then supplies candidate bounding boxes and confidence scores for comparison with the tracker output.
  • C. Tracking with detection: A detected bounding box replaces the tracker result only when its highest confidence exceeds τd and the tracker score.Otherwise, the tracker bounding box remains the current result.
  • C. Tracking with detection: The default thresholds τt and τd are both set to 0.9.The paper varies the two hyper-parameters separately to assess their effects.
  • C. Tracking with detection: Detection fusion significantly improves all evaluated trackers across combinations with five detector types and 14 detector versions.The evaluation combines eight trackers with the detector variants.
  • C. Tracking with detection: SiamFC+Faster-RCNN(VGG16) increases Success by 23.4% over baseline SiamFC.Faster-RCNN, particularly its VGG16 version, is generally the strongest fusion choice, whereas ATSS provides little additional benefit.
  • C. Tracking with detection: Large changes in τt and τd cause tracking fluctuations of less than 1%.This indicates robustness of the fusion method to the tested parameter values.

V. CONCLUSION

The paper introduces the DUT Anti-UAV dataset for visible-light UAV detection and tracking, evaluates existing detectors and trackers, and proposes a tracker–detector fusion strategy. Experiments show that the strategy significantly improves tracking performance across the evaluated trackers.

  • V. CONCLUSION: DUT Anti-UAV contains 10,000 images with 10,109 annotated objects and 20 tracking sequences averaging 1240 frames.The images and frames are manually and precisely annotated.
  • V. CONCLUSION: The detection images are divided into training, testing, and validation subsets, while the tracking data consists of 20 sequences.The dataset is released for academic research.
  • V. CONCLUSION: Fourteen detector versions combine five detection algorithms with three backbone-network types.These detectors are retrained on the detection-training set and evaluated on the detection-testing set.
  • V. CONCLUSION: The paper reports tracking results for eight trackers and evaluates combinations of those trackers with 14 detectors.Success and precision plots summarize tracker performance on the tracking dataset.
  • V. CONCLUSION: The proposed fusion strategy significantly improves the tracking performance of all evaluated trackers.The conclusion presents detection fusion as the paper’s main tracking contribution.
Loading 2205.10851v1…