Car Detection using Unmanned Aerial Vehicles: Comparison between Faster R-CNN and YOLOv3

Bilel Benjdira; Taha Khursheed; Anis Koubaa; Adel Ammar; Kais Ouni

Source-linked AI summary

Car Detection using Unmanned Aerial Vehicles: Comparison between Faster R-CNN and YOLOv3

Bilel Benjdira, Taha Khursheed, Anis Koubaa, Adel Ammar, Kais Ouni

arXiv:1812.10968v1Dec 28, 2018cs.ROcs.CVcs.LG

TL;DR

Accurate, real-time car detection from UAV imagery is important for surveillance and traffic monitoring, but CNN architectures can perform differently across scenarios. The paper trains and tests Faster R-CNN and YOLOv3 on a labeled UAV car dataset and compares their detection metrics and processing time. YOLOv3 outperforms Faster R-CNN in recall and processing time, while the two are comparable in precision.

Problem
The paper addresses how accurately and quickly CNN-based detectors can detect and count cars in UAV imagery for surveillance and traffic monitoring.
Method
The authors theoretically compare Faster R-CNN and YOLOv3, then train and test both models on a labeled car dataset from UAV images.
Results
YOLOv3 achieves higher recall than Faster R-CNN, while precision is comparable between the algorithms.
Takeaways & Limitations
For UAV car detection, YOLOv3 is the stronger choice when recall and processing time are prioritized, whereas precision is similar for both models.

Abstract

from arXiv · show

Unmanned Aerial Vehicles are increasingly being used in surveillance and traffic monitoring thanks to their high mobility and ability to cover areas at different altitudes and locations. One of the major challenges is to use aerial images to accurately detect cars and count them in real-time for traffic monitoring purposes. Several deep learning techniques were recently proposed based on convolution neural network (CNN) for real-time classification and recognition in computer vision. However, their performance depends on the scenarios where they are used. In this paper, we investigate the performance of two state-of-the-art CNN algorithms, namely Faster R-CNN and YOLOv3, in the context of car detection from aerial images. We trained and tested these two models on a large car dataset taken from UAVs. We demonstrated in this paper that YOLOv3 outperforms Faster R-CNN in sensitivity and processing time, although they are comparable in the precision metric.

I. INTRODUCTION

UAVs support wide-area, high-resolution surveillance and traffic monitoring, but accurate real-time vehicle detection remains challenging. This paper compares Faster R-CNN and YOLOv3 for car detection from aerial images using accuracy and processing time.

Motivation: Traditional machine-learning object detectors limited UAV adoption through low accuracy, while CNN-based deep learning improved classification and object-recognition accuracy.GPU parallelization and cloud offloading also support real-time processing of computation-intensive video analysis.
Motivation: UAVs provide flexible, mobile coverage for surveillance and traffic monitoring, including vehicle detection and counting from aerial video streams.Their high-resolution imagery supports wide fields of view and operation at different altitudes.
Related approaches: Faster R-CNN replaces slow selective-search region proposals with an object-detection algorithm, addressing a computational limitation of earlier R-CNN approaches.R-CNN extracts approximately 2000 regions through selective search before classification.
Related approaches: YOLO processes the image once through a fully convolutional network, making it fast and suitable for real-time detection; YOLOv3 further improves accuracy.YOLO divides the image into grids and predicts bounding boxes and class probabilities.
Study objective: The study evaluates Faster R-CNN and YOLOv3 as representative region-based and single-stage CNN architectures for aerial car detection, focusing on accuracy and processing time.The paper compares the latest approaches from the two categories.

II. RELATED WORKS

Earlier CNN-based car-detection methods improved accuracy but often relied on computationally expensive localization or multistage pipelines. The paper therefore compares Faster R-CNN and YOLOv3 for this task.

Earlier CNN methods: Related work on UAV car detection includes CNN and hybrid deep neural-network methods designed to recognize cars across different scales.The hybrid model combined sliding windows with deep CNN features and modified window placement around cars.
Earlier CNN methods: A two-stage method combined mean-shift segmentation, VGG16 feature extraction, and SVM classification for car detection.Its localization and classification stages formed a computationally heavy pipeline.
Earlier CNN methods: The earlier two-stage approach required around 12 minutes to process a 3456*5184 image, preventing real-time application.The main computational load came from mean-shift segmentation for object localization.
Study rationale: The study selects Faster R-CNN and YOLOv3 as state-of-the-art CNN object-detection algorithms to compare their performance for aerial car detection.The paper provides a theoretical overview before evaluating the two approaches.

III. THEORETICAL OVERVIEW FASTER R-CNN AND YOLOV3

Faster R-CNN and YOLOv3 represent two state-of-the-art CNN object-detection approaches examined for their differing designs and use in car detection.

Comparison focus: The paper aims to examine differences between Faster R-CNN and YOLOv3, particularly for car detection.Both algorithms had been adapted to many recognition problems before this focused comparison.
Faster R-CNN: Faster R-CNN combines a region proposal network with a Fast R-CNN detector in a unified object-detection network.The RPN generates proposals at multiple scales and aspect ratios, while shared convolutional layers feed the detector.
Faster R-CNN: Faster R-CNN uses ROI pooling and fully connected layers to classify proposed regions and predict object coordinates.For this application, the classifier estimates car and background probabilities.
YOLOv3: YOLOv3 is presented as an improvement over YOLOv1 and YOLOv2 within the single-stage YOLO family.The supplied overview identifies YOLOv3 as the latest version considered in the paper.

1) Description of YOLO v1 algorithm:

YOLO detects objects in a single pass by mapping an input image to grid-based bounding boxes, confidence scores, and class probabilities. Its speed comes with a documented localization trade-off.

Architecture: YOLO uses 24 convolutional layers followed by 2 fully connected layers, while Fast YOLO reduces the architecture to 9 convolutional layers at an accuracy cost.The general architecture is described as the basis for the YOLO model.
Prediction formulation: YOLO divides the input into an S × S grid, with each cell predicting B bounding boxes and class probabilities over C classes.Each bounding box contains x, y, w, h, and a confidence score.
Prediction formulation: YOLO outputs an S × S × (B ∗5 + C) tensor and retains high-confidence boxes with the most probable object class.For PASCAL VOC, the example uses S=7, B=2, and C=20, producing a (7,7,30) tensor.
Training: YOLO trains with confidence, localization, and classification losses based on sum-squared error against ground-truth predictions.The selected box is the prediction with the highest IoU with the ground truth.
Post-processing: YOLO uses non-maximal suppression to remove duplicate detections when predictions overlap with IoU ≥0.5, deleting the lowest-confidence prediction.This post-processing step reduces multiple predictions for the same object.
Reported properties: YOLO was reported to outperform other CNN architectures in speed while maintaining or exceeding state-of-the-art mAP.The paper also notes more localization errors but fewer false-positive predictions.
Training: YOLO training uses a pretrained classifier followed by end-to-end convolutional training for object detection.The classifier is pretrained on ImageNet before its fully connected layers are replaced with a convolution layer.

2) Improvements made in YOLO v2:

YOLOv2 introduced several changes to improve accuracy, recall, localization stability, small-object detection, and performance across input dimensions.

Batch normalization improved mAP by 2% and allowed dropout removal without overfitting.
Replacing 224 × 224 inputs with a 448 × 448 high-resolution classifier increased mAP by 4%.
Anchor boxes slightly decreased mAP by 0.3% but increased recall from 81% to 88%.
Dimension clusters used K-means with IoU scores to automatically find suitable anchor boxes.
Direct location prediction estimated offsets to anchors and increased mAP by 5% together with the fourth improvement.
Fine-grained features concatenated high- and low-resolution features through a pass-through layer, improving mAP by 1%.
Multi-scale training randomly changed image dimensions every 10 batches to support prediction across varied input sizes.

3) Improvements made in YOLO v3:

YOLOv3 added multi-label classification, revised anchor assignment, predictions at three scales, and the Darknet-53 feature extractor.

YOLOv3 used multi-label classification with logistic classifiers and binary cross-entropy loss for each label.
YOLOv3 assigned each ground-truth object to the anchor with the highest overlap while ignoring others above a 0.7 threshold.
Feature pyramid networks made predictions at 3 different scales and encoded boxes, objectness scores, and class predictions in a 3-D tensor.
Darknet-53 is a 53-layer CNN using skip connections inspired by ResNet.

IV. EXPERIMENTAL COMPARISON BETWEEN FASTER R-CNN AND YOLOV3

The experimental section evaluates Faster R-CNN and YOLOv3 using five metrics and includes real-time UAV car-detection demonstrations.

The algorithms were evaluated using five metrics described in the paper.
The section includes a video demonstration of real-time car detection from UAV imagery.
Faster R-CNN and YOLOv3 detections on UAV images are shown in Figures 3 and 4, respectively.

A. Description of the Dataset

The study used a UAV car dataset and trained Faster R-CNN and YOLOv3 with specified model configurations and computing hardware.

A. Description of the Dataset: The dataset contained 218 training images with 3,365 labeled cars and 52 test images with 737 cars.
A. Description of the Dataset: Images came from Prince Sultan University campus UAV flights and an open-source GitHub dataset, covering different environments and scales.
B. Description of the Hardware and Software tools: Faster R-CNN used Inception ResNet v2 through the TensorFlow Object Detection API, with 200K training steps and batch size one.
B. Description of the Hardware and Software tools: Faster R-CNN was trained to recognize only the car class, using up to 300 total proposals and 100 proposals per class.
B. Description of the Hardware and Software tools: YOLOv3 used its provided code and default configuration with 608 × 608 inputs, batch size 64, and 16*16 grids.
B. Description of the Hardware and Software tools: Experiments used an Intel Core i9-8950HK CPU, Nvidia GTX 1080 graphics card, 32 GB RAM, and Ubuntu 16.04.

C. Performance evaluation and metrics

The evaluation uses five metrics to compare car-detection performance: precision, recall, F1 score, quality, and processing speed.

C. Performance evaluation and metrics: Five metrics—precision, recall, F1 score, quality, and processing speed—are used to evaluate the two algorithms.The first four metrics quantify detection quality, while processing speed measures computational performance.
C. Performance evaluation and metrics: Precision measures the proportion of detected cars that are true cars, while recall or sensitivity measures detection of all car instances.Precision uses true and false positives; recall uses true positives and false negatives.
C. Performance evaluation and metrics: F1 score is the harmonic average of precision and recall, providing a combined indication of detection robustness.Quality is separately defined using true positives, false positives, and false negatives.

D. Comparison between Faster R-CNN and YOLO v3

Both algorithms achieve very high precision, but YOLOv3 detects more car instances and processes individual images far faster than Faster R-CNN.

D. Comparison between Faster R-CNN and YOLO v3: 99.73% precision for YOLOv3 versus 99.66% for Faster R-CNN shows comparable performance in correctly classifying detected cars.The corresponding non-car false-positive rates are 0.27% and 0.34%, respectively.
D. Comparison between Faster R-CNN and YOLO v3: 99.07% recall for YOLOv3 versus 79.40% for Faster R-CNN indicates that YOLOv3 detects more car instances.False negatives number 2 for YOLOv3 and 150 for Faster R-CNN.
D. Comparison between Faster R-CNN and YOLO v3: 0.057 ms average processing time for YOLOv3 versus 1.39 s for Faster R-CNN reveals a large per-image processing-time gap.The measurements cover 15 image sizes from 100px×100px to 1500px×1500px.
D. Comparison between Faster R-CNN and YOLO v3: Processing time remains approximately independent of image size for both algorithms in the tested range.The reported average processing time is the same independently from image size.

V. CONCLUSION

The paper experimentally compares Faster R-CNN and YOLOv3 for car detection in UAV images using labeled training and test data and five evaluation metrics. YOLOv3 matches Faster R-CNN in precision while outperforming it in sensitivity and processing time, but the study’s scope remains limited to car detection and should be extended across vehicle types and conditions.

V. CONCLUSION: The study trains and tests Faster R-CNN and YOLOv3 on a labeled UAV car dataset using precision, recall, F1 score, quality, and processing time.The paper combines theoretical model descriptions with an experimental comparison.
V. CONCLUSION: YOLOv3 achieves 99.07% sensitivity and outperforms Faster R-CNN in sensitivity and one-image processing time, while precision remains comparable.Both algorithms are reported to have high capability to correctly classify car objects.
V. CONCLUSION: The authors present the comparison as guidance for researchers choosing an object-detection algorithm according to their needs.The supported choice is between the evaluated algorithms and their measured performance characteristics.
V. CONCLUSION: The study should be extended to bicycles, motorcycles, buses, and trucks and to varied lighting and environmental conditions.Suggested conditions include day and night settings, urban and rural environments, crowded traffic, winter, and summer.