Source-linked AI summary
Real-Time Flying Object Detection with YOLOv8
Dillon Reis, Jordan Kupec, Jacqueline Hong, Ahmad Daoudi
TL;DR
Flying-object detection remains difficult because objects vary in size, aspect ratio, speed, occlusion, and background, while existing detection methods can be unreliable. The paper trains a generalized YOLOv8 model across 40 flying-object classes, then transfers its weights to a refined model for realistic environments. The refined model reaches 99.1% mAP50 and 98.7% Precision at 50 fps on a three-class dataset.
Problem
Flying-object detection remains challenging because objects vary in spatial size, aspect ratio, speed, rotation, occlusion, and background, while existing detection methods can be unreliable.
Method
The paper trains YOLOv8 on 40 flying-object categories for generalized representations, then transfer-learns those weights on a more realistic dataset to produce a refined model.
Results
99.1% mAP50, 98.7% Precision, 98.8% Recall, and 50 fps are reported for the refined model on a three-class dataset, while the generalized model reaches 0.685 mAP50-95 and 50 fps.
Takeaways & Limitations
The resulting generalized model is intended for transfer learning or further research, while the refined model is presented as ready for out-of-the-box implementation.
Takeaways & Limitations
YOLOv8 is slightly slower than YOLOv5, and its selection rests partly on an assumption because no official paper had been released.
Abstract
from arXiv · showhide
This paper presents a generalized model for real-time detection of flying objects that can be used for transfer learning and further research, as well as a refined model that achieves state-of-the-art results for flying object detection. We achieve this by training our first (generalized) model on a data set containing 40 different classes of flying objects, forcing the model to extract abstract feature representations. We then perform transfer learning with these learned parameters on a data set more representative of real world environments (i.e. higher frequency of occlusion, very small spatial sizes, rotations, etc.) to generate our refined model. Object detection of flying objects remains challenging due to large variances of object spatial sizes/aspect ratios, rate of speed, occlusion, and clustered backgrounds. To address some of the presented challenges while simultaneously maximizing performance, we utilize the current state-of-the-art single-shot detector, YOLOv8, in an attempt to find the best trade-off between inference speed and mean average precision (mAP). While YOLOv8 is being regarded as the new state-of-the-art, an official paper has not been released as of yet. Thus, we provide an in-depth explanation of the new architecture and functionality that YOLOv8 has adapted. Our final generalized model achieves a mAP50 of 79.2%, mAP50-95 of 68.5%, and an average inference speed of 50 frames per second (fps) on 1080p videos. Our final refined model maintains this inference speed and achieves an improved mAP50 of 99.1% and mAP50-95 of 83.5%
1. Introduction
The paper targets reliable, real-time visual detection of flying objects, especially drones that evade existing detection methods. It develops generalized and refined YOLOv8-based models while explaining YOLOv8’s architecture.
- Drone use in assassination attempts, prison drug deliveries, and border surveillance highlights the need for reliable detection.
- Small electromagnetic signatures, maneuverability, low noise, and accessibility allow drones to evade radar and support illicit use.
- Radio and acoustic detection methods are currently inaccurate, motivating visual detectors for real-time monitoring systems.
- The generalized model is trained on 40 flying-object categories to learn abstract representations, then transfer-learned for realistic occlusion, scale, and rotation conditions.
- The paper uses YOLOv8 as a fast single-shot detector and explains its architecture because no official YOLOv8 paper had yet been released.
2. Materials and Methods
The study trains and evaluates YOLOv8 for real-time flying-object detection under substantial variation in object appearance, scale, motion, and background. It combines generalized multi-class training, transfer learning, model-size selection, hyperparameter tuning, and feature-activation analysis.
- Challenges: Flying-object detection requires fast inference and evaluation across changing object scale, rotation, trajectory, background, and aspect ratio.Low inter-class variance also makes visually similar aircraft difficult to distinguish.
- Datasets: The initial dataset contains 15,064 images, 24,769 annotations, and an 80% train–20% validation split without augmentation.Images average 1.6 annotated objects and have a median ratio of 416x416.
- Model and training: YOLOv8 is initialized with COCO-pretrained weights, and small, medium, and large variants are compared to balance inference speed against mAP50-95.The implementation uses the Ultralytics repository and transfer learning on custom datasets.
- Model and training: 50 frames per second is achieved at 1080p with the medium model, whose 19.75 ms total processing time motivates its selection for hyperparameter tuning.The total comprises 0.5 ms preprocessing, 17.25 ms inference, and 2 ms post-processing; medium-to-large mAP50-95 improvement is only 0.002.
- Loss and evaluation: The YOLOv8 loss combines CIoU box loss, binary cross-entropy classification loss, and distribution focal loss, while the update procedure uses momentum and weight decay.mAP50-95 averages class AP across IoU thresholds from 0.5 through 0.95 in 0.05 increments.
- Confusion diagnosis: Activation maps progress from broad aircraft structure to components and fine-grained textures, while similar F-14 and F-18 activations may explain their confusion.The analysis uses four CSPDarkNet53 c2f stages to inspect increasingly detailed representations.
3. Results
The generalized and refined models are evaluated on difficult flying-object conditions, including tiny objects, blended backgrounds, underrepresented classes, and feature localization. The refined model achieves strong detection performance across three flying-object classes.
- Generalized Model: The generalized model is tested on extremely small objects, background blending, and classification of different flying-object types.The evaluation includes bird, drone, passenger-airplane, and V22 examples.
- Generalized Model: 0.026% of the image: the generalized model correctly detects and classifies a small drone blending into its background.
- Generalized Model: 0.063% of the image: the generalized model correctly identifies a minute passenger airplane blended into its surroundings.
- Generalized Model: 0.14% of the image and 3.57% of the dataset: the model identifies an underrepresented V22 with a confidence score of 0.83.The V22’s vertical propeller configuration can make it resemble a drone.
- Refined Model: The refined model is initialized from generalized-model weights and trained for 199 epochs on real-world data emphasizing extremely small objects.
- Refined Model: The refined model correctly classifies tiny, visually difficult objects, including a 0.02%-sized drone, a 0.034%-sized airplane, and a 0.01%-sized helicopter.Reported confidence scores are 0.81 for the drone, 0.85 for the airplane, and 0.73 for the helicopter.
- Refined Model: The feature maps progressively shift from object segmentation toward background and granular features, while later layers recover the drone’s outline.
- Refined Model: The model achieves mAP50 of 0.991 and mAP50-95 of 0.835 across plane, helicopter, and drone classes.
4. Discussion
The authors use transfer learning from a generalized flying-object model to a refined model and report strong performance at real-time speed on three classes.
- Transfer learning from generalized-model weights produces a refined model with 99.1% mAP50, 98.7% precision, 98.8% recall, and 50 fps.The evaluation covers drone, plane, and helicopter classes.
- The refined model surpasses the cited YOLOv5 comparison, which reports 90.40% mAP50, 91.8% precision, 87.5% recall, and 31 fps on a different dataset.
- The cited prior models report 84.9% AP on UAVs and 86.5% AP on aircraft, using datasets containing small objects and clustered backgrounds.
5. Model Architecture
YOLO frames object detection as a single-pass regression task, while later versions organize detection through backbone, neck, and head components. YOLOv8 retains this structure but adds multi-scale feature aggregation, updated training support, and Soft-NMS, outperforming YOLOv5 on RF100 comparisons.
- YOLOv1: YOLOv1 predicts bounding boxes and class probabilities in one neural-network evaluation after dividing the image into an S x S grid.Its output tensor has shape S x S x (B x 5 + C).
- YOLOv1: YOLOv1 uses 24 convolutional layers followed by two fully connected layers, with ImageNet pretraining applied to the first 20 convolutional layers.The remaining layers are added and randomly initialized during inference.
- YOLOv5: YOLOv5 uses Darknet53 with cross-stage partial connections, a feature-aggregation neck, anchor boxes, and three output scales for detection.Anchor boxes provide predefined aspect ratios and scales that are adjusted to fit object instances.
- YOLOv5: YOLOv5 standardizes the detector into a backbone, neck, and head, using multi-scale branches that predict boxes, class probabilities, and confidence scores.Its heads use 13 x 13, 26 x 26, and 52 x 52 grid sizes, followed by Non-maximum Suppression.
- YOLOv8: YOLOv8 combines Feature Pyramid Network and Path Aggregation Network modules, generates multi-scale features, and introduces a labeling tool for annotation.The FPN creates feature maps at different scales, while PAN aggregates features across network levels.
- YOLOv8: YOLOv8 uses Soft-NMS instead of YOLOv5’s NMS, reducing redundant detections by softly thresholding overlapping boxes rather than discarding them outright.The paper reports similar or better results than YOLOv5 across RF100 categories and a higher mAP with fewer outliers.