Source-linked AI summary
Fast YOLO: A Fast You Only Look Once System for Real-time Embedded Object Detection in Video
Mohammad Javad Shafiee, Brendan Chywl, Francis Li, Alexander Wong
TL;DR
Real-time video object detection with YOLOv2 remains difficult on embedded devices constrained by computation and memory. Fast YOLO evolves a compact YOLOv2 architecture and uses motion-adaptive inference, reporting ∼3.3X faster runtime and fewer deep inferences than original YOLOv2.
Problem
YOLOv2 can run in real time on powerful GPUs but remains difficult to use for real-time video object detection on embedded devices with limited computational power and memory.
Method
Fast YOLO combines evolutionary optimization of YOLOv2 into O-YOLOv2 with motion-adaptive inference that reduces deep-inference frequency based on temporal motion.
Results
∼3.3X faster average runtime is reported for Fast YOLO than original YOLOv2, alongside 38.13% fewer deep inferences and a ∼2.8X more compact network.
Takeaways & Limitations
Fast YOLO is presented as a framework for real-time embedded object detection in video with reduced model compactness and deep-inference frequency.
Abstract
from arXiv · showhide
Object detection is considered one of the most challenging problems in this field of computer vision, as it involves the combination of object classification and object localization within a scene. Recently, deep neural networks (DNNs) have been demonstrated to achieve superior object detection performance compared to other approaches, with YOLOv2 (an improved You Only Look Once model) being one of the state-of-the-art in DNN-based object detection methods in terms of both speed and accuracy. Although YOLOv2 can achieve real-time performance on a powerful GPU, it still remains very challenging for leveraging this approach for real-time object detection in video on embedded computing devices with limited computational power and limited memory. In this paper, we propose a new framework called Fast YOLO, a fast You Only Look Once framework which accelerates YOLOv2 to be able to perform object detection in video on embedded devices in a real-time manner. First, we leverage the evolutionary deep intelligence framework to evolve the YOLOv2 network architecture and produce an optimized architecture (referred to as O-YOLOv2 here) that has 2.8X fewer parameters with just a ~2% IOU drop. To further reduce power consumption on embedded devices while maintaining performance, a motion-adaptive inference method is introduced into the proposed Fast YOLO framework to reduce the frequency of deep inference with O-YOLOv2 based on temporal motion characteristics. Experimental results show that the proposed Fast YOLO framework can reduce the number of deep inferences by an average of 38.13%, and an average speedup of ~3.3X for objection detection in video compared to the original YOLOv2, leading Fast YOLO to run an average of ~18FPS on a Nvidia Jetson TX1 embedded system.
1 Introduction
Fast YOLO addresses the difficulty of running accurate, real-time video object detection on embedded devices, where YOLOv2’s GPU real-time performance does not readily transfer. It combines an evolved YOLOv2 architecture with motion-adaptive inference.
- 1 Introduction: Object detection requires both localizing objects in scenes and assigning labels to their bounding boxes.
- 1 Introduction: Traditional sliding-window detection suffers from high computational complexity and high detection error rates.
- 1 Introduction: YOLO reduced computational complexity by jointly computing bounding-box coordinates and class probabilities as a single regression problem.
- 1 Introduction: YOLOv2 further used anchor boxes, k-means-selected priors, and Darknet-19 to improve modeling accuracy and speed.
- 1 Introduction: YOLOv2 remains difficult to deploy for real-time video detection on embedded devices with limited computation, memory, and power.
- 1 Introduction: Fast YOLO combines evolutionary architecture optimization with motion-adaptive inference to accelerate embedded video detection.
2 Methodology
Fast YOLO uses two components: an optimized YOLOv2 architecture and motion-adaptive inference. The first reduces model size, while the second avoids deep inference on frames judged sufficiently similar to a reference frame.
- 2 Methodology: Fast YOLO consists of an optimized YOLOv2 architecture and a motion-adaptive inference component.
- 2.1 Optimized Network Architecture: Evolutionary deep intelligence models architectural traits probabilistically and synthesizes offspring networks using inherited architecture information and environmental factors.
- 2.1 Optimized Network Architecture: Environmental factors are configured to reduce parameters across generations because parameters drive neural-network computational and memory requirements.
- 2.1 Optimized Network Architecture: ∼2.8X fewer parameters are obtained in O-YOLOv2 than in original YOLOv2, reducing computational, memory, and processor power requirements.
- 2.2 Motion-adaptive Inference: Each current frame is compared with a reference frame through a 1×1 convolutional layer that produces a motion probability map.
- 2.2 Motion-adaptive Inference: Deep inference is performed only when the motion-adaptive module determines that the current frame is sufficiently unique; otherwise, the stored reference probability map is reused.
3 Results & Discussion
Experiments compare O-YOLOv2 with original YOLOv2 on Pascal VOC 2007 and evaluate Fast YOLO runtime on a Nvidia Jetson TX1. The optimized model is substantially smaller, while Fast YOLO is substantially faster.
- 3 Results & Discussion: 2.8X smaller O-YOLOv2 incurs only a 2% drop in IOU compared with original YOLOv2 on Pascal VOC 2007.
- 3 Results & Discussion: Table 1 compares the architectural and performance characteristics of original YOLOv2 and O-YOLOv2.
- 3 Results & Discussion: ∼3.3X speed-up is achieved by Fast YOLO, with 56ms average runtime versus 184ms for original YOLOv2 on Nvidia Jetson TX1.
- 3 Results & Discussion: Table 2 reports average runtime and deep-inference frequency for Fast YOLO, O-YOLOv2, and original YOLOv2 on Nvidia Jetson TX1.
4 Conclusion
The paper concludes that Fast YOLO combines an evolved compact YOLOv2 architecture with motion-adaptive inference for real-time embedded video detection. It reports faster runtime and fewer deep inferences than original YOLOv2.
- 4 Conclusion: Fast YOLO targets real-time object detection in video on embedded devices.
- 4 Conclusion: ∼3.3X faster average runtime, 38.13% fewer deep inferences, and a ∼2.8X more compact network are reported relative to original YOLOv2.