PaperlayerView exact revision on arXiv

Source-linked AI summary

Edge YOLO: Real-Time Intelligent Object Detection System Based on Edge-Cloud Cooperation in Autonomous Vehicles

Siyuan Liang, Hao Wu

arXiv:2205.14942v1cs.CVcs.LGeess.SP

TL;DR

Existing object-detection schemes struggle to meet the timeliness, resource, and energy requirements of autonomous-vehicle edge systems. The paper proposes Edge YOLO, an edge-cloud cooperative lightweight detector based on a reconstructed YOLOV4, and reports its performance on COCO2017 and KITTI using Jetson-based verification.

  • Problem

    Existing object-detection algorithms face real-time, resource, and energy constraints on intelligent-transportation edge devices with limited power, GPU performance, and RAM.

  • Method

    Edge YOLO combines edge-cloud cooperation with a lightweight YOLOV4 redesign using backbone pruning and reduced feature fusion, deployed on Jetson-based edge equipment.

  • Results

    On COCO2017, Edge YOLO reaches 47.3% mAP with a 25.27MB model and 11.4 FPS inference on Jetson Nano; KITTI mAP is [0.7904, 0.8473].

  • Takeaways & Limitations

    Edge YOLO provides an edge-deployed object-detection system that balances detection speed and accuracy while using cloud assistance for model updates.

Abstract

from arXiv · show

Driven by the ever-increasing requirements of autonomous vehicles, such as traffic monitoring and driving assistant, deep learning-based object detection (DL-OD) has been increasingly attractive in intelligent transportation systems. However, it is difficult for the existing DL-OD schemes to realize the responsible, cost-saving, and energy-efficient autonomous vehicle systems due to low their inherent defects of low timeliness and high energy consumption. In this paper, we propose an object detection (OD) system based on edge-cloud cooperation and reconstructive convolutional neural networks, which is called Edge YOLO. This system can effectively avoid the excessive dependence on computing power and uneven distribution of cloud computing resources. Specifically, it is a lightweight OD framework realized by combining pruning feature extraction network and compression feature fusion network to enhance the efficiency of multi-scale prediction to the largest extent. In addition, we developed an autonomous driving platform equipped with NVIDIA Jetson for system-level verification. We experimentally demonstrate the reliability and efficiency of Edge YOLO on COCO2017 and KITTI data sets, respectively. According to COCO2017 standard datasets with a speed of 26.6 frames per second (FPS), the results show that the number of parameters in the entire network is only 25.67 MB, while the accuracy (mAP) is up to 47.3%.

I. INTRODUCTION

The paper targets real-time, lightweight object detection for edge-based intelligent transportation systems, where cloud-centric processing faces latency, resource, and energy constraints. Edge YOLO adapts YOLOV4 through network simplification and edge-cloud cooperation for traffic monitoring and driving assistance.

  • Motivation: Edge computing can reduce transmission dependence while supporting offline availability, device miniaturization, and user privacy in intelligent transportation systems.The introduction presents edge-layer processing as a response to congestion and delay from large image and video streams.
  • Motivation: Mobile edge devices have limited power, GPU performance, and RAM, making lightweight processing of YOLO networks necessary.The paper contrasts these devices with high-performance servers and PCs.
  • Motivation: Cloud-based object detection can suffer from poor real-time performance, high resource occupancy, and excessive energy consumption in intelligent transportation systems.These issues are especially relevant to traffic accident detection, assisted driving, and image-data transmission.
  • Proposed approach: The system deploys inference on edge nodes, stores collected images locally, and uploads data when devices are idle for cloud-assisted updating.This workflow is designed for an AIoT system adapted to existing intelligent transportation computing equipment.
  • Proposed approach: Edge YOLO is based on YOLOV4 and removes redundant network components and repeated feature extraction parts for edge computing scenarios.The stated target applications are traffic safety monitoring and driving assistance.

2) Novel efficient Edge YOLO system based on E-CC:

The proposed E-CC architecture separates edge inference from cloud training and updating, combining complementary computing roles for intelligent transportation. It builds on lightweight-network and object-detection developments to address embedded-device constraints.

  • E-CC architecture: E-CC assigns model inference and idle-condition data uploading to the edge, while the cloud performs timed training and weight updating.The paper contrasts these clearly separated responsibilities with federated learning and evaluates E-CC against cloud computing.
  • E-CC architecture: E-CC is presented as a response to isolated cloud and edge layers and the need for more integrated intelligent transportation systems.The architecture is linked to managing edge-node isolation and transport-network pressure.
  • Related developments: Edge-computing research focuses on optimizing CNN-based object detectors and balancing edge-device power consumption with computing capability.The paper characterizes AI and edge computing as complementary.
  • Related developments: Object detection identifies predefined categories and their image coordinates, with deep-learning detectors developing across one-stage and two-stage approaches.The related work traces the field from traditional handcrafted features toward CNN-based detection.
  • Related developments: YOLOV4 combines CSPDarknet-53, PANet, SPP, and a YOLO head, but its reasoning still requires desktop-level GPUs for large-scale edge deployment.This limitation motivates adapting the network for edge scenarios.
  • Related developments: Lightweight networks reduce deployment demands through smaller models, optimized convolution operations, and related architectural strategies.The paper places Edge YOLO within this broader lightweight-network direction.
  • Embedded platforms: NVIDIA Jetson platforms are characterized as compact, energy-efficient, and capable of low-latency, high-throughput inference on embedded systems.The cited power range is 5 to 15 watts for Jetson integrated GPUs versus 100 to 250 watts for traditional GPUs.

III. SYSTEM PLATFORM AND ALGORITHM

The system combines an edge-deployed detection platform with cloud-assisted training and updates, then redesigns YOLOV4 to balance accuracy and speed on embedded hardware. Its workflow is compared with a cloud-computing path.

  • III. SYSTEM PLATFORM AND ALGORITHM: The proposed algorithm reconstructs YOLOV4 by pruning the backbone and reducing GPU-intensive feature-fusion techniques with limited improvement.The resulting architecture is intended to balance accuracy and speed in edge AI scenarios.
  • A. System Platform Design and Comparison: The autonomous-vehicle platform uses Jetson Xavier and Jetson Nano computers with onboard cameras for real-time inspection detection.These platforms are selected as representative, cost-effective Jetson embedded systems.
  • A. System Platform Design and Comparison: Figure 2 contrasts the E-CC and cloud-computing working paths for object detection based on edge devices.The E-CC route is marked purple and the cloud-computing route green.
  • A. System Platform Design and Comparison: In the E-CC path, a camera sends video to NVIDIA edge equipment for CNN inference, saves results locally, and receives cloud-trained weight updates.The camera frame rate is set to 30 FPS.
  • A. System Platform Design and Comparison: In the cloud-computing path, the camera transmits video to a central server, which allocates resources and QPS for CNN-based detection before returning results locally.The path represents the paper’s comparison scheme for cloud-based processing.

B. Algorithm and Network Structure

The algorithm extracts image features through a backbone and uses a neck network to fuse multi-scale information for detection. This organization supports prediction across targets of different sizes.

  • B. Algorithm and Network Structure: The backbone extracts necessary image features before object classification and localization.It forms the feature-extraction stage of the object-detection pipeline.
  • B. Algorithm and Network Structure: The neck fuses information from different feature maps to enable independent detection across scales, especially for small targets.The fused features support multi-scale prediction and improve detection performance.

1) The Trimmed Backbone of YOLOV4:

Edge YOLO trims YOLOV4’s CSPDarknet53 backbone to reduce computational and memory demands while preserving feature-processing capacity. The section also describes anchor-based bounding-box prediction and a simplified network structure.

  • Backbone motivation: CSPDarknet53 replaces YOLOV3’s Darknet53 backbone, but its extensive feature extraction requires substantial computation and memory.The backbone uses CSPNet’s split-and-merge structure, while the original feature extractor contains many convolutional layers and parameters.
  • CSPBlock redesign: CSPBlock divides feature channels into two paths, reducing repeated gradients and memory usage while retaining gradient information and improving convolutional learning.One path passes through transition layers, while the other connects directly before fusion.
  • Network simplification: 10% of the original network’s parameter quantity is retained in the simplified Edge YOLO structure with a 416×416×3 input image.A COCO2017 verification-set example was detected despite background brightness interference, although the absent shoe label prevented high-heel detection.
  • Bounding-box prediction: Anchor-box prediction uses target-center offsets and prior box dimensions to estimate bounding-box coordinates and sizes for multi-scale detection.The described variables include center offsets, anchor-center coordinates, and anchor width and height.

2) Improved Feature Fusion Network in Neck:

Edge YOLO simplifies the Neck feature-fusion design while retaining multi-scale prediction and refining bounding-box processing.

  • Improved Feature Fusion Network in Neck:: 13 × 13, 26 × 26, and 52 × 52 feature maps support large-, middle-, and small-object predictions from 416 × 416 inputs.The three output resolutions are used for multi-scale detection.
  • Improved Feature Fusion Network in Neck:: SPP combines 1 × 1, 5 × 5, 9 × 9, and 13 × 13 maximum-pooling outputs to enlarge the backbone feature receptive field.The pooled maps are concatenated before subsequent feature fusion.
  • Improved Feature Fusion Network in Neck:: FPN integrates low-resolution semantic features with high-resolution features through bottom-up and top-down paths for multi-scale prediction.The paths use down-sampling and up-sampling with a step size of two, alongside lateral connections.
  • Improved Feature Fusion Network in Neck:: Soft-NMS adjusts overlapping detection scores according to IoU instead of directly zeroing lower-confidence boxes.This modification addresses missed detections and low recall associated with direct suppression.
  • Improved Feature Fusion Network in Neck:: Edge YOLO uses three loss terms for localization offset, object confidence, and object classification, with CIoU for localization offset loss.Binary cross-entropy is used for classification loss summation.
  • Improved Feature Fusion Network in Neck:: 17 layers replace YOLOV4’s 61-layer Neck by removing repeated PANet components and retaining a reduced FPN-based fusion design.The omitted components include bottom-up path augmentation, adaptive feature pooling, and fully connected fusion.

IV. RESULTS AND DISCUSSION

The evaluation trains Edge YOLO and compares it with YOLO and lightweight detectors on COCO2017 and a constructed COCO dataset using edge-device metrics.

  • IV. RESULTS AND DISCUSSION: Edge YOLO is evaluated against YOLOV3, YOLOV4, YOLOV3-Tiny, and MobileNetV3 SSD using mAP, model size, BFLOPS, and edge-device FPS.The IoU threshold is set to 0.5 for determining true positives.

A. Dataset Description

The study uses COCO2017 as a broad object-detection benchmark and KITTI as a vehicle-environment dataset containing varied target scales and conditions.

  • Dataset Description: COCO2017 is selected as a public, authoritative object-detection benchmark with multiple annotation types and many targets per image.Its annotations include object instances, key points, and image captions.
  • Dataset Description: KITTI contributes 7,481 training images and 7,518 test images for vehicle-environment object detection across nine target categories.The categories include cars, vans, trucks, pedestrians, cyclists, trams, and related classes.
  • Dataset Description: KITTI includes multi-scale targets, truncation and occlusion annotations, and 1242 × 375 images captured from the driver’s perspective.Examples include nearby pedestrians and trams alongside distant small vehicles.

B. Model Training

Training uses separate software and hardware environments, while the evaluation compares Edge YOLO with established YOLO and lightweight detection networks.

  • Model Training: The training setup uses Ubuntu 18.04, PyTorch 1.6 for MobileNetV3 SSD, and Darknet with CUDA and cuDNN acceleration for YOLO models.CUDA 10.1 accelerates training, while CUDA 10.2 supports Jetson inference.
  • Model Training: YOLOV3 and YOLOV4 are trained for nearly 150,000 batches, while the other object-detection networks are trained for nearly 100,000 batches.Figure 9 reports the loss relationship over the final 10,000 batches.

C. Model Performance Analysis

Edge YOLO combines lightweight network design with embedded-device evaluation to balance detection accuracy, model size, inference speed, and robustness across COCO2017 and KITTI scenarios.

  • COCO2017 performance: 25.27MB model size is 10.2% of YOLOV4 while achieving 47.3% mAP on COCO2017.On Jetson Nano, Edge YOLO reaches 11.4 FPS inference speed.
  • Embedded-device performance: 3–5s faster loading on Jetson Xavier and 5s–8s faster loading on Jetson Nano are reported for the compared dataset condition.
  • KITTI performance: KITTI evaluation reports mAP values from 0.7904 to 0.8473, with mildly improved reasoning speed and slightly reduced model scale and computation.The compressed trunk and reconstructed neck reduce Edge YOLO parameters for vehicle-edge multi-object detection.
  • Detection robustness: Edge YOLO maintains robust detection when targets are incomplete or occluded, while higher-resolution images benefit small-target detection.

D. Comparation With Cloud Computing

The comparison evaluates cloud and edge-cloud deployments across embedded platforms and detection networks, focusing on detection behavior, upload volume, and delay as image counts increase.

  • Comparison setup: Five object-detection networks are evaluated across COCO2017 and KITTI datasets on Jetson Xavier and Jetson Nano platforms.
  • Detection comparison: Edge YOLO, YOLOV4, and YOLOV3-Tiny are compared on the KITTI 2D detection test set for generalization and small-target detectability.
  • Comparison setup: Cloud computing, Jetson Nano edge-cloud computing, and Jetson Xavier Nx edge-cloud computing are compared by uploaded pictures and delays.
  • Delay comparison: Cloud deployment experiences increasing transmission delay as the number of images rises, making real-time and fast performance difficult to guarantee.The edge-cloud architecture is reported to preserve satisfactory computing performance and improve timeliness at the edge.

V. CONCLUSION

The conclusion presents Edge YOLO as an edge-suitable object-detection system that combines local inference with cloud-assisted updating. It reports compact deployment, real-time detectability, and low average power consumption.

  • Conclusion: 8 million parameters fit small embedded devices with a single GPU while balancing YOLO-series speed and accuracy at the edge.
  • Conclusion: 16.8W average power consumption is reported for Edge YOLO.
  • Conclusion: Edge devices perform model reasoning and upload data during idle conditions, while the cloud handles timed training and weight updating.
  • Conclusion: The system is intended for broader edge-computing devices and additional intelligent transportation platforms and scenarios.
Loading 2205.14942v1…