PaperlayerView exact revision on arXiv

Source-linked AI summary

One-Stage Object Detectors in Autonomous Driving

Jonel Roman, Ryan Sirjue, Peter Nguyen, Daniel Krutky, Juan Jesus, Sudip Dhakal

arXiv:2608.19014v1cs.CVcs.AI

TL;DR

Autonomous-driving perception must detect road users accurately under real-time latency constraints, but one-stage detectors make different speed, accuracy, and deployment trade-offs. This survey organizes and compares these detectors, finding that no single model simultaneously optimizes all three dimensions and that dependable deployment still requires system-level support beyond detector improvements.

  • Problem

    Autonomous-driving detection must balance accurate road-object recognition with real-time latency in fast-changing scenes.

  • Method

    The paper surveys one-stage detectors, comparing their architectures, reported performance, deployment trade-offs, and benchmark results across autonomous-driving datasets.

  • Results

    No single detector simultaneously optimizes accuracy, speed, and deployment feasibility, with YOLO and SSD strongest for latency-sensitive real-time performance.

  • Takeaways & Limitations

    Reliable autonomous-driving detection depends on detector design alongside training data, robust sensing, and deployment choices.

  • Takeaways & Limitations

    Practical deployment remains limited by small-object detection, occlusion, environmental variability, computational constraints, and system integration complexity.

Abstract

from arXiv · show

Autonomous vehicles depend on fast and reliable perception systems to detect surrounding vehicles, pedestrians, cyclists, traffic signs, and other road objects in real time. This paper presents a comprehensive survey and analysis of one-stage object detectors for autonomous driving rather than an implementation of a new detection system. The survey reviews the evolution of major one-stage detectors, including YOLOv1, SSD, RetinaNet, EfficientDet, anchor-free detectors such as FCOS and CenterNet, and recent real-time models such as YOLOv10. It compares these architectures through their design choices, feature-fusion strategies, loss functions, deployment trade-offs, and reported benchmark performance. The paper also summarizes commonly used autonomous-driving datasets, evaluation metrics, open challenges, and future research directions. Overall, this survey highlights how one-stage detectors balance speed, accuracy, efficiency, and robustness, while also emphasizing the remaining gap between benchmark results and dependable real-world autonomous-driving performance.

I. INTRODUCTION … C. Architectural and Methodological Innovations

The paper surveys one-stage object detectors for autonomous driving, organizing their evolution, design trade-offs, and reported performance across real-time perception needs. It emphasizes that practical deployment must balance latency, accuracy, efficiency, robustness, and edge-hardware constraints.

  • I. INTRODUCTION: Autonomous-driving detectors must rapidly identify surrounding road objects whose boxes, labels, and confidence scores influence tracking, planning, control, and risk estimation.The required objects include vehicles, pedestrians, cyclists, traffic signs, lane obstacles, and other road users.
  • I. INTRODUCTION: The survey organizes foundational one-stage detectors chronologically, compares reported speed–accuracy performance on KITTI, Waymo, and BDD100K, and examines edge-hardware deployment limitations.It is a survey and comparative analysis rather than a newly implemented detection system.
  • I. INTRODUCTION: Recent models such as EfficientDet and YOLOv10 extend one-stage detection beyond simple speed improvements through better feature fusion, reduced post-processing overhead, and more efficient deployment.The remaining challenges include small objects, occlusion, lighting variation, weather, and hardware limitations.
  • II. LITERATURE REVIEW: The literature review connects prior studies through shared design goals, trade-offs, and unresolved challenges rather than reviewing each paper separately.The section organizes work on one-stage object detectors in autonomous driving around major recurring themes.
  • A. Motivation for One-Stage Detection in Autonomous Driving: One-stage detection is presented as a practical autonomous-driving foundation because models increasingly combine low latency with improved localization and robustness under dynamic or degraded scenes.EfficientDet emphasizes feature fusion and scaling efficiency, ReFPN-FCOS contributes architectural improvements, and adverse-weather studies test dependability in degraded driving conditions.
  • B. Real-Time Efficiency and Speed–Accuracy Trade-Offs: YOLO-style models generally offer the strongest real-time deployment potential by maintaining high throughput and usable detection accuracy, but speed alone is insufficient for autonomous vehicles.Application-specific studies include pothole detection, motorbike detection, and autonomous-driving benchmarks.
  • C. Architectural and Methodological Innovations: Architectural research has progressed from speed-focused anchor-based YOLOv3 and SSD designs toward deliberate backbone, neck, head, and feature-fusion changes that balance efficiency with detection quality.The reported design direction seeks faster inference without sacrificing localization accuracy through revised loss and assignment strategies.

D. Detection Accuracy in Autonomous Driving Environments … A. Core Concepts to Define

The paper frames one-stage detection accuracy as an optimization problem involving loss design, label assignment, and classification-localization alignment, while emphasizing that benchmark gains do not eliminate real-world autonomous-driving limitations. Its technical taxonomy highlights multi-scale feature fusion, confidence–localization distinctions, class imbalance, and end-to-end detection.

  • D. Detection Accuracy in Autonomous Driving Environments: Detection research has shifted from speed-centered early YOLO methods toward optimization choices that improve prediction quality and real-time detection.The cited methodological changes include better loss design, label assignment, and alignment between classification confidence and localization precision.
  • E. AV-Specific Challenges and Practical Limitations: Real-world autonomous-driving perception requires detector accuracy alongside robust sensing, diverse training data, and careful deployment.The selected studies address accuracy, difficult road conditions, and perception reliability as interconnected challenges.
  • F. Emerging Trends and Research Gaps: Recent one-stage detectors increasingly target alignment, efficiency, scaling, and real-time optimization rather than speed alone.TOOD addresses classification-confidence and localization-precision misalignment, while newer methods shift rather than eliminate trade-offs.
  • F. Emerging Trends and Research Gaps: Benchmark accuracy and efficiency continue improving, but dependable real-world autonomous-driving performance remains an open gap.This unresolved gap motivates connecting architectural progress with deployment-oriented limitations.
  • A. Core Concepts to Define: EfficientDet’s BiFPN uses bidirectional cross-scale connections and weighted fusion to integrate multi-scale features for complex object detection.Feature fusion is presented as a core concept in the technical taxonomy.
  • A. Core Concepts to Define: High classification confidence does not always guarantee spatial precision, making localization accuracy critical for safety-critical driving tasks.This distinction is highlighted in ReFPN-FCOS research.
  • A. Core Concepts to Define: Class imbalance occurs when background data outweighs object data, motivating Dynamic Loss functions that focus learning on hard examples.The taxonomy identifies class imbalance and training convergence as related challenges.
  • A. Core Concepts to Define: YOLOv10 exemplifies end-to-end detection by eliminating post-processing bottlenecks such as Non-Maximum Suppression (NMS) for real-time inference.End-to-end detection is defined as a paradigm that removes this post-processing step.

B. Three Sub-Families of One-Stage Detectors … 3) YOLOv3:

The survey organizes one-stage detectors into three sub-families and discusses selected models chronologically, emphasizing their innovations, architectures, improvements, limitations, and relevance to autonomous driving. Within the YOLO family, YOLOv3 advances multi-scale detection and backbone strength while retaining limitations in localization and heavily occluded scenes.

  • B. Three Sub-Families of One-Stage Detectors: The survey groups one-stage detectors into three sub-families for systematic comparison.
  • IV. OVERVIEW OF ONE-STAGE DETECTORS: Selected one-stage detectors are presented chronologically, with discussion covering original publication, innovations, architecture, improvements, limitations, and autonomous-driving relevance.
  • A. YOLO Family: The YOLO family is influential because successive versions seek to balance real-time speed and detection accuracy for autonomous driving.
  • 1) YOLOv1:: YOLOv1 framed detection as a single regression from image pixels to bounding boxes and class probabilities, establishing real-time detection without region proposals.
  • 1) YOLOv1:: YOLOv1 predicts boxes, confidence scores, and class probabilities in one forward pass but struggles with small objects, multiple objects, and precise localization.
  • 2) YOLOv2:: YOLOv2 improved YOLOv1’s accuracy while preserving speed through anchor boxes, batch normalization, dimension clusters, high-resolution classification, and multi-scale training.
  • 2) YOLOv2:: YOLOv2 uses Darknet-19 and anchor-based final-stage predictions, but still has difficulty with small objects and complex multi-scale scenes.
  • 3) YOLOv3:: YOLOv3 combines Darknet-53 residual connections with three-scale predictions, improving handling of different-sized and distant small objects while trailing stronger detectors in localization and heavily occluded scenes.

4) YOLOv4: … 7) YOLOv10:

The surveyed detectors progressively modernize YOLO-style real-time detection through practical training improvements, anchor-free design, architectural optimization, and reduced post-processing. Across YOLOv4, YOLOX, YOLOv7, and YOLOv10, these advances improve speed-accuracy efficiency while leaving challenges in crowded, occluded, small-object, and adverse-driving conditions.

  • 4) YOLOv4:: YOLOv4 combined CSP connections, Mosaic augmentation, Mish activation, and CIoU-based optimization to improve detection accuracy without requiring certain hardware.Its contribution was a collection of practical training and architectural modules rather than one single module.
  • 4) YOLOv4:: YOLOv4 used CSPDarknet53, PANet-style neck components, and a YOLO detection head, but remained anchor-based and struggled with crowded scenes and small objects.Its training losses and optimization strategies targeted localization and generalization under certain conditions.
  • 5) YOLOX:: YOLOX modernized the YOLO family with an anchor-free formulation, decoupled head, and SimOTA label assignment while retaining practical real-time speed.These changes were intended to close the gap with stronger contemporary dense detectors.
  • 5) YOLOX:: YOLOX separates classification and regression in its head and suits streaming perception, although it must balance latency, small-object sensitivity, weather, and occlusion robustness.Its training setup was identified as a major reason for its performance gains.
  • 6) YOLOv7:: YOLOv7 introduced a trainable set of tools and architectural refinements that improved speed and accuracy across a broad FPS range without abandoning YOLO practicality.It aimed to outperform existing real-time detectors while preserving deployment practicality.
  • 6) YOLOv7:: YOLOv7 refined the backbone, neck, and training strategy to raise performance efficiently, yet remained vulnerable to severe occlusion, domain shift, and very small objects.The paper identifies YOLOv7 as one of its generation’s strongest high-speed detectors for autonomous driving.
  • 7) YOLOv10:: YOLOv10 improved efficiency and accuracy through consistent dual assignments and a non-maximum-suppressionfree detection pipeline that reduced post-processing overhead.The model aimed to strengthen the speed-accuracy tradeoff for real-time detection.
  • 7) YOLOv10:: YOLOv10 refines multiple detector components for end-to-end detection, reduced latency, and efficiency across model scales, while facing occlusion, small distant objects, and adverse weather.Its design emphasizes computational efficiency and detection performance rather than relying on one major architectural change.

B. SSD-Based Detectors … 3) RefineDet:

SSD-based detectors established dense multi-scale prediction as a practical foundation for real-time detection in complex road scenes. SSD, DSSD, and RefineDet progressively addressed scale, context, and localization limitations while introducing trade-offs in complexity and speed.

  • B. SSD-Based Detectors: SSD-based detectors established dense multi-scale prediction as a major branch of one-stage detection for complex visual environments such as road scenes.They helped form the practical foundation for later real-time detectors.
  • 1) SSD:: SSD predicts category scores and box offsets from multiple feature maps at different scales using default boxes in a one-stage framework.SSD was introduced in 2016 as the Single Shot MultiBox Detector.
  • 1) SSD:: SSD combines classification and localization losses over matched default boxes, but remains fast while weaker on small-object detection.Its architecture uses a base network and extra convolutional layers to generate predictions at different resolutions.
  • 2) DSSD:: DSSD extends SSD with deconvolution-based modules and extra context to improve small-object handling and richer-scene understanding.DSSD was introduced in 2017.
  • 2) DSSD:: DSSD improves small-object accuracy through a stronger backbone and deconvolution layers, but increased complexity sacrifices speed.Its detection objective remains a multi-task combination of classification and localization losses.
  • 3) RefineDet:: RefineDet narrows the efficiency gap between one-stage and two-stage detectors through a two-step refinement strategy within a one-stage pipeline.An anchor refinement module adjusts anchors before the object detection module performs final classification and regression.
  • 3) RefineDet:: RefineDet combines anchor refinement, transfer connection blocks, and end-to-end multi-task training to improve accuracy and road-user localization quality.Compared with SSD-style detectors, it is more complex and less conceptually simple than earlier predictors.

C. Anchor-Free One-Stage Detectors … 1) RetinaNet:

The section traces anchor-free one-stage detectors from keypoint-based representations to per-pixel predictions, emphasizing reduced anchor complexity for varied autonomous-driving objects. It then presents RetinaNet’s focal-loss solution to foreground-background imbalance alongside its accuracy–deployment trade-off.

  • C. Anchor-Free One-Stage Detectors: Anchor-free detectors reduce anchor-box design complexity and offer flexible strategies for road objects with varied sizes, shapes, and spatial arrangements.This motivation is particularly relevant to autonomous driving.
  • 1) CornerNet:: CornerNet represents objects as paired top-left and bottom-right key points, avoiding predefined anchor boxes and grouping detected corners into bounding boxes.It introduced an alternative anchor-free formulation, though matching many corners can complicate deployment in crowded scenes.
  • 2) CenterNet:: CenterNet represents each object as a center point, combining center-heatmap prediction with regression of size, offset, and sometimes 3D attributes.This reduces anchor adaptation and heavy post-processing, but nearby centers in crowded scenes remain difficult.
  • 3) FCOS:: FCOS eliminates anchor boxes and proposal generation through fully convolutional per-pixel box predictions with centerness estimation.Its design reduces anchor-related hyperparameter sensitivity while retaining dense classification and localization.
  • 3) FCOS:: FCOS uses multi-level FPN-style features and shared heads for class, box-regression, and centerness outputs, but ambiguous assignments can still hinder complex-scene performance.Its anchor-free formulation supports adaptation to varied road-object scales and aspect ratios.
  • D. Modern Dense and Efficiency-Oriented Detectors: Modern one-stage detectors increasingly target localization quality, confidence calibration, multi-scale feature fusion, and deployment efficiency to meet autonomous-driving accuracy and real-time constraints.These priorities reflect the need to balance perception quality with real-time operation.
  • 1) RetinaNet:: RetinaNet addresses foreground-background imbalance with focal loss, downweighting easy negatives so training emphasizes hard examples.This focal-loss mechanism was its most notable contribution.
  • 1) RetinaNet:: RetinaNet combines a backbone and feature pyramid network with separate classification and box-regression subnetworks, achieving strong accuracy but potentially heavier deployment than real-time YOLO-style models.Its autonomous-driving deployment may therefore depend on hardware constraints.

2) EfficientDet: … V. BENCHMARK DATASETS

EfficientDet, GFL, VFNet, and RTMDet advance one-stage detection through efficient multi-scale design, quality-aware prediction, and real-time optimization. Benchmark datasets differ substantially in sensors, classes, scenes, annotations, and evaluation protocols, so no single dataset captures every deployment condition.

  • 2) EfficientDet:: EfficientDet combines BiFPN weighted bidirectional multi-scale fusion with compound scaling across the backbone, feature network, head, and resolution.This makes the detector family scalable and adaptable.
  • 2) EfficientDet:: EfficientDet uses EfficientNet backbones, lightweight heads, and depthwise separable convolutions, but is difficult to customize despite suitability for embedded platforms and onboard accelerators.Its design retains standard dense detection objectives while emphasizing efficiency.
  • 3) GFL:: GFL jointly represents class confidence and localization quality while modeling box locations as distributions rather than fixed point estimates.These choices improve detection ranking and express localization uncertainty.
  • 3) GFL:: GFL’s benefits depend on a strong base detector and careful implementation, while its ranking and uncertainty-aware localization support safety-critical autonomous-driving scenes.The method changes prediction representation and loss design more than the detector architecture.
  • 4) VFNet:: VFNet introduces IoU-Aware Classification Score and Varifocal Loss to align classification confidence with localization quality and improve final detection ranking.It directly addresses the mismatch between high class scores and poorly localized boxes.
  • 4) VFNet:: VFNet adds quality-aware scoring and box-refinement mechanisms, improving ranking quality and accuracy while introducing additional design complexity.Its novelty lies primarily in loss and scoring design rather than a radically new backbone.
  • 5) RTMDet:: RTMDet combines a balanced backbone-neck design, efficient large-kernel depthwise convolutions, and soft-label matching costs for real-time detection.Its design was developed through systematic empirical study.
  • V. BENCHMARK DATASETS: Benchmark datasets vary in sensor setup, object classes, scene diversity, annotation style, and evaluation protocol, and no single benchmark captures all deployment conditions.KITTI remains historically important but is smaller than newer datasets such as Waymo, nuScenes, and BDD100K.

VI. EVALUATION METRICS · VII. PERFORMANCE COMPARISON · VIII. ADVANTAGES, DISADVANTAGES AND LIMITATIONS

The survey evaluates one-stage detectors using both accuracy and deployment metrics, presents reported performance as non-controlled comparisons, and emphasizes that autonomous-driving deployment depends on robustness, computational cost, and edge-hardware suitability beyond speed alone.

  • VI. EVALUATION METRICS: Accuracy metrics, including AP, mAP, IoU, precision, and recall, assess object detection and localization quality.mAP evaluates performance across classes, AP targets one class, and IoU measures predicted-box overlap with the ground-truth box.
  • VI. EVALUATION METRICS: Deployment metrics such as FPS, latency, parameter count, and FLOPs indicate whether detectors satisfy real-time and hardware constraints.A high-mAP detector may remain impractical with excessive latency, whereas a fast detector may be unsafe if it misses small or distant road users.
  • VIII. ADVANTAGES, DISADVANTAGES AND LIMITATIONS: Speed alone does not establish deployment reliability because autonomous-driving detectors must balance inference efficiency with detection performance.The survey frames deployment as a trade-off involving real-time operation, missed road users, and hardware constraints.
  • VII. PERFORMANCE COMPARISON: The performance comparison reports values from original papers, but differing datasets, input sizes, hardware, and reporting conventions prevent a controlled benchmark interpretation.Missing values are marked N/R when they were not reported or were not directly comparable.
  • VIII. ADVANTAGES, DISADVANTAGES AND LIMITATIONS: One-stage detectors’ practical autonomous-driving usefulness depends on small-object detection, occlusion robustness, computational cost, and suitability for edge hardware.These factors determine whether a detector can be reliably deployed in an autonomous-vehicle perception pipeline.
  • VIII. ADVANTAGES, DISADVANTAGES AND LIMITATIONS: FPS measures images processed per second, latency measures prediction delay, parameter count reflects model size, and FLOPs estimate image-processing computation.Together, these metrics characterize computational demands relevant to deployment.
  • VIII. ADVANTAGES, DISADVANTAGES AND LIMITATIONS: Recall measures how many real objects are found, precision measures how many predicted detections are correct, and RMSE measures average prediction error.These metrics provide additional views of detector correctness and prediction deviation.

A. Cross-Cutting Patterns and Observations

One-stage detectors occupy different trade-off positions because no model simultaneously optimizes accuracy, speed, and deployment feasibility. Real-world autonomous-driving deployment therefore requires latency-focused, hardware-aware, and robustness-driven design beyond benchmark accuracy or FPS alone.

  • Accuracy–Speed–Deployment Trade-offs: No single detector simultaneously optimizes accuracy, speed, and deployment feasibility; YOLO and SSD prioritize real-time performance for latency-sensitive autonomous-driving tasks.This speed advantage can reduce accuracy in challenging scenarios, particularly for small objects.
  • Deployment Constraints: Deployment depends on memory, power, embedded resources, environmental robustness, and integration complexity, not inference speed alone.SSD may achieve the highest FPS but has weaker small-object detection, limiting suitability for safety-critical perception.
  • Benchmark-to-Reality Gap: Benchmark performance does not ensure real-world readiness under poor lighting, weather variability, or dense traffic, motivating hardware-aware, robustness-driven, system-level optimization.Future work must connect algorithmic innovations such as anchor-free detection with measurable gains in real-world autonomous-driving perception.
  • Deployment Readiness: YOLO remains the most deployment-ready architecture, while FCOS and CenterNet provide competitive-accuracy, simpler alternatives that still need hardware-aware optimization.The comparison attributes YOLO’s advantage to sustained speed optimization and architectural efficiency.

IX. OPEN CHALLENGES IN ONE-STAGE DETECTION FOR AVS · A. Performance on Autonomous-Driving-Specific Data · X. FUTURE DIRECTIONS

One-stage detectors remain promising for autonomous-driving perception, but benchmark gains do not ensure safe deployment under small, distant, occluded, or weather-degraded conditions. Future evaluation therefore requires driving-specific datasets, hardware validation, and explicit analysis of accuracy–efficiency trade-offs.

  • IX. OPEN CHALLENGES IN ONE-STAGE DETECTION FOR AVS: Benchmark progress does not automatically guarantee safe autonomous-driving deployment, because detectors may fail on small, distant, occluded, or weather-degraded objects.The surveyed literature identifies this as the central unresolved pattern in one-stage detection for autonomous vehicles.
  • A. Performance on Autonomous-Driving-Specific Data: Autonomous-driving evaluation should prioritize road-scene datasets such as KITTI, Waymo, nuScenes, BDD100K, Cityscapes, and Argoverse over general-purpose benchmarks alone.These datasets better reflect safety-critical objects and difficult driving conditions.
  • A. Performance on Autonomous-Driving-Specific Data: The strongest detector on a general benchmark is not always the most suitable choice for an autonomous-driving pipeline.Detector suitability depends on deployment requirements beyond general benchmark performance.
  • A. Performance on Autonomous-Driving-Specific Data: YOLO-style detectors are attractive for high FPS and practical deployment options, but they may miss small or distant objects without careful training and multi-scale design.Speed and deployment practicality do not eliminate small-object detection weaknesses.
  • A. Performance on Autonomous-Driving-Specific Data: SSD is lightweight and fast, yet its small-object weakness threatens detection of pedestrians, cyclists, and road signs.These object categories are especially important in autonomous-driving perception.
  • X. FUTURE DIRECTIONS: RetinaNet, FCOS, EfficientDet, and RTMDet improve different parts of the accuracy-efficiency trade-off but still require validation on driving-specific datasets and hardware.The passage links deployment readiness to both data-domain and hardware validation.

A. Improving Small Object Detection in Road Scenes … XI. CONCLUSION

Future work targets safer and more deployable one-stage detection by improving small-object and adverse-condition robustness, edge efficiency, evaluation, pipeline integration, and anchor-free stability. The conclusion emphasizes balancing real-time efficiency with accurate detection through better feature fusion, loss functions, and model designs.

  • A. Improving Small Object Detection in Road Scenes: Small and distant road users require stronger multi-scale features, better feature fusion, and training that emphasizes small objects for earlier, more reliable detection.Examples include pedestrians, cyclists, traffic signs, and motorcycles, which may appear very small when far away.
  • B. Building More Robust Models for Bad Weather: Detectors should be trained and tested in rain, fog, haze, glare, and nighttime conditions, potentially combining image enhancement with object detection.Models that perform well on clear images may lose accuracy in poor weather and lighting.
  • C. Designing Faster Models for Edge Hardware: Lightweight designs, pruning, quantization, and hardware optimization can reduce model size and inference time while retaining enough accuracy for safety-critical real-time deployment.This addresses the limits of onboard hardware and the need for fast vehicle decisions.
  • D. Creating Better Real-World Evaluation Metrics: Real-world evaluation should supplement mAP and FPS with latency, critical-object misses, weather-change performance, and hardware usage.Benchmark strength alone may not reveal failures in dense traffic, bad weather, or rare safety-critical situations.
  • E. Connecting Detection with the Full AV Pipeline: Detectors should support tracking, localization, planning, and control with outputs such as uncertainty estimates, object distance, and motion information.Detection is one component of the autonomous-driving pipeline, and its results affect safer driving decisions.
  • F. Improving Anchor-Free Detection for Real-Time Use: Anchor-free detectors need faster and more stable real-time performance through simpler heads, improved center-point or keypoint matching, and better handling of overlapping objects.CornerNet, CenterNet, and FCOS remove anchor-box tuning but can still face speed or localization problems in crowded road scenes.
  • XI. CONCLUSION: One-stage detectors have evolved from speed-focused architectures toward models balancing real-time efficiency with accurate detection through better feature fusion, stronger loss functions, and efficient designs.These improvements address trade-offs that matter in autonomous driving.
Loading 2608.19014v1…