PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey of Autonomous Driving: Common Practices and Emerging Technologies

Ekim Yurtsever, Jacob Lambert, Alexander Carballo, Kazuya Takeda

arXiv:1906.05113v3cs.ROeess.SY

TL;DR

The paper addresses unresolved robustness and safety challenges that limit automated driving despite its promised benefits. It surveys architectures, core functions, emerging methods, datasets, and tools, and implements selected algorithms for comparison. Its conclusion is that major gaps remain across sensing, algorithms, assessment, adverse conditions, connectivity, and human-machine interaction.

  • Problem

    Robust automated driving remains unresolved: urban operation, safety certification, perception reliability, adverse conditions, and driver handover still present open challenges.

  • Method

    The paper conducts a structured survey of ADS challenges, architectures, core functions, emerging technologies, datasets, and tools, supplemented by implementation and real-world algorithm comparisons.

  • Results

    The survey finds clear research gaps, including disagreement over sensing modalities, insufficient algorithm accuracy and efficiency, unresolved online assessment, adverse-road and weather challenges, immature vehicle communication, and missing real-world handover systems.

  • Takeaways & Limitations

    The paper concludes that directed efforts toward robustness across all ADS levels are needed before safe and efficient roads are achievable.

Abstract

from arXiv · show

Automated driving systems (ADSs) promise a safe, comfortable and efficient driving experience. However, fatalities involving vehicles equipped with ADSs are on the rise. The full potential of ADSs cannot be realized unless the robustness of state-of-the-art improved further. This paper discusses unsolved problems and surveys the technical aspect of automated driving. Studies regarding present challenges, high-level system architectures, emerging methodologies and core functions: localization, mapping, perception, planning, and human machine interface, were thoroughly reviewed. Furthermore, the state-of-the-art was implemented on our own platform and various algorithms were compared in a real-world driving setting. The paper concludes with an overview of available datasets and tools for ADS development.

I. INTRODUCTION

The paper surveys automated driving systems against their promised societal benefits and persistent robustness challenges. It aims to address a gap by combining coverage of architectures, core functions, emerging trends, development resources, and implemented algorithm comparisons.

  • Motivation: ADSs are pursued to prevent accidents, reduce emissions, transport mobility-impaired people, and reduce driving-related stress.Projected annual social benefits could approach $800 billion by 2050 through congestion mitigation, casualty reduction, lower energy consumption, and increased productivity.
  • Motivation: Robust automated driving in urban environments has not yet been achieved, while accidents caused by immature systems undermine trust and cost lives.
  • Technical landscape: Common ADS practice divides automated driving into modules, while end-to-end driving has emerged as an alternative and deep learning dominates many tasks.
  • Terminology: The paper follows SAE terminology, using ADS for hardware-software systems that execute dynamic driving tasks on a sustainable basis.
  • Scope and contribution: The paper fills a literature gap by surveying challenges, architectures, core functions, emerging trends, datasets, software stacks, and simulation tools together.
  • Scope and contribution: It also implements selected state-of-the-art algorithms on the authors’ platform and compares alternative approaches in a real-world driving setting.

B. CHALLENGES

Automated driving remains difficult because ADSs operate in indeterministic environments, face demanding safety and ethical requirements, and must integrate competing architectural approaches. The survey contrasts modular and end-to-end designs while identifying unresolved reliability, connectivity, and deployment barriers.

  • B. CHALLENGES: No production vehicle is capable of level-four or level-five driving automation, and level five must operate across any road network and weather condition.
  • B. CHALLENGES: Level-four-and-above urban driving remains an open challenge because weather and surrounding human behavior are highly indeterministic, while failures have caused documented collisions and fatalities.
  • B. CHALLENGES: Consumer concerns about safety and control are reinforced by fatalities from immature technology, while excessively cautious ADS behavior can also create negative impressions.
  • B. CHALLENGES: Risk and reliability certification remains unresolved because the industry and rule-makers have not established a design-simulation-test-redesign-certification procedure.
  • A. SYSTEM ARCHITECTURE: Modular systems separate sensor inputs from actuator outputs through components for localization, mapping, perception, assessment, planning, control, and human-machine interaction.
  • A. SYSTEM ARCHITECTURE: Modular decomposition supports transferable expertise and redundant safety constraints, but errors can propagate through the pipeline until failure.
  • A. SYSTEM ARCHITECTURE: End-to-end systems generate ego-motion directly from sensory inputs using supervised deep learning, neuroevolution, or deep reinforcement learning.
  • A. SYSTEM ARCHITECTURE: Connected systems offer potential benefits but face security, mobility, routing, naming, caching, reliability, and computing challenges; no operational connected system exists yet.

B. SENSORS AND HARDWARE

ADS platforms combine diverse onboard sensors, actuators, computational units, and vehicle configurations to support robust environmental perception and driving operations. Sensor choices involve trade-offs among color, depth, illumination robustness, resolution, accuracy, cost, and physical size.

  • Hardware categories: ADS hardware includes exteroceptive and proprioceptive sensors, communication arrays, actuators, and computational units.Exteroceptive sensors perceive the environment, while proprioceptive sensors monitor internal vehicle state.
  • Camera sensing: Cameras provide color and passive sensing but perform poorly under changing illumination, while monocular depth estimation remains difficult.Their passive operation avoids interference with other systems, but additional modalities remain necessary for dynamic driving tasks.
  • Camera sensing: Omnidirectional cameras provide 360° vision for navigation, localization, and mapping, although panoramic imagery remains distorted near the periphery.The paper illustrates this trade-off with Ricoh Tetha V images collected on the Nagoya University campus.
  • Emerging modalities: Event cameras asynchronously record brightness changes with microsecond response times, supporting dynamic-object sensing but retaining resolution limitations.The DAVIS240 example shows events triggered by cyclist and vehicle motion.
  • Active sensors: Radar and lidar provide depth information without illumination dependence, but radar is cheaper and longer-range while lidar is more accurate and larger.Lidar performance is negatively affected by fog or snow, whereas radar can fit inside side mirrors.
  • Vehicle platforms: Research platforms range from full-size sensor-rich cars to large vehicles and trailers, where jackknifing adds a control-safety challenge.The Nagoya University Prius was used for core automated driving operations, while platooning targets reduced drag and fuel consumption.

IV. LOCALIZATION AND MAPPING

Localization determines the vehicle’s position relative to a reference frame and is fundamental for lane positioning and global navigation. The paper compares GPS-IMU fusion, SLAM, and a priori map-based localization techniques.

  • Localization overview: Localization finds the ego-vehicle’s position relative to an environmental reference frame and supports accurate lane positioning and global navigation.The section identifies three common onboard-sensor approaches and presents their comparison in Table 5.

A. GPS-IMU FUSION

GPS-IMU fusion corrects dead-reckoning drift using intermittent absolute position readings, but GPS-IMU systems alone do not satisfy urban automated-driving performance requirements. They are therefore used mainly for route planning or initial pose estimation alongside other sensors.

  • Fusion principle: GPS-IMU fusion corrects accumulated dead-reckoning errors at intervals using absolute GPS position readings.IMU measurements provide changes in position and orientation for dead reckoning.
  • Limitations: GPS-IMU systems alone fail to meet urban localization requirements because errors accumulate and GPS accuracy degrades or becomes intermittent in dense environments.The paper cites a GPS error that caused Carnegie Mellon’s red team to fail the 2004 DARPA Grand Challenge.
  • System role: GPS-IMU is used for high-level route planning and initial pose estimation alongside lidar and other sensors in state-of-the-art systems.It is not used alone for full vehicle localization.

B. SIMULTANEOUS LOCALIZATION AND MAPPING

SLAM jointly builds a map and localizes the vehicle without prior environmental information, whereas map-based methods match online readings against pre-built maps. SLAM is broadly deployable but currently less accurate and efficient, while map-based approaches face maintenance and environmental-change constraints.

  • SLAM: SLAM simultaneously constructs a map and localizes the vehicle without requiring a priori environmental information.Its outdoor use is less efficient than localization with a pre-built map because of computational and environmental challenges.
  • SLAM: SLAM can work anywhere, but its accuracy and efficiency remain below those of a priori methods.The paper notes that SLAM could replace a priori techniques if its performance improves further.
  • Map-based localization: A priori map-based localization matches online sensor readings to a detailed pre-built map, often using an initial GPS pose estimate.The best match determines the vehicle’s location, while mapping methods and sensor modalities vary.
  • Map-based localization: Map-based methods degrade when environmental changes make stored maps diverge from reality and require an additional map-making step.Map maintenance is time- and resource-consuming, especially across large and rapidly changing road networks.
  • Landmark search: Landmark search is computationally cheaper and robust when sufficient poles, curbs, signs, or road markers are available.Its dependence on landmarks makes the system prone to failure where landmark density is insufficient.
  • Point cloud matching: Point-cloud matching iteratively translates and rotates online scans against larger a priori maps to estimate sensor position and orientation.The paper used this state-of-the-art approach with GPS and dead reckoning for initial pose estimation.
  • Emerging approaches: Camera-based 2D-to-3D matching reduces vehicle hardware requirements relative to lidar, but the map still must be created with lidar.Vision matching can increase localization computational load, while camera hardware may reduce system cost.

V. PERCEPTION

Perception combines camera, lidar, radar, and emerging sensing modalities to detect and interpret the driving environment. The central design trade-off is balancing accuracy, robustness to environmental conditions, and computational cost for real-time operation.

  • Scope of perception: Perception covers object detection, semantic segmentation, 3D detection, road and lane detection, and object tracking across multiple sensing modalities.Cameras remain the most common perception sensor, while 3D vision provides an important alternative or supplement.
  • 2D object detection: Single-stage detectors favor fast inference and low memory cost, whereas region-proposal methods generally provide higher recognition and localization accuracy at greater computational cost.YOLO illustrates the single-stage trade-off, with the full model operating at 45 FPS and a smaller model at 155 FPS with a small accuracy trade-off.
  • Perception limitations: Camera perception is vulnerable to illumination and changing appearance, while lidar and radar also have weather or resolution limitations.Infrared sensing can help with low-light pedestrian detection, but unpredictable illumination remains a challenge for widespread ADS deployment.
  • 3D object detection: 3D detection bridges camera-based image-space outputs to metric space, but depth estimation and image matching add substantial processing cost.Lidar directly provides sparse 3D points and is less dependent on lighting, although point sparsity and range-dependent appearance complicate detection.
  • 3D object detection: VoxelNet established convincing point-cloud-only 3D bounding-box estimation, while SECOND improved accuracy and efficiency by exploiting lidar sparsity.Despite continuing accuracy improvements, 3D convolutional complexity remains an issue for real-time use.
  • 3D object detection: Bird’s-eye-view projections reduce computational expense, and sparsity-aware methods are approaching the accuracy needed for ADS integration.KITTI and nuScenes provide labeled 3D scenes for evaluating these methods.

B. OBJECT TRACKING

Object tracking in ADSs estimates dynamic objects’ positions, headings, velocities, and future trajectories, using data association, filtering, physical models, and increasingly deep learning.

  • Tracking must estimate heading and velocity, not just location, to predict trajectories and help avoid collisions.These trajectories are estimated in the vehicle frame for downstream use.
  • Common trackers associate detections and apply traditional filtering, with nearest-neighbor association often sufficient for high-rate 3D tracking.Image-based methods additionally use appearance features, while point-cloud methods can use density and Hausdorff-distance similarity.
  • Multiple-hypothesis tracking helps recover from incorrect data associations over time.
  • Particle filters incorporate physical object models and can track geometry-related parameters alongside dynamics.Rao-Blackwellized particle filters have been used for dynamic variables and vehicle geometry in an L-shaped vehicle model.
  • Deep learning enables real-time monocular-image tracking and emerging multi-task networks that estimate object dynamics.These developments support generalized networks spanning multiple perception tasks.

C. ROAD AND LANE DETECTION

Road and lane detection progresses from drivable-area estimation toward lane topology and semantics, combining preprocessing, feature extraction, model fitting, and temporal integration. General systems for complex road topologies remain underdeveloped.

  • Road understanding must determine drivable surfaces, lanes, lane connectivity, and complex semantics such as merging and turning lanes.These tasks correspond to progressively richer levels of driving automation.
  • Road-understanding pipelines preprocess exteroceptive data before extracting road and lane features.Camera correction, lidar filtering, dynamic-object removal, lane-marking detection, and road estimation are examples of this sequence.
  • Model fitting establishes road and lane continuity using lines, splines, non-parametric models, parallel-lane assumptions, and topology-aware representations.Topology-aware models incorporate lane splitting and merging.
  • Temporal integration combines vehicle dynamics with road tracking and filtering to produce smoother road and lane segmentation.Kalman and particle filtering are cited as temporal-smoothing approaches.
  • Most current methods retain assumptions and limitations, while robust systems handling complex road topologies have yet to be developed.Standardized topological maps and machine-learning-based classification are helping address this gap.

VI. ASSESSMENT

A robust ADS should continuously assess scene risk and uncertainty while predicting the intentions of nearby drivers and pedestrians.

  • Assessment covers overall risk and uncertainty, human driving behavior, and driving style recognition.

A. RISK AND UNCERTAINTY ASSESSMENT

Risk and uncertainty assessment quantifies scene hazards and supports prediction of human behavior, but intention and driving-style modeling remain constrained by limited observations and real-time requirements. Integration into real-world ADS pipelines is not yet established.

  • Quantifying scene uncertainty and risk is presented as a way to assess driving situations within ADS development.
  • Bayesian deep learning can propagate probability distributions through ADS modules instead of exact outcomes.The surveyed work reports an advantage over conventional approaches in a hypothetical scenario.
  • Separate risk-inference frameworks use sensory inputs, HMMs, language models, and deep spatiotemporal networks to detect or infer scene risk.The authors used an open-source deep spatiotemporal method to assess lane-change risk.
  • Human intention modeling is relevant to medium- and long-term prediction, but it is not yet common practice and has no established state of the art.Learning human driving traits extended one target vehicle’s prediction horizon by 56% in one cited study.
  • Short observation windows and real-time high-frequency computation constrain the use of complex human-behavior models.ADSs may observe surrounding vehicles for only seconds, limiting models requiring longer observation periods.
  • Driving style lacks an established common definition and remains severely understudied for surrounding human drivers.Existing definitions commonly focus on aggressiveness or fuel consumption.
  • Driving-style recognition has not yet been successfully integrated into a real-world ADS pipeline.The surveyed studies are described as promising despite this integration gap.

VII. PLANNING AND DECISION MAKING

Planning divides automated driving into global route planning and local path planning. Global planners find routes on road networks using graph-based methods, while modern systems combine algorithms for millisecond queries.

  • Planning structure: Planning separates automated driving into global route planning and local path planning.Global planning selects a road-network route; local planning executes it while avoiding obstacles.
  • Global planning: Global planners find routes from an origin to a user-defined destination using GPS and offline maps.Modern production cars commonly use these systems for global navigation.
  • Global planning: Route planning is formulated as a point-to-point shortest-path problem in a directed graph.Conventional approaches include goal-directed, separator-based, hierarchical, and bounded-hop techniques; A* is a standard goal-directed method.
  • Global planning: Separator-based methods remove graph vertices or arcs and build overlay graphs to accelerate shortest-path queries.The overlay graph supports faster route calculations.
  • Global planning: Modern route planners can answer queries in milliseconds by combining separator-based and bounded-hop techniques.Transit Node Routing with Arc Flags is an example of this combined strategy.

B. LOCAL PLANNING

Local planning generates trajectories that execute a global plan while avoiding obstacles and satisfying configuration-space criteria. The survey covers graph-based, sampling-based, curve-interpolation, optimization, and learning-based approaches, noting unresolved deployment barriers.

  • Local planning: Local planners generate trajectories that avoid obstacles and satisfy optimization criteria between start and destination configurations.Their objective is to execute the global plan without failing.
  • Graph-based planners: Graph-based planners discretize paths or decision spaces, but discrete outputs can produce jerky trajectories.State lattices use high-dimensional nodes and precomputed path primitives for dynamic environments.
  • Sampling-based planners: Sampling-based planners build configuration-space connectivity through random path samples using methods such as PRM and RRT.RRT is faster than PRM, while RRT* improves path optimality at the cost of computational efficiency; sampling-based methods can still yield jerky trajectories.
  • Curve and optimization planners: Interpolating curve planners create collision-free paths by fitting curves to waypoints or newly sampled points.The new path can deviate from and later re-enter the initially planned trajectory.
  • Curve and optimization planners: Optimization-based planners improve existing paths with optimization functions, including nonlinear optimization of A* trajectories and Newton-based smoothing of potential fields.The cited potential-field method addresses oscillation while obtaining C1 continuity.
  • Learning-based planners: Deep-learning planners can generate trajectories from sensory data, but safety measures, generalization, and labeled-data requirements remain unresolved.They are promising but not yet widely used in real-world systems.

IX. DATASETS AND AVAILABLE TOOLS

Datasets, software frameworks, and simulation environments support ADS training, testing, and development. The survey reviews driving and naturalistic datasets, open-source platforms, and simulators while concluding that major ADS research gaps remain.

  • Datasets and benchmarks: Datasets are crucial because ADS algorithms and tools must be trained and tested before road deployment.Common practice is to validate individual functions separately on annotated datasets.
  • Datasets and benchmarks: KITTI provided a large labeled driving-scene dataset, while Oxford RobotCar collected over 1000 km using cameras, lidar, GPS, and INS without annotation.KITTI remains widely used, but it has fewer data and labeled classes than generic image databases.
  • Datasets and benchmarks: Naturalistic datasets focus on drivers, including SHRP2 with over 3000 participants collected across three years.Other examples include the 100-Car study, euroFOT, and NUDrive.
  • Available tools: Open-source ADS frameworks include Autoware, Apollo, Nvidia DriveWorks, and openpilot.The study used Autoware to realize core automated-driving functions.
  • Available tools: Simulation supports algorithm development before road tests because instrumented vehicles are costly, public-road experiments are regulated, and dangerous scenarios can be tested safely.CARLA is an urban driving simulator developed for this purpose.
Loading 1906.05113v3…