PaperlayerView exact revision on arXiv

Source-linked AI summary

Autonomous Vehicles on the Edge: A Survey on Autonomous Vehicle Racing

Johannes Betz, Hongrui Zheng, Alexander Liniger, Ugo Rosolia, Phillip Karle, Madhur Behl, Venkat Krovi, Rahul Mangharam

arXiv:2202.07008v1cs.ROcs.SE

TL;DR

Autonomous racing challenges autonomous-driving systems to operate near vehicle limits in fast, dynamic, uncertain, and adversarial environments. This paper presents the first holistic survey of autonomous racecar research across software, hardware, competitions, and open challenges. It concludes that the field is growing, with substantial planning and control research and emerging opportunities for reinforcement learning.

  • Problem

    Existing autonomous-driving software operates autonomously only at limited velocity, while autonomous racing requires software to replace expert drivers near high-speed vehicle limits.

  • Method

    The paper surveys and classifies autonomous-racing software and hardware across perception, planning, control, end-to-end methods, platforms, and competitions.

  • Results

    The survey provides a comprehensive overview of autonomous-racing research and identifies open research challenges across the field.

  • Takeaways & Limitations

    The identified challenges are presented as guidelines for future autonomous-racing researchers, while reinforcement learning is identified as an emerging applicable area.

Abstract

from arXiv · show

The rising popularity of self-driving cars has led to the emergence of a new research field in the recent years: Autonomous racing. Researchers are developing software and hardware for high performance race vehicles which aim to operate autonomously on the edge of the vehicles limits: High speeds, high accelerations, low reaction times, highly uncertain, dynamic and adversarial environments. This paper represents the first holistic survey that covers the research in the field of autonomous racing. We focus on the field of autonomous racecars only and display the algorithms, methods and approaches that are used in the fields of perception, planning and control as well as end-to-end learning. Further, with an increasing number of autonomous racing competitions, researchers now have access to a range of high performance platforms to test and evaluate their autonomy algorithms. This survey presents a comprehensive overview of the current autonomous racing platforms emphasizing both the software-hardware co-evolution to the current stage. Finally, based on additional discussion with leading researchers in the field we conclude with a summary of open research challenges that will guide future researchers in this field.

I. INTRODUCTION

Autonomous racing extends autonomous driving toward high-speed, high-dynamic, software-only racecars operating near vehicle limits. This survey categorizes and reviews the field’s software, hardware, competitions, and open challenges.

  • Motivation: Motorsport historically transfers innovations such as disc brakes, turbochargers, carbon-fibre production, hybrid powertrains, and vehicle connectivity to road vehicles.Racing targets faster lap times while also motivating performance and safety innovation.
  • Motivation: Autonomous racing replaces the human pilot with software that must perceive competitors and the track, localize the vehicle, and plan dynamic trajectories at high speeds.The field inherits motorsport’s demanding dynamics while requiring autonomous operation.
  • Scope and contribution: The survey provides a first holistic review of autonomous vehicle racing by summarizing, classifying, and evaluating software and hardware developments.It also highlights future research directions and open questions.
  • Scope and contribution: The review covers perception, planning, control, end-to-end learning, vehicle dynamics, simulation environments, hardware platforms, and autonomous racing competitions.Its categorization follows a perception–planning–control pipeline spanning hardware and software.
  • Scope and contribution: The survey focuses on four-wheeled autonomous racecars with combustion or electric power, including full-size and small-scale vehicles clearly connected to automobile racing.Applications such as drone racing are outside its scope.

A. Perception

Autonomous-racing perception must support fast detection, localization, and state estimation on tracks whose geometry is simple but landmark-poor. Research therefore emphasizes software-based localization, cone detection, and high-speed estimation.

  • Perception challenges: The perception problem includes object and free-space detection, mapping, and vehicle localization in high-speed racing environments.The survey identifies high-speed detection, localization, and state estimation as fundamental problems.
  • Racetrack environment: Racetracks contain a single driveable lane bounded by inner and outer limits, with run-off areas and surrounding walls that vary across racing series.These features differ from normal streets and define the environment represented in the racetrack model.
  • Perception challenges: Wide tracks with few landmarks make perception difficult, and surveyed systems generally do not use precreated HD maps for localization.Available open-source racetrack maps provide 2D bounds useful for planning but not localization.
  • Localization and estimation: Software-based localization research uses sensors such as cameras, LiDAR, IMUs, and wheel encoders, despite many vehicles also carrying differential GPS.Approaches include cone-feature odometry and onboard LiDAR mapping.
  • Localization and estimation: Pose error varies with speed, averaging 0.1 m laterally at 60 km/h and 1.48 m laterally at 200 km/h.Longitudinal error likewise rises from 1.9 m at 100 km/h to 4.92 m at 200 km/h.
  • Localization and estimation: 10 m/s was achieved during simultaneous mapping, localization, and planning with an RMSE error of 0.29 m on an FSD vehicle.A recurrent neural network using multiple vehicle sensors achieved 15x better performance than an EKF, with RMSE values of vx = 0.141 m/s and vy = 0.059 m/s.
  • Object detection: Formula Student Driverless teams detect cone color and form to reconstruct the track boundaries, including blue left-side and yellow right-side cones.Reported approaches include CNN-based detectors, YOLO variants, pose estimation, and synchronized stereo or monocular camera data.

B. Planning

Planning in autonomous racing separates global raceline optimization from local, behavioral, and interaction planning. Global methods optimize objectives such as lap time or geometric properties, while local planners avoid obstacles over a fixed horizon without straying far from the raceline.

  • Global Planning: Global planning computes a raceline around the complete racetrack, commonly optimizing for the lowest lap time under generation constraints.The raceline is the global reference path for fast vehicle motion.
  • Local Planning: Local planning generates fixed-horizon motion that avoids environmental or adversarial collisions while remaining fast, feasible, and close to the global raceline.Strategies modify the global plan, select among dynamically feasible trajectories, or search free space around obstacles.
  • Behavioral Planning: Behavioral planning handles higher-level decisions including overtaking, energy management, vehicle interaction, and responses to race-control inputs.Planning challenges include high-speed obstacle avoidance, frequent replanning, long horizons, and uncertainty.
  • Global Planning: Global raceline methods optimize lap time, geometric properties, or energy, producing different trajectories and lap times.Compared approaches include shortest-path, minimum-curvature, and minimum-time optimization.
  • Global Planning: Evolutionary algorithms search populations of complete racing configurations, evaluating simulated lap times and mutating candidates toward an optimal raceline.Each individual can encode racing-environment, vehicle-hardware, and software parameters.
  • Global Planning: Nonlinear optimal-control formulations minimize lap time while respecting track geometry, friction, vehicle dynamics, and control limits.Proposed solvers and vehicle models differ across formulations.

C. Control

Autonomous-racing control tracks planned paths and velocity profiles while operating near handling limits. The surveyed approaches include MPC, nonlinear and stochastic variants, learning-based methods, and iterative correction across repeated laps.

  • Control Objective: Control uses path and velocity references to reduce lateral and heading errors while following the raceline.Steering, throttle, and braking commands are computed from the vehicle state and reference trajectory.
  • Model Predictive Control: Model Predictive Control forecasts vehicle motion over a finite horizon, repeatedly optimizing actions to track a reference or minimize lap time.Its forecast supports proactive action, while feedback is incorporated through repeated updates.
  • Model Predictive Control: Tube-MPC avoids dynamical-constraint violations in qualitative comparison while MPC outperforms LQR tracking.The comparison concerns maximum lateral and longitudinal acceleration constraints.
  • Robust and Stochastic Control: Stochastic MPC variants address uncertainty by learning model uncertainty online and tightening constraints while preserving standard MPC structure.One reported approach solves its optimization in 20 ms for a 1:43 and achieves faster lap times and fewer constraint violations than standard MPC for a desired violation.
  • Nonlinear Control: Nonlinear MPC captures vehicle and tire nonlinearities, but sufficiently long prediction horizons can increase computation time and prevent real-time operation.Hierarchical designs and simpler models are used to balance fidelity and computational demands.
  • Learning-Based Control: Learning-based controllers use repeated-lap data or online Gaussian-process regression to reduce tracking error and compensate for gaps between planned and driven trajectories.ILC computes feedforward corrections from prior laps, while GP learning estimates the trajectory gap during driving.

D. End-to-End

End-to-end autonomous-racing systems replace some or all conventional software modules with data-driven models, but partial designs retain an intermediate representation for classical control. The survey reports promising results alongside persistent challenges in architecture selection, data requirements, generalization, simulation transfer, and high-speed out-of-distribution events.

  • Definition: End-to-end driving replaces partial modules or the entire autonomous-driving software pipeline with data-driven approaches such as DNNs.Partial end-to-end systems retain some conventional modules, whereas full end-to-end systems replace all software modules.
  • Architecture: Partial end-to-end systems produce low-dimensional racetrack representations, such as trajectories, for classical controllers instead of predicting actuator outputs directly.This architecture combines learned perception or representation with controllers such as PID.
  • Challenges: End-to-end systems require diverse, high-volume training data, while generalization, simulation-to-reality transfer, and rare high-speed events remain major problems.Out-of-distribution events are especially concerning because high-speed driving is rare in training data, making correct reactions difficult to learn.
  • Reported results: A partial end-to-end approach with DNN-parameterized trajectories outperforms a full end-to-end approach in laptime and failures.The comparison is reported in the DeepRacing simulation environment and virtual testbed.
  • Reinforcement learning: Reinforcement-learning methods include Q-Learning, DDPG, SAC, and model-based approaches, with performance varying across tracks, simulators, and vehicles.Q-Learning combined with state representation learning generalizes better to new racetracks than single RL, while MPC outperforms SAC in one small-scale-vehicle comparison.

E. Applied Autonomous Racing Studies

Applied studies evaluate autonomous racing methods through simulations, real-vehicle experiments, complete software stacks, and vehicle-dynamics modeling. The survey also catalogs simulators and evaluation tools used to test planning, control, and learned models.

  • Evaluation: Studies with racecars compare autonomous systems with human driving, including an evaluation of an autonomous racecar crash.
  • Complete Software Stack: Complete autonomous racing requires integrated perception, planning, and control algorithms evaluated on real hardware or in simulation.
  • Vehicle Dynamics Modeling: Vehicle-dynamics models support simulation and model-based trajectory planning or control, but more detailed models require more parameters that may be unavailable.
  • Vehicle Dynamics Modeling: Deep neural networks identify vehicle-model parameters more accurately than purely parametric models and generalize better to unknown dynamics.
  • Simulation: TORCS, Learn-to-Race, and SVL provide simulation environments with tracks, vehicle physics, sensors, and interfaces for control, planning, or reinforcement-learning research.
  • Evaluation: Researchers use dedicated multi-vehicle scenarios, including overtaking maneuvers, to evaluate trajectory planners and safety-assessment algorithms.

III. AUTONOMOUS RACING HARDWARE: VEHICLES AND COMPETITIONS

Autonomous racing research evaluates methods across simulation and multiple classes of physical vehicles. About half of the surveyed papers additionally evaluate their methods on real vehicle hardware.

  • Almost all surveyed papers evaluate their proposed methods in a specific simulation environment.
  • About half of the papers additionally conduct evaluations on real vehicle hardware.
  • Physical evaluation platforms range from passenger sports cars and research prototypes to small-scale race vehicles and real racing cars.

A. Small-Scale Autonomous Racing Vehicles

Small-scale autonomous racecars provide accessible platforms for developing and testing autonomy software, spanning reduced-scale vehicles, 1:10 cars, 1:5 cars, and autonomous go-karts. Their low cost and setup requirements aid research, although scaling creates mismatches with full-size racecars.

  • Small-Scale Vehicles: Small-scale vehicles are commonly modified remote-controlled cars with added sensors and electronic control hardware for autonomous-racing software research.
  • 1:10 Vehicles: 1:10 platforms include Berkeley Autonomous Racecar2, MIT Racecar, MuSHR, RoSCAR, and F1TENTH vehicles released with hardware and setup documentation.
  • 1:5 Vehicles: The 1:5 AutoRally is a 22 kg autonomous racecar with a top speed of ∼90 km/h, using two monocular cameras as main environmental sensors.
  • eV Grand Prix Autonomous: The eV Grand Prix Autonomous competition uses standardized electric go-kart chassis that teams may modify, while its race format consists of single-vehicle time trials.
  • Formula Student Driverless: Formula Student Driverless lets student teams choose vehicle hardware and supports research across perception, localization, planning, control, and holistic software pipelines.

B. Real Autonomous Race Vehicles

Real autonomous race vehicles provide larger-scale platforms for testing autonomy under race-relevant performance and hardware conditions. The survey describes both the mismatch of small-scale platforms and full-size electric and combustion racecars.

  • Small-scale platforms are attractive for research but remain mismatched with real racecars in maximum speed, acceleration, tire limits, sensors, and computation.
  • Roborace: Roborace’s Devbot 2.0 uses an LMP chassis, electric rear-wheel drive, camera, LiDAR, radar, and Nvidia PX2 and Speedgoat ECUs.
  • Roborace: Roborace provides equal hardware platforms so teams can focus on autonomous software and compete in an annual competition.
  • Indy Autonomous Challenge: The Indy Autonomous Challenge uses an Indy Lights chassis with rear-wheel drive, a combustion engine, six-speed sequential transmission, camera, LiDAR, radar, and one main ECU.

IV. OPEN RESEARCH QUESTIONS AND CHALLENGES

The survey identifies unresolved challenges spanning high-speed perception, adversarial driving, trajectory prediction, real-time vehicle dynamics, and complete software-stack execution. These challenges concern adapting autonomy to high-speed, dynamic racing conditions while managing latency, uncertainty, and vehicle limits.

  • High-Speed Perception: High-speed perception still lacks racing-specific methods for motion blur, sensor synchronization, multimodal fusion, and reliable detection beyond 100 m.The survey highlights reduced computational delay, improved camera-Radar-LiDAR fusion, and higher object-detection quality as needed directions.
  • Behavior Planning and Adversarial Driving: Behavior planning lacks sophisticated maneuvers that model competing agents while preserving the ego vehicle’s dynamic capabilities.The proposed direction combines likely-outcome trajectory models with closed-form trajectory-likelihood evaluation.
  • Behavior Planning and Adversarial Driving: Adversarial driving research must address risk calculation, risk evaluation, and high-precision local behavioral and trajectory planning.Racetracks provide settings for testing high-risk maneuvers and crowded multi-vehicle scenarios.
  • Vehicle Dynamics and Future Research: Autonomous vehicles operating at the handling limits require more accurate real-time vehicle-dynamics modelling and software that accounts for competition-specific constraints.The survey frames these requirements as part of broader unresolved research questions for future autonomous-driving algorithms.
  • Overall Software Application: Overall software applications remain underexplored, including module synchronization, sensor and actuator delays, execution scheduling, and CPU/GPU orchestration.These factors affect system latency, reaction time, and vehicle performance under real-time conditions.

V. SUMMARY AND CONCLUSIONS

The survey synthesizes autonomous-racing research across algorithms, vehicle dynamics, simulation, hardware, and racing platforms. It covers 233 papers, reports increased publication activity in the last four years, and derives open challenges for a growing field.

  • Summary and Conclusions: The survey covers perception, planning, control, end-to-end algorithms, vehicle-dynamics modelling, simulation environments, hardware, and autonomous-racing platforms.It also describes competitions as platforms for testing and evaluating autonomy software.
  • Summary and Conclusions: 233 papers are covered in the survey, with publication activity increasing during the last four years.The publication trend is identified through Figure 12.
  • Summary and Conclusions: Planning and control receive disproportionate research attention, while reinforcement learning is identified as readily applicable to autonomous racing.The authors use these observations to derive open research challenges.
  • Summary and Conclusions: The surveyed work is intended to support safer autonomous passenger vehicles and the development of advanced autonomous-driving algorithms.The paper presents its challenge list as guidance for future researchers.
  • Summary and Conclusions: The survey’s paper list is maintained in a GitHub repository to provide open, structured access to research in autonomous racing.The repository is updated regularly.

CONTRIBUTIONS AND ACKNOWLEDGEMENT

The contribution and acknowledgement material records the paper’s collaborative authorship and credits invited researchers whose workshop talks informed the open-challenges section.

  • Contributions and Acknowledgement: The authors contributed across the survey’s overall structure, path planning, control, and open-research-challenges sections.The contribution statement assigns different sections and critical revision responsibilities among the listed authors.
  • Contributions and Acknowledgement: Six researchers are thanked for workshop talks that contributed to creation of Section IV on autonomous-racing opportunities and challenges.The talks were given at the 2021 IEEE ICRA workshop.

APPENDIX

The appendix contains a table listing abbreviations used in the paper.

  • Appendix: Table VII provides the paper’s list of abbreviations.
Loading 2202.07008v1…