PaperlayerView exact revision on arXiv

Source-linked AI summary

Autonomous Drone Racing: A Survey

Drew Hanover, Antonio Loquercio, Leonard Bauersfeld, Angel Romero, Robert Penicka, Yunlong Song, Giovanni Cioffi, Elia Kaufmann, Davide Scaramuzza

arXiv:2301.01755v4cs.RO

TL;DR

Autonomous drone racing addresses how drones can navigate complex tracks rapidly and safely with limited onboard sensing and computation. This survey synthesizes model-based and learning-based approaches across the flight stack, finding a field increasingly reliant on learning components while retaining traditional methods. It highlights simulation dependence and adaptation to uncertain environments as important boundaries for future progress.

  • Problem

    Autonomous drone racing demands real-time perception, planning, control, and state estimation under high-speed motion, limited computation, uncertainty, and potential opponents.

  • Method

    The survey reviews the evolution of model-based and learning-based autonomous drone racing approaches across perception, planning, control, estimation, modeling, and system engineering.

  • Results

    Learning-based components are present in the best-performing drone-racing algorithms, while competitive real-world systems remain hybrid rather than fully data-driven.

  • Takeaways & Limitations

    Drone racing has advanced from walking-speed flight to performance surpassing world champions through algorithmic innovations and system-engineering improvements across the navigation pipeline.

  • Takeaways & Limitations

    Most learning-based approaches rely heavily on simulators, while real-world training and sim-to-real alternatives have so far been limited largely to legged locomotion.

Abstract

from arXiv · show

Over the last decade, the use of autonomous drone systems for surveying, search and rescue, or last-mile delivery has increased exponentially. With the rise of these applications comes the need for highly robust, safety-critical algorithms which can operate drones in complex and uncertain environments. Additionally, flying fast enables drones to cover more ground which in turn increases productivity and further strengthens their use case. One proxy for developing algorithms used in high-speed navigation is the task of autonomous drone racing, where researchers program drones to fly through a sequence of gates and avoid obstacles as quickly as possible using onboard sensors and limited computational power. Speeds and accelerations exceed over 80 kph and 4 g respectively, raising significant challenges across perception, planning, control, and state estimation. To achieve maximum performance, systems require real-time algorithms that are robust to motion blur, high dynamic range, model uncertainties, aerodynamic disturbances, and often unpredictable opponents. This survey covers the progression of autonomous drone racing across model-based and learning-based approaches. We provide an overview of the field, its evolution over the years, and conclude with the biggest challenges and open questions to be faced in the future.

I. INTRODUCTION

Autonomous drone racing uses fast, onboard flight through ordered gates as a benchmark for advancing perception, planning, control, and related flight-stack capabilities. This survey organizes the field’s evolution, applications, competitions, and open challenges.

  • Motivation and Applications: Faster flight can increase the operating radius available from a given battery, strengthening autonomous drones’ utility for inspection, delivery, and other applications.The survey connects racing-derived capabilities with disaster response, aerial delivery, and complex-structure inspection.
  • Open Challenges: The survey identifies safety and generalization across tasks and environments as remaining challenges despite major progress in autonomous racing.It presents itself as the first survey of the field’s state of the art and future directions.
  • Why Autonomous Drone Racing?: Drone racing provides a challenging benchmark for evaluating complex perception, planning, and control algorithms on computationally limited platforms at tens-of-milliseconds timescales.Lap time supplies a single measure of progress, while recent work has achieved superhuman performance in controlled conditions.
  • Why Autonomous Drone Racing?: Autonomous drone racing requires quadrotors to fly through ordered gates in minimum time while avoiding collisions.Professional pilots can exceed 100 kph, but the required sensorimotor skills take years to acquire.
  • Why Autonomous Drone Racing?: The task requires detecting opponents and waypoints, estimating their three-dimensional location and orientation, and controlling nonlinear dynamics at the performance limits.Perception, planning, and control can determine outcomes separated by less than a tenth of a second.
  • Survey Scope: The survey covers drone modeling, classical perception-planning-control methods, and system components, emphasizing that these parts must work together near the performance envelope.It reviews aerodynamics, batteries, motors, cameras, nonlinearities, and the flight stack.

IROS ADR II

Autonomous racing competitions progressed from cautious classical-method demonstrations to substantially faster autonomous flight. The section also relates racing performance to increasingly detailed vehicle and aerodynamic models.

  • Competition Progression: 0.7 m/s was the winning speed in a Vancouver autonomous drone racing competition using classical onboard methods.The course was challenging, and teams navigated it with onboard computation.
  • Competition Progression: Top speeds approaching 10 m/s marked a significant jump in the 2019 Lockheed Martin-sponsored autonomous racing competition.TU Delft’s MAVLAB team won the $1M-prize competition.
  • Vehicle Modeling: The vehicle model represents position, attitude, velocity, body rates, motor speeds, gravity, and control inputs in the drone’s dynamic evolution.The model calculates the total force and torque, or wrench, acting on the drone.
  • Aerodynamic Modeling: The common assumption that propeller thrust and drag torque scale with squared rotational speed breaks down at racing speeds because it omits several aerodynamic effects.The cited omissions include linear rotor drag, dynamic lift, rotor interactions, and additional effects discussed in the survey.
  • Aerodynamic Modeling: Blade Element Momentum models capture single-propeller wrench effects more accurately than simpler models and can run in real time when implemented efficiently.These models have been used to test algorithms in simulation.
  • Aerodynamic Modeling: Full CFD can capture remaining aerodynamic and interaction effects but is impractical for racing because of its extreme computational demands.Data-driven approaches are used to retain computational simplicity while approaching CFD accuracy.

C. Motor and Battery Models

Motor, battery, sensor, and classical navigation models must account for the demanding conditions of drone racing. The section contrasts model-based architecture with perception and state-estimation methods used under high-speed constraints.

  • Motor and Battery Models: Motor speed depends on throttle, battery voltage, and rotor drag torque, so motor dynamics require a battery-voltage model.For racing lithium-polymer batteries, discharge currents can exceed 100 A, making Peukert models poorly applicable; one-time-constant graybox models are more suitable.
  • Sensor Models: Drone racing pushes mechanical, electrical, and sensing components toward their limits, while cameras and IMUs remain common because they are low-cost, lightweight, and mechanically robust.These sensors form the typical onboard sensing suite for autonomous drones.
  • Sensor Models: Accurate camera and IMU simulation requires modeling camera calibration, sensor placement, IMU bias and noise, and camera-IMU extrinsic calibration and timing.These characteristics influence perception and controller-development validation before real-world deployment.
  • Sensor Models: High-frequency propeller vibrations create IMU aliasing and additional camera motion blur, while their effects are difficult to model and correct.The survey therefore identifies suitable hardware design as imperative.
  • Classical Pipeline: The classical autonomous-navigation stack separates perception, planning, and control, with perception estimating vehicle state and environment from onboard sensors.The survey’s model-based approaches rely on first-principles modeling and optimization techniques.
  • Visual-Inertial Odometry: Visual-inertial odometry is common because it balances accuracy, onboard sensing, and computational requirements, using camera images and inertial measurements through frontend and backend blocks.Direct frontends are robust in featureless scenes, whereas feature-based methods are more vulnerable to motion blur during high-speed flight.

2) Additional sensor modalities in VIO:

VIO for drone racing is extended with event-camera sensing and dynamics-aware estimation, while datasets and learned approaches address high-speed motion challenges. Classical VIO remains reliable only at limited speeds, where racing conditions induce drift and blur.

  • Additional sensor modalities in VIO: Event cameras provide microsecond temporal resolution and 140 dB dynamic range, complementing standard cameras against motion blur.Their high temporal resolution and dynamic range support increased robustness in onboard VIO at high speed.
  • Additional sensor modalities in VIO: Dynamics-aware VIO adds transitional-dynamics constraints, improved noise models, and learned aerodynamics to improve high-speed state estimates.The learned aerodynamics component specifically improves VIO estimates at high speeds.
  • Additional sensor modalities in VIO: Classical VIO provides reliable estimates at limited speeds but accumulates large drift under motion blur, low texture, and high dynamic range typical of drone racing.The EuRoC benchmark reaches only 0.9 m/s average linear velocity and 0.75 rad/s average angular velocity, below racing conditions.
  • Additional sensor modalities in VIO: The UZH-FPV Drone Racing Dataset combines standard-camera images, event data, and IMU measurements from human-piloted quadrotor flights with racing-like visual challenges.It is intended to support VIO research for drone racing tasks.
  • Additional sensor modalities in VIO: Learned end-to-end odometry could specialize to drone racing but remains early-stage and computationally costly for online onboard estimation.Customizing these methods for racing remains an open research question.
  • Additional sensor modalities in VIO: Classical racing pipelines plan feasible, time-optimal trajectories from state estimates, while planning research spans path and trajectory planning.The pipeline predicts future states while respecting platform and environmental constraints.

1) Path planning:

Path and trajectory planning methods range from sampling and combinatorial path search to polynomial, spline, and optimization-based formulations. Racing demands time-optimal, dynamically feasible trajectories, but computational cost and representation limits remain important constraints.

  • Path planning: Sampling-based planning uses configuration-space sampling and collision detection, whereas combinatorial planning explicitly represents obstacle or free space for graph search.RRT and PRM are common sampling-based methods; A* and Dijkstra operate on graph representations of decomposed space.
  • Path planning: RRT* and PRM* can find optimal paths given infinite time, and variants support trajectory generation for fast flight and drone racing.RRT* supplies waypoints for polynomial planning, while PRM* guides sampling-based trajectory planning.
  • Path planning: Polynomial and spline methods exploit quadrotor differential flatness to plan four flat outputs whose derivatives encode dynamically feasible trajectories.Typical flat outputs are 3D position and heading.
  • Path planning: 29: The proposed polynomial representation achieves close-to-time-optimal trajectories through arbitrarily shaped gates while improving computational efficiency over.The method targets minimum-time racing trajectories through gates.
  • Path planning: Traditional polynomial planning cannot represent true time-optimal trajectories because its smooth single-segment structure restricts sampled control inputs.Direct collocation can approach optimal performance using more segments, joint optimization, and polynomial approximations of the dynamics.
  • Path planning: Optimization-based planning independently selects states and inputs over time while incorporating time minimization, quadrotor dynamics, and input constraints.Approaches use point-mass, simplified-quadrotor, or full-state models.
  • Path planning: Time-optimal path-parameterization methods maximize velocity along a given path but do not further optimize that path.The quadrotor formulation accounts for translational and rotational dynamics.
  • Path planning: Waypoint-constrained time-optimal methods can be computationally costly and intractable in real time.Recent formulations add progress constraints, actuator saturation, rotor-thrust inputs, and singularity-free orientation representations.

4) Search-based Trajectory Planning:

Search-based and sampling-based methods pursue minimum-time flight under model and computational constraints, while explicit planning offers modularity but can fail when execution deviates from the plan. The survey also tracks rising demonstrated speeds and contrasts model-based with coupled perception-control approaches.

  • Search-based Trajectory Planning: Search-based planning discretizes state and time, uses motion primitives and graph search, and optimizes flight time only up to discretization.Its per-axis limits do not fully represent the quadrotor model.
  • Search-based Trajectory Planning: Search-based methods become computationally demanding for complex models and are currently limited to planning between two states, which does not suit multi-waypoint racing.They also suffer from reduced plan quality when per-axis limits poorly represent true quadrotor dynamics.
  • Sampling-based Trajectory Planning: Sampling-based planning has produced minimum-time trajectories for full quadrotor models, but the earliest cluttered-environment method was offline and intractable in real time.A later online method replans a point-mass path and uses full-model Model Predictive Contouring Control to maximize progress.
  • Discussion: Explicit planning is modular and reusable across tasks, but open-loop execution can compound timing or state-estimation deviations into complete system failure.Online replanning is one response to this disconnection between planning and deployment.
  • Discussion: Increasing top speeds have been demonstrated in autonomous-drone literature over the last decade.The survey presents these trends using literature and competition data in Figure 4.
  • Discussion: Controllers must make real-time decisions despite poor sensor information and model mismatch, using inputs ranging from velocity commands to direct rotor thrust.The survey distinguishes model-based control from coupled perception and control.

1) Model-Based Control:

Model-based control uses explicit dynamics and optimization to command agile drones, with methods spanning differential flatness, NMPC, adaptive models, contouring, and perception-aware control. These approaches trade computational efficiency, modeling complexity, tracking performance, and perception objectives in high-speed flight.

  • Control approaches: Differential-flatness and INDI controllers enable aggressive quadrotor flight, reaching nearly 13 m/s and accelerations above 2g while tolerating aerodynamic disturbances.Differential flatness provides the reference trajectory structure, while INDI tracks angular-acceleration commands.
  • Control approaches: NMPC outperforms differential-flatness methods on racing tasks, although both cascaded controllers track trajectories near 20 m/s and above 4g.The comparison contrasts computationally efficient differential flatness with the stronger racing-task performance of NMPC.
  • Computational considerations: MPPI can navigate complex forest environments, but sampling over high-dimensional quadrotor states can create performance problems.The cited analysis reports this scaling issue for quadrotors, while most successful MPPI demonstrations concern ground robots.
  • Model adaptation: Adaptive nonlinear MPC learns residual dynamics online, demonstrating nearly 20 m/s flight, above 4g acceleration, and minimal degradation under unknown payloads.Its adaptive inner loop adds minimal computational overhead and improves tracking performance over Gaussian-Process MPC by 70% on high-speed flights.
  • Integrated objectives: Model-based controllers can combine tracking accuracy, minimum flight time, and perception objectives through contouring and perception-aware costs.Perception-aware methods keep gates, landmarks, or dynamic obstacles visible while optimizing motion, tracking, or flight time.
  • Perception-aware control: Perception-aware racing controllers include real-time sequential-quadratic programming and MPPI with deep optical flow, but some demonstrations remain limited to simulation.The MPPI approach predicts relevant pixel motion and was deployed in a drone-racing simulator rather than on real hardware.

3) Discussion:

The discussion contrasts model-based and learning-based approaches for drone racing. Model-based methods offer explicit constraints and predictions but face modeling and online-optimization burdens, while learning-based methods reduce computation yet require substantial data and careful attention to generalization.

  • Model-based limitations: Model-based control degrades with inaccurate dynamics models, and obtaining and tuning sufficiently accurate aerodynamic models requires substantial expertise.Complex aerodynamic forces are difficult to capture in real-time-capable models.
  • Model-based limitations: Drone-racing optimal control includes dynamics, thrust, state, and gate-order constraints, but predicting through the entire race makes online optimization intractable.The formulation uses a horizon extending to task completion, with T denoting the number of discrete race time steps.
  • Learning-based approaches: Learning-based methods replace parts of the planner, controller, or perception stack with neural networks that accept high- or low-dimensional inputs.The survey emphasizes their representation power and relative ease of development and hardware deployment.
  • Learning-based approaches: Learning-based methods can reduce computational effort and support low-latency replanning, while training remains limited by high sample complexity.They can incorporate system latencies and sensor noise during training, but require extensive experience.
  • Learning-based limitations: Simulation makes training-data collection more practical, whereas real-world data generalize more easily but are slow, tedious, and expensive to collect.Simulation itself requires significant engineering to achieve generalization beyond the training environment.
  • Survey scope: This survey extends existing learning-based flight surveys by covering recent advances and comparing learning-based with traditional drone-racing methods.Its stated purpose is to connect existing work and identify meaningful directions for future progress.

A. Learned Perception

Learned perception methods detect gates and produce navigation representations, while learned planning-perception stacks increasingly map sensory observations toward trajectories. Event cameras address motion blur, but learned odometry remains computationally costly and traditional state estimation remains common.

  • Learned perception networks detect landmarks from RGB, depth, or event cameras and output representations such as waypoints or gate locations.
  • Gate-detection systems predict gate geometry or position and may fuse these outputs with visual-inertial odometry through an Extended Kalman Filter.
  • Lightweight architectures such as GateNet and Pencil-Net reduce processing demands for gate detection on computationally limited drones.
  • Event cameras mitigate motion-blur issues during rapid flight, making them a promising modality for high-speed navigation.
  • Deep gate detectors are standard, but learning-based odometry has not replaced traditional VIO because of high computational cost and online-use constraints.
  • Tightly coupled learned planning and perception can reduce mapping and computational requirements, yet effective systems still commonly use metric or locally consistent state estimation.

D. Learned Planning & Control

Learned planning and control increasingly replace explicit trajectory generation or control with policies trained from data. These methods can handle track variation and remove planning-deployment discrepancies, but remain limited by guarantees, generalization, computation, and reliance on simulation.

  • D. Learned Planning & Control: Learned controllers can map state inputs directly to control commands, eliminating the need for a high-level trajectory planner.
  • D. Learned Planning & Control: Reinforcement-learning policies can handle large track changes and scalable random layouts while solving planning and control simultaneously.
  • D. Learned Planning & Control: Differentiable optimizers combine model-based and learning-based approaches by enabling gradients through controllers and prediction-planning-control pipelines.
  • D. Learned Planning & Control: Policy learning removes the discrepancy between planning and deployment but still commonly requires globally consistent state estimation and an environmental map.
  • E. End-to-End Flight: End-to-end systems either replace modular blocks with neural networks or directly map sensor observations to commands, with completely end-to-end racing remaining rare.
  • E. End-to-End Flight: End-to-end navigation remains simulation-bound because of weak generalization, computational complexity, and inferior performance relative to modular methods.
  • F. Discussion: Learning-based methods are prominent in high-performing racing systems and can incorporate modeling uncertainties such as latency through training.
  • F. Discussion: Major open limitations include heavy simulator reliance and difficulty adapting quickly to new, uncertain environments.

V. DRONE RACING SIMULATORS

Drone racing simulators range from physics-focused environments to photorealistic and massively parallel platforms, supporting perception, control, reinforcement learning, and hardware-in-the-loop evaluation. Competitions have progressively tested autonomous systems in increasingly realistic and competitive settings.

  • Simulation environments: Simulation environments recreate drone flight conditions to accelerate autonomous-flight research.They support development and evaluation before deployment on real vehicles.
  • Simulation environments: RotorS extends Gazebo for multirotor simulation with plugins, physics modeling, and contact-force support.Its capabilities include Bullet-based dynamics and linear-drag simulation.
  • Photorealistic simulation: AirSim and FlightGoggles provide photorealistic rendering for simulating perception and estimation, with AirSim also supporting reinforcement-learning training.FlightGoggles additionally interfaces simulated imagery with real-world vehicles through motion capture.
  • Scalable and hardware-in-the-loop simulation: Flightmare combines photorealistic graphics with swappable physics engines and hardware-in-the-loop support, while Aerial Gym simulates millions of multirotors in parallel.Aerial Gym also models obstacles and generates RGB, depth, segmentation, and optical-flow data.
  • Competitions: Drone racing competitions have progressed from gate navigation to simulated head-to-head racing and autonomous flight exceeding 100 kph.At Swiss Drone Days 2022, autonomous drones achieved faster lap times than expert human pilots with motion capture and won races without it using onboard sensing and computing.

VII. DATASETS, HARDWARE, AND OPEN SOURCE CODE

The field provides datasets, aerodynamic models, and open-source software to support perception, modeling, and system development. Hardware coverage is intentionally brief, while the survey emphasizes community-accessible resources.

  • Section scope: This section surveys datasets, open-source code bases, and hardware considerations for autonomous drone racing.The code bases are grouped by use case and the hardware discussion is brief.
  • Datasets: The MIT aggressive-flight dataset contains over 10 hours of multimodal data for evaluating visual-inertial odometry and SLAM.It includes camera, IMU, motor-speed, and motion-capture measurements at different sampling rates.
  • Datasets: The UZH-FPV Drone Racing Dataset contains real-world agile-flight imagery, inertial and event-camera data, and millimeter-level motion-capture ground truth.It was later extended with data recorded onboard an autonomous racing drone.
  • Models: NeuroBEM combines blade-element-momentum models with learned aerodynamic representations and reduces model prediction errors by over 50% versus exclusively first-principles approaches.The model is fit to a specific quadrotor platform but is described as applicable to any quadrotor platform.
  • Open-source software: Open-source repositories cover controllers, planners, sensor calibration, and complete drone-racing software stacks, lowering implementation barriers for researchers.The survey presents these resources in Table I and encourages community contributions.
  • Hardware scope: The survey does not rigorously cover racing-drone hardware design and instead provides only a self-contained overview.An in-depth hardware discussion is referred to elsewhere.

1) Racing Drone Design:

Racing-drone design balances agility, acceleration, power, efficiency, computational needs, and durability, while standardized hardware can ensure fair competition. The survey also notes possible future expansion beyond quadcopters and identifies persistent high-speed estimation challenges.

  • Racing Drone Design: Onboard-compute racing drones are constrained by computer size, with current systems commonly built on 6 inch frames.Lightweight construction is needed to maximize agility and acceleration.
  • Racing Drone Design: Brushless motors provide high specific power, while propeller size trades motor response against energy efficiency and batteries must sustain racing-level power demand.Larger propellers improve efficiency, whereas smaller propellers respond faster.
  • Competition hardware: Standardized hardware platforms can ensure fair competition by preventing participants from independently optimizing vehicle design.This approach is used in human racing and was adopted by AlphaPilot.
  • Beyond Quadcopters: VTOL and fixed-wing aircraft could extend autonomous drone racing beyond multicopters by combining higher speed with maneuverability.The survey focuses on multicopters but identifies these platforms as future research alternatives.
  • High-Speed State Estimation: Robust state estimation remains difficult at racing speeds because motion blur, low texture, high dynamic range, and calibration changes degrade visual-inertial odometry.Event cameras and additional vehicle sensors are proposed as possible ways to reduce drift, but high-speed demonstrations remain limited.
  • Vision-only flight: Human pilots use only first-person-view video, motivating research into vision-only autonomous racing systems.Such systems could reduce inertial-sensing data throughput, power consumption, and cost, although real-world learning remains challenging.

C. Challenge 3: Multiplayer Racing

Multiplayer racing requires anticipating opponent behavior and balancing speed with safety, extending autonomous racing beyond time-optimal flight. Transfer to real-world applications remains constrained by limited environmental knowledge and the difficulty of continual learning on agile drones.

  • Challenge 3: Multiplayer Racing: Existing autonomous racing research has largely optimized flight time without modeling how opponents alter competition dynamics.Multiplayer racing requires estimating opponents from sparse forward-facing observations and predicting overtaking or collision scenarios.
  • Safety and performance: Safety research is shifting from collision-free trajectory generation toward explicit trade-offs between racing performance and risk.These approaches generally formulate safety-performance balancing as constrained optimization.
  • Safety and performance: Risk-aware driving methods such as chance constraints, control barrier functions, and reachability analysis may inform safer agile-flight algorithms.Learning-based approaches remain limited by the difficulty of scaling to high-dimensional perception without robust state estimation.
  • Challenge 5: Transfer to Real-World Applications: Real-world transfer is limited by unknown environments, including the need to estimate the drone state while mapping an initially unknown environment.Drone-racing research has nevertheless developed adaptive control and decision-making strategies for reacting to changing conditions.
  • Challenge 5: Transfer to Real-World Applications: Continual reinforcement learning is difficult onboard agile drones because data collection is risky and the vehicle must immediately know how to hover to avoid crashing.Safe reinforcement learning is proposed as a way to enable exploration without catastrophic system failure.
  • Conclusions and Summary: The survey concludes that autonomous drones have progressed from walking-speed racing to surpassing world champions through innovations spanning perception, planning, control, and system engineering.It characterizes competitive real-world systems as hybrid rather than fully data-driven, with state estimation retaining strong reliance on prior knowledge.
Loading 2301.01755v4…