PaperlayerView exact revision on arXiv

Source-linked AI summary

AMZ Driverless: The Full Autonomous Racing System

Juraj Kabzan, Miguel de la Iglesia Valls, Victor Reijgwart, Hubertus Franciscus Cornelis Hendrikx, Claas Ehmke, Manish Prajapat, Andreas Bühler, Nikhil Gosala, Mehak Gupta, Ramya Sivanesan, Ankit Dhall, Eugenio Chisari, Napat Karnchanachari, Sonja Brits, Manuel Dangel, Inkyu Sa, Renaud Dubé, Abel Gawel, Mark Pfeiffer, Alexander Liniger, John Lygeros, Roland Siegwart

arXiv:1905.05150v1cs.RO

TL;DR

Autonomous racing requires vehicles to operate near handling limits while navigating previously unknown tracks. This paper integrates perception, estimation, mapping, planning, and control into one software stack, which achieved first place in every attended competition.

  • Problem

    Full autonomy requires vehicles to operate near their handling limits, including avoidance maneuvers and slippery surfaces, rather than only standard driving situations.

  • Method

    The paper integrates perception, state estimation, SLAM, planning, and vehicle dynamics control into a complete autonomous racing platform.

  • Results

    The system secured overall first place at every attended Formula Student Driverless competition and demonstrated up to 1.5g lateral acceleration.

  • Takeaways & Limitations

    The results support the capabilities and scalability of the proposed software stack for autonomous racing on unknown tracks.

  • Takeaways & Limitations

    About 300 ms of perception-to-control delay can cause instability near handling limits, so the current system limits top speed and wheel torques.

Abstract

from arXiv · show

This paper presents the algorithms and system architecture of an autonomous racecar. The introduced vehicle is powered by a software stack designed for robustness, reliability, and extensibility. In order to autonomously race around a previously unknown track, the proposed solution combines state of the art techniques from different fields of robotics. Specifically, perception, estimation, and control are incorporated into one high-performance autonomous racecar. This complex robotic system, developed by AMZ Driverless and ETH Zurich, finished 1st overall at each competition we attended: Formula Student Germany 2017, Formula Student Italy 2018 and Formula Student Germany 2018. We discuss the findings and learnings from these competitions and present an experimental evaluation of each module of our solution.

Supplementary Material

The supplementary material provides access to a video and open-source software and datasets, while identifying the authors’ ETH Zurich affiliations.

  • The paper provides a supplementary video demonstrating the system.
  • Open-source software and datasets are available through the AMZ Driverless GitHub repository.
  • The authors are affiliated with ETH Zurich’s Autonomous Systems Lab and Automatic Control Laboratory.

1 Introduction

The paper presents a complete autonomous racing platform that integrates perception, estimation, mapping, planning, and control for racing on unknown cone-defined tracks. Its system was evaluated in real competitions, achieved strong racing performance, and is accompanied by open-source tools and datasets.

  • System contribution: The system combines LiDAR and vision-based cone perception, multi-sensor motion estimation with outlier rejection, particle-filter SLAM, track-boundary-aware planning, and constrained lap-time control.The control framework minimizes lap time while respecting traction limits and track boundaries.
  • Extensions and resources: The paper extends earlier subsystem work with path planning, control, a testing framework, and a holistic evaluation of the complete system.An open-source simulation framework and an extensive dataset are also provided for autonomous racing development and testing.
  • System contribution: The platform covers environment perception, state estimation, SLAM, planning, and vehicle dynamics control in one autonomous racing system.The contribution spans the full software stack from sensing to vehicle control.
  • Experimental results: 1.5g (14.7 m s−2) lateral acceleration was demonstrated on a full-scale autonomous racecar.
  • Experimental results: The system was evaluated in real-world autonomous racing competitions, with performance assessed for each subsystem.

2 Main Concept

The racecar is designed to complete ten laps around an unknown cone-defined track using onboard sensing, redundant hardware, and software modules for perception, estimation, mapping, and control.

  • Problem Statement – The Race: All sensing and computation must occur onboard because the track is completely unknown before the race.The vehicle must build sufficient track knowledge during operation rather than relying on a pre-existing map.
  • Hardware Concept – The Racecar: The system uses redundant LiDAR and camera perception pipelines, supported by multiple sensors for motion estimation and fault-tolerant operation.The architecture was motivated by robustness requirements and observed sensor failures during testing.
  • Hardware Concept – The Racecar: The vehicle combines LiDAR, monocular and stereo cameras, resolvers, a ground-speed sensor, steering encoder, INS, and IMU to perceive cones and estimate motion.LiDAR is mounted on the front wing, while stereo cameras target nearby cones and the monocular camera supports longer-range estimation.
  • Hardware Concept – The Racecar: Two computing units and an ECU distribute computation and safety functions so the car can continue racing after failure of the non-critical slave unit.The ECU alone interfaces with actuators and handles low-level controls and safety checks.
  • Software Concept: The software stack separates perception, motion estimation and mapping, and control, with control computing steering, throttle, and braking actions.Perception supplies cone estimates, mapping builds the track representation, and control operates the vehicle within handling limits.

3 Perception

The perception system detects and colors track cones using complementary LiDAR and camera pipelines. LiDAR processing removes ground points, reconstructs sparse cone returns, and classifies color with intensity patterns, while vision estimates cone positions from monocular or stereo images.

  • Overview: The perception pipeline provides real-time cone positions, colors, and uncertainties while using sensor redundancy for robustness.Independent LiDAR and camera pipelines can support operation when one sensing modality fails.
  • LiDAR-based Perception: LiDAR processing consists of pre-processing, cone detection, and color estimation, producing cone locations and colors in the car reference frame.The pipeline accepts raw point clouds and applies velocity-based undistortion before later processing.
  • LiDAR-based Perception: Adaptive ground removal can discard cone points, so Euclidean clusters are expanded with cylindrical regions reconstructed from the pre-filtered point cloud.This recovery step restores falsely removed returns and improves cone detection and color estimation.
  • LiDAR-based Perception: A four-convolutional-layer and five-fully-connected-layer CNN classifies 32 × 32 grayscale cone images as blue, yellow, or unknown using LiDAR intensity patterns.Yellow-black-yellow and blue-white-blue cone patterns produce different vertical intensity gradients.
  • Camera-based Perception: The camera pipeline detects cones, regresses keypoints, and estimates 3D pose using PnP for monocular images or feature matching and triangulation for stereo images.YOLOv2 supplies bounding boxes and confidence scores for blue, yellow, and orange cones.
  • LiDAR-based Results: On same-type nearby cones, CNN and rule-based color classification each achieved 96% accuracy, while the CNN was less affected by altered intensities from additional stickers.The comparison indicates the CNN generalized better than the rule-based method to cones differing from training examples.
  • LiDAR-based Results: A LiDAR-only SLAM map colored all cones accurately except near the start line, where cones were deliberately left uncolored to avoid orange-cone misclassification.The map was used to validate color estimation over one lap.

4 Motion Estimation and Mapping

The system combines robust multi-sensor velocity estimation with particle-filter SLAM to support reliable localization and mapping during autonomous racing. Experimental results show accurate velocity estimates, effective sensor-failure rejection, and low mapping error.

  • Velocity estimation: A 9-state Extended Kalman Filter fuses heterogeneous sensor measurements with a vehicle model for robust velocity estimation.The estimator propagates velocities with acceleration and incorporates tire-slip dynamics, while updating asynchronously with velocity, acceleration, yaw-rate, and wheel-speed measurements.
  • Velocity estimation: Acceleration is included in the state and fused from tire-slip dynamics and IMU measurements to improve redundancy and robustness.The design reduces uncertainty from changing surface inclination and time-varying bias.
  • Failure detection: The estimator remains observable when either GSS or GNSS fails, while simultaneous IMU and velocity-sensor failures require a reduced partial-kinematic model with zero-slip updates.The sensor-availability configurations and use of the reduced model are summarized in Table 2.
  • Failure detection: Chi-square testing rejects outliers, while variance-based drift detection removes sensors contributing excessively to measurement variance.The two methods are implemented together because chi-square detection is effective for outliers but can detect drift only after the error becomes large.
  • Mapping: FastSLAM 2.0 represents cone tracks with a particle-filter map that supports uncertain data association and adjustable runtime.Its particle structure provides multiple association hypotheses, and runtime can be traded against performance through the number of particles.
  • Mapping: The SLAM implementation integrates velocity at 200 Hz for localization and combines landmark observations with velocity estimates to update the map.The architecture separates a high-frequency localizer from the mapping algorithm.
  • Experimental results: The estimated velocity produced less than 1.5 m of position error over a 310 m track, corresponding to less than 0.5% drift, even with substantial wheel slip.The estimate was evaluated against GSS and GPS-related ground-truth measurements.
  • Experimental results: Using chi-square and drift detection together removes most sensor failures, including continuous failures that chi-square testing alone cannot reject.The chi-square method rejects short-lived failures, whereas drift detection also discards continuous sensor failures.

2 Failure detection

The evaluation examines sensor-failure detection, mapping accuracy, path generation, and boundary prediction. Results show low mapping errors and robust boundary estimates, with errors increasing when cone spacing becomes large.

  • Failure detection: Chi-square testing alone cannot detect the full GNSS failure, whereas drift detection identifies the failure and supports rejection of continuous sensor faults.Figure 24 contrasts GNSS failure, drift detection, and chi-square-only velocity estimates.
  • Mapping evaluation: Vision-only mapping achieves an RMSE of 0.25 m, while LiDAR-only mapping achieves an RMSE of 0.23 m against ground-truth measurements.These values compare estimated maps with ground-truth measurements.
  • Mapping evaluation: Combined LiDAR-and-vision localization differs from tracked ground truth by an RMSE of 0.2 m on a 230 m track.A Totalstations prism attached to the car provided positional ground truth.
  • Boundary estimation: The path-planning pipeline generates candidate trajectories through cone-defined track geometry and selects one using a cost function.The visualization shows Delaunay triangulation, a trajectory tree, and the selected path.
  • Boundary estimation: Only 4.2% of iterations produced a path outside the real track, and those paths left it only more than 7 m from the car.The receding-horizon controller and 3 m s−1 SLAM-mode speed prevented these distant errors from causing the car to leave the track.
  • Boundary estimation: Boundary-estimation error increased when consecutive cones were separated by more than 5 m.The algorithm was tested across varied track layouts and road conditions, remaining robust on rule-compliant tracks.

5 Control

The control system uses a nonlinear MPC formulation built around a blended vehicle model, path-contouring objectives, and track constraints. It is designed to control the racecar near its performance limits while remaining suitable for real-time execution.

  • 5 Control: After the first lap, SLAM supplies the complete track layout, enabling MPC to maximize progress around the known track while respecting vehicle and track constraints.The formulation uses the track center line as its reference path and requires only the track layout and vehicle model, without predetermined driving logic.
  • 5.1 Vehicle Model: The vehicle dynamics use a nonlinear bicycle model with tire-force laws that match racing performance while remaining tractable for real-time MPC.The model includes lateral tire forces, drivetrain force, and torque-vectoring effects.
  • 5.1 Vehicle Model: A blended dynamic-kinematic model addresses the dynamic model’s ill-defined behavior at slow velocities while retaining tire-interaction modeling at high velocities.The two models share states and are linearly blended across a vehicle-speed range; below and above that range, purely kinematic or dynamic modeling is used.
  • 5.1 Vehicle Model: The controller uses input dynamics so steering and drivetrain commands are controlled through their derivatives, allowing input-rate constraints and penalties.The derivatives are introduced through the Δ formulation, with ΔD and Δδ as new control inputs.
  • 5.2 Contouring Formulation: The contouring formulation balances path-following errors against lap-time progress by maximizing velocity along the reference path.A double-integrator arc-length model tracks position, velocity, and acceleration relative to the path, while weighted objectives set the trade-off.
  • 5.4 Runtime Analysis: 90% of solve times during FSG trackdrive were below the sampling time, with a maximum nonlinear-optimization solve time of 0.065 s.Experiments used a 50 ms sampling time and a 40-step prediction horizon corresponding to 2 s.

6 Testing Framework

The Automated Testing System combines automated simulation, log handling, distributed processing, and web-based visualization to verify implementations efficiently. It addresses the time and manpower limits of repeated real-world racecar testing.

  • 6 Testing Framework: The testing framework formalizes verification procedures and uses automated simulations to test the autonomous-racing software stack.The framework is called Automated Testing System (ATS).
  • 6 Testing Framework: Automated simulation reduces dependence on time-consuming and manpower-intensive race-track testing by catching problems before track deployment.Simulations are triggered by pull requests and run nightly.
  • 6.1 Simulation: FSSIM uses an integrated first-principles model and a flat constant-friction track approximation to match actual vehicle performance in simulation.Tracks can be automatically generated from log data, while perception sensors are not simulated.
  • 6.2 Data Handling: ATS automatically archives, indexes, and visualizes large volumes of ROS log data through a web interface.The system uses rosbag logs and enables users to check results quickly.
  • 6.2 Data Handling: A distributed service assigns queued log-processing tasks to worker nodes that poll for scheduled work, keeping the system simple and stateless.Bookkeeping is stored in a PostgreSQL database, and visualizations can be manually queued or scheduled when new logs are discovered.

7 The Race Results

The racecar completed ten consecutive FSG 2018 laps autonomously with near-limit lateral acceleration and consistent lap times. The authors emphasize real-world testing, simulation support, modular interfaces, safety margins, and architecture delays as key engineering lessons.

  • Race Results: 10 consecutive laps were completed autonomously at FSG 2018, ending with a safe stop within the allowed distance.The vehicle achieved 1.5g lateral acceleration during the run.
  • Race Results: 1.5g lateral acceleration was achieved over the run, close to the tire limit.Conservative torque limits produced a squared GG plot and prioritized safety over pure performance.
  • Race Results: The optimization-based controller produced fast and consistent lap times during the ten-lap run.Table 4 reports the achieved FSG 2018 lap times and attributes their consistency to the controller’s optimization nature.
  • Lessons Learned: Real-world testing is essential for evaluating the whole system, while simulation makes testing more efficient and helps identify problems.The ATS automatically simulates multiple trials, supports parameter selection, and analyzes real-world and simulation tests.
  • Lessons Learned: Fixed software interfaces enable independent subsystem development, simulation, and dataset-based testing, but the full pipeline still requires regular integration tests.Whole-system testing verifies both end-to-end operation and computational constraints.
  • Lessons Learned: About 300 ms elapse between cone detection and the corresponding control command reaching the car.At the handling limit, these delays can cause instabilities; limiting top speed and wheel torques is used as a workaround.

8 Conclusion

The system enabled autonomous driving and mapping on unknown tracks, then supported faster driving after loop closure detection. Extensive testing culminated in competition victories, while future work targets delays, perception, estimation, control, fault recovery, and testing automation.

  • Conclusion: The autonomous racecar can drive on unknown tracks, create their maps, and drive faster after loop closure detection.The conclusion presents these capabilities as outcomes of the complete design.
  • Conclusion: In 2017, AMZ Driverless was the only team among 15 FSG entrants to finish all disciplines.In 2018, the team won both FSI and FSG while improving on previous results and setting numerous score records.
  • Conclusion: The results demonstrated the capabilities and scalability of the software stack and the potential of the created tools.The conclusion links these findings to the system’s competition use.
  • Future Work: The 2019 development focus included reducing delays, extending perception and estimation, improving control efficiency, and automating failure recovery and testing.The planned racecar aimed to match human-driver performance.

A.1 Observability Analysis

The observability analysis evaluates whether the vehicle state can be reconstructed from measurements and control inputs, including under sensor failures. Full nine-state estimation requires velocity measurements; without them, a partial kinematic model assumes negligible slip below 6 m s−1.

  • Observability Analysis: A system is observable when its state can be fully reconstructed from measurements and control inputs.The analysis examines whether the state estimate converges to the true value after sensor failure.
  • Observability Analysis: For the nonlinear vehicle model, observability is tested by applying a rank test to an observability matrix constructed using Lie derivatives.The matrix is defined recursively in the analysis.
  • Observability Analysis: Full nine-state estimation requires a velocity measurement for the observability matrix to have full rank.Iteratively removing measurements identifies velocity as necessary for estimating all nine states.
  • Observability Analysis: If all velocity measurements fail, the model switches to a partial kinematic model assuming slip ratio is sufficiently close to zero.This assumption holds in practice for speeds under 6 m s−1.

A.2 Slip Ratio Estimation

Wheel slip biases velocity estimates during high acceleration and braking, so the estimator models slip and corrects wheel-odometry information. Wheel-speed updates jointly estimate hub velocity and slip ratio within the EKF measurement model.

  • Slip Ratio Estimation: Wheel slip can reach 30% at high acceleration and become higher under heavy braking, biasing wheel-odometry velocity estimates.The system therefore estimates slip ratio to correct misleading odometry information.
  • Slip Ratio Estimation: Slip ratio is defined separately for each wheel using its angular velocity, linear velocity, and wheel radius.The wheel indices distinguish front or rear and left or right wheels.
  • Slip Ratio Estimation: Wheel-speed measurement updates are split into linear hub velocity and slip-ratio components in the measurement model.Both components are updated simultaneously in the EKF using wheel-speed measurements.
  • Slip Ratio Estimation: The wheel-specific measurement equations incorporate steering angle, yaw rate, axle distances, track width, wheel radius, and slip ratios.The equations describe the four wheel-speed measurement components.

A.3 FastSLAM 2.0 Algorithm Details

The mapping phase uses FastSLAM 2.0 to build a colored 2D landmark map while simultaneously localizing the vehicle. Particles maintain map and pose hypotheses, updated through EKF-based proposals, landmark association, observation updates, and coherence-based weighting.

  • Map representation and particle state: The mapping phase creates a 2D map of landmarks while simultaneously localizing the vehicle within a global frame.Each particle carries both a map and a pose, with the global frame initialized at mapping start.
  • Map representation and particle state: Each landmark stores a Gaussian position estimate, a categorical color estimate, and counters for observed and missing detections.Colors include yellow, blue, orange, and unknown; the map is defined as a set of N landmarks.
  • Pose and landmark updates: For each observation update, particle poses are proposed from an EKF-based Gaussian using the previous pose composed with odometry, with observation delays compensated by integrated odometry.Matched observations are then incorporated iteratively into the pose proposal EKF.
  • Pose and landmark updates: Landmark positions are updated with EKF equations, new landmarks are initialized from their observed locations, and landmark colors are updated through Dirichlet-based Bayesian inference with discrete counters.The color model uses K = 3 possible outcomes for Bayesian updates.
  • Particle weighting and resampling: Particle weights measure map coherence with new observations by rewarding new landmarks and matched landmarks while penalizing missed landmarks and color mismatches.Resampling is enforced when the effective sample size falls below Neff < 0.6N.
Loading 1905.05150v1…