Source-linked AI summary
LGSVL Simulator: A High Fidelity Simulator for Autonomous Driving
Guodong Rong, Byung Hyun Shin, Hadi Tabatabaee, Qiang Lu, Steve Lemke, Mārtiņš Možeiko, Eric Boise, Geehoon Uhm, Mark Gerow, Shalin Mehta, Eugene Agafonov, Tae Hyung Kim, Eric Sterner, Keunhae Ushiroda, Michael Reyes, Dmitry Zelenkovsky, Seonman Kim
TL;DR
Real-vehicle testing and manual dataset labeling are costly, while open-source AD stacks have limited simulator choices. The paper introduces an open, Unity-based LGSVL Simulator that integrates with Autoware and Apollo and supports extensible, end-to-end simulation. Its applications include software- and hardware-in-the-loop testing, digital twins, generated scenarios, and custom sensors.
Problem
Real-vehicle testing is extremely costly, and manually labeled autonomous-driving data is limited, slow, costly, and error-prone for complex ground truths.
Method
The paper introduces an open, Unity-based high-fidelity simulator with communication bridges, sensor and environment tools, digital twins, and extensible simulation components.
Results
LGSVL Simulator integrates with Autoware and Apollo for end-to-end tests and supports applications including SIL, HIL, digital-twin, scenario-generation, and custom-sensor simulation.
Takeaways & Limitations
The open ecosystem lets users apply LGSVL Simulator to autonomous-driving and related systems and add contributions for different applications.
Abstract
from arXiv · showhide
Testing autonomous driving algorithms on real autonomous vehicles is extremely costly and many researchers and developers in the field cannot afford a real car and the corresponding sensors. Although several free and open-source autonomous driving stacks, such as Autoware and Apollo are available, choices of open-source simulators to use with them are limited. In this paper, we introduce the LGSVL Simulator which is a high fidelity simulator for autonomous driving. The simulator engine provides end-to-end, full-stack simulation which is ready to be hooked up to Autoware and Apollo. In addition, simulator tools are provided with the core simulation engine which allow users to easily customize sensors, create new types of controllable objects, replace some modules in the core simulator, and create digital twins of particular environments.
I. INTRODUCTION
The paper motivates simulation as a safer, scalable alternative to costly real-vehicle testing and manual data labeling, then introduces LGSVL Simulator as an open, high-fidelity system integrated with major open-source AD stacks.
- Motivation: Simulation enables faster and safer testing of autonomous-driving algorithms, including rare dangerous scenarios that are difficult to encounter on real roads.Examples include pedestrian jaywalking and extreme weather.
- Motivation: Real-world autonomous-driving datasets are costly and error-prone to label, while some ground-truth types are extremely difficult or impossible to annotate manually.This particularly affects pixel-wise segmentation and optical-flow labels.
- Contribution: LGSVL Simulator is a Unity-based, open-source simulator with a communication bridge supporting ROS, ROS2, and Cyber RT messages.These interfaces support integration with Autoware and Baidu Apollo.
- Contribution: The simulator provides map tools for importing and exporting HD maps in Lanelet2, OpenDRIVE, and Apollo HD Map formats.
- Contribution: LGSVL Simulator is presented as a response to limited open-source simulator choices for available autonomous-driving stacks.
II. RELATED WORK
Prior simulators cover vehicle dynamics, robotics, synthetic data, or virtual environments, but differ in environment complexity, sensor support, AD-stack integration, and customizability.
- Autonomous-driving simulators: Earlier autonomous-driving simulators lacked important capabilities such as diverse sensor support and pedestrian simulation.
- Robotics platforms: Gazebo supports pluggable sensor models and physics engines but is difficult to use for large, complex environments and lacks modern game-engine rendering advances.
- Open-source simulators: AirSim, CARLA, and Deepdrive were typically built for reinforcement learning or synthetic-data generation and often require substantial AD-stack integration work.
- Commercial simulators: Commercial automotive simulators can be difficult to customize because their source code is not open.
- Driving games: Driving games provide realistic environments but generally require hacking, may violate licenses, and offer limited support for non-camera sensors and deterministic control.
III. OVERVIEW OF LGSVL SIMULATOR
LGSVL Simulator connects a user’s AD stack to a simulation workflow through runtime-specific communication bridges, while supporting multiple stacks and simultaneous interaction in one environment.
- A. User AD Stack: The workflow centers on the user AD stack, which is developed, tested, and verified through simulation.
- A. User AD Stack: LGSVL Simulator provides out-of-the-box integration with Apollo, Autoware.AI, and Autoware.Auto.
- A. User AD Stack: Communication bridges are selected according to the AD stack runtime, including Cyber RT for Apollo and ROS or ROS2 bridges for Autoware.
- A. User AD Stack: Custom runtime frameworks can be connected through additional bridge plug-ins.
- A. User AD Stack: Multiple AD systems can connect simultaneously through dedicated bridges and interact within one unified simulation environment.
B. Simulation Engine
The simulation engine uses Unity to create photorealistic environments and combines environment, sensor, and ego-vehicle dynamics and control simulation, with open-source distribution.
- B. Simulation Engine: Unity technologies including HDRP support photorealistic virtual environments intended to match the real world.
- B. Simulation Engine: The engine comprises environment simulation, sensor simulation, and ego-vehicle dynamics and control simulation.
- B. Simulation Engine: Environment simulation covers traffic, weather, and time of day, all controllable through the Python API.
- B. Simulation Engine: The simulation engine is open source, with publicly available source code and a free downloadable executable.
C. Sensor and Vehicle Models
LGSVL provides configurable physical and virtual sensors, supports real-world sensor models, and allows custom sensor plugins and external vehicle-dynamics integration.
- Sensor Models: Users configure sensor type, placement, publishing rate, topics, reference frames, and specifications such as LiDAR beam count through JSON.
- Sensor Models: The simulator offers camera, LiDAR, Radar, GPS, IMU, and virtual ground-truth sensors, with custom sensors supported through plugins.The default sensor set includes physical and virtual ground-truth options.
- Sensor Models: Segmentation sensors combine semantic and instance segmentation, with configurable instance-level labeling for selected object semantics.
- Sensor Models: Real-world sensor plugins match counterpart parameters and output formats, while users can implement new sensor variations or types.Velodyne VLP-16 LiDAR is given as an example.
- Vehicle Models: LGSVL includes basic ego-vehicle dynamics and supports third-party dynamics through FMI, shared libraries, or separate IPC interfaces.
D. 3D Environment and HD Maps
LGSVL combines 3D environments, digital twins, and editable HD maps to provide realistic simulation inputs and interoperable map data for autonomous-driving systems.
- 3D Environment: The virtual environment supplies inputs to perception, prediction, tracking, vehicle control, localization, planning, and environmental simulation.It also supports weather, time of day, traffic agents, and other dynamic objects.
- 3D Environment: Digital twins replicate real-world locations from logged images and point-cloud data, including Borregas Avenue and GoMentum Station.GoMentum testing used both simulation and a real test vehicle at the facility.
- HD Maps: LGSVL supports creating, editing, and exporting HD-map annotations within existing 3D environments.Annotations include lanes, boundaries, signals, signs, and pedestrian routes.
- HD Maps: HD maps can be exported as Apollo 5.0 HD Map, Autoware Vector Map, Lanelet2, or OpenDrive 1.4, and supported real-world maps can be imported.
E. Test Scenarios
LGSVL test scenarios vary environments and agent behavior to verify autonomous-driving-stack behavior, with scripting and deterministic physics enabling repeatable testing.
- Scenario Construction: Scenarios vary time of day, weather, road conditions, and the distribution and movement of vehicles and pedestrians.
- Scenario Construction: The Python API lets users spawn and control NPC vehicles and pedestrians and set environmental parameters.
- Scenario Construction: Deterministic physics makes scripted scenarios repeatable for simulation testing.
- Scenario Evaluation: SCENIC generated thousands of varied test cases in the GoMentum digital twin, and simulation results informed real-world scenario selection.
IV. APPLICATIONS
LGSVL supports multiple autonomous-driving simulation applications, with its open ecosystem intended to enable further domain extensions.
- Applications: LGSVL supports various simulation applications for autonomous driving and other domains.
A. SIL and HIL Testing
LGSVL Simulator supports both software-in-the-loop and hardware-in-the-loop testing, enabling end-to-end, unit-level, and vehicle-control testing of autonomous-driving stacks.
- LGSVL Simulator supports both software in the loop (SIL) and hardware in the loop (HIL) testing of AD stacks.
- For SIL testing, the simulator generates camera, LiDAR, GPS, and IMU data for end-to-end testing of perception and localization modules.
- The simulator can also generate module inputs such as 3D bounding boxes, allowing users to bypass perception for planning unit tests.
- Chassis commands let an AD stack on another machine control the simulated ego vehicle for HIL testing as it would control a real car.
- Thousands of Scenic-generated scenarios were tested in a GoMentum Station digital twin, with representative cases also tested at the real station.
B. Machine Learning and Synthetic Data Generation
LGSVL Simulator supports synthetic-data generation, reinforcement learning, V2X testing, and smart-city traffic applications through programmable APIs and sensor plugins.
- The Python API generates camera and LiDAR data with ground truth such as occlusion, truncation, bounding boxes, and segmentation labels.
- Users can configure sensor parameters and generate labeled perception-training data in custom formats, including KITTI format.
- Out-of-the-box OpenAI Gym integration makes LGSVL Simulator an environment for reinforcement learning with OpenAI Gym.
- Custom V2X sensor plugins can provide information from vehicles, pedestrians, and infrastructure for V2X testing and synthetic-data generation.
- A user-defined sensor can count vehicle stops from exact simulated speed and location information for smart-city traffic evaluation.
- Controllable plugins let users customize traffic lights and other special traffic signs through the Python API.
V. CONCLUSIONS
LGSVL Simulator is an open, Unity-based high-fidelity simulator integrated with Autoware and Apollo and designed for extension across autonomous-driving applications.
- LGSVL Simulator is a Unity-based high-fidelity simulator for autonomous driving and related systems.
- It integrates with Autoware and Apollo for end-to-end tests and can be extended to other similar autonomous-driving systems.
- Application examples demonstrate the capabilities of the LGSVL Simulator.
- The open-source simulation engine and open ecosystem allow users to apply the simulator to different applications and contribute additions.