Source-linked AI summary
AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles
Shital Shah, Debadeepta Dey, Chris Lovett, Ashish Kapoor
TL;DR
Autonomous-vehicle development needs large, diverse training data and safer, less costly testing than real-world operation provides. AirSim offers an extensible, modular simulator with realistic physics, visuals, sensors, and vehicle models; simulated and real flight tracks were fairly close, while several sensing and environmental effects remain unmodeled.
Problem
Autonomous-vehicle learning and testing require large annotated datasets across conditions, but real-world operation is expensive and unsafe and simulation-to-reality transfer is difficult.
Method
AirSim combines Unreal Engine rendering with modular, extensible physics, vehicle, environment, sensor, API, and firmware interfaces.
Results
Simulated and real-world flight tracks were fairly close, with symmetric Hausdorff distances of 1.47 m for a circle and 0.65 m for a square.
Takeaways & Limitations
AirSim supports generating large quantities of training data and developing simulator-based algorithms intended for deployment on real vehicles without change.
Takeaways & Limitations
AirSim does not yet simulate richer collision response, advanced ground interaction, several camera sensor effects, GPS degradation from obstacles, advanced wind effects, or thermal simulations.
Abstract
from arXiv · showhide
Developing and testing algorithms for autonomous vehicles in real world is an expensive and time consuming process. Also, in order to utilize recent advances in machine intelligence and deep learning we need to collect a large amount of annotated training data in a variety of conditions and environments. We present a new simulator built on Unreal Engine that offers physically and visually realistic simulations for both of these goals. Our simulator includes a physics engine that can operate at a high frequency for real-time hardware-in-the-loop (HITL) simulations with support for popular protocols (e.g. MavLink). The simulator is designed from the ground up to be extensible to accommodate new types of vehicles, hardware platforms and software protocols. In addition, the modular design enables various components to be easily usable independently in other projects. We demonstrate the simulator by first implementing a quadrotor as an autonomous vehicle and then experimentally comparing the software components with real-world flights.
1 Introduction
AirSim addresses the high cost, time, and data demands of autonomous-vehicle development by narrowing the simulation–reality gap with an open-source platform.
- Autonomous-vehicle learning requires prohibitively large amounts of training data, while real-world operation is unsafe and expensive during training.
- Realistic transfer from simulation remains difficult because simulated perception, environments, and actuators often lack real-world richness and diversity.
- The simulator generates real-time depth, object-segmentation, and front-camera streams during aerial flight in urban environments.
- AirSim is an open-source platform designed to narrow the simulation–reality gap for autonomous-vehicle development.
2 Related Work
Prior simulators provide modularity, robotics integration, or hardware testing, but their rendering, environment richness, perception capabilities, or openness remain limited in different ways.
- Gazebo offers modular physics, sensors, and worlds for general robots, but creating large-scale visually rich environments has been difficult.
- Hector integrates ROS and Gazebo with flight dynamics, sensor bias drift, and software-in-loop, but lacks Pixhawk and MavLink support.
- RotorS provides a modular micro-aerial-vehicle framework and Pixhawk HITL, but its Gazebo foundation limits perception-related capabilities.
- jMavSim supports PX4 firmware and devices but uses simpler sensors and rendering without environmental objects.
- Many game-like simulators and training applications are commercial closed-source systems with little public information about models, accuracy, or development APIs.
3 Architecture
AirSim uses a modular architecture linking vehicle, physics, environment, sensor, rendering, API, and firmware interfaces to simulate autonomous aerial-vehicle operation.
- AirSim’s core architecture contains environment, vehicle, physics, sensor, rendering, public API, and vehicle-firmware interface components.
- Flight-controller firmware receives simulated sensor data and returns actuator signals consumed by the simulated vehicle model.
- The vehicle model converts actuator signals into forces and torques, which the physics engine uses to compute the next kinematic state.
- The resulting kinematic state and environmental quantities provide ground truth for simulated sensor models.
- Detailed simulated environments support companion-computer workloads involving vision, lidar, waypoint selection, and SLAM.
3.1 Vehicle Model
AirSim models vehicles as extensible rigid bodies whose actuator controls generate forces and torques, with physical parameters supporting dynamic simulation.
- Vehicle parameters include mass, inertia, drag, friction, and restitution coefficients used by the physics engine for rigid-body dynamics.
- A vehicle is defined by K vertices with positions, normals, and vehicle-specific scalar controls that generate forces and torques along the normals.
- Quadrotor controls drive propeller rotational speeds, whose forces and torques are computed from a propeller model.
- Moving actuator vertices during flight supports vehicles such as VTOL aircraft and reconfigurable quadrotors.
- The vehicle interface exposes cross-sectional area so the physics engine can compute linear and angular drag.
3.2 Environment
AirSim models environmental phenomena with computationally efficient approximations intended to support real-time hardware-in-the-loop operation. Its environment models cover gravity, magnetic fields, atmospheric pressure, and air density across relevant spatial conditions.
- Environment: AirSim prioritizes environmental models that are accurate enough for real-time hardware-in-the-loop operation.The paper explicitly balances model accuracy against computational cost.
- 3.2.1 Gravity: Gravity is approximated as a function of altitude using Newton’s law and a binomial expansion that neglects higher-order terms.The approximation uses Earth’s radius and surface gravitational acceleration as parameters.
- 3.2.2 Magnetic Field: The tilted dipole model approximates Earth’s magnetic field while capturing variation with geographic position and altitude, including polar regions.The model computes magnetic co-latitude, field intensity, inclination, declination, and field components.
- 3.2.3 Air Pressure and Density: Atmospheric pressure and density are modeled with standard-atmosphere formulations below 51 kilometers and an alternative model up to 86 kilometers.Air density is computed from pressure, temperature, and the specific gas constant.
3.3 Physics Engine
The physics engine computes vehicle kinematics from forces, torques, environmental effects, collisions, and numerical integration. It targets high-frequency updates while using physically motivated approximations for drag and orientation dynamics.
- Physics Engine: The physics engine computes each body’s next kinematic state from applied forces and torques at an update frequency of 1000 Hz.The state includes position, orientation, linear and angular velocities, and linear and angular accelerations.
- 3.3.1 Forces and Torques: Linear drag is computed from air-drag parameters, vehicle cross-sectional area, and air density, acting opposite the velocity vector.The drag model accounts for the vehicle’s motion through air.
- 3.3.1 Forces and Torques: Angular drag is approximated by integrating linear drag over the vehicle surface, with the body simplified as connected faces or a rectangular box.This replaces the computationally intensive calculation of angular drag for arbitrary shapes.
- 3.3.1 Forces and Torques: Net forces and torques combine drag with forces and torques applied at vehicle vertices, after which Newtonian and Euler rigid-body equations produce accelerations.Linear acceleration includes gravity, while angular acceleration uses the inertia tensor and body-frame angular velocity.
- 3.3.3 Integration: Velocity Verlet integration updates position with second-order accuracy while avoiding the rapid unbounded error associated with first-order Euler integration.The method is selected over Runge–Kutta for computational inexpensiveness and stability.
- 3.3.3 Integration: Orientation updates use numerically stable, normalizable quaternions and compose the body-frame incremental rotation with the world-frame orientation quaternion.The incremental rotation is obtained from the angular velocity’s angle-axis representation.
- 3.3.2 Collision Detection: Collision responses use Unreal Engine impact data and Coulomb friction to modify both linear and angular kinematics.The engine receives impact position, normal, and penetration depth for collisions during each render interval.
3.4 Sensors
AirSim provides modular sensor models for common navigation and inertial sensors, generating measurements from environmental and vehicle ground truth. The models add realistic noise, bias, drift, latency, and update-rate effects.
- Sensors: AirSim provides independently usable, replaceable models for accelerometer, gyroscope, barometer, magnetometer, and GPS sensors.The models are implemented as header-only C++ libraries and exposed through abstract interfaces.
- 3.4.1 Barometer: The barometer combines detailed atmospheric ground-truth pressure with Gaussian Markov bias drift and sensor-specific white noise before converting pressure to altitude.The drift process uses a one-hour time constant in the described model.
- 3.4.2 IMU: Gyroscope and accelerometer measurements add white noise and time-varying bias drift to ground truth, with accelerometer processing removing gravity and transforming into the body frame.Gyroscope parameters can be obtained from Allan variance plots or sensor datasheets.
- 3.4.3 Magnetometer: The magnetometer uses a tilted dipole Earth model to compute body-frame magnetic-field components and adds datasheet-specified white noise.The model varies with geographic position and altitude.
- 3.4.4 GPS: The GPS model simulates typically 200 ms latency, typically 50 Hz updates, and separately parameterized horizontal and vertical position-fix decay.Fix decay is modeled with first-order low-pass filters.
3.5 Visual Rendering
AirSim uses Unreal Engine 4 as its rendering platform to support advanced rendering and detailed environments for autonomous-vehicle simulation.
- Visual Rendering: Unreal Engine 4 provides AirSim’s rendering platform for detailed environments and physically based visual effects.Relevant features include physically based materials, photometric lights, planar reflections, distance-field shadows, and lit translucency.
4 Experiments
Experiments compare AirSim’s quadrotor simulation with real-world flights and sensor measurements, finding close trajectory and sensor characteristics while identifying modeling sources of residual differences.
- Experimental setup: The experiments configured AirSim and a real quadrotor with matched physical parameters and sensor settings, using repeatable MavLinkTest offboard control.The real platform used a Pixhawk v2 controller, Flamewheel frame, and Gigabyte 5500 Brix; simulated sensor models were configured from datasheets.
- Trajectory evaluation: Simulated and real flights followed identical square and circular commands, with local NED positions and timestamps collected for both.The square trajectory had 5 m sides and the circle had a 10 m radius.
- Trajectory evaluation: 1.47 m and 0.65 m Hausdorff distances separated simulated and real-world tracks for the circle and square trajectories, respectively.The reported distances indicate that the tracks were fairly close in both patterns.
- Trajectory evaluation: The simulated and real trajectories were qualitatively close in visual comparisons of the circle and square patterns.Figures 4(a) and 4(b) use purple for simulated trajectories and red for real trajectories.
- Sensor models: Sensor experiments evaluated barometer, magnetometer, and IMU models using manufacturer datasheets, while the GPS model was excluded because it was simplistic.The evaluation included stationary and inflight IMU readings and repeated barometer height changes.
- Sensor models: Inflight accelerometer variance was much higher in the real world than in simulation, likely because AirSim did not model airframe vibration from running motors.The reported accelerometer variances were 1.75e−3 m2/s4 simulated versus 9.46 m2/s4 real-world.
- Sensor models: Simulated barometer and magnetometer signals closely resembled real-world characteristics, with pressure offsets attributed to different absolute pressures and temperature effects not simulated.The reported barometer comparison found that the simulated sensor characteristics matched the real sensor well.
5 Conclusion and Future Work
AirSim provides high-fidelity physical and visual simulation with modular, extensible components intended for autonomous-vehicle development and inexpensive machine-learning data generation. Future work targets richer physical, sensor, GPS, wind, and thermal modeling.
- Conclusion: AirSim provides high-fidelity physical and visual simulation for generating large quantities of machine-learning training data cheaply.The platform is intended to support autonomous agents and reinforcement-learning development.
- Conclusion: Its physics engine, vehicle, environment, and sensor models are independently usable with minimal dependencies and are designed for extensibility.The extensibility APIs are intended to accommodate future vehicle types and related development.
- Future work: AirSim currently lacks richer collision response, advanced ground interaction, several camera effects, advanced noise and lens models, and GPS degradation from obstacles.The paper identifies these as areas for improvement in real-world mimicry.
- Future work: Planned extensions include advanced wind effects, thermal simulations for fixed-wing vehicles, and future physics and ray-tracing capabilities.These plans are framed as uses of the simulator’s extensibility interfaces.