Source-linked AI summary
Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, Gavriel State
TL;DR
Robotics reinforcement learning is limited by the computational and data-transfer costs of conventional CPU–GPU simulation pipelines. Isaac Gym instead keeps physics simulation, policy training, and their buffers on the GPU through a Tensor API, achieving fast training across challenging robotic tasks and demonstrating sim-to-real transfer. The paper reports 2-3 orders of magnitude of training speed-up and single-GPU performance that previously required large heterogeneous clusters.
Problem
Conventional robotics RL requires substantial computation and incurs CPU–GPU transfer bottlenecks, limiting simulation speed for challenging tasks.
Method
Isaac Gym combines GPU-accelerated PhysX simulation with a PyTorch tensor-based API that keeps observations, rewards, and actions on the GPU during end-to-end training.
Results
2-3 orders of magnitude of training speed-up are achieved for continuous-control tasks, with challenging environments trained on a single NVIDIA A100 GPU.
Takeaways & Limitations
Isaac Gym enables high-fidelity, fast training across challenging simulated robotic environments and supports sim-to-real transfer demonstrations on ANYmal and TriFinger.
Takeaways & Limitations
GPU simulation alone can retain performance bottlenecks when physics state must be copied back to the CPU for observations and rewards.
Abstract
from arXiv · showhide
Isaac Gym offers a high performance learning platform to train policies for wide variety of robotics tasks directly on GPU. Both physics simulation and the neural network policy training reside on GPU and communicate by directly passing data from physics buffers to PyTorch tensors without ever going through any CPU bottlenecks. This leads to blazing fast training times for complex robotics tasks on a single GPU with 2-3 orders of magnitude improvements compared to conventional RL training that uses a CPU based simulator and GPU for neural networks. We host the results and videos at \url{https://sites.google.com/view/isaacgym-nvidia} and isaac gym can be downloaded at \url{https://developer.nvidia.com/isaac-gym}.
1 Introduction
Isaac Gym addresses the computational and data-transfer bottlenecks of conventional robotics reinforcement learning with an end-to-end GPU pipeline. Its Tensor API keeps simulation and policy-training data on the GPU, enabling fast training across diverse robotic environments on a single GPU.
- Motivation: Conventional robotics RL is constrained by computational demands and limited simulation speed, especially for long-horizon behaviors and high-degree-of-freedom robots.CPU–GPU transfers during physics simulation, reward computation, and policy training add inefficiency.
- Approach: Isaac Gym provides a high-fidelity GPU-accelerated simulator and a Python Tensor API that exposes physics buffers as PyTorch tensors without CPU bottlenecks.The platform uses NVIDIA PhysX and directly feeds observation tensors to policies while returning action tensors to the physics system.
- Approach: Tens of thousands of simultaneous environments can run on a single GPU because observation, reward, and action buffers remain on the GPU throughout learning.Isaac Gym duplicates environments for parallel simulation while supporting variations such as domain randomization.
- Approach: 100-1000x speedup is enabled by stepping PhysX and accessing simulator states directly on the GPU through the Tensor API.The pipeline also supports high-fidelity simulation and interfaces with existing robot models.
- Results: 20 seconds, 4 minutes, under 2 minutes, 6 minutes, and 35 minutes achieve performant training for Ant, Humanoid, ANYmal, AMP animation, and Shadow Hand cube rotation respectively on one NVIDIA A100 GPU.The paper also reports reproducing OpenAI Shadow Hand performance in about 1 hour and 6 hours for feed-forward and LSTM policies, versus 30 hours and 17 hours in the conventional setup.
2 Background
Isaac Gym targets the throughput limits of robotics reinforcement learning by parallelizing physics on GPUs and keeping simulation data in tensor-based interfaces. Its design supports many concurrent environments while avoiding repeated CPU-device transfers.
- GPU simulation parallelizes computation across shapes, bodies, and joints, making it well suited to thousands of environment instances.
- Isaac Gym keeps physics stepping, observation and reward computation, and action application on the GPU without copying large data quantities between devices.
- The interface supports actors, rigid bodies, and degrees of freedom, with physics states and controls exposed through tensors.
- The tensor API exposes physics data in flat global buffers, reducing user-code overhead from iterating over many individual actors.
- Multiple environment instances can be represented as tensor dimensions, enabling vectorized observation and reward computation through parallel GPU kernels.
- Tensor-wrapping utilities share native CPU or GPU buffers with Python frameworks such as PyTorch without copying data.
3 Physics Simulation
Isaac Gym models robot bodies and tendons with PhysX-based mechanics and uses a Temporal Gauss Seidel solver to compute future object states. The simulator exposes tuning parameters for controlling its behavior.
- Robots are simulated as PhysX reduced-coordinate articulations, while individual rigid bodies may use maximal-coordinate bodies or single-link articulations.
- Fixed Tendon mechanics model tendon actuation of degrees of freedom, and their dynamics are tested in the Shadow Hand environment.
- The Temporal Gauss Seidel solver computes future object states by combining sub-stepping effects with per-iteration accumulated velocity updates.
- The simulator exposes parameters that users can tune.
4 Environments
The evaluation covers diverse locomotion, manipulation, animation, and robotic-hand environments using GPU-vectorized PPO training. Experiments use standardized hardware, repeated seeds, and actor-critic configurations across tasks.
- All environments are trained with PPO using a GPU end-to-end implementation that vectorizes observations and actions.
- The environment suite includes locomotion, Franka cube stacking, humanoid character animation, and Shadow, Allegro, and Trifinger hand tasks.
- The suite emphasizes complex robotic-hand environments beyond relatively simple Ant and Humanoid benchmarks.
- Unless stated otherwise, experiments use one NVIDIA A100 GPU and one 3.7GHz Intel i7-8700K CPU.
- Training runs are averaged over 5 seeds, with reward curves plotted using µ ± σ regions.
- The default setup uses a symmetric actor-critic with shared observations and a shared policy-value network, while Shadow Hand and TriFinger also use asymmetric critics with privileged simulation state.
5 Characterising Simulation Performance
Simulation performance improves with parallel environments, but the best configuration depends on task complexity and horizon length. Ant reaches the highest throughput, while Humanoid and Shadow Hand remain scalable at lower rates.
- Ant: 1000 seconds to 100 seconds: increasing Ant environments from 256 to 8192 reduces time to reach 7000 reward by an order of magnitude.
- Ant: 700K parallel environment steps per second: Ant reaches this rate, with no further gain from increasing environments beyond 8192 under the tested horizon.
- Humanoid: 104 seconds to 103 seconds: increasing Humanoid environments from 256 to 4096 reduces time to reach 7000 reward by an order of magnitude.
- Humanoid: 8192 and 16384 environments remain viable for Humanoid when horizon lengths are reduced to 32 and 16.
- Humanoid: 200K parallel environment steps per second: Humanoid achieves a lower rate than Ant because it has more degrees of freedom.
- Shadow Hand: 150K parallel environment steps per second: Shadow Hand reaches its maximum effective frame rate with 16384 agents, while horizon length 8 still permits learning.
6 Characterising Environment Performance
Isaac Gym evaluates scalable GPU-based training across locomotion, animation, and dexterous manipulation environments. The experiments report rapid learning and, for selected tasks, transfer from simulation to real robots.
- 6.1 Environment Performance: 540K environment steps per second enabled Ant to exceed reward 3000 in 20 seconds and fully converge in under 2 minutes.The experiment used 4096 agents simulated in parallel on an A100 GPU.
- 6.1 Environment Performance: Reward threshold 5000 was reached by Humanoid in less than 4 minutes, 4x faster than the previous results using the same threshold.The Humanoid environment has 21 DOFs and used 4096 parallel agents.
- 6.1 Environment Performance: Reward 5000 was achieved by the simplified Ingenuity environment in just under 30 seconds with 4096 agents.The model applied forces directly to two rotors and used martian gravity of -3.721 m/s2.
- 6.1 Environment Performance: ANYmal learned to follow randomized target velocities in under 2 minutes with 4096 agents simulating in parallel.The rough-terrain variant was transferred to a real robot that walked on uneven surfaces, slopes, stairs, and obstacles.
- 6.2 Humanoid Character Animation: 300x faster training was achieved for humanoid motion imitation: approximately 39 million samples required 6 minutes versus 30 hours in PyBullet.AMP enabled imitation of motions ranging from walking and running to spin-kicks and dancing.
- 6.4 Dexterous Manipulation: More than 20 consecutive Shadow Hand successes took less than 1 hour, while 37 successes with LSTMs were achieved in just under 6 hours.The OpenAI-observation asymmetric actor-critic setting was suited for sim-to-real transfer; TriFinger achieved a 55% mean real-world success rate.
7 Summary
Isaac Gym is presented as a high-performance, high-fidelity simulator for challenging robotic environments. It enables fast training on a single NVIDIA A100 GPU and supports contact-rich manipulation with sim-to-real demonstrations.
- 7 Summary: Isaac Gym trains challenging simulated robotic environments on a single NVIDIA A100 GPU that previously required heterogeneous CPU and GPU clusters.The framework is described as both high performance and high fidelity.
- 7 Summary: Sim-to-real demonstrations with ANYmal locomotion and TriFinger cube reposing confirm the backend's suitability for contact-rich manipulation.The conclusion connects the simulation backend to both locomotion and manipulation transfer demonstrations.
- 7 Summary: The paper positions Isaac Gym as a framework for fast training across many challenging robotic environments rather than a single task.Its stated scope spans multiple environments and robot-learning applications.
A.1 Tendons
The appendix describes tendon simulation within the Shadow Hand environment. It directs readers to the following section for the simulation details.
- A.1 Tendons: Tendons are simulated as part of the Shadow Hand environment.The appendix identifies Shadow Hand as the test environment for tendon simulation.
- A.1 Tendons: The tendon simulation details are described in the appendix section that follows.The passage serves as a transition to the fixed-tendon description.
- A.1 Tendons: The Shadow Hand environment is the concrete setting used to test the tendon dynamics.The passage links the tendon model to this manipulation environment.
A.1.1 Fixed Tendons
Fixed tendons couple articulation degrees of freedom through a tree of tendon joints. Their recursively computed lengths generate spring, damping, and limit forces in the connected links.
- A.1.1 Fixed Tendons: Fixed tendons couple degrees of freedom in an articulation through a tree of tendon joints.Each tendon joint is associated with exactly one axis of a link's incoming articulation joint.
- A.1.1 Fixed Tendons: Tendon length is computed recursively as parent length plus joint position scaled by that joint's coefficient.Each tendon joint coefficient determines the contribution of rotational or translational position to tendon length.
- A.1.1 Fixed Tendons: Deviation from the tendon-wide rest length produces spring force or torque on the child link and an equal opposing force on the root parent link.The tendon joint also applies damping, while separate length limits add proportional force or torque near the limits.
A.1.2 Spatial Tendons
Spatial tendons impose weighted line-of-sight distance constraints between articulation links through positioned attachments. They can branch into sub-tendons, with constraints evaluated separately along each root-to-leaf path and forces applied only at endpoint attachments.
- Spatial tendons constrain line-of-sight distances between links of a single articulation through positioned attachments.Their length is a weighted sum of distances between tendon attachments and may use multiple attachments per link.
- Spatial tendon topology need not follow the articulation topology, unlike fixed tendons.
- Branching spatial tendons split into conceptual sub-tendons for each root-to-leaf path, with length and limit constraints evaluated per sub-tendon.Spring-damper dynamics can contract or extend each sub-tendon; suitable limits can create a one-sided, string-like constraint.
- Sub-tendon forces act on leaf and root attachments but do not propagate through intermediate attachments.
A.2 Observations & Rewards
The environments define task-specific observations and reward functions across locomotion, manipulation, and dexterous rotation tasks. These formulations combine task progress, pose alignment, action-related terms, success conditions, and domain-randomization settings, with Shadow Hand experiments reporting rapid training outcomes.
- Ant and Humanoid: Ant and Humanoid use a shared reward formulation combining heading, velocity, effort, action, degree-of-freedom, and termination terms.The termination contribution is gated by whether torso height is at or below the termination height.
- Observations: Observation definitions are provided separately for Ant, Humanoid, Ingenuity, and ANYmal environments.The paper points to dedicated observation tables for each environment and defines reward terms and symbols in supporting tables.
- Franka Cube Stack: Franka Cube Stack rewards stacking, alignment, lifting, and reaching, with weights w_stack = 16.0, w_align = 2.0, w_lift = 1.5, and w_reach = 0.1.Stacking requires height, alignment, and gripper-separation conditions; alignment is gated on lifting.
- Shadow Hand: Shadow Hand reward computation combines object-goal distance, orientation alignment, action regularization, success bonuses, and fall penalties.The OpenAI variant uses separate actor and critic observation configurations, while the success code awards a bonus inside the orientation tolerance and penalizes large goal distance.
- Trifinger: Trifinger rewards include a logistic-kernel object-goal term, angular pose error, fingertip-distance change, and action-related terms.The logistic kernel converts Euclidean tracking error into a bounded reward, using a = 50 in the described formulation.
- Shadow Hand results: 20 consecutive successful cube rotations are achieved in just under 1 hour with feed-forward networks, while LSTMs reach 37 in just under 6 hours.Training with a 0.1-rad tolerance and testing with a 0.4-rad tolerance reaches 44 consecutive rotations.