PaperlayerView exact revision on arXiv

Source-linked AI summary

Gymnasium: A Standard Interface for Reinforcement Learning Environments

Mark Towers, Ariel Kwiatkowski, Jordan Terry, John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo Perez-Vicente, Andrea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Tan, Omar G. Younis

arXiv:2407.17032v4cs.LGcs.DL

TL;DR

OpenAI Gym had stopped receiving updates, and Gymnasium is presented as its maintained successor. The paper describes an environment API with built-in and compatible environments, functional and vectorized interfaces, and expanded spaces; the authors report broad adoption and identify limits in scope and execution speed.

  • Problem

    OpenAI Gym was no longer maintained, with no updates since October 2022.

    Source: §1 · p. 2

  • Method

    Gymnasium provides a maintained RL environment API with built-in and compatible environments, functional and vectorized interfaces, and expanded spaces for multimodal observations and actions.

    Source: §1 · p. 2

  • Results

    Gymnasium had over a million downloads in April 2025 and over 18 million from its November 2023 release through May 2025.

    Source: §1 · p. 2

  • Takeaways & Limitations

    The authors hope Gymnasium removes barriers from DRL research and accelerates development of safe, socially beneficial AI.

    Source: §1 · p. 2

    Gymnasium is only an environment API, so users need a separate training library, and its Python implementation can run slower than custom C, C++, or Cython environments.

    Source: §5 · p. 10

Abstract

from arXiv · show

Reinforcement Learning (RL) is a continuously growing field that has the potential to revolutionize many areas of artificial intelligence. However, despite its promise, RL research is often hindered by the lack of standardization in environment and algorithm implementations. This makes it difficult for researchers to compare and build upon each other's work, slowing down progress in the field. Gymnasium is an open-source library that provides a standard API for RL environments, aiming to tackle this issue. Gymnasium's main feature is a set of abstractions that allow for wide interoperability between environments and training algorithms, making it easier for researchers to develop and test RL algorithms. In addition, Gymnasium provides a collection of easy-to-use environments, tools for easily customizing environments, and tools to ensure the reproducibility and robustness of RL research. Through this unified framework, Gymnasium significantly streamlines the process of developing and testing RL algorithms, enabling researchers to focus more on innovation and less on implementation details. By providing a standardized platform for RL research, Gymnasium helps to drive forward the field of reinforcement learning and unlock its full potential. Gymnasium is available online at https://github.com/Farama-Foundation/Gymnasium

1 Introduction

After OpenAI Gym stopped receiving updates, Gymnasium emerged as its maintained successor. The paper presents a broad environment API, including functional and vectorized interfaces and support for multimodal observation/action spaces.

1 Introduction

  • Gymnasium provides a maintained environment API with built-in and compatible external environments and support from training libraries.The authors report over a million downloads in April 2025 and over 18 million from November 2023 to May 2025.Source: §1 · p. 2
  • A functional environment API supports theoretically aligned research, search algorithms, and vectorized hardware acceleration.Source: §1 · p. 2
  • Expanded vectorized-environment support includes specialized wrappers, custom implementations, alternative vectorization methods, and autoreset modes.Source: §1 · p. 2
  • Expanded algebraic spaces include text, graphs, disjoint unions, and variable-length sequences for multimodal observation/action environments.Source: §1 · p. 2

2 Related Work

Gymnasium is used alongside libraries for RL training, dataset management, and domain-specific environments. Alternative APIs differ in scope and focus, including lightweight and multi-agent designs.

2.1 Training Libraries

2.2 Other Tooling utilising Gymnasium

2.3 Alternative API Libraries

3 Library Structure

Gymnasium organizes RL environments through flexible interfaces that can be reused across environments and algorithms. Its library includes environment abstractions, tools for customization and reproducibility, and built-in and external environments.

3 Library Structure

  • Env represents an RL environment through observation and action spaces plus reset and step methods for starting episodes and advancing them.The spaces specify expected observation and action structures; reset begins an episode and step advances the environment after an action.Source: §3 · p. 4
  • VectorEnv batches results from independently running environments, making it possible to execute hundreds or thousands in parallel for higher steps per second.Its structure and core methods closely match Env, easing algorithm upgrades; it supports workloads using millions or billions of observations.Source: §3 · p. 5
  • Spaces specify the expected structure of observations and actions, with dtype and shape attributes and a method for generating random examples.Their attributes can help define replay-buffer structure and neural-network inputs and outputs.Source: §3 · p. 5
  • The registry identifies environments by namespace, name, and version, while version increments for behavior changes support reproducing historic research.Registered environments can be initialized with gymnasium.make using registered arguments or wrappers.Source: §3 · p. 5
  • Gymnasium supplies 33 wrappers for modifying reset or step behavior, plus utilities for API checking, keyboard control, and OpenAI Gym compatibility.Built-in wrappers include reward clipping, observation normalization, and MP4 recording.Source: §3 · p. 5

3.1 Implemented environments

  • Built-in environments span classic control, tabular tasks, Box2D simulations, and MuJoCo robotics; external projects include first- and third-party environments.These environments serve as implementation references or testing grounds, with uses ranging from gridworlds and arcade games to medical and UAV-control tasks.Source: §3.1 · pp. 5–6

4 Novel features

This section presents four core API changes in Gymnasium, developed through more than 800 pull requests from over 40 contributors.

4.1 Functional Environment API

  • FuncEnv expresses environments through functional components for initialization, transitions, observations, rewards, and terminal-state checks, mirroring POMDP elements.Its components correspond to initial-state sampling, state transitions, observations, rewards, and whether a state ends an episode.Source: §4.1 · p. 6
  • Direct state access supports theoretical and search research, while JAX-compatible stateless functions enable scalable hardware vectorization through FunctionalJaxVectorEnv.Existing Env implementations must be reimplemented as FuncEnv to use these advantages.Source: §4.1 · pp. 6–7

4.2 Termination and Truncation

  • Termination is state-dependent, truncation an external cutoff; because their value-estimation consequences differ, Gymnasium returns separate booleans where Gym used one.Truncation can stop an episode that otherwise would have continued accumulating rewards; Gymnasium changed Env.step to return distinct termination and truncation signals.Source: §4.2 · pp. 7–8

4.3 Vectorization

  • Async improves Lunar Lander throughput on high-RAM M3 hardware but degrades it on Colab, while Sync batches iteratively and Async uses subprocesses.The relative performance of SyncVectorEnv and AsyncVectorEnv depends on environment complexity and hardware.Source: Figure 3 · p. 8, §4.3 · p. 8
  • Custom CartPole vectorization uses parallel tensor operations, runs an order of magnitude faster than Sync or Async without their memory overhead, but requires custom implementation.The approach suits simple dynamics and is not normally compatible with more complex environments.Source: §4.3 · p. 8
  • Gymnasium supports next-step, same-step, and disabled autoreset: next-step is the default and fastest, same-step returns the final observation in info, and disabled gives users reset control.Next-step mode may include “dead” actions on reset timesteps; same-step mode prevents rendering the episode's final observation.Source: §4.3 · p. 9

4.4 Algebraic spaces

  • New Text, Sequence, Graph, and OneOf spaces expand built-ins for multimodal observations and actions, including connected-device graphs and variable-length obstacle sequences.The paper identifies these spaces as newly added and unique to Gymnasium.Source: §4.4 · p. 9

5 Summary

Gymnasium builds on OpenAI Gym as an RL environment API and benchmark suite, adding tools for vectorized environments, multimodal spaces, and functional environments. Its scope is limited to environment interfaces, and its Python implementation can run slower than custom C, C++, or Cython environments.

5 Summary

  • Gymnasium provides a standardized RL environment API and benchmark suite spanning finite MDPs to robotic simulations.The paper reports wide-scale adoption and compatibility with training libraries and external environments.Source: §5 · p. 9
  • Figure 5 organizes Gymnasium's built-in spaces into Fundamental and Composite categories.Composite spaces comprise subspaces, while Fundamental spaces cannot be divided into subspaces.Source: Figure 5 · p. 10
  • New features include vectorized environments, spaces for multimodal observations or actions, and a functional interface designed for hardware acceleration.These features build upon OpenAI Gym.Source: §5 · pp. 9–10
  • Gymnasium requires a separate training library, and its Python environments can be slower than custom C, C++, or Cython implementations.Gymnasium is an environment API rather than an agent-training library.Source: §5 · p. 10
Loading 2407.17032v4…