Source-linked AI summary
Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks
Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers, Rodrigo de Lazcano, Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro, Jordan Terry
TL;DR
RL benchmark libraries often emphasize control and visual understanding rather than logical reasoning or instruction following. This paper presents minimal, customizable Minigrid and Miniworld libraries for 2D and 3D goal-oriented tasks, documents their design and API, and uses their unified API to study transfer across observation spaces. The libraries have been widely adopted, while the case studies demonstrate transfer-learning capabilities and the paper identifies simplicity and Python performance as limitations.
Problem
Existing RL environment suites focus on control and complex visual observations, while Minigrid and Miniworld lacked an academic account of their design, API, and user case studies.
Method
The paper documents two minimal, customizable libraries for 2D and 3D goal-oriented environments, their environment-generation API, and unified-API case studies spanning RL-agent and human transfer learning.
Results
Minigrid and Miniworld have been widely adopted, and transfer-learning behavior improved when critic and mission-embedding weights were not frozen; transferring only those weights outperformed also transferring actor weights.
Takeaways & Limitations
The unified API supports studying transfer learning between different observation spaces and human decision-making across Minigrid and Miniworld.
Takeaways & Limitations
The libraries’ simplicity limits the types of environments they can create, and their Python implementation is slower than highly optimized C++ game-engine environments.
Abstract
from arXiv · showhide
We present the Minigrid and Miniworld libraries which provide a suite of goal-oriented 2D and 3D environments. The libraries were explicitly created with a minimalistic design paradigm to allow users to rapidly develop new environments for a wide range of research-specific needs. As a result, both have received widescale adoption by the RL community, facilitating research in a wide range of areas. In this paper, we outline the design philosophy, environment details, and their world generation API. We also showcase the additional capabilities brought by the unified API between Minigrid and Miniworld through case studies on transfer learning (for both RL agents and humans) between the different observation spaces. The source code of Minigrid and Miniworld can be found at https://github.com/Farama-Foundation/{Minigrid, Miniworld} along with their documentation at https://{minigrid, miniworld}.farama.org/.
1 Introduction
Minigrid and Miniworld provide modular 2D and 3D goal-oriented environments designed for broad RL research use. Their minimal, extensible libraries emphasize accessibility, customization, visualization, and scalable complexity.
- Minigrid and Miniworld provide 2D and 3D environments for goal-oriented, navigation-based, and instruction-based tasks.
- Their easily extendable API lets users implement novel research-specific environments.
- Minimal dependencies support easy installation for a wide range of users.
- Users can create new environments or add functionality to existing ones.
- Top-down views make learned policies easier to visualize and understand.
- Environments with different complexity levels help users understand learned-policy limitations.
2 Minigrid & Miniworld Libraries
Minigrid and Miniworld provide modular 2D and 3D goal-oriented environments built around simple, extensible APIs and standardized RL interaction. Their configurable observations, actions, rewards, and environment generation support research across curriculum learning, exploration, meta-learning, and transfer learning.
- Design Philosophy: The libraries use Python and Gym’s RL environment API to prioritize simplicity and customizability for broad use.They were designed with few external dependencies, using NumPy for Minigrid and Pyglet for Miniworld graphics.
- Minigrid Environments: Minigrid environments are deterministic 2D GridWorlds composed of n × m tiles containing objects such as walls, keys, and goals.Different tile configurations construct tasks with varying complexity.
- Minigrid Environments: Minigrid agents receive image, direction, and mission observations, with seven discrete actions shared across environments.The default reward is sparse and can be customized by overriding MiniGridEnv._reward.
- Miniworld Environments: Miniworld environments are configurable 3D worlds of connected rooms containing objects, with default first-person RGB observations sized 80×60.The observation size can be changed through obs_width and obs_height, and the action space adds move-back to Minigrid’s actions.
- Constructing and Extending Environments: Both libraries support compact environment-generation functions, tutorials, direct integration with existing RL libraries, and wrappers for stochastic actions and varied observation spaces.Their common API also supports applications including automatic curricula, exploration, meta-learning, and transfer learning.
- Adoption: Around 2400 GitHub stars and 620 forks indicate substantial adoption of Minigrid and Miniworld by the RL research community.The libraries have been used for applications including curriculum learning, exploration, meta-learning, and transfer learning.
3 Case Studies for Utilizing the Unified API
The unified API supports transfer-learning case studies across Minigrid and Miniworld observation spaces for RL agents and humans. The studies cover policy-weight transfer, human experience transfer, and lightweight custom implementations.
- Case studies: The case studies evaluate RL-agent and human transfer learning between Minigrid and Miniworld observation spaces.The RL study transfers policies, while the human study transfers experience between related environments.
- RL agent transfer: The two agent environments use the same instruction-following task: go to a randomly selected colored object.Instructions combine one of six colors with one of three object types.
- RL agent transfer: The study trained PPO on Minigrid, transferred selected policy components to Miniworld, and tested 12 weight-transfer options.The policy includes mission-instruction, image, actor, and critic components.
- RL agent transfer: Transfer improved when critic and mission-embedding weights were not frozen, while transferring only those components outperformed also transferring actor weights.The transfer metric compares reward-curve AUC for transferred and randomly initialized Miniworld agents over 200k time steps.
- Human transfer: The human study used four-room Minigrid and Miniworld environments with partial observations and a goal of reaching a target in as few steps as possible.Subjects first collected Minigrid experience for 10 episodes before transferring to Miniworld; each subject adapted to random key assignments during the first episode.
4 Related Works
Related simulation libraries support RL benchmarking, GridWorld research, robotic tasks, or 3D scenarios, but differ in integration, complexity, or customizability. Minigrid and Miniworld are positioned around simpler, more extensible goal-oriented environments.
- Simulation environments: Many simulation environments emphasize fixed control tasks, making custom objectives such as sequential key-door-goal tasks difficult to add.Examples include swinging up a pendulum and making a humanoid stand up.
- Robot learning benchmarks: Robotic benchmarks such as pixmc and Franka kitchen target goal-oriented tasks, but may be less suitable for RL studies focused solely on decision-making.The passage notes that these benchmarks often use robotic simulation settings rather than isolated decision-making platforms.
- 2D GridWorld libraries: MazeBase supports GridWorld-like 2D games but is difficult to integrate with deep-learning libraries because it uses Lua and lacks the OpenAI Gym API.Griddly offers greater functionality and flexibility, but its higher complexity makes understanding and customization more difficult.
- 3D simulation libraries: ViZDoom enables tactical and strategic decisions in Doom-based environments, whereas DeepMind Lab provides 3D simulation with a steep customization learning curve.ViZDoom customizability is limited because its scenarios are similar; DeepMind Lab uses a C engine and Lua levels.
5 Conclusion
Minigrid and Miniworld provide modular, customizable goal-oriented RL environments, with a unified API supporting transfer-learning and human decision-making studies. Their simplicity and Python implementation impose limits on environment types and computational speed, while idealized dynamics constrain direct real-world applicability.
- The libraries provide modular and customizable reinforcement-learning environments for goal-oriented tasks.
- Their unified API enables studies of transfer learning across observation spaces and human decision-making.
- Minimal environment functions simplify creation but limit the types of environments that can be created.
- Python implementations are computationally slower than environments using highly optimized C++ game engines.
- Idealized system dynamics mean learned policies may require safeguard mechanisms before direct real-world application.
A Dataset Documentation & URL
The Minigrid and Miniworld source code and documentation are available through their GitHub repositories and project websites.
- Source code is available on GitHub, with documentation hosted at the Minigrid and Miniworld project websites.
B Implementation Details for Transfer Learning Between Different Observations Spaces for 10 Human Subjects
The transfer-learning experiments added functionality for saving and plotting human trajectories in MiniGrid-FourRooms-v0 and MiniWorld-FourRooms-v0. The complete implementation required 149 lines of code.
- The experiments implemented human trajectory saving for MiniGrid-FourRooms-v0 and MiniWorld-FourRooms-v0.
- Data saving and plotting were implemented for both MiniGrid-FourRooms-v0 and MiniWorld-FourRooms-v0.
- 149 lines of code were required to implement the new functionality.
C Hosting, Licensing, and Maintenance Plan
Minigrid and Miniworld are hosted on GitHub, use Apache-2.0 licenses, and are planned for continued maintenance by the Farama Foundation.
- Both libraries are hosted on GitHub and distributed under Apache-2.0 licenses.
- The Farama Foundation plans to maintain both libraries in the foreseeable future.
D Author Statement
The authors accept responsibility for rights violations and state that both libraries use Apache-2.0 licenses.
- The authors bear responsibility for any violation of rights.
- Both libraries are distributed under Apache-2.0 licenses.
- The author statement addresses rights responsibility and licensing for both libraries.
E Case Study Implementation
The case studies provide implementation resources, specify the RL training setup, and describe human transfer-learning trials across environments.
- Implementations for the RL-agent and human transfer-learning case studies are available through the listed repositories.
- The RL-agent study used one NVIDIA RTX A4000 GPU and default Stable-Baselines 3 PPO hyperparameters.
- Human transfer learning: Human participants played two goal-oriented games for 10 rounds each using number keys 1–9 for control.
H GitHub Stars & Citations
By June 12, 2023, Minigrid and Miniworld had substantial and continuing community uptake, reflected in GitHub stars and Minigrid citations.
- Around 2400 GitHub stars had been recorded for the two libraries by June 12, 2023.
- The number of GitHub stars was still increasing at the recording date.
- Minigrid had received 470 Google Scholar citations by June 12, 2023.