Source-linked AI summary
Obstacle Tower: A Generalization Challenge in Vision, Control, and Planning
Arthur Juliani, Ahmed Khalifa, Vincent-Pierre Berges, Jonathan Harper, Ervin Teng, Hunter Henry, Adam Crespi, Julian Togelius, Danny Lange
TL;DR
Obstacle Tower addresses the need for benchmarks that evaluate generalization in high-fidelity, procedurally generated environments from raw pixels. The paper defines the environment and evaluates human and deep reinforcement-learning agents, finding that current methods remain far below human performance.
Problem
Existing benchmarks provide limited variability or simpler representations, making evaluation of generalization in complex visual control and planning settings important.
Method
The paper introduces a high-fidelity 3D benchmark with procedurally generated floors, raw visual observations, configurable rewards, and tasks requiring vision, control, and planning.
Results
Current state-of-the-art methods perform far below human level across all experimental conditions, while Rainbow shows only limited generalization.
Takeaways & Limitations
Obstacle Tower provides a challenging benchmark for studying unsolved problems in vision, control, planning, and generalization.
Takeaways & Limitations
Whether any method can perform well with Obstacle Tower’s sparse reward function remains an open question.
Abstract
from arXiv · showhide
The rapid pace of recent research in AI has been driven in part by the presence of fast and challenging simulation environments. These environments often take the form of games; with tasks ranging from simple board games, to competitive video games. We propose a new benchmark - Obstacle Tower: a high fidelity, 3D, 3rd person, procedurally generated environment. An agent playing Obstacle Tower must learn to solve both low-level control and high-level planning problems in tandem while learning from pixels and a sparse reward signal. Unlike other benchmarks such as the Arcade Learning Environment, evaluation of agent performance in Obstacle Tower is based on an agent's ability to perform well on unseen instances of the environment. In this paper we outline the environment and provide a set of baseline results produced by current state-of-the-art Deep RL methods as well as human players. These algorithms fail to produce agents capable of performing near human level.
1 Introduction
Obstacle Tower is introduced as a high-fidelity benchmark designed to test vision, control, planning, and generalization on procedurally varied environments. It addresses limitations of earlier game benchmarks by requiring agents to act from pixels and solve unseen layouts.
- 1 Introduction: Earlier benchmarks often used limited branching factors, processed observations, fast forward models, fixed topologies, or 2D representations.
- 1 Introduction: Procedural content generation is important for evaluating agent generalization rather than performance on a single deterministic problem.
- 1 Introduction: Obstacle Tower extends raw-pixel game benchmarks with varied 3D visual themes and layouts, including Ancient, Moorish, and Industrial examples across floor levels.
- 1 Introduction: Obstacle Tower combines high-fidelity 3D visuals, procedurally generated floors, and physics-driven interactions to create a broad reinforcement-learning challenge.The environment is designed to require both dexterity and planning while varying appearance and layout across instances.
- 1 Introduction: Procedural variation in textures, lighting, and object geometry requires agents to generalize their understanding of object appearance.
2 Obstacle Tower Environment
Obstacle Tower provides configurable observations, actions, and rewards within an environment whose procedural generation supports evaluation of generalization.
- 2 Obstacle Tower Environment: Obstacle Tower offers configurable observation spaces, action spaces, and reward functions while relying heavily on procedural generation.
2.1 Environment Specifications
The environment runs as a long, progressively complex episode in which agents navigate procedurally structured floors using visual and auxiliary observations. Its multi-discrete actions and reward configurations support varied control and learning setups.
- 2.1 Environment Specifications: The environment runs on Mac, Windows, and Linux and integrates with reinforcement-learning frameworks through the OpenAI Gym interface.
- 2.1 Environment Specifications: Episodes can contain up to 100 floors, with increasingly complex rooms containing puzzles, enemies, obstacles, or keys and locked doors.
- 2.1 Environment Specifications: The episode ends after a hazard collision, timer expiration, or arrival at the top floor, creating a trade-off between collecting time orbs and quickly completing floors.
- 2.1 Environment Specifications: The 168 × 168 RGB third-person image can be downscaled to 84 × 84 and is paired with keys held and remaining time.
- 2.1 Environment Specifications: The multi-discrete action space combines movement, camera rotation, and jumping, with an alternative flattened space containing 54 actions.
- 2.1 Environment Specifications: Obstacle Tower supports sparse rewards of +1 per completed floor and dense rewards of +0.1 for opening doors, solving puzzles, or collecting keys.
2.2 Procedural Generation of Floors
Obstacle Tower generates floors procedurally across visual appearance, mission structure, room layouts, and increasing complexity. This proceduralism requires agents to learn general-purpose representations for vision, low-level control, and high-level planning.
- Procedural generation varies lighting, textures, room layouts, and floor plans, requiring general-purpose representations across vision, control, and planning.
- Mission graphs encode level progression, such as obtaining a key, solving a puzzle, unlocking a door, and reaching the stairs.
- Graph grammar rules and randomized graph recipes generate increasingly complex floor layouts across level groups.
- Generated mission graphs are transformed into 2D room grids using shape grammar before being used to construct the virtual scene.
- Template-based room generation combines finite templates with probabilistic module and item selection to create many room configurations.
2.3 Evaluation Criteria
Obstacle Tower proposes evaluation schemes that distinguish fixed-instance performance from generalization to unseen configurations and visual themes. The recommended evaluations test held-out seeds and, for strong generalization, an unseen theme.
- The evaluation framework provides three schemes and recommends weak or strong generalization because Obstacle Tower explicitly tests generalization.
- No-generalization evaluation uses one fixed tower version and reports performance across five random dynamics seeds.
- Weak generalization trains on 100 environment seeds and tests five held-out configuration seeds, with five dynamics-seed evaluations per test configuration.
- Strong generalization tests held-out configuration seeds and a visual theme separate from training, with training on Ancient and Moorish and testing on Industrial.
2.4 Value as a Research Benchmark
Obstacle Tower is designed as a pixels-to-control benchmark combining vision, control, planning, and generalization. Its procedural visual and structural variation prevents straightforward reuse of learned appearances or high-level plans across instances.
- Obstacle Tower combines vision, control, planning, and generalization in a benchmark for pixels-to-control agents.
- High-fidelity RGB observations include real-time lighting, complex 3D shapes, high-resolution textures, and multiple visual themes.
- Agents must handle rooms containing obstacles, enemies, moving platforms, and puzzles while controlling continuous object dynamics through discrete actions.
- Procedural layouts and stochastic platforms and enemies make methods that exploit deterministic training environments poorly suited to test instances.
- Procedurally generated dependencies require planning for efficient routes between rooms, while changing layouts prevent reusing one high-level plan across floors or instances.
3 Preliminary Results
Evaluations compared environment performance, human play, and Deep RL agents across fixed, varied, and generalization conditions. Humans transferred knowledge to unseen configurations with substantial success, whereas PPO and Rainbow agents remained far below human performance despite Rainbow’s relative advantage.
- Environment Performance: The environment achieved roughly 350 simulation steps per second on floor 20 and around 500 steps per second on the simplest floor.Reported environment-step measurements correspond to five internal simulation steps and exclude model inference and training time.
- Human Performance: Humans solved an average of 15 floors in no-generalization and weak-generalization tests, and reached up to 20 floors under strong generalization.Strong generalization reduced human performance relative to the other conditions, but participants still transferred knowledge successfully.
- Agent Performance: Rainbow outperformed PPO in both fixed and varied training, while both algorithms performed better in fixed training than varied training.Fixed training required learning one tower instance, whereas varied training required learning a distribution of instances.
- Agent Performance: Agents failed to reach an average floor completion score of 10 in fixed evaluation and floor 5 in varied evaluation, consistently underperforming humans.Varied agents stalled when the locked-door mechanic was introduced at floor 5.
- Agent Performance: Agents trained with varied Rainbow outperformed all other algorithms and training conditions on weak and strong generalization evaluations.Rainbow under varied training was the strongest agent configuration on the two generalization conditions.
4 Discussion
Obstacle Tower remains substantially beyond current state-of-the-art agents, with generalization far below human performance and sparse-reward success unresolved. The authors position the benchmark as a challenge intended to drive advances relevant beyond this environment.
- Current state-of-the-art methods perform far below human level across all experimental conditions, while Rainbow’s limited generalization remains significantly worse than even the lowest-performing humans.This gap persists even when using a dense reward function.
- Whether any method can perform well with Obstacle Tower’s sparse reward function remains an open question.
- Fundamental improvements over the state of the art are required for learned agents to perform better on the task.
- The authors expect such improvements to generalize beyond Obstacle Tower to domains including robotic navigation and planning.
- Future releases aim to make Obstacle Tower a customizable environment by adding state information, reward-function modification, and new procedural modules and items.The reported results correspond to version 1.0, while later planned extensions broaden the environment’s uses.
- Obstacle Tower is intended to focus research on unsolved problems in vision, control, planning, and generalization, similarly to earlier influential game benchmarks.