Source-linked AI summary
ViZDoom Competitions: Playing Doom from Pixels
Marek Wydmuch, Michał Kempka, Wojciech Jaśkowski
TL;DR
Complex first-person 3D environments remain a challenge for reinforcement learning, and suitable benchmarks are scarce. The paper presents two Visual Doom AI Competition editions and reports that newer bots improved, though they remained weaker than humans.
Problem
Reinforcement learning research has focused heavily on 2D games partly because suitable environments and benchmarks for harder 3D settings are scarce.
Source: §I · p. 1Method
The paper describes the 2016 and 2017 Visual Doom AI Competitions, where bots played multiplayer Doom deathmatches using raw screen-buffer information.
Source: §I · p. 1, §V · p. 11Results
The new 2017 submissions defeated agents from the previous competition, and the new Track 1 champion scored 50% more points than the former champion, but bots remained weaker than humans.
Source: 4) Discussion · p. 10, §V · p. 11Takeaways & Limitations
Deathmatches based on raw visual input in this FPS game remain an open problem with research opportunities.
Source: §V · p. 11The bots remained weaker than human players, who could exploit their weaknesses.
Source: §V · p. 11
Abstract
from arXiv · showhide
This paper presents the first two editions of Visual Doom AI Competition, held in 2016 and 2017. The challenge was to create bots that compete in a multi-player deathmatch in a first-person shooter (FPS) game, Doom. The bots had to make their decisions based solely on visual information, i.e., a raw screen buffer. To play well, the bots needed to understand their surroundings, navigate, explore, and handle the opponents at the same time. These aspects, together with the competitive multi-agent aspect of the game, make the competition a unique platform for evaluating the state of the art reinforcement learning algorithms. The paper discusses the rules, solutions, results, and statistics that give insight into the agents' behaviors. Best-performing agents are described in more detail. The results of the competition lead to the conclusion that, although reinforcement learning can produce capable Doom bots, they still are not yet able to successfully compete against humans in this game. The paper also revisits the ViZDoom environment, which is a flexible, easy to use, and efficient 3D platform for research for vision-based reinforcement learning, based on a well-recognized first-person perspective game Doom.
I. INTRODUCTION
The paper motivates visual reinforcement learning in complex 3D environments and presents the Visual Doom AI Competition as a way to study that challenge. It also describes ViZDoom’s design rationale and research features.
I. INTRODUCTION
- Complex 3D first-person environments remain far from human-level play, while suitable benchmarks for harder environments are scarce.These settings demand multiple skills, including navigation, localization, memory, exploration, and precision.Source: §I · p. 1
- The 2016 and 2017 competitions tested bots in multiplayer Doom deathmatches using only screen-buffer information, requiring exploration, navigation, resource gathering, dodging, and accurate shooting.The setting also involves sparse and often delayed rewards.Source: §I · p. 1
A. Design Requirements
- ViZDoom’s design requirements include a lightweight, fast, customizable, multiplayer platform with total game-process control, scenario tools, scripting, language bindings, and portability.Doom was selected for its low requirements, simple architecture, cross-platform support, and support for efficient rendering and custom scenarios.Source: A. Design Requirements · pp. 1–2
B. Features
- ViZDoom 1.1.5 supports multiple control modes, custom scenarios, and access to scene information including depth, visible objects, and a top-down map.These features extend the initial 1.0 version.Source: B. Features · p. 2
1) Control Modes:
ViZDoom offers control modes for agents and human observers, supporting synchronous interaction and multiplayer play. Its scenario tools let researchers configure maps, mechanics, rewards, and constraints.
1) Control Modes:
- Synchronous modes let the game wait for an agent’s decision, avoiding the timing constraints of asynchronous play; they also run deterministically.Asynchronous play runs at 35 frames per second, so slow decisions can miss frames and fast decisions may wait for the next frame.Source: 1) Control Modes · p. 2
- In player modes, the agent acts during the game; in spectator modes, a human acts while the agent observes..Source: 1) Control Modes · p. 2
- ViZDoom supports single- and multiplayer games with up to sixteen agents, including deathmatch, team deathmatch, and cooperative scenarios.Multiplayer is available in all modes as of ViZDoom 1.1.Source: 1) Control Modes · p. 2
2) Scenarios:
- Custom scenarios let researchers define maps, mechanics, terminal conditions, and rewards, with constraints on available actions and episode duration.More than a dozen predefined scenarios support training fundamental skills such as shooting and navigation; Doom community tools support scenario and map creation.Source: 2) Scenarios · pp. 2–3, 1) Control Modes · p. 2
3) Automatic Objects Labeling:
ViZDoom supplements screen input with object labels, depth, and top-down maps, and includes tools for efficient experimentation and replay. Its rendering can be fast and lightweight.
3) Automatic Objects Labeling:
- Object labeling supplies visible objects as an additional input channel and exposes their bounding boxes, names, positions, orientations, and movement directions.The object information can support supervised-learning research.Source: 3) Automatic Objects Labeling · p. 3
4) Depth Buffer:
- The depth buffer can simulate mobile-robot distance sensors and help test whether agents learn where objects are..Source: 4) Depth Buffer · p. 3
5) Top-Down View Map:
- The top-down map can show the whole environment or only discovered areas, and can display objects, their facings, and sizes.It supports navigation research and can turn ViZDoom into a 2D environment for simpler scenarios.Source: 5) Top-Down View Map · p. 3
6) Off-Screen Rendering and Frame Skipping:
- Off-screen rendering avoids the burden of displaying the game, while frame skipping omits frames an effective bot may not need to see.These features facilitate computationally heavy experiments and server-based runs without a graphical interface.Source: 6) Off-Screen Rendering and Frame… · p. 3
7) ViZDoom’s Performance:
- Up to 2500 frames per second at 320 × 240 and 7000 at 160 × 120 were measured on a modern single-threaded CPU; one instance uses only a dozen MBs of RAM.Rendering resolution and computation of additional buffers affect performance; at low resolutions, rendering time is negligible compared with backpropagation for a reasonably complex neural network.Source: 7) ViZDoom’s Performance · p. 3
8) Recording and replaying episodes:
- Saved episodes can be replayed with access to buffers, rewards, game variables, and player actions, and rendering settings can be changed during playback.This supports learning from demonstration and preparing high-resolution demonstration movies.Source: 8) Recording and replaying episodes · p. 3
C. Application Programming Interface (API)
ViZDoom provides an API for controlling Doom and accessing visual and game-state information. The paper also compares it with other visual research platforms and notes its computational efficiency.
C. Application Programming Interface (API)
- The API provides screen-buffer access, keyboard and mouse actions, and selected game-state variables such as health and ammunition.The C++ API also has Python, Lua, Java, and Julia bindings, with numerous control and rendering options.Source: C. Application Programming Interface… · p. 3
III. VISUAL RL RESEARCH PLATFORMS
- DeepMind Lab and Project Malmo are first-person counterparts, while UnrealCV and Unity ML-Agents provide broader scenario-building approaches.ViZDoom uniquely provides detailed visible-object information, depth, and a top-down map among the three platforms discussed in p24; UnrealCV lacks an RL-specific API abstraction layer.Source: §III · pp. 3–4
- ViZDoom’s comparatively simple graphics make it lightweight and computationally efficient, allowing multiple environment instances to run with limited resources.A detailed environment comparison is presented in Table I.Source: §III · p. 4
IV. VISUAL DOOM AI COMPETITIONS (VDAIC)
VDAIC asks bots to play Doom deathmatches from screen pixels, combining real-time perception and planning in a 3D multi-agent setting. Its rules restrict in-match information to visual input and HUD data, distinguishing it from competitions that expose higher-level states.
A. Motivation
- VDAIC addresses the gap between bots’ access to internal game data and humans’ reliance on the screen, by requiring play from raw visual information.The competition was held at the IEEE Conference on Computational Intelligence and Games in 2016 and 2017.Source: A. Motivation · p. 4
B. Other AI competitions
- Unlike contests centered on high-level states or planning, VDAIC requires bots to perceive and plan in real time in a 3D multi-agent environment.GVGAI offers a forward model for planning, while Starcraft AI focuses on planning from high-level representations.Source: B. Other AI competitions · p. 5
1) Rules :
- During matches, bots may use only the screen buffer and HUD information, although participants may prepare offline using other methods and environment features.The objective is to maximize frags: opponent kills minus the bot’s suicides.Source: 1) Rules · p. 5
2) Tracks:
The 2016 competition used a known-map rocket-only track and a harder full deathmatch on three unseen maps. Track 1 produced several capable bots, while IntelAct dominated Track 2; leading solutions used distinct learning strategies, including curriculum training, future-measurement prediction, and separate navigation and aiming modules.
2) Tracks:
- Track 1 used one known, constricted map and rocket launchers, whereas Track 2 used three unseen, more spacious maps with pistols initially and multiple weapons available.Track 2 was harder because agents could not memorize the maps and needed strategic choices about navigation, weapons, and exploration.Source: 2) Tracks · pp. 5–6
3) Results:
- IntelAct significantly surpassed competitors across all Track 2 maps; in Track 1, F1 led by a large margin, while Arnold’s 413 frags narrowly exceeded Clyde’s 393.Seven of nine Track 1 submissions could systematically eliminate opponents; F1 had the fewest suicides, while Arnold was gunned down least often.Source: 3) Results · p. 6, Table II · p. 6, Table III · p. 7
4) Notable Solutions:
- F1, the Track 1 winner, used A3C with curriculum learning from easier to harder tasks and hardcoded some behaviors, including faster movement when not firing.The competition’s algorithm and framework overview is in Table IV.Source: 4) Notable Solutions · p. 6, Table IV · p. 7
- IntelAct, the Track 2 winner, used Direct Future Prediction to predict future measurements for actions and optimize a changeable linear goal; weapon selection was hardcoded.Other behaviors were learned by playing against bots on multiple maps.Source: 4) Notable Solutions · p. 6
- Arnold placed second in both tracks using DQN navigation and DRQN aiming, with an enemy-visibility classifier routing action choice between the modules.It crouched constantly, which the authors say gave it a significant advantage, and was trained on author-created maps beginning with weak enemies.Source: 4) Notable Solutions · p. 6
- Clyde used vanilla A3C, while TUHO paired Dueling DQN navigation with a Haar-detector aiming system; top agents also addressed sparse delayed rewards with auxiliary signals or curriculum training.TUHO trained on two supplied maps and two bundled with ViZDoom.Source: 4) Notable Solutions · pp. 6–7
5) Discussion:
The competition bots could handle tactical combat but remained far from human competence. Their strategic weaknesses were especially apparent on larger Track 2 maps, alongside specific gaps in aiming, movement, and memory.
5) Discussion:
- The strongest bots handled tactical aiming and shooting but remained well below human competence, particularly in strategic play.The discussion identifies defense, escaping, navigation, and exploration as weaknesses.Source: 5) Discussion · p. 7
- On larger Track 2 maps, bots often circled the same area waiting for targets instead of showing situational awareness.The maps were larger and called for better situational awareness.Source: 5) Discussion · p. 7
- No agent could aim vertically; Arnold's hardcoded crouching helped it avoid projectiles, while other bots may not have encountered crouching players during training.The paper presents the training explanation for other bots as probable.Source: 5) Discussion · p. 7
- Bots used strafing poorly, did not dodge missiles, and appeared not to use memory to chase opponents leaving view.These behaviors contrast with actions the discussion says humans performed easily.Source: 5) Discussion · p. 7
- Most submissions used A3C or DQN, while leading agents also addressed sparse, delayed rewards with auxiliary signals or curriculum training.IntelAct and Arnold used auxiliary signals; F1 used curriculum training.Source: 5) Discussion · p. 7
6) Logistics:
The organizers used warm-up rounds and made evaluation results public, but submission compatibility problems created substantial work. For 2017, they changed competition rules and infrastructure, including an elimination round, new match and map conditions, and Docker-based execution.
6) Logistics:
- Three optional warm-up rounds preceded evaluation, and tabular results and video recordings were made public.The rounds helped participants become familiar with submission and evaluation and test bots against each other.Source: 6) Logistics · pp. 7–8
- Dependency and compatibility problems made submission testing strenuous and time-consuming, prompting organizers to use Docker containers in the subsequent edition.The organizers said Docker would relieve them from handling those issues.Source: 6) Logistics · p. 8
1) Changes Compared to the Edition 2016:
- For 2017, matches lasted 10 minutes across 10 matches per track; Track 2 used five unseen maps selected from 88 candidates, and deaths incurred a 10-second respawn.The player limit rose from 8 to 16; crouching was disabled, and the 2017 environment and features were used.Source: 1) Changes Compared to the Edition 2016 · p. 8
- An obligatory elimination round limited final-round entry to teams that participated and presented sufficiently competent bots; winning 2016 bots were also included as baselines.Matches were launched by a host program and agent processes ran in participant-submitted Docker containers.Source: 1) Changes Compared to the Edition 2016 · p. 8
2) Results:
The 2017 competition showed strong Track 2 results and a close Track 1 finish, with the two leading bots in Track 2 separated by only two frags. The engine also recorded movement, attack, accuracy, damage, and item statistics to characterize bot performance.
2) Results:
- The engine recorded movement speed, attack counts, shooting and detection precision, hits, damage taken, and items picked up.Shooting precision measured damaging attacks divided by all attacks; detection precision measured attacks made while another player was visible divided by all attacks.Source: 2) Results · p. 8
- Marvin won Track 1 with 248 frags, only 3 more than Arnold2; the previous champion F1 placed fifth with 164.The paper describes the 2017 Track 1 bots as substantially stronger than those of 2016.Source: 2) Results · p. 8
- All Track 2 bots scored more than 50 frags; Arnold4 won with 275, just ahead of YanShi's 273.The results are reported in Table VI.Source: 2) Results · pp. 8–9, Table VI · p. 9
- Arnold4 was the competition's most accurate bot and the only one without a suicide; YanShi had 275 kills but two suicides.YanShi had the highest frags/death ratio and detection precision, while IntelAct scored 221 frags.Source: 2) Results · p. 9
3) Notable Solutions: •
The 2017 agents improved but remained weaker than humans, while their solutions combined reinforcement learning, demonstrations, and hand-built components. The discussion also notes persistent perception and memory problems alongside faster training support.
3) Notable Solutions: •
- Marvin won Track 1 using A3C pre-trained with human-game replays, then trained through self-play, with a handcrafted policy overriding uncertain aim choices.It placed fourth in Track 2; its policy separated movement and aiming outputs, including vertical aiming.Source: 3) Notable Solutions: • · p. 9
- Arnold2 and Arnold4 modified the earlier Arnold with strafing, no crouching, and a hardcoded run after prolonged stillness to address low mobility.They differed only in track-specific details, such as manual weapon selection.Source: 3) Notable Solutions: • · p. 9
- Axon used A3C in stages, moving from human-generated data to skill tasks and then competition against F1, with a five-scale visual input.Its input included the original image, three center-screen crops, and a crosshair zoom.Source: 3) Notable Solutions: • · p. 9
- YanShi combined supervised resource-and-enemy detection with SLAM, MCTS, hardcoded rules, and components from earlier F1 and IntelAct agents.It placed second in Track 2 and used RPN-based aiming plus rules for scanning, dodging, and avoiding corners.Source: 3) Notable Solutions: • · pp. 9–10
4) Discussion:
- The 2017 Track 1 champion scored 50% more points than the former champion, but bots remained weaker than humans, especially in Track 2.An author could consistently defeat all of them by a considerable margin.Source: 4) Discussion · p. 10
- A3C was the 2017 RL method of choice, while Marvin and Axon used demonstrations to bootstrap learning and address sparse, delayed rewards.Agents also advanced in strafing and vertical aiming, but did not show tactics such as aiming at opponents’ legs.Source: 4) Discussion · p. 10
- Synchronized multiplayer support in ViZDoom 1.1 enabled faster training, while new features and prior winning solutions broadened supervision and training against competing agents.YanShi trained against other solutions as well as built-in bots.Source: 4) Discussion · p. 11
- Bots continued to shoot inaccurately, sometimes killing themselves, and their enemy-detection precision fell on unknown maps; they also often passed opponents without pursuit.The paper connects suicides to low attack precision and poor understanding of surroundings, and notes that agents had little or no memory.Source: 4) Discussion · p. 11
5) Logistics:
The paper describes competition logistics and revisits ViZDoom as a flexible research platform. It concludes that bots were capable but still weaker than humans, leaving raw-visual FPS play an open problem.
5) Logistics:
- Docker images eased software-environment preparation and unified 2017 evaluation, but manual building and troubleshooting remained necessary for some submissions.The authors argue for a more automated submission and testing process with immediate feedback.Source: 5) Logistics · p. 11
V. CONCLUSIONS
- The paper presents the first two Visual Doom AI Competition editions, held in 2016 and 2017, where bots played multiplayer Doom deathmatches using raw pixels only.The contests attracted teams from leading AI labs around the world.Source: §V · p. 11
- Winning bots used contemporary RL algorithms, and 2017 submissions strengthened them through fine-tuning, added supervision, and separate navigation and fighting approaches.The paper concludes that these bots remained weaker than human players, leaving raw-visual FPS deathmatch an open research problem.Source: §V · p. 11
- Deathmatch is relatively easy compared with completing original single-player Doom levels, which require navigation and localization on more complex maps with switches and keys.The paper anticipated that the competition would remain difficult and described a 2018 track involving randomly generated single-player levels.Source: §V · p. 11
- ViZDoom 1.1.5 is presented as a flexible, efficient, lightweight platform for vision-based RL, with customizable scenarios and multiple learning modes.Unlike Atari 2600, it offers a 3D, semi-realistic, first-person world and full environment control through its API.Source: §V · p. 11
APPENDIX
The appendix provides online resource links for ViZDoom, competition submissions, maps, and videos.
APPENDIX
- The listed online resources include the ViZDoom repository and submission files from the 2016 and 2017 competitions.The page also lists 2017 map packs and videos from both competition years.Source: APPENDIX · p. 12
- Map packs used in 2017 are linked alongside videos from the 2016 and 2017 competitions.The resources are provided as URLs in the appendix.Source: APPENDIX · p. 12
- The online resources list three video links from 2016 and two from 2017, alongside links to ViZDoom, 2016 and 2017 submissions, and 2017 map packs.Source: APPENDIX · p. 12