PaperlayerView exact revision on arXiv

Source-linked AI summary

Pommerman: A Multi-Agent Playground

Cinjon Resnick, Wes Eldridge, David Ha, Denny Britz, Jakob Foerster, Julian Togelius, Kyunghyun Cho, Joan Bruna

arXiv:1809.07124v2cs.MA

TL;DR

Multi-agent research lacks suitable benchmarks for general-sum and more-than-two-player settings. Pommerman provides a Bomberman-style environment with cooperative and competitive scenarios, and early agents demonstrated both a novel bomb-kicking strategy and FFA performance from DAgger-trained agents.

  • Problem

    Suitable benchmarks for general-sum games and settings with more than two players are lacking, despite multi-agent research needs.

    Source: Why Pommerman · p. 1, Multi-Agent Learning · p. 2

  • Method

    Pommerman adapts Bomberman into scenarios with at least four agents, combining cooperative and competitive play and supporting communication.

    Source: Multi-Agent Learning · p. 1, Game Information · p. 3

  • Results

    Published agents discovered bomb-kicking and achieved high success; DAgger agents played at or above a single SimpleAgent’s FFA win rate of ∼20%.

    Source: Early results · p. 4

  • Takeaways & Limitations

    The authors present Pommerman as a challenging, extensible testbed for multi-agent research and competitions over the long term.

    Source: High Quality Benchmark · p. 2

    Agents can settle on avoiding bombs to prevent self-explosion, but must learn to use bombs for the environment’s multi-agent benefits to emerge.

    Source: High Quality Benchmark · p. 2

Abstract

from arXiv · show

We present Pommerman, a multi-agent environment based on the classic console game Bomberman. Pommerman consists of a set of scenarios, each having at least four players and containing both cooperative and competitive aspects. We believe that success in Pommerman will require a diverse set of tools and methods, including planning, opponent/teammate modeling, game theory, and communication, and consequently can serve well as a multi-agent benchmark. To date, we have already hosted one competition, and our next one will be featured in the NIPS 2018 competition track.

Why Pommerman

Pommerman is proposed to address the lack of benchmarks for general-sum and three-or-more-player research. Its game variants and design choices target diverse multi-agent questions, while its extensibility is intended to sustain a challenging benchmark and competition.

Why Pommerman

Multi-Agent Learning

  • At least four agents navigate a grid, plant bombs, and pursue their team’s survival across adversarial and cooperative scenarios.FFA has at most one winner; team variants include communication and cooperation with previously unseen teammates.Source: Multi-Agent Learning · p. 1
  • Pommerman supports research beyond 1v1 settings, including game theory, communication, reinforcement learning, planning, and opponent or teammate modeling.Its FFA variant does not reduce to a 1v1 game, while team variants support research with and without explicit communication.Source: Multi-Agent Learning · pp. 1–2
  • Explicit communication, symbolic rather than sensor-based input, discrete controls, and two-agent teams shape the research setting, with trade-offs depending on research goals.Source: Multi-Agent Learning · pp. 1–2

High Quality Benchmark

  • Pommerman aims to meet benchmark needs through intuitive, enjoyable gameplay and symbolic state input that avoids requiring large amounts of compute for agent-building.The paper also notes that benchmark quality depends on ease of integration and a learning problem suited to current methods.Source: High Quality Benchmark · p. 2
  • Potential extensions include partial observability, random teammates, agent communication, power-ups, and play with humans, supporting a challenging testbed over time.The authors note that N-player learning has few mathematical guarantees.Source: High Quality Benchmark · p. 2
  • A basic gameplay hurdle is a local optimum in which agents avoid bombs to prevent self-explosion, although bombing is needed to destroy opponents.The paper reports that players have successfully solved this challenge.Source: High Quality Benchmark · p. 2

Description

Pommerman is a four-agent grid environment with team and free-for-all variants, bomb-based destruction, and optional communication. Early results show both challenges for reinforcement learning and agents that can learn effective play.

Description

  • The paper introduces Pommerman as an environment for multi-agent research, with code and training tools available in its repository.Its description also covers the environment’s design and early results.Source: Description · p. 2

Game Information

  • Battles begin on a randomly drawn symmetric 11x11 grid, with four agents in the corners and teammates on opposite corners; bombs destroy nearby walls, agents, power-ups, and bombs.Bombs have a ten-step life, and chained explosions can occur.Source: Game Information · pp. 2–3
  • Agents choose among six actions and receive board and agent-state observations; communicative scenarios additionally relay two dictionary words from a teammate each turn.Partially observed variants mask squares outside the agent’s 5x5 purview; observations include position, ammunition, blast strength, and nearby bomb information.Source: Game Information · p. 3

Early results

  • Agents have used bombs as projectiles by laying and then kicking them toward opponents, achieving a high success rate with a strategy novice humans do not attempt.Published work also indicates that optimal solutions remain unknown across the variants.Source: Early results · p. 4
  • Out of the box, Deep Q-Learning and Proximal Policy Optimization did not beat SimpleAgent without a very large batch size and shaped reward.Bomb use is highly correlated with losing but must be used effectively to win.Source: Early results · p. 4
  • Agents trained with DAgger could play at or above a SimpleAgent’s approximately 20% FFA win rate, while four SimpleAgents often draw.The result was somewhat sensitive to hyperparameter choices.Source: Early results · p. 4

Competitions

The paper reports a preliminary free-for-all competition and describes the planned NIPS 2018 event. Agent submissions use Docker containers and must meet the competition’s response-time requirements.

FFA competition

  • The June 3, 2018 preliminary competition drew eight competitors, whose working agents were submitted by the May 31 deadline despite limited advertising and no prizes.It used the four-player FFA variant.Source: FFA competition · p. 4
  • In the preliminary FFA competition, Yichen Gong’s redesigned Finite State Machine Tree-Search agent was strongest, winning 22 of 35 matches; Gӧrӧg Márton’s agent won 8.The remaining matches included ties.Source: FFA competition · p. 4

NIPS Competition

  • The NIPS 2018 competition was planned as a live event featuring the partially observable team variant without communication.Competitors were required to submit two agents by November 21, 2018.Source: NIPS Competition · p. 4

Submitting Agents

  • Submissions were Docker-based, with one agent container for FFA and two for team play; agents had to expose an act endpoint and return an action within 100 ms.Late responses trigger the Stop action and, where applicable, message (0, 0); the timeout is a competition constraint, not part of the game itself.Source: Submitting Agents · p. 4

Conclusion

The paper presents Pommerman as an environment for multi-agent research and points readers to its code and competition information.

Conclusion

  • The paper introduces Pommerman, explains its multi-agent research setup, and describes early results and competitions.The code is available in the project repository, with competition information on the website.Source: Conclusion · p. 4
Loading 1809.07124v2…