Source-linked AI summary
The Multi-Agent Reinforcement Learning in Malm\"O (MARL\"O) Competition
Diego Perez-Liebana, Katja Hofmann, Sharada Prasanna Mohanty, Noboru Kuno, Andre Kramer, Sam Devlin, Raluca D. Gaina, Daniel Ionita
TL;DR
Multi-agent RL faces scaling challenges in realistic general-reward settings, and research often focuses on few tasks and a single learning agent. MARLÖ proposes a competition across multiple Minecraft games, using varied tasks and opponents to encourage generalization. The competition also provides shared tasks and evaluation resources to support comparisons and progress-sharing.
Problem
Multi-agent RL remains challenging in general-reward games, and much current empirical work focuses on few tasks with a single learning agent rather than generalization across tasks and opponents.
Source: §1 · p. 1Method
Results
The competition is designed to test and encourage agent generalization across games, tasks, and opponent types, rather than reporting empirical agent-performance results.
Source: §3 · p. 2Takeaways & Limitations
The competition makes multi-agent tasks publicly available and provides shared baselines and evaluation to ease comparison and support progress-sharing.
Source: §3 · pp. 2–3
Abstract
from arXiv · showhide
Learning in multi-agent scenarios is a fruitful research direction, but current approaches still show scalability problems in multiple games with general reward settings and different opponent types. The Multi-Agent Reinforcement Learning in Malm\"O (MARL\"O) competition is a new challenge that proposes research in this domain using multiple 3D games. The goal of this contest is to foster research in general agents that can learn across different games and opponent types, proposing a challenge as a milestone in the direction of Artificial General Intelligence.
1 Introduction
Multi-agent reinforcement learning is promising, but learning remains difficult in competitive, collaborative, and general-reward games, and current solutions scale empirically only in relatively small environments or with simplifying assumptions. The paper also identifies generalization across tasks and opponent types as an area needing further research.
1 Introduction
- The paper frames multi-agent settings as Stochastic N-player Games, where agents observe states, act on the environment and other players, and receive rewards.Reinforcement learning is presented as a common approach to learning in these games.Source: §1 · p. 1
- Current multi-agent RL solutions scale empirically only in relatively small environments or under simplifying assumptions, even in competitive and collaborative settings.General-reward games are described as more realistic for many applications and notoriously challenging.Source: §1 · p. 1
- Most current empirical multi-agent RL work focuses on a few individual tasks with a single learning agent, leaving generalization across tasks and opponent types in need of research.The passage notes a risk of overfitting to individual tasks and specific opponent types in single-agent RL.Source: §1 · p. 1
2 MARLÖ: Multi-Agent Reinforcement Learning in MalmÖ
MARLÖ introduces a multi-game challenge in which participants build agents for 3D games within Minecraft. The games offer varied interaction and reward structures, while parameterized tasks, starter materials, and a staged tournament shape training and evaluation.
2 MARLÖ: Multi-Agent Reinforcement Learning in MalmÖ
- MARLÖ introduces a challenge for learning agents to play multiple 3D games within Minecraft.The first competition was scheduled for 2018 and featured three games.Source: §2 · p. 2
- Mob Chase is collaborative: agents can corner a mob for 1 point or leave through an exit for 0.2 points.The game gives a high reward for capturing the mob and a smaller reward for exiting.Source: §2 · p. 2
- Build Battle combines competition and collaboration as two teams construct a cuboid, rewarding correct block actions with 0.2 points and incorrect actions with −0.2 points.Source: §2 · p. 2
- Treasure Hunt assigns collectors and fighters to retrieve treasure and survive; a team member's death gives every teammate −1 points.Winning-team rewards depend on whether the collector gets the treasure or reaches the exit.Source: §2 · p. 2
- Games are parameterizable into potentially endless task variants, and final evaluation uses organizer-designed customizations that remain hidden until the competition deadline.Examples include weather, block types, entity number and position, and playing-area size.Source: §2 · p. 2
- The starter kit provides framework instructions, local testing, simple tasks, and default challenge agents as opponents and examples.The passage says this approach has helped entrants get started and iterate in previous competitions.Source: §2 · p. 2
- A staged round-robin playoff requires teams to perform across all three games, with league rankings based on the sum of scores across tasks.Top entries progress through successive stages to a final league that determines the winner.Source: §2 · p. 2
3 Conclusions
The competition is designed to encourage agents that generalize across games, tasks, and opponents, with task configurations hidden from participants. It also offers shared tasks and evaluation resources to support access, comparison, and progress-sharing.
3 Conclusions
- MARLÖ is designed to encourage approaches that generalize across games, tasks, and opponent types, including through hidden final task configurations and play against multiple agents.Source: §3 · p. 2
- The competition makes multi-agent learning tasks publicly available with a low barrier to entry and room to increase task difficulty as progress is made.Source: §3 · p. 2
- Shared baselines and an evaluation setup ease comparison, while the competition provides a platform for testing and sharing progress in multi-agent RL.Source: §3 · p. 3
- Future competition versions are expected to introduce different games with new dynamics, richer interactions, and further multi-agent RL challenges.Source: §3 · p. 3