Source-linked AI summary
MAgent: A Many-Agent Reinforcement Learning Platform for Artificial Collective Intelligence
Lianmin Zheng, Jiacheng Yang, Han Cai, Weinan Zhang, Jun Wang, Yong Yu
TL;DR
Many-agent reinforcement learning requires scales beyond the dozens of agents supported by existing platforms. MAgent addresses this gap with a scalable, configurable platform and demonstrates learned cooperation, competition, and hybrid strategies in three environments. It supports studying AI populations at both individual-agent and society levels.
Problem
Many-agent reinforcement learning for artificial collective intelligence often requires hundreds to millions of agents, while existing platforms and algorithms are generally limited to dozens.
Method
MAgent provides a large-scale configurable gridworld platform with scalable simulation, reward descriptions, agent customization, interactive rendering, and reinforcement-learning baselines.
Results
Three learned environments exhibit local cooperation in pursuit, competition in gathering, and combined cooperation and competition in battle.
Takeaways & Limitations
MAgent supports research on many-agent reinforcement learning and the study of collective behavior at individual-agent and society levels.
Abstract
from arXiv · showhide
We introduce MAgent, a platform to support research and development of many-agent reinforcement learning. Unlike previous research platforms on single or multi-agent reinforcement learning, MAgent focuses on supporting the tasks and the applications that require hundreds to millions of agents. Within the interactions among a population of agents, it enables not only the study of learning algorithms for agents' optimal polices, but more importantly, the observation and understanding of individual agent's behaviors and social phenomena emerging from the AI society, including communication languages, leaderships, altruism. MAgent is highly scalable and can host up to one million agents on a single GPU server. MAgent also provides flexible configurations for AI researchers to design their customized environments and agents. In this demo, we present three environments designed on MAgent and show emerged collective intelligence by learning from scratch.
Introduction
Artificial collective intelligence depends on many-agent reinforcement learning, but existing platforms and algorithms are generally limited to populations of only dozens of agents.
- Artificial collective intelligence is emerging as AI agents work together across domains such as trading, games, and transportation optimization.
- Multi-agent reinforcement learning for artificial collective intelligence typically requires hundreds to millions of agents.
- Existing experimentation platforms are basically designed for no more than dozens of agents, creating a need for large-population multi-agent reinforcement learning infrastructure.
- State-of-the-art multi-agent reinforcement learning algorithms are also limited to scales of dozens of agents, posing a major research challenge.
The MAgent Platform
MAgent combines scalable many-agent simulation with configurable environments, reward descriptions, agent representations, baseline algorithms, and interactive visualization.
- The MAgent Platform: MAgent can host up to one million agents on a single GPU server while supporting configurable environments, agents, rewards, and interactive rendering.Network sharing and ID embedding support scalability; users can also slide or zoom the viewing window and manipulate agents.
- Gridworld Fundamentals: A large-scale gridworld provides configurable states, actions, attributes, heterogeneous agents, and fast C++ simulation.
- Reward Description Language: The reward description language assigns rewards when event expressions become true and supports logical operations including and, or, and not.
- Baseline Algorithms: The platform implements parameter-sharing DQN, DRQN, and A2C, with DQN performing best in the reported settings.Agent-specific ID embeddings provide diversity, and users can benchmark their own algorithms against these baselines.
Live and Interactive Part
The demo uses pursuit, gathering, and battle to show learned cooperation, competition, and their combination among large populations of agents.
- Interactive Demonstration: Visitors can explore the gridworld through a renderer and control several agents while cooperating or competing with reinforcement-learning agents.
- Pursuit: Pursuit produces local cooperation: predators form closures with nearby teammates to lock preys and receive rewards every subsequent step.
- Gathering: Gathering produces competition in a limited-resource environment, with agents first eating food and then attempting to kill nearby rivals.
- Battle: Battle combines cooperation and competition as two armies of hundreds of agents learn encirclement attacks and guerrilla warfare through self-play.
Conclusion
MAgent supports studying many-agent reinforcement learning at both individual-agent and society levels, with continuous environments and additional algorithms identified as future work.
- MAgent enables researchers to study populations of AI agents at both individual-agent and society levels.
- Future work will add continuous environments and more algorithms to MAgent.