Multi-agent Embodied AI: Advances and Future Directions

• Figure 1 is identified as an overview of the survey.Source: Figure 1 · p. 2

1 Introduction

• Embodied AI combines AI, robotics, and cognitive science to enable robots to perceive, plan, decide, act, and adapt; DL, RL, LLMs, and VLMs have advanced perception, planning, and control.Reported advances include image recognition, speech interaction, and robotic control.Source: §1 · p. 2
• Symbolic approaches struggled with dynamic perception-action, imitation learning depends on expert data and can face distribution shifts, while RL learns decision strategies through interaction-driven feedback.Deep RL handles high-dimensional sensory inputs, and large models have advanced visual understanding, multimodal perception, and task planning.Source: §1 · p. 2
• Multi-agent embodied tasks require heterogeneous agents to collaborate, while joint action and planning complexity, decentralized partial observability, learning-driven non-stationarity, and credit assignment complicate coordination.Real-world tasks may include confronting unknown adversaries or addressing complex dynamic challenges.Source: §1 · p. 2
• Research on multi-agent embodied AI remains nascent, and existing reviews—including embodied MARL and foundation-model integration [209]—cover specific dimensions rather than the whole field.Specialized benchmarks for embodied multi-agent scenarios have begun to emerge.Source: §1 · p. 3
• This paper systematically reviews multi-agent embodied AI, covering foundational concepts, single-agent methods and benchmarks, multi-agent techniques and benchmarks, and future directions.The review aims to stimulate further research and innovation in the field.Source: §1 · p. 3

2.1 Embodied AI

• Embodied AI agents use physical bodies to perceive, act, and adapt through continuous environmental interaction [111].Their perception, cognition, and action form a feedback cycle; embodiment shapes possible interactions, while ongoing engagement supports adaptation and continual improvement.Source: §2.1 · pp. 3–4

2.2 Multi-Agent Systems

• Multi-agent systems decentralize decision-making among autonomous agents that coordinate locally toward global goals.This design supports scalability, fault tolerance, and adaptability in dynamic or partially observable settings; MARL methods including CTDE address coordination and non-stationarity, while LLMs add language-based communication and high-level reasoning.Source: §2.2 · pp. 4–5

2.3 Optimal Control

• Optimal control selects inputs to optimize criteria such as time, energy, or cost under system constraints.Methods include calculus of variations, Pontryagin’s maximum principle, dynamic programming, and MPC; MPC repeatedly solves a finite-horizon problem and applies its first action, supporting multi-agent coordination.Source: §2.3 · p. 5

2.4 Reinforcement Learning

• Reinforcement learning learns reward-maximizing actions through environmental interaction, while cooperative MARL models decentralized agents with local observations and shared rewards.Single-agent RL is commonly formalized as an MDP; cooperative MARL often uses Dec-POMDPs. Value-based methods include DQN and QMIX, while policy-based approaches such as MADDPG and MAPPO use centralized training with decentralized execution.Source: §2.4 · pp. 6–7

2.5 Hierarchical Learning

• Hierarchical learning divides complex tasks into abstract goals and concrete subtasks.Low-level skills can use MPC or end-to-end RL; high-level policies sequence them using environment rewards or plans generated by LLMs.Source: §2.5 · p. 7

2.6 Imitation Learning

• Imitation learning acquires behavior from expert demonstrations, avoiding manual reward design when rewards are ambiguous or infeasible.Behavior cloning directly maps observed states to expert actions and is simple and efficient, but covariate shift can cause errors to accumulate in long-horizon tasks.Source: §2.6 · p. 7
• Inverse reinforcement learning recovers an expert’s reward for policy learning, whereas GAIL learns imitation through adversarial training [69].IRL can improve generalization and interpretability but adds computational overhead; GAIL avoids explicit reward modeling and can improve sample efficiency, at the cost of reduced interpretability.Source: §2.6 · pp. 7–8

2.7 Generative Models

• Generative modeling spans attention-based Transformers, noise-driven diffusion models, and Mamba state-space models [70] [60].Transformers use attention to capture sequence context, while diffusion models learn to reverse gradual noise corruption. Mamba uses continuous-time state-space models and selective scanning for linear-time long-sequence processing and improved efficiency.Source: §2.7 · pp. 8–9

3.1 Classic Control and Planning

• Classic planning includes constraint-, sampling-, and optimization-based approaches, but feasibility-focused constraint solving and computationally intensive planning limit performance in complex settings.In, a collision-inclusive planner using ICRRT and trajectory optimization substantially reduced task completion time versus conventional collision-free methods.Source: §3.1 · pp. 10–11

3.2 Learning based Methods

• End-to-end RL learns policies through interaction but can be sample-inefficient; hierarchical RL improves efficiency by dividing tasks, while imitation learning avoids explicit reward design.Hierarchical systems can pair high-level RL planning with low-level controllers; imitation learning instead acquires policies from expert demonstrations, supporting faster practical deployment.Source: §3.2 · pp. 11–12

3.3 Generative Model Based Methods

• Generative models support direct action generation and high-level planning, but low-level control and generalization beyond pretraining remain challenges.Pre-trained models can plan through iterative reasoning and reflection; domain-specific fine-tuning or additional networks are used to improve direct controllers' generalizability.Source: §3.3 · p. 13
• Generative models also support perception, reward design, and data-efficient learning, with reward-function generation reducing repeated model calls during training.World models can generate training data, while alternative methods create simulated environments for agent interaction.Source: §3.3 · pp. 13–14

3.4 Benchmarks

• Benchmarks standardize evaluation and reduce real-world testing costs, spanning household activities, embodied question answering, instruction following, and physical reasoning.Examples include Behavior [170], OpenEQA, ReALFRED, and PhysBench [32].Source: §3.4 · pp. 14–16

4.1 Multi-agent Control and Planning

• Grouped control and sequential decentralized planning address scalability and conflict resolution challenges in multi-agent teams.EMAPF clusters nearby agents for centralized control within groups and independent control between groups; Swarm-Formation orders agents to plan from their predecessors’ trajectories.Source: §4.1 · p. 17, Figure 15 · p. 17

4.2 Learning for Multi-agent Interaction

• Asynchronous timing and agent heterogeneity require learning methods beyond single-agent approaches, including macro-actions and agent-specific policies.ACE uses macro-action-based MAPPO to handle delayed environmental feedback; HetGPPO [10] and COMAT use separate observation and policy networks connected through graph-based communication.Source: §4.2 · p. 18
• Open environments demand continual adaptation as tasks, conditions, and teammates or opponents change unpredictably.Proposed approaches include robust training, continual coordination, scalable GNN or transformer architectures, distributed policies, and self-play or self-reflection [240].Source: §4.2 · pp. 18–19

4.3 Generative Models based Multi-agent Interaction

• Generative models support explicit task allocation, execution-aware reassignment, dependency-aware planning, and implicit assignment through imitation.SMART-LLM [86] decomposes tasks and groups agents by capability; later methods incorporate execution outcomes [188] [109] [213] [28] or sub-task dependency graphs [205] [135].Source: §4.3 · p. 20
• Distributed generative agents communicate and negotiate, while centralized evaluation, shared reflection, and hierarchical topologies address coordination challenges and scale.Agents can request help or complete missing observations [64], and negotiation can evolve team membership, organization, and leadership [113] [27] [64]; designed topologies can scale to thousands of agents [145].Source: §4.3 · pp. 20–21
• Generative models can improve human-agent teamwork through adaptive strategies, active human guidance, inferred intentions, and evolving agent behavior.Agents may query humans when uncertain [51], infer intentions from verbal and behavioral cues, and refine collaboration through interaction [106].Source: §4.3 · p. 21
• Generative world models aim to improve sample efficiency, but modeling agent interactions and inferring global state from local observations remain difficult.Approaches use observation fusion and decoupled global/local modeling [162] [6] [112], including traffic dynamics conditioned on destinations and personalities and diffusion-based shared representations with intention inference.Source: §4.3 · pp. 19, 21, 22

4.4 Benchmarks

• Multi-agent embodied benchmarks remain less standardized and often narrowly scoped; the paper systematically reviews existing benchmarks to identify gaps and guide broader evaluation frameworks.The reviewed benchmarks span specialized settings including pathfinding, manipulation, driving, collaborative perception, navigation, and human-robot collaboration.Source: Table 4 · p. 22, §4.4 · p. 23, Figure 23 · p. 23
• PARTNR [25] evaluates natural-language human–AI collaboration with 100,000 tasks across 60 houses, and highlights a performance gap between current LLM planners and human collaborators.Its tasks cover constraint-free, spatial, temporal, and heterogeneous coordination, with automatically generated simulation-grounded evaluation functions.Source: §4.4 · p. 26
• EmbodiedBench [223] evaluates 19 multimodal LLMs on 1,128 instances across four environments and six capability subsets, while S3E tests C-SLAM across four trajectory patterns and levels of spatial overlap.S3E’s patterns are concentric circles, intersecting circles, an intersection curve, and rays; they test intra- and inter-loop closures.Source: §4.4 · p. 26
• MRMG-Planning [65] evaluates multi-modal, multi-robot, multi-goal planning in continuous spaces, including dynamic changes to robot assignments and scene geometry.Its 21 scenarios include cases with up to five heterogeneous robots and 74 sequential goals, and use RRT* and PRM* baseline planners.Source: §4.4 · p. 26
• CARIC [22] benchmarks heterogeneous multi-UAV inspection, requiring LiDAR-equipped explorers and camera-only photographers to map unknown structures and capture imagery under realistic constraints.Its metrics decompose inspection scores into visibility, motion blur, and spatial resolution.Source: §4.4 · p. 26

Theory for complex Embodied AI interaction

• Embodied multi-agent theory must address asynchronous sensing, delayed actions, limited observability, communication constraints, heterogeneity, and the unclear stability and generalization of LLMs.The authors propose causal and complex-systems perspectives and hierarchical, bio-inspired coordination for real-world complexity.Source: §5 · pp. 26–27

New algorithm design

• QMIX [149] and MAPPO show promise in SMAC and GRF [238], but CTDE [3] assumptions and poor generalization and scalability limit embodied deployment.The section points to hierarchical coordination, agent grouping, structured priors, and classic control theory as directions for robust, scalable algorithms under noisy sensing, limited actuation, delayed feedback, and partial observability.Source: §5 · p. 27

Effective and efficient learning

• Embodied multi-agent learning faces low sample efficiency and costly joint exploration, while single-agent efficiency strategies often falter under complex interactions and non-stationarity.Proposed directions include interactive multi-agent world models, prior-informed exploration, rapid initialization through IL, meta-learning, and multi-task and sim-to-real transfer.Source: §5 · p. 27

Large generative models assisted learning

• Foundation models offer semantic and cross-modal priors, but static single-agent origins and embodied task complexity challenge their direct use in multi-agent settings.The section proposes scalable multi-agent pretraining, integration with reinforcement learning and graph-based coordination, and stronger sim-to-real transfer; it also notes few-shot adaptation, multimodal joint pretraining, and simulation-driven augmentation as transferable insights [117].Source: §5 · p. 27

General multi-agent Embodied AI framework

• Task-specific policies and fixed-team assumptions constrain generalization, motivating equilibrium-aware training, modular scalable architectures, and robust multimodal representations.The section identifies variable tasks, dynamics, and team compositions [238], and the added complexity of extending generalist models such as Gato [150] and RT-X to multi-agent settings; MAT offers permutation-invariant representations, but multimodal integration remains challenging.Source: §5 · pp. 27–28

Adaptation to open environments

• Open environments expose policies to shifting tasks, rewards, observations, and agent populations, requiring robust policies, continual learning, contextual reasoning, and real-time adaptation.The section highlights policy reuse, latent-variable modeling, and meta-learning for unseen circumstances, while calling for better modeling of open-environment dynamics and heterogeneous agent and human behavior [238].Source: §5 · p. 28

Evaluation and verification

• Current benchmarks often lack multimodality, ecological validity, heterogeneous teams, realistic execution conditions, and standardized interpretable metrics.Future evaluation should include modular scalable frameworks, physical testbeds, reproducibility, open leaderboards, robustness, scalability, energy efficiency, behavioral diversity, and formal verification.Source: §5 · p. 28

Applications and implementations; Other future directions

• Practical deployment requires domain-specific coordination, validation, privacy, ethics, safety, and human oversight across applications from robotics and healthcare to smart cities.The section also calls for cross-modal fusion and efficient communication, stronger foundations for stable social learning [131], and explicit safety constraints, ethical guidelines, and explainability; gaming platforms OpenAI Five [8] and AlphaStar face compute and transfer limitations.Source: §5 · pp. 28–29

• The survey bridges single-agent and multi-agent embodied AI through a comparative review intended to give researchers an integrated perspective and facilitate innovation.Source: §6 · p. 29
• It examines foundational concepts and methodologies spanning classic control and planning, learning-based techniques, and generative-model-based frameworks.Source: §6 · p. 29
• It discusses widely adopted benchmarks for evaluating embodied AI in both single-agent and multi-agent contexts, then highlights significant challenges and promising future directions.Source: §6 · p. 29