PaperlayerView exact revision on arXiv

Source-linked AI summary

Multi-agent Embodied AI: Advances and Future Directions

Zhaohan Feng, Ruiqi Xue, Lei Yuan, Yang Yu, Ning Ding, Meiqin Liu, Bingzhao Gao, Jian Sun, Xinhu Zheng, Gang Wang

arXiv:2505.05108v2cs.AIcs.MA

TL;DR

Embodied AI research lacks a comprehensive account of multi-agent systems operating beyond simplified, closed settings. This paper systematically reviews foundational concepts, methods, benchmarks, advances, and future directions, concluding that generalization, scalability, and open-environment adaptation remain central challenges.

  • Problem

    Existing embodied AI research largely centers on single agents and simplified settings, while multi-agent systems face complex coordination challenges in dynamic environments.

  • Method

    The paper systematically reviews embodied AI foundations, optimal control, reinforcement learning, imitation learning, generative models, multi-agent methods, and relevant benchmarks.

  • Results

    The survey bridges single-agent and multi-agent embodied AI while organizing current methodologies, benchmarks, challenges, and future research directions.

  • Takeaways & Limitations

    Progress toward practical multi-agent embodied AI requires attention to generalization, scalability, changing tasks and teams, and open environments.

  • Takeaways & Limitations

    Current methods remain constrained by task-specific scenarios, limited generalizability and scalability, and the mismatch between foundation models and embodied multi-agent settings.

Abstract

from arXiv · show

Embodied artificial intelligence (Embodied AI) plays a pivotal role in the application of advanced technologies in the intelligent era, where AI systems are integrated with physical bodies that enable them to perceive, reason, and interact with their environments. Through the use of sensors for input and actuators for action, these systems can learn and adapt based on real-world feedback, allowing them to perform tasks effectively in dynamic and unpredictable environments. As techniques such as deep learning (DL), reinforcement learning (RL), and large language models (LLMs) mature, embodied AI has become a leading field in both academia and industry, with applications spanning robotics, healthcare, transportation, and manufacturing. However, most research has focused on single-agent systems that often assume static, closed environments, whereas real-world embodied AI must navigate far more complex scenarios. In such settings, agents must not only interact with their surroundings but also collaborate with other agents, necessitating sophisticated mechanisms for adaptation, real-time learning, and collaborative problem-solving. Despite increasing interest in multi-agent systems, existing research remains narrow in scope, often relying on simplified models that fail to capture the full complexity of dynamic, open environments for multi-agent embodied AI. Moreover, no comprehensive survey has systematically reviewed the advancements in this area. As embodied AI rapidly evolves, it is crucial to deepen our understanding of multi-agent embodied AI to address the challenges presented by real-world applications. To fill this gap and foster further development in the field, this paper reviews the current state of research, analyzes key contributions, and identifies challenges and future directions, providing insights to guide innovation and progress in this field.

1 Introduction

Embodied AI integrates perception, planning, decision-making, and action with active environmental interaction and adaptation. This survey extends the field from single-agent systems toward multi-agent coordination in complex, dynamic settings.

  • Embodied AI equips agents to perceive, plan, decide, and act while interacting with and adapting to their environments.
  • Its central challenge is combining robust perception and decision-making with continuous learning in dynamic, evolving environments.
  • Multi-agent embodied AI faces enlarged joint action spaces, partial observability, learning-induced non-stationarity, and difficult credit assignment.
  • Research remains nascent, often adapting single-agent methods while newly developing benchmarks for embodied multi-agent scenarios.
  • The paper systematically reviews foundations, methods, benchmarks, recent multi-agent advances, and future directions for embodied AI.

2 Preliminaries

This section introduces Embodied AI, multi-agent systems, and core methods including optimal control, reinforcement learning, and generative architectures. It emphasizes continuous perception-action interaction, decentralized coordination, and methods for real-time control and scalable sequence processing.

  • Embodied AI: Embodied AI agents perceive, act on, and adapt to environments through continuous interaction using integrated perception, cognition, and action components.Sensors provide context-dependent information, cognitive modules support reasoning and planning, and actuators translate decisions into environmental changes.
  • Embodied AI: Embodied intelligence is grounded in physical bodies and closed-loop interaction, where actions alter future sensory inputs and support adaptive learning.The body constrains and enables behavior, while continual perception, decision-making, and action create a feedback-rich learning cycle.
  • Embodied AI: Large-scale multimodal models and LLMs extend embodied agents with semantic understanding, instruction following, contextual reasoning, and few-shot adaptation.These capabilities have supported applications in robotics, autonomous driving, intelligent manufacturing, and healthcare.
  • Multi-Agent Systems: Multi-agent systems use decentralized agents that interact locally while coordinating globally, supporting scalability, fault tolerance, and adaptability in dynamic environments.Applications include parallel sensing, distributed planning, and real-time coordination in robotics, autonomous driving, and smart infrastructure.
  • Multi-Agent Systems: MARL addresses multi-agent coordination through methods including centralized training with decentralized execution, parameter sharing, credit assignment, and opponent modeling.These techniques target non-stationarity, coordination, and partial observability in learning-based multi-agent systems.
  • Optimal Control: Optimal control selects policies that optimize criteria such as time, energy, or cost under system dynamics and control constraints.MPC repeatedly solves a finite-horizon optimization online, implements only the first action, and supports real-time coordinated control of robot swarms and UAV formations.
  • Reinforcement Learning: RL learns sequential decisions through trial-and-error interaction, while MARL extends this framework to settings where multiple agents coordinate or compete.Single-agent RL is commonly formalized as an MDP, whereas fully cooperative MARL is often modeled with Dec-POMDPs.
  • Generative Models: Mamba uses continuous-time state-space models and selective scanning to improve efficiency and scalability for long-sequence processing.Its architecture is presented as an alternative to Transformers and diffusion models for long-range dependencies and real-time computation.

3 Single-Agent Embodied AI

Single-agent embodied AI methods have progressed from manually designed planning and control toward learning-based, hierarchical, demonstration-driven, and generative-model approaches. These methods address computational, generalization, sample-efficiency, and representational challenges through different combinations of direct policy learning, structured control, demonstrations, and generative reasoning.

  • Overview: Single-agent embodied AI research has shifted from classic task-specific planning toward learning-based methods that adapt across diverse tasks and environments.Manual design becomes infeasible as agents operate in dynamic and unstructured environments.
  • Classic Planning and Control: Classic planning methods include constraint-based, sampling-based, and optimization-based approaches for generating trajectories subject to task and environmental requirements.Optimization-based methods formulate objectives and feasibility conditions explicitly, including approaches such as polynomial trajectory planning, MPC, and OC.
  • End-to-End Learning: End-to-end RL maps perceptual information directly to action decisions, avoiding manual decomposition and producing stable, high-frequency control outputs.The learned policy directly interacts with the environment, while optimization maximizes expected cumulative rewards.
  • Hierarchical Learning: Hierarchical learning combines high-level RL planning or policy generation with robust low-level controllers to address complex objectives and large state-action spaces.This structure targets end-to-end RL’s low sample efficiency and prolonged training caused by expansive exploration spaces.
  • Learning from Demonstrations: Learning from demonstrations reduces reliance on costly trial-and-error interaction and manually specified rewards by using expert behavior as direct supervision.The paper illustrates demonstration-based learning through acquired writing and painting skills.
  • Generative Models: Generative models are explored to improve representational capacity, prior-knowledge use, multimodal processing, task planning, perception, and reward design.Large VLMs can act as direct decision-makers or support high-level planning, while other generative methods assist perception and reward construction.

4 Multi-agent Embodied AI

Multi-agent embodied AI extends embodied intelligence to settings where agents must coordinate in shared, dynamic environments. The section reviews control-, learning-, and generative-model-based approaches, then surveys benchmarks and their limitations.

  • Multi-agent embodied tasks require collaboration because real-world environments involve multiple agents, human-AI interaction, and emergent group behaviors.
  • Control and planning methods: Control-based methods support precise, real-time decisions, but centralized planning has scalability challenges and distributed control addresses larger multi-agent systems.
  • Learning-based methods: Learning-based methods must handle asynchronous decisions, heterogeneous agents, and open environments beyond the challenges of single-agent learning.
  • Learning-based methods: Heterogeneous-agent methods use separate observation and policy networks connected through graph communication, while advantage decomposition improves credit assignment.
  • Learning-based methods: Open environments change objectives, environmental factors, rewards, and collaboration patterns unpredictably, motivating robust training and continual coordination.
  • Generative model-based methods: Generative models assign tasks, support distributed decision-making and negotiation, and use pretrained knowledge to reduce individual exploration spaces.
  • Benchmarks: Existing multi-agent benchmarks often target specialized tasks or narrow scenarios, limiting their general applicability and generalizability.
  • Benchmarks: RoCo evaluates six multi-robot manipulation tasks involving decomposition, observation symmetry, workspace overlap, zero-shot adaptation, and feedback-driven replanning.

5 The Challenges and Future Works

The paper identifies theoretical, learning, foundation-model, generalization, open-environment, evaluation, application, and safety challenges for multi-agent embodied AI. It calls for stronger theories, more efficient learning, general frameworks, realistic benchmarks, and safer multimodal systems.

  • Theory for complex Embodied AI interaction: Embodied multi-agent intelligence introduces theoretical challenges from asynchronous sensing, delayed actions, limited observability, communication constraints, and agent heterogeneity.
  • New algorithm design: Future algorithm design must move beyond benchmark successes that predominantly rely on centralized training with decentralized execution assumptions.
  • Effective and efficient learning: Embodied multi-agent tasks reduce sample efficiency because joint exploration spans vast state and policy spaces, while real-world interactions incur time, financial, and hardware costs.
  • Large generative models assisted learning: Foundation models lack inductive biases for asynchronous communication, partial observability, tightly coupled policies, and non-stationarity in embodied multi-agent scenarios.
  • General multi-agent Embodied AI framework: General multi-agent frameworks must address variable tasks, dynamics, team compositions, non-stationarity, multiple equilibria, and assumptions of fixed team sizes or homogeneous agents.
  • Adaptation to open environments: Open environments produce shifting observations, rewards, tasks, populations, and human behaviors, undermining consistency between training and deployment.
  • Evaluation and verification: Current evaluation frameworks often lack multimodal signals, heterogeneous embodiments, partial observability, communication latency, adversarial behavior, and asynchronous execution.
  • Other future directions: Future directions include multimodal fusion, efficient communication, theories of emergent social learning, and explicit safety and ethical constraints.

6 Conclusion

The survey systematically reviews embodied AI foundations, methodologies, benchmarks, and the transition from single-agent to multi-agent settings, while identifying challenges and future directions.

  • The survey bridges single-agent and multi-agent embodied AI through a systematic review of methods, benchmarks, challenges, and future directions.It covers classic control and planning, learning-based techniques, generative-model frameworks, and evaluation benchmarks.
Loading 2505.05108v2…