PaperlayerView exact revision on arXiv

Source-linked AI summary

Large Language Models for Multi-Robot Systems: A Survey

Peihan Li, Zijian An, Shams Abrar, Lifeng Zhou

arXiv:2502.03814v5cs.ROcs.AI

TL;DR

MRS require coordination, communication, and decision-making among embodied robots, but existing reviews do not adequately cover these physical-world constraints. This survey synthesizes LLM applications across planning, motion, action generation, and human intervention, then examines deployment challenges and future research directions.

  • Problem

    Existing surveys do not specifically address how LLMs integrate into physically embodied MRS with their distinctive communication, coordination, and real-world constraints.

  • Method

    The paper systematically reviews LLM use in MRS across task allocation and planning, motion planning, action generation, human intervention, communication, benchmarks, and deployment factors.

  • Results

    The survey identifies hybrid coordination patterns, deployment constraints, the sim-to-real gap, and the absence of LLM-specific metrics in current MRS benchmarks.

  • Takeaways & Limitations

    Future MRS research should prioritize benchmarking, reasoning capabilities, real-time performance, fine-tuning, and task-specific models to support real-world deployment.

  • Takeaways & Limitations

    Current MRS evaluations rarely report hallucination, token consumption, API-call count, or other LLM-specific metrics, while most studies remain simulation-based.

Abstract

from arXiv · show

The rapid advancement of Large Language Models (LLMs) has opened new possibilities in Multi-Robot Systems (MRS), enabling enhanced communication, task allocation and planning, and human-robot interaction. Unlike traditional single-robot and multi-agent systems, MRS poses unique challenges, including coordination, scalability, and real-world adaptability. This survey provides the first dedicated review of LLM integration into MRS. It systematically categorizes their applications across high-level task allocation, mid-level motion planning, low-level action generation, and human intervention. We highlight key applications in diverse domains, such as household robotics, construction, formation control, target tracking, and robot games, showcasing the versatility and transformative potential of LLMs in MRS. Furthermore, we examine the challenges that limit adapting LLMs to MRS, including mathematical reasoning limitations, hallucination, latency issues, and the need for robust benchmarking systems. Finally, we outline opportunities for future research, emphasizing advancements in fine-tuning, reasoning techniques, and task-specific models. This survey aims to guide researchers in the intelligence and real-world deployment of MRS powered by LLMs. Given the rapidly evolving nature of research in the field, we continuously update the paper list in the open-source GitHub repository.

1 Introduction

LLMs are being integrated into MRS to improve communication, coordination, decision-making, and human interaction in dynamic real-world tasks. This survey organizes the emerging literature across four application levels and distinguishes embodied MRS challenges from those of single robots and abstract multi-agent systems.

  • LLMs help robots interpret complex instructions, coordinate behavior, collaborate with teammates, interact with humans, and adapt to dynamic environments.
  • MRS uses multiple autonomous robots to achieve scalable, resilient, and efficient collective behavior across applications such as monitoring, warehousing, and exploration.
  • LLM integration addresses MRS challenges in communication, coordination, and collective decision-making under dynamic and uncertain conditions.Natural-language interfaces can support high-level information exchange instead of predefined communication structures and protocols.
  • LLMs can provide operators with a shared natural-language interface for commanding robots and completing complex real-world missions.
  • The survey categorizes LLM applications in MRS into task allocation and planning, motion planning, action generation, and human intervention.It addresses the need for a dedicated MRS review because prior multi-agent surveys emphasize virtual agents and provide limited MRS coverage.
  • The survey examines communication, benchmarks, simulation environments, deployment-relevant factors, challenges, limitations, and future directions for LLM-based MRS.

2 Background

MRS coordinate multiple embodied robots, while LLM techniques provide language-based reasoning, retrieval, multimodal perception, and action interfaces. Transferring methods from abstract multi-agent systems remains difficult because physical robots face continuous dynamics, imperfect perception, latency, and communication constraints.

  • MRS distribute complex tasks across multiple robots to improve efficiency, reliability, flexibility, scalability, and resilience in applications exceeding a single robot’s capabilities.Teams may be homogeneous or heterogeneous depending on whether robots share the same functionality.
  • Centralized MRS control can bottleneck as group size increases, whereas decentralized control improves resilience and scalability but complicates communication and coordination.
  • LLMs are pretrained language models whose specialized-task performance can be limited when their training data lacks domain-specific coverage.Fine-tuning and retrieval-augmented generation are presented as ways to adapt models without relying only on general pretraining.
  • RAG retrieves relevant information from external databases or knowledge repositories at runtime to improve zero-shot accuracy and applicability on specialized tasks.
  • Multimodal LLMs integrate images, video, audio, or structured sensor data with language, while VLMs and VLAs connect perception and reasoning to executable actions.
  • MRS differs from abstract MAS because physical robots must collectively perceive, decide, and act under dynamic, uncertain, and embodied conditions.
  • Embodied settings invalidate common MAS assumptions about atomic successful actions, perfect observability, lossless synchronous communication, and negligible deliberation latency.
  • Transferred MAS techniques can perform poorly in MRS: fully decentralized dialog degrades as teams grow, and one-shot chain-of-thought fails in the reported MAPF maze setting.

3 Communication Types for LLMs in Multi-robot Systems

LLM performance in multi-robot systems depends strongly on communication architecture and orchestration. Comparisons of centralized, decentralized, and hybrid designs show that hybrid structures can improve scalability and adaptability for complex operations.

  • LLM-augmented agents vary in consistency, coordination, and efficiency according to how their communication architecture integrates individual models.
  • Agent architectures can progress from zero-shot inference to self-thinking loops and few-shot prompting that incorporate prior observations or example actions.
  • Chen et al. compare fully decentralized DMAS, fully centralized CMAS, and hybrid HMAS-1 and HMAS-2 communication frameworks.
  • For teams of six or fewer agents, CMAS and HMAS-2 perform comparably, while HMAS-2 outperforms CMAS on more complex tasks and DMAS and HMAS-1 perform worse.

4 LLMs for Multi-robot Systems

This section categorizes LLM applications in multi-robot systems across task planning, motion planning, action generation, and human intervention. Across these levels, surveyed systems commonly combine LLM reasoning with centralized or hybrid coordination, downstream controllers or solvers, and feedback mechanisms to address scalability and reliability.

  • LLM-based MRS applications span high-level task allocation and planning, mid-level motion planning, low-level action generation, and human intervention.These categories cover reasoning and decision-making, navigation, actuator-level control, and interaction with human operators.
  • High-Level Task Allocation and Planning: Hybrid communication architectures combine central allocation with local validation or reassignment, outperforming purely decentralized coordination in reported warehouse tasks.HMAS-2 achieved 82.5% success on BoxNet1 versus 25% for fully decentralized DMAS, while purely centralized systems degrade as robot counts increase.
  • High-Level Task Allocation and Planning: LLMs commonly perform semantic decomposition or structure extraction, while formal planners, optimization solvers, or schedulers enforce executable constraints and correctness.This division reduces hallucination through constrained representations but can produce optimal solutions to semantically incorrect LLM-generated structures.
  • Mid-Level Motion Planning: Motion-planning systems use runtime global planners, offline encoders, or sparse event-triggered conflict resolvers, with event-triggered designs better matching deliberative LLM latency.Most evaluations involve fewer than ten robots and remain limited by partial observability, fixed communication policies, and limited real-world validation at scale.
  • Low-Level Action Generation: For low-level action generation, successful systems produce persistent executable artifacts for fast classical or physics-based controllers instead of querying LLMs at every control step.Per-step querying degrades with obstacle-rich environments or larger teams because of spatial arithmetic errors, context growth, and geometric misinterpretation.
  • Human Intervention: Human intervention ranges from command-and-report cycles to approval, verification, and on-demand help, primarily mitigating hallucinations, unsafe actions, and out-of-distribution events.The surveyed systems generally treat human oversight as reactive rather than as peer negotiation that actively shapes plans.

5 Applications

LLM-integrated multi-robot systems are applied across household settings and open-world domains, including construction, formation, target tracking, and games. These studies address navigation, task allocation, task decomposition, and team coordination across indoor and outdoor scenarios.

  • Application Scope: Across these domains, LLMs support navigation, task allocation, and dynamic decision-making for diverse indoor and outdoor scenarios.The survey frames these applications as leveraging language-model understanding, planning, and task coordination.
  • Household: Household applications address navigation, task allocation, and task decomposition in complex indoor environments.Studies include multi-target localization and object identification across multiple rooms, emphasizing spatial awareness and adaptability.
  • Other Applications: Construction, Formation, Target Tracking, and Games: LLM-based MRS are also studied for construction, formation, target tracking, and robot games in open-world environments.The survey highlights applications ranging from excavation and transportation to drone formations and other coordination tasks.
  • Comparison Dimensions: Table 1 compares LLM-based MRS by communication structure, robot homogeneity, input modality, evaluation setting, and model type.Its abbreviations distinguish centralized, decentralized, and hierarchical communication; homogeneous and heterogeneous systems; text, image/video, and audio modalities; simulation and real-world evaluation; and VLMs versus LLMs.
  • Overall Observation: Collectively, the surveyed studies demonstrate potential for LLM-driven MRS to address diverse and complex challenges across application domains.

6 LLMs, Simulations, and Benchmarks

The survey examines how LLMs, VLMs, and VLAs support multi-robot reasoning, perception, coordination, and action, alongside simulation platforms and benchmarks for evaluating these systems. Existing benchmarks measure task success but largely omit deployment-relevant LLM metrics and broad physical-team coverage.

  • Multimodal extensions: VLMs embed visual features with language for flexible perception and reasoning, whereas VLAs map observations and commands to executable control sequences.These architectures shorten the perception–action cycle and can support faster responses in dynamic multi-robot settings.
  • LLM capabilities: LLMs support multi-step reasoning, task allocation, motion planning, and human interaction, while VLMs and VLAs add perception grounding and direct action generation.LLMs target long-horizon and symbolic coordination; VLMs interpret visual context; VLAs connect perception and reasoning to executable control.
  • Integration patterns: Integration patterns include visual grounding for language reasoning, joint multimodal prompting, and perception-to-action mapping.Hybrid designs can assign mission strategy to a central LLM, visual interpretation to robot-level VLMs, and low-latency control to VLAs.
  • Simulation environments: Simulation platforms such as AI2-THOR, PyBullet, Pygame, SimRobot, and ARGoS evaluate coordination, manipulation, planning, formation control, and swarm behaviors.These platforms differ in physical realism, scale, and target task, ranging from household-like manipulation to decentralized swarm experiments.
  • Benchmarks: Benchmarks provide standardized comparisons for coordination and adaptability, including robotics environments such as ALFRED, RoCoBench, and COHERENT-Benchmark, plus the multi-agent BOLAA benchmark.Their tasks cover collaborative planning, sequential execution, task distribution, and real-time adaptation across physical-robot or software-agent settings.
  • Benchmark gaps: Task success rate is universal across robotics benchmarks, but hallucination, token use, API calls, latency, reasoning quality, and perception-noise robustness are largely unreported.Only RoCoBench and COHERENT-Benchmark are natively multi-robot; ALFRED is single-agent and BOLAA evaluates software agents.

7 Challenges and Opportunities

The survey identifies reasoning fragility, hallucination, deployment constraints, latency, sim-to-real mismatch, and inadequate benchmarking as barriers to reliable LLM-based MRS. It proposes multimodal, task-specific, efficient, and deployment-aware research directions to improve robustness and evaluation.

  • Challenges: Irrelevant added clauses can reduce state-of-the-art reasoning performance by up to 65%, threatening precise calculations required for trajectories, spatial planning, and task execution.The survey characterizes current reasoning as vulnerable to pattern matching rather than genuine logical deduction.
  • Challenges: Hallucinations can be factual or faithfulness-related, producing incorrect or instruction-inconsistent outputs that may cause coordination errors and faulty decisions.The survey emphasizes detection and mitigation so outputs remain factually accurate and contextually appropriate.
  • Challenges: Server-based models depend on reliable connectivity and server availability, whereas local models avoid those dependencies but require sufficiently powerful onboard hardware.These trade-offs are especially important for remote field deployments and teams whose coordination depends on continuous LLM guidance.
  • Challenges: GPT-4 response time ranged from 15 to 30 seconds per step in one multi-agent path-finding scenario, limiting real-time feasibility.Local processing can reduce latency but increases cost and becomes less scalable as team size grows.
  • Deployment readiness: Deployment evidence remains limited: wall-clock latency is rarely reported, many systems move inference outside the online loop, and deployment-relevant factors are inconsistently evaluated.The survey compares invocation, replanning, communication, observability, and evaluation scale across representative systems.
  • Sim-to-real gap: Most studies rely on idealized simulation, while real deployments face sensing noise, localization errors, actuation limits, communication loss, and external inference latency.The survey describes the LLM-MRS sim-to-real gap as primarily involving perception, representation, and latency rather than only robot dynamics.
  • Benchmarking opportunities: Current benchmarks focus mainly on indoor and household settings, limiting evaluation across diverse and evolving MRS scenarios.The survey calls for unified benchmarking with consistent metrics to improve comparison and standardization.
  • Opportunities: Promising directions include fine-tuning and RAG, task-specific datasets, advanced reasoning methods, lightweight models, multimodal architectures, unstructured-environment evaluation, and realistic simulators.Recommended simulators should model LLM latency and variance, API cost, intermittent communication, and perceptual noise rather than treating inference as instantaneous.

8 Conclusion

This survey establishes a dedicated framework for understanding LLM integration into MRS, spanning planning levels, applications, deployment readiness, and future research needs. It connects current capabilities with challenges that must be addressed for practical deployment.

  • Survey scope: The survey provides the first dedicated review of integrating LLMs into MRS and identifies crosspaper design patterns across planning levels.It also assesses deployment readiness through latency, replanning, communication, and evaluation scale.
  • Structured framework: Its framework spans high-level task allocation and planning, mid-level motion planning, low-level action generation, and human intervention.These levels organize the roles LLMs can play in MRS.
  • Applications: The reviewed applications include household tasks, construction, formation control, target tracking, and games and competitions.These domains illustrate the range of MRS functionalities covered by the survey.
  • Implications: Integrating LLMs can augment individual and collective intelligence for autonomous and collaborative operation in complex environments.The conclusion frames this capability as a basis for additional innovation and efficiency in MRS.
  • Future research: Near-term research must address benchmarking, reasoning capabilities, and real-time performance to narrow the gap between laboratory simulations and real-world applications.Longer-term opportunities include disaster response, space exploration, and large-scale autonomous operations.
  • Future research: The survey aims to support researchers by identifying current gaps and opportunities for moving MRS from theoretical studies toward practical deployments.The authors emphasize continued progress and collaboration across disciplines.
Loading 2502.03814v5…