PaperlayerView exact revision on arXiv

Source-linked AI summary

Learn as Individuals, Evolve as a Team: Multi-agent LLMs Adaptation in Embodied Environments

Xinran Li, Chenjia Bai, Zijian Li, Jiakun Zheng, Ting Xiao, Jun Zhang

arXiv:2506.07232v1cs.MAcs.AIcs.LG

TL;DR

LLM planners may lack embodied-context understanding and mechanisms for effective multi-agent cooperation. LIET adapts them through individual utility learning and evolving team communication; it outperforms existing baselines on C-WAH and TDW-MAT, while relying on pretrained vision modules to convert visual information into text.

  • Problem

    Off-the-shelf LLM planners can lack the embodied-context understanding and dynamic cooperation needed for multi-agent embodied planning.

    Source: §1 · p. 2

  • Method

    LIET combines individual learning of environment-specific utility with team-level communication that evolves through shared experience and reflection.

    Source: §1 · p. 2, §3.3 · p. 5

  • Results

    LIET outperforms baselines on C-WAH and TDW-MAT benchmarks, with results reported across LLaMA 3.1-70B and ChatGPT-4o.

    Source: §4.2 · p. 8

  • Takeaways & Limitations

    The authors conclude that individual utility learning and evolving communication enhance planning performance on multi-agent embodied benchmarks.

    Source: §5 · p. 9

    LIET relies on pretrained vision modules to convert visual information into text for LLM planning; integrated multimodal processing remains future work.

    Source: §5 · p. 10

Abstract

from arXiv · show

Large language models (LLMs) possess extensive knowledge bases and strong reasoning capabilities, making them promising tools for complex, multi-agent planning in embodied environments. However, despite LLMs' advanced abilities and the sophisticated modular design of agentic methods, existing LLM-based planning algorithms remain limited by weak adaptation capabilities to multi-agent embodied scenarios. We address this limitation by introducing a framework that enables LLM agents to learn and evolve both before and during test time, equipping them with environment-relevant knowledge for better planning and enhanced communication for improved cooperation. Inspired by centralized training with decentralized execution in multi-agent reinforcement learning, we propose a \textit{Learn as Individuals, Evolve as a Team (LIET)} paradigm for multi-agent LLMs adaptation. At the individual level, LLM agents learn a local utility function from exploratory datasets to better comprehend the embodied environment, which is then queried during test time to support informed decision-making. At the team level, LLM agents collaboratively and iteratively maintain and update a shared cooperation knowledge list based on new experiences, using it to guide more effective communication. By combining individual learning with team evolution, LIET enables comprehensive and flexible adaptation for LLM agents. Our experiments on Communicative Watch-And-Help and ThreeD-World Multi-Agent Transport benchmarks demonstrate that LIET, instantiated with both LLaMA and GPT-4o, outperforms existing baselines and exhibits strong cooperative planning abilities.

1 Introduction

Off-the-shelf LLM planners can lack embodied-context understanding and dynamic cooperation, prompting the question of whether agents can adapt through interaction. LIET combines individual learning with team collaboration, and is reported to outperform baselines on multi-agent planning benchmarks.

1 Introduction

  • LIET adapts agents through individual learning and team collaboration, drawing on prior experience and test-time interactions.Its semi-centralized approach has agents explore individually, then collaborate through communication and reflection.Source: §1 · p. 2
  • LIET significantly outperforms existing baselines on the C-WAH and TDW-MAT multi-agent planning benchmarks.The reported results highlight its effectiveness for long-horizon collaborative planning.Source: §1 · p. 2
  • Its individual utility function estimates candidate-plan costs, while evolving prompts use a shared knowledge list to improve communication strategies.The list is refined through receiver-side reflection on whether messages improve planning outcomes in new situations.Source: §1 · p. 2

2 Background

The paper frames fully cooperative embodied decision-making as a Dec-POMDP and reviews LLM planning and multi-agent collaboration approaches. LIET is positioned as a semi-centralized alternative to decentralized and centralized collaboration methods.

2.1 Problem Setting

  • The problem is modeled as a fully cooperative Dec-POMDP with partial environmental observations, messages, embodied and communication actions, and a shared goal.Experiments use long-horizon household tasks; high-level actions can fail when not grounded in the current environment state.Source: §2.1 · p. 2

2.2 Related Work

  • Single-agent LLM planning work grounds actions with value functions, thinking modules, world abstractions, demonstrations, or vision-language alignment.Examples include SayCan, grounded decoding, and Text2Motion [14] [21].Source: §2.2 · p. 3
  • Decentralized collaboration can lack global consensus, while centralized discussion can struggle with partial observability, scaling agents, and communication costs.LIET instead combines individual planning with an evolving communication scheme that uses past experiences.Source: §2.2 · p. 3
  • Realism-oriented work includes LLaMAR’s imperfect simulator observations and verifier, and SMART-LLM’s deployment on real robots.The paper describes these efforts as orthogonal and potentially combinable with LIET.Source: §2.2 · p. 3

3 Method

LIET addresses embodied multi-agent planning with decentralized utility-guided decisions and team communication that evolves through interaction. Individual agents estimate action costs from local descriptions, while receiver feedback updates shared communication guidance.

3 Method

  • LIET comprises a decentralized utility module for independent decisions and an evolving communication scheme for cooperation and mutual understanding.The framework adapts prompts to address limited environment-relevant knowledge and collaboration gaps.Source: §3 · p. 3, §3.1 · p. 4

3.1 Overall Framework of LIET

  • Agents plan independently using a utility function and maintain a shared list of cooperation tips to guide messages, with the list refined through reflection.The framework is illustrated with two agents but is not limited to that configuration.Source: §3.1 · p. 4, Figure 1 · p. 4

3.2 Learn as Individuals

  • The goal-agnostic utility function maps local-observation and action descriptions to estimated environmental-step costs for test-time plan evaluation.A fine-tuned LLM with a value head learns cost estimates as a regression task from exploratory description-cost pairs using MSE; cost collection requires no task-specific expertise or human expert labeling.Source: §3.2 · pp. 4–5

3.3 Evolve as a Team

  • At test time, agents use a shared knowledge list to generate messages, and receivers reflect on message usefulness to update that list for later interactions.The sender’s local observation includes task progress, action history, and dialogue history; receiver reflection can use additional sender details, such as current plans.Source: §3.3 · pp. 5–6

4 Experimental Results

LIET is evaluated on two cooperative embodied-planning benchmarks and compared with traditional and LLM-based planners. The reported results favor LIET across both benchmarks, with visualizations, ablations, and multi-agent tests examining its behavior and components.

4.1 Environmental Setups

  • C-WAH evaluates household chores over 10 episodes and five tasks, while TDW-MAT measures object transport across food and stuff tasks.C-WAH episodes have a 250-timestep horizon; TDW-MAT includes 24 episodes of up to 3000 time frames and scores transport rate.Source: §4.1 · p. 7
  • The baselines include hierarchical traditional planners and LLM-based methods that are decentralized or centralized.The LLM-based methods are CoELA, ProAgent, and RoCo; the traditional methods are MHP and RHP.Source: §4.1 · p. 7

4.2 Results

  • LIET outperforms baseline methods on both C-WAH and TDW-MAT benchmarks.The reported comparisons use LLaMA 3.1-70B and ChatGPT-4o; Table 1 covers C-WAH and Table 2 covers TDW-MAT.Source: Table 1 · p. 7, Table 2 · p. 7, §4.2 · p. 8
  • In a C-WAH episode, agents weigh task progress, utility-estimated costs, and dialogue history, while the cooperation list corresponds directly to generated messages.Alice proposes a division of labor and Bob selects the more efficient subtask; Figure 3 displays plans, thoughts, the cooperation list, and a generated message.Source: §4.2 · pp. 8–9, Figure 3 · p. 8
  • All three C-WAH ablations degrade performance, covering removal of individual learning, prompted cost estimation, and removal of team evolution.Table 3 reports the ablation comparison for GPT-4o with symbolic perception.Source: §4.2 · p. 9, Table 3 · p. 9
  • On C-WAH with GPT-4o, LIET outperforms baselines across agent counts using utility functions trained only on two-agent data.The reported explanation highlights the combination of local cost estimation and global cooperation through evolving communication prompts.Source: §4.2 · p. 9, Table 4 · p. 9

5 Conclusions

LIET combines individual utility learning with team-level refinement of communication, and the paper reports strong performance and robustness when adding agents. Its visual-information processing relies on pretrained vision modules that convert images to text.

5 Conclusions

  • LIET combines utility functions learned during exploration with team prompts refined through mutual feedback, and the reported benchmarks show superior performance and robustness with additional agents.The conclusion contrasts these adaptation mechanisms with relying solely on zero-shot capabilities.Source: §5 · p. 9
  • The reported conclusion emphasizes adaptation mechanisms over relying solely on LLMs’ zero-shot capabilities in complex embodied scenarios.Source: §5 · p. 9
  • The conclusion identifies pretrained vision modules that convert visual information into text as a limitation in multimodal processing.Comprehensive integration of multimodal information in embodied environments is left as future research.Source: §5 · p. 10

A.1 Implementation Details on LIET

LIET implementation combines utility-function fine-tuning on exploratory data with prompts for planning, message generation, and updating a shared cooperation knowledge list.

A.1 Implementation Details on LIET

  • 10 C-WAH episodes and 24 TDW-MAT episodes supplied exploratory data; 80% was used to fine-tune a LLaMA 3.2-1B model with LoRA adapters and an MLP value head.Table 5 lists architectural and training hyperparameters.Source: §A.1 · p. 13, Table 5 · p. 13
  • The C-WAH planner prompt supplies the shared goal, dialogue, progress, prior actions, and available actions to guide action selection.The message-generator prompt requests a short message informed by the goal, progress, action history, and dialogue; generated messages can use the shared knowledge list as hints.Source: §A.1 · pp. 13–14
  • A shared knowledge list is updated from new experiences and accumulated knowledge to guide future communication during decentralized planning.The reflector prompt supplies the dialogue, prior actions, current plans, and existing list for updating.Source: §A.1 · p. 14

A.2 Additional Details on Environments

The appendix follows evaluation schemes from prior work [38, 22] and describes the TDW-MAT environment, tasks, and agent interfaces.

A.2.1 TDW-MAT

  • Agents transport as many target objects as possible to goal positions within 3000 frames, using containers that hold up to three objects or carrying up to two objects without one.Success is measured by the number of objects transported.Source: §A.2.1 · p. 15
  • TDW-MAT has food- and stuff-transport tasks, with 10 targets and 2–5 containers per scenario across four floorplans and three layout variations each.Two floorplans are training environments and two are reserved for testing; objects are placed in four room types, with food mainly in kitchens and stuff mainly in offices.Source: §A.2.1 · p. 15
  • Agents observe forward-facing RGB and depth images, with optional oracle perception, plus their simulated position and rotation.The RGB view is 512 × 512 with a 90-degree field of view; the depth image uses the same camera parameters.Source: §A.2.1 · p. 15
  • TDW-MAT provides seven action types, including movement, turning, grasping, putting targets into containers, dropping objects, and sending messages.Messages are limited to 500 characters per frame.Source: §A.2.1 · p. 15

A.2.2 C-WAH

C-WAH evaluates household-task completion through specified subgoals, with defined observation and action spaces.

A.2.2 C-WAH

  • C-WAH includes five household scenarios with 3–5 subgoals each, and success requires completing all subgoals within 250 steps.The tasks are Prepare afternoon tea, Wash dishes, Prepare a meal, Put groceries, and Set up a dinner table.Source: §A.2.2 · p. 15
  • Agents receive egocentric RGB and depth images, optional color-coded oracle perception, their position, and messages from other agents.The RGB view is 256 × 512 with a 60-degree field of view.Source: §A.2.2 · p. 16
  • The eight action types include navigation, turning, grasping, opening and closing containers, placing objects, and sending messages.Messages are limited to 500 characters.Source: §A.2.2 · p. 16

A.3 Experimental Infrastructure

Experiments used NVIDIA A100 GPUs and hosted LLaMA 3.1-70B and ChatGPT-4o APIs.

A.3 Experimental Infrastructure

  • Each experimental run took less than 2 days; LLaMA 3.1-70B used TogetherAI APIs and ChatGPT-4o used OpenAI APIs.Experiments were conducted using NVIDIA A100 GPUs.Source: §A.3 · p. 16

B More Experimental Results

A visualized C-WAH episode shows LIET combining task progress, efficiency, and dialogue history, while communication shifts toward increasingly specific task division guided by the cooperation knowledge list.

B More Experimental Results

  • The illustrated decision process integrates task progress, efficiency, and dialogue history.The authors present the episode as additional visualization evidence for conclusions in Section 4.2.Source: B More Experimental Results · p. 16
  • Agents’ proposals shift from room-level to object-level task division as the episode progresses, aligning with the knowledge list’s recommendation for flexible task allocation.The authors describe this as an example of effective collaboration through adaptive communication.Source: B More Experimental Results · p. 17
  • Early concentration in the kitchen and living room coincides with more discussion of labor division and progress updates.These rooms contain most target objects.Source: B More Experimental Results · p. 17
  • Figure 4 depicts a C-WAH delivery episode with annotated plans and thoughts, LIET-generated dialogue, and the current cooperation knowledge list.The task is to put wine, cupcakes, and pudding on the coffee table.Source: Figure 4 · p. 17
Loading 2506.07232v1…