PaperlayerView exact revision on arXiv

Source-linked AI summary

Towards Collaborative Intelligence: Propagating Intentions and Reasoning for Multi-Agent Coordination with Large Language Models

Xihe Qiu, Haoyu Wang, Xiaoyu Tan, Chao Qu, Yujie Xiong, Yuan Cheng, Yinghui Xu, Wei Chu, Yuan Qi

arXiv:2407.12532v1cs.CLcs.AI

TL;DR

LLM agent frameworks face limits in single-agent execution, inter-module communication, and grounding, complicating cooperative coordination. REMALIS propagates intentions and uses execution feedback to adapt planning and grounding; it improves performance on complex collaborative tasks and outperforms several baselines.

  • Problem

    Existing agent frameworks often depend on single-agent execution, lack inter-module communication, and use static grounding, while cooperative MARL may converge on suboptimal policies.

    Source: §1 · p. 1

  • Method

    REMALIS propagates teammate-relevant intentions and connects execution feedback with planning and grounding adjustments.

    Source: §1 · p. 2, §3.1 · p. 4

  • Results

    REMALIS improves performance on complex collaborative tasks and significantly outperforms several baseline methods.

    Source: §1 · p. 2, §5 · p. 8

  • Takeaways & Limitations

    The paper concludes that frameworks enabling cooperative skills analogous to human team members are a promising direction for LLM agents.

    Source: §5 · p. 8

    REMALIS uses centralized training, which may hinder scalability in fully decentralized environments.

    Source: §6 · p. 9

Abstract

from arXiv · show

Effective collaboration in multi-agent systems requires communicating goals and intentions between agents. Current agent frameworks often suffer from dependencies on single-agent execution and lack robust inter-module communication, frequently leading to suboptimal multi-agent reinforcement learning (MARL) policies and inadequate task coordination. To address these challenges, we present a framework for training large language models (LLMs) as collaborative agents to enable coordinated behaviors in cooperative MARL. Each agent maintains a private intention consisting of its current goal and associated sub-tasks. Agents broadcast their intentions periodically, allowing other agents to infer coordination tasks. A propagation network transforms broadcast intentions into teammate-specific communication messages, sharing relevant goals with designated teammates. The architecture of our framework is structured into planning, grounding, and execution modules. During execution, multiple agents interact in a downstream environment and communicate intentions to enable coordinated behaviors. The grounding module dynamically adapts comprehension strategies based on emerging coordination patterns, while feedback from execution agents influnces the planning module, enabling the dynamic re-planning of sub-tasks. Results in collaborative environment simulation demonstrate intention propagation reduces miscoordination errors by aligning sub-task dependencies between agents. Agents learn when to communicate intentions and which teammates require task details, resulting in emergent coordinated behaviors. This demonstrates the efficacy of intention sharing for cooperative multi-agent RL based on LLMs.

1 Introduction

Existing LLM agent frameworks face limits in execution, inter-module communication, and grounding, while cooperative MARL can produce suboptimal policies. REMALIS propagates intentions and uses feedback to support coordinated planning and grounding, with improved performance reported on complex collaborative tasks.

1 Introduction

  • Existing frameworks often rely on a single execution agent, lack inter-module communication, and ground plans without execution feedback.The passage also notes that MARL methods can converge on suboptimal policies that lack cooperation.Source: §1 · p. 1
  • REMALIS propagates intentions between LLM agents and links execution feedback with planning and grounding adjustments.Execution agents can prompt collaborative re-planning, while coordination patterns guide grounding changes.Source: §1 · p. 2
  • Intention propagation reduces misaligned sub-tasks, while adjusted grounding strategies improve LLM scene comprehension.Planning re-planning guided by execution feedback is also reported to increase goal-oriented coordination.Source: §1 · p. 2
  • REMALIS improves performance on complex collaborative tasks across traffic-flow prediction and web-based activities datasets.The reported results also describe intention communication, strategic adjustment, and collaborative re-planning.Source: §1 · p. 2

2 Preliminary

This section describes planning and grounding components alongside cooperative execution by specialized agents. Their training uses task-specific losses and optimized grounding-task weights to support coherent team outputs.

2 Preliminary

  • The planning module predicts the next pending sub-goal from the current sub-goal, intention, grounded embedding, and agent feedback.It is trained to maximize the likelihood of sub-goals along decision sequences, enabling feedback-based re-planning of sub-task dependencies.Source: §2 · p. 2
  • The grounding module contextualizes symbol embeddings using states, intentions, and feedback, with a GNN grounding function and uncertainty modeling.The latent variable is modeled as a normal distribution to capture multimodal grounding uncertainties.Source: §2 · pp. 2–3
  • Cooperative execution assigns distinct semantic domains to specialized agents, which are trained simultaneously to foster collaboration.The design distributes tasks and reduces overlap in capabilities.Source: §2 · p. 3
  • Joint training optimizes agent outputs against ground-truth labels and adjusts soft module weights for grounding-task decomposition.The objective is optimized collectively across agents to improve individual output accuracy and team-level coherence.Source: §2 · p. 3

3 Approach

REMALIS combines intention propagation, bidirectional coordination, and recursive reasoning agents across planning, grounding, and cooperative execution.

3.1 Planning with Intention Propagation

  • Each agent's private intention stores its current goal, related sub-goals, a next-sub-goal distribution, and desired teammate assignments.Source: §3.1 · p. 4
  • A recurrent communication channel infers teammate intentions from messages and is trained to maximize coordination reward, propagating relevant sub-task dependencies.Source: §3.1 · p. 4

3.2 Grounding with Bidirectional Coordination Channels

  • Execution feedback summarizes coordination patterns, including errors, grounding uncertainty, and replanning triggers, to guide upstream planning and grounding adjustments.Source: §3.2 · pp. 4–5

3.3.1 Agent Policy Generation

  • The action policy conditions on state, intention, and coordination feedback; a feedback-conditioned policy approximation supplies auxiliary regularization during training.Source: §3.3.1 · p. 5

3.3.2 Grounding Strategy Adjustment

  • Graph-based action-dependency modeling captures relational structure for coordinated action generation conditioned on communication.Source: §3.3.2 · p. 5
  • Grounding adjustment uses coordination feedback, concept-confusion estimates, sensitivity-weighted regularization, and episodic uncertainty memory to adapt against miscoordination.Source: §3.3.2 · p. 5

3.4 Collective Learning and Adaptation

  • Collective learning optimizes reward alongside weighted grounding uncertainty and coordination errors, with regularization intended to support stable and robust learning.Source: §3.4 · p. 6

4 Experiments

Experiments evaluate REMALIS on traffic-flow prediction and web activities, across task difficulties and against single-agent baselines. Results report stronger performance and coordination, with ablations examining intention propagation, feedback, recursive reasoning, and sharing strategies.

4.1 Datasets

  • The evaluation uses 100,000 TFP scenarios and over 500,000 web-interaction examples spanning 150 domains.TFP examples include road, vehicle, weather, and traffic-control information; web examples cover multi-step activities such as bookings and reservations.Source: §4.1 · p. 6

4.2 Implementation Details

  • Tasks are divided into four difficulty levels per dataset, and examples are split 80%/10%/10% for training, validation, and testing.TFP includes Easy, Medium, Hard, and Hell; web activities uses Easy, Medium, Hard, and All.Source: §4.2 · p. 6

4.3 Results and Analysis

  • Across a range of complexities, REMALIS achieved 55.37%, surpassing the second-highest-scoring method, LUMOS, which scored 51.49%. Additionally, REMALIS (7B) outperformed 13B AUTOACT by over 3 percentage points, scoring 52.87%.Source: §4.3 · p. 7
  • On web “Hard” tasks, REMALIS scored 21.42%, exceeding GPT-3.5 Turbo’s 17.36% by over four points and LUMOS by over two.The passage also reports that REMALIS exceeded GPT-3.5 Turbo across all difficulty levels.Source: §4.3 · p. 7
  • On TFP, REMALIS outperformed the next-best methods at Easy (89.15%), Medium (77.62%), Hard (64.53%), and Hell (55.37%).The reported margins were 1.26, 1.33, 2.19, and 0.56 percentage points, respectively.Source: §4.3 · p. 7

4.4 Ablation Studies

  • Removing intention propagation lowers accuracy by over 6% across both datasets; removing bidirectional coordination channels lowers performance by 4.37%.Without feedback coordination, agents are less responsive to scenarios requiring replanning.Source: §4.4 · p. 8, Table 2 · p. 8
  • Replacing recursive reasoning with convolutional and recurrent networks reduces contextual inference accuracy by 5.86%.Full Intention Sharing aligns sub-tasks at 91%, 71%, and 62% across Easy, Medium, and Hard, versus 31%, 23%, and 17% with no communication; on Hard tasks it takes 521 ms versus 1198 ms.Source: §4.4 · p. 8

5 Conclusion

REMALIS combines intention propagation, bidirectional coordination channels, and recursive reasoning in a multi-agent LLM framework. The paper reports that it outperforms several baselines and frames the approach as a step toward flexible coordination in complex collaborative environments.

5 Conclusion

  • REMALIS combines intention propagation, bidirectional coordination channels, and recursive reasoning to support coordinated multi-agent LLM collaboration.The paper reports that the framework significantly outperforms several baseline methods.Source: §5 · p. 8

6 Limitiation

The section identifies centralized training as a potential scalability boundary for REMALIS and reviews related single-agent, MARL, and integrated-learning approaches.

6 Limitiation

  • REMALIS uses centralized training, which may hinder scalability in fully decentralized environments; it also lacks explicit handling of agents arriving or departing during execution.The authors add that recursive reasoning may struggle with long-term dependencies and planning horizons beyond a certain time frame.Source: §6 · p. 9

A.1 Single Agent Frameworks

  • Modular frameworks such as LUMOS and FiReAct separate planning from execution, but still rely on a single agent for final action execution.REMALIS extends this design by using a cooperative team of execution agents.Source: §A.1 · p. 12

A.2 Multi-Agent Reinforcement Learning

  • Prior MARL methods support decentralized execution with centralized training, while REMALIS emphasizes modular LLMs for complex language tasks.Its execution agents specialize in semantic domains and are coordinated through a communication module.Source: §A.2 · p. 12

A.3 Integrated & Collaborative Learning

  • REMALIS jointly trains semantically consistent specialized execution agents and learns a communication module to aggregate and propagate messages across them.The paper describes this integrated and collaborative learning as synergizing individual skills into collective intelligence.Source: §A.3 · p. 12

B Methodology and Contributions

REMALIS applies integrated learning and communication to coordinate specialized execution agents. Its contributions include attentive message propagation, dynamic feedback to grounding and planning, and collaboration on complex tasks.

B Methodology and Contributions

  • REMALIS is presented as an integrated-learning framework for communication and collaboration among specialized agents.Its execution agents are trained cooperatively and specialize in different semantic domains while understanding peer abilities.Source: B Methodology and Contributions · p. 12
  • An attentive communication module propagates informative cues across agents and coordinates execution strategies without explicit supervision.The module is described as acting in the role of a team leader.Source: B Methodology and Contributions · pp. 12–13
  • REMALIS is designed for more complex tasks than single-agent counterparts through specialized knowledge and close communicative coordination.The paper describes the resulting team intelligence as emergent.Source: B Methodology and Contributions · p. 13
  • Dynamic feedback from communication reaches grounding and replanning in planning, increasing adaptability when execution difficulties arise.Source: B Methodology and Contributions · p. 13

C Key variables and symbols

  • Table 4 summarizes symbols for REMALIS’s planning, grounding, execution policies, intentions, goals, sub-goals, and intention-propagation channel.Source: C Key variables and symbols · p. 13

D Tasks Setup

Traffic-control tasks are divided into four difficulty levels.

D.1 Traffic Control

  • Traffic-control tasks are classified as Easy, Medium, Hard, and Hell.Source: §D.1 · p. 13

D.2 Web Tasks

Web tasks are grouped into Easy, Medium, Hard, and All difficulty levels, ranging from simple interactions to open-ended exploration.

D.2 Web Tasks

  • Easy web tasks involve a single-link click or short phrase on a simple interface with clear options.Source: §D.2 · p. 13
  • Medium tasks require multi-page action sequences, such as selecting filters or submitting forms, and test understanding of site flow.Source: §D.2 · p. 13
  • Hard tasks require open-ended exploration through dense, ambiguous sites, chaining obscure links and controls to reach goals.Source: §D.2 · p. 13
  • All combines tasks across the difficulty spectrum to assess generalized web-agent skills.The passage says performance at this level correlates with readiness for real-world web use cases.Source: §D.2 · p. 13

E Experimental Setups

The experimental setup compares REMALIS with LUMOS, AgentLM, and GPT-3.5, and describes model and specialized-module configurations.

E Experimental Setups

  • The comparison includes REMALIS, LUMOS, AgentLM, and GPT-3.5, whose sizes, architectures, and training configurations differ.Source: E Experimental Setups · p. 13
  • REMALIS is a 7-billion-parameter GNN model trained for 15 epochs with AdamW, a 1e-4 learning rate, and batch size 32.Source: E Experimental Setups · p. 13
  • LUMOS has 13 billion parameters and uses a Transformer architecture, while AgentLM has 6 billion parameters and also uses a Transformer.Source: E Experimental Setups · p. 13
  • Table 4 lists key variables and symbols in the proposed recursive multi-agent learning framework, while Table 5 compares traffic-network complexity levels.Source: Table 4 · p. 14, Table 5 · p. 14
  • REMALIS uses seven specialized and integrated execution agents, while LUMOS uses a single integrated execution agent.Source: E Experimental Setups · p. 14

F Pseudo-code

The pseudo-code describes hierarchical planning and grounding, intention inference through a propagation channel, and execution feedback that guides upstream modules.

F Pseudo-code

  • The planning module encodes the current sub-goal, intention, grounded embedding, and feedback to predict the next sub-goal through a GNN and softmax.Source: F Pseudo-code · pp. 14–15
  • The grounding module encodes state, intention, and feedback, then applies vocabulary cross-attention and convolution to produce a grounded embedding.Source: F Pseudo-code · pp. 14–15
  • The recurrent intention-propagation channel maps a teammate’s message to a belief about that teammate’s intention and is trained to maximize coordination reward.Source: F Pseudo-code · p. 15
  • Execution policies sample actions from agent state and intention; execution encodings are summarized as coordination feedback to guide planning and grounding.Source: F Pseudo-code · p. 16
  • The execution objective maximizes team reward and an auxiliary loss used for additional regularization or constraints.Source: F Pseudo-code · p. 16

G Discussion

REMALIS outperformed single-agent baselines and contemporary methods across difficulty levels on traffic flow prediction and web activities. The supplementary framework description outlines planning, grounding, and execution processes that use feedback and shared intentions to coordinate agents.

G Discussion

  • REMALIS outperformed single-agent baselines and contemporary methods across difficulty levels on both traffic flow prediction and web activities datasets.Source: G Discussion · p. 16
  • Intention propagation aligns agents’ actions toward shared objectives, while execution feedback shapes planning and grounding strategies based on observed coordination patterns.The discussion links shared local beliefs to fewer misaligned sub-tasks and miscoordination errors, and describes dynamic adjustment of comprehension and planning policies.Source: G Discussion · p. 16
  • Centralized training may hinder scalability in fully decentralized environments, and the implementation does not explicitly handle agents arriving or departing during execution.The authors suggest decentralized training and mechanisms for adapting to team changes as future work.Source: G Discussion · p. 17

H Supplementary application description of the overall framework

  • The planning module analyzes traffic conditions, sets congestion-related goals, decomposes them into dependent subgoals, and assigns subtasks to specialized agents.Execution-agent feedback also guides planning-module replanning.Source: H Supplementary application… · p. 17
  • The grounding module maps traffic concepts and entities to physical counterparts, resolves context-dependent ambiguity, and adjusts strategies using execution feedback.It supplies grounded embeddings for execution-agent decisions.Source: H Supplementary application… · p. 17
  • Specialized execution agents share local goals with relevant teammates, align actions, perform assigned subtasks, and adapt to feedback and changing traffic conditions.The module description also includes monitoring vehicle arrivals and traffic patterns and coordinating to alleviate congestion.Source: H Supplementary application… · p. 17
Loading 2407.12532v1…