A Comprehensive Survey on Multi-Agent Cooperative Decision-Making: Scenarios, Approaches, Challenges and Perspectives

• The survey focuses on intelligent decision-making in multi-agent systems and cooperative environments.Source: A Comprehensive Survey on Multi-Agent… · p. 2
• Multi-agent reinforcement learning is one of the survey’s stated topics.Source: A Comprehensive Survey on Multi-Agent… · p. 2
• Large language models are also among the survey’s stated topics.Source: A Comprehensive Survey on Multi-Agent… · p. 2

1.1. Research Backgrounds of Multi-Agent Decision-Making

• Multi-agent cooperative decision-making is a machine-learning and AI subfield in which agents collaborate on tasks across simulated environments and complex real-world systems.Applications include smart agriculture, collaborative robotics, self-driving obstacle avoidance, autonomous navigation, and joint rescue tasks.Source: §1.1 · p. 4

1.2. Overview of Previous Multi-Agent Surveys

• Prior surveys cover varied topics, including MARL training and applications, utility-based cooperation, opponent-induced non-stationarity, and communication in MARL systems.They also address connected and automated vehicles, multi-robot systems, methodologies, challenges, and future directions.Source: §1.2 · pp. 4–5

1.3. Motivations of the Current Survey

• The survey identifies gaps in prior reviews and responds with broader method coverage, equal attention to environments, and practical implementation details.It highlights limited scope, neglected simulation platforms, and under-emphasis of implementation, including code structures, system architecture, and development challenges.Source: §1.3 · p. 5

1.4. The Survey Overview / Contents Organization

• The survey focuses on Deep MARL- and LLM-based methods, then covers simulation platforms, practical applications, challenges, and future research directions.Its applications include autonomous driving, UAV navigation, and collaborative robotics.Source: §1.4 · pp. 5–6

• The taxonomy groups cooperative decision-making methods into rule-based, game theory-based, evolutionary algorithm-based, MARL-based, and LLM-based approaches.Source: §2 · p. 7
• Rule-based, game theory-based, and evolutionary approaches rely heavily on pre-designed strategies and assumptions, limiting adaptability in complex, dynamic, and uncertain environments.Source: §2 · p. 7
• DRL-based and LLM reasoning-based methods can learn and adapt strategies on the fly, motivating special research attention for dynamic and uncertain environments.Source: §2 · p. 7
• The section covers mainstream cooperative decision-making paradigms, MARL-based methods, and LLM-based multi-agent systems in Sections 2.1, 2.2, and 2.3, respectively.Source: §2 · p. 7

2.1.1. Rule-based

• Rule-based systems offer transparent, consistent decision-making but face adaptability and scalability challenges.The section distinguishes fuzzy logic, reinforcement-learning-integrated rules, expert systems, hybrid intelligence, and ensemble learning, with trade-offs including uncertainty handling, domain reliability, and policy robustness.Source: §2.1.1 · pp. 8–9

2.1.2. Game Theory-based

• Game theory analyzes cooperative, competitive, and mixed strategic interactions through equilibrium-based optimization, with learning and inference extending its adaptability.Applications described include path planning, wireless resource allocation, microgrid energy management, production scheduling, fault-tolerant control, and distributed computing.Source: §2.1.2 · pp. 10–11

2.1.3. Evolutionary Algorithms-based

• Evolutionary algorithms optimize agent strategies through selection, mutation, and recombination, supporting continuous learning, large-scale coordination, and self-organization.The section classifies methods as genetic algorithms, deep neuroevolution, coevolution, hybrid evolutionary reinforcement learning, and evolutionary game theory.Source: §2.1.3 · pp. 11–12

2.1.4. MARL-based Multi-Agent Systems

• MARL is organized into CTCE, DTDE, and CTDE paradigms, which address coordination, scalability, and policy optimization through different training and execution arrangements.CTCE centralizes control but is limited in large systems; DTDE scales through local learning but faces non-stationarity and credit assignment challenges; CTDE centralizes training and decentralizes execution, while communication-based algorithms support coordination.Source: §2.1.4 · p. 14

2.1.5. LLMs-based Multi-Agent Systems

• LLM-based multi-agent systems decompose tasks among collaborating agents that communicate through natural language or symbolic representations.A typical hierarchy assigns planning and task decomposition to global agents and execution and feedback to local agents; the section also describes supporting frameworks, adaptive re-planning, and evaluation environments spanning household tasks and gaming interactions.Source: §2.1.5 · pp. 14–15
• LLM-based systems support coordinated work across robotics, autonomous driving, disaster response, and complex decision-making tasks.Frameworks include AutoGen, Chain-of-Experts, AgentVerse, AutoAgents, DyLAN, CoELA, MetaGPT [109], XAgent, PlanAgent [113], LangGraph [111] [112], CrewAI [114] [115], and CoAct [97], with designs for role specialization, dynamic collaboration, planning, and feedback.Source: §2.1.5 · pp. 15–16
• As agent populations grow, maintaining coherent language-based coordination becomes harder, while LLMs’ black-box reasoning complicates trust and debugging.Balancing agent autonomy with effective collaboration is also a significant challenge.Source: §2.1.5 · p. 16
• LLM-based systems can improve task decomposition, collaboration, and feedback mechanisms, offering flexibility for decision-making that requires coordination and adaptability.The survey identifies scalability and reasoning transparency as unresolved qualifications to this potential.Source: §2.1.5 · p. 16

2.2. MARL-based Multi-Agent Decision-Making Taxonomies

• The taxonomy frames MARL as a response to cooperative, dynamic, and competitive settings, with paradigms distinguished by how agents share information during learning and deployment.The section identifies CTCE, DTDE, and CTDE as its three primary paradigms.Source: §2.2 · p. 16

2.2.1. Centralized Training with Decentralized Execution (CTDE)

• CTDE trains decentralized agent policies using centralized information, then has agents act independently from local observations at execution.Its value-decomposition methods factor joint values into agent values under the IGM principle; actor-critic and PPO-based methods use centralized critics or policy updates. HATRPO and HAPPO remove parameter sharing through sequential agent updates, while CTDE extensions also address communication and policy learning.Source: §2.2.1 · pp. 17, 19
• CTDE value-decomposition methods address the high-dimensional joint action-value problem by decomposing joint values into agent values, with IGM ensuring compatible maximizing actions.QMIX uses a monotonic mixing network, but its monotonicity constraint limits representable policies; Weighted QMIX instead emphasizes better joint actions during projection, while QTRAN introduces a more flexible factorization.Source: §2.2.1 · pp. 17–18
• CTDE actor-critic and PPO-based methods use centralized critics or controlled policy updates to address multi-agent learning challenges while retaining decentralized execution.MAPPO uses a centralized value function for training, whereas HATRPO and HAPPO use individualized policies and sequential updates; their improvement guarantees depend on specified conditions, including full observability and deterministic environments for HATRPO.Source: §2.2.1 · pp. 18–19

2.2.2. Decentralized Training with Decentralized Execution (DTDE)

• DTDE lets each agent train independently from local observations and rewards, supporting scalability and robustness where communication or global coordination is limited.Its independence can cause non-stationarity and credit-assignment difficulties, and performance may suffer on tasks requiring strong coordination; approaches include independent Q-learning and decentralized policy gradients.Source: §2.2.2 · pp. 20–21

2.2.3. Centralized Training with Centralized Execution (CTCE)

• CTCE centralizes decision-making: agents send observations, actions, and rewards to a controller, which issues directives instead of having agents use their own policy networks.Multi-Agent DQN/PS-DQN uses parameter sharing and curriculum learning to scale task complexity; cited extensions include counterfactual rewards for credit assignment and DQN for wireless resource allocation.Source: §2.2.3 · pp. 21–22

2.2.4. Addition Taxonomies and E

• Communication-based MARL work addresses a shared challenge: CTCE can scale poorly, DTDE may lack global coordination, and CTDE execution can be bottlenecked by agent communication.CTDE is widely adopted because it combines centralized information during training with distributed execution, motivating efforts to improve communication strategies.Source: §2.2.4 · p. 22
• Communication protocols comprise broadcasting, targeted, and networked communication, while algorithmic approaches focus on value functions, policy search, or communication efficiency.Broadcasting sends messages to all agents, targeted communication selects recipients under supervisory control, and networked communication supports localized interaction among neighbors.Source: §2.2.4 · p. 22, Figure 7 · p. 23
• Value-function communication MARL includes DIAL [201], which exchanges Q-function gradients, and DDRQN [202], which shares Q-values or hidden states using recurrent networks.DDRQN is presented as addressing partial observability and supporting adaptive decisions in dynamic settings.Source: §2.2.4 · p. 22
• Policy-search communication MARL includes CommNet, BiCNet [205], MD-MADDPG, Intrinsic A3C, and MACC.Their mechanisms include shared encoded signals, bidirectional recurrent layers, centralized training with decentralized execution, shared intrinsic rewards for exploration, and adaptive communication for stable coordination.Source: §2.2.4 · pp. 22–23
• Communication-efficiency methods include ATOC, TarMAC, and IC3Net, using attention or gating to regulate when, with whom, and how agents communicate.These mechanisms seek to limit redundant messages while optimizing interaction frequency and quality.Source: §2.2.4 · p. 23
• These advances enable information sharing and coordinated decision-making in multi-agent systems, supporting MARL deployment in real-world settings where communication must be efficient and effective.Source: §2.2.4 · p. 23

2.3.1. Architectural Design

• Adaptive autonomy lets agents adjust task-specific behavior within architect-defined constraints as requirements or environments change [179] [113].Examples include changing an information-retrieval strategy based on result relevance or adapting communication style to social context.Source: §2.3.1 · pp. 23–24
• Self-organizing autonomy allows agents to adapt without predefined structures, allocate tasks by state and skills, and form coalitions or emergent strategies [102] [187].This autonomy is presented as useful in highly dynamic and unpredictable environments.Source: §2.3.1 · p. 24

2.3.2. Suitable Scenarios

• LLM economic agents model individual and group decisions and can produce results qualitatively similar to traditional economic models.Examples include market-trend and policy-impact simulation.Source: §2.3.2 · p. 24
• The Social-network Simulation System uses LLM agents to reproduce attitudes, emotions, and behaviors and model population-level information, attitude, and emotion diffusion [213].It can identify influential nodes and model the evolution of social norms and behaviors.Source: §2.3.2 · p. 24
• LLMs simulate recommender-system users with reliable behaviors and outperform baseline simulation systems, improving recommendation accuracy [97] [111] [112].Applications include generating user profiles and simulating engagement to optimize recommendations and retention.Source: §2.3.2 · p. 24
• Natural-science applications use LLM agents to simulate complex systems, including macroeconomic policy effects and generative agent-based models of system dynamics [100].Generative agent-based modeling combines mechanistic models with generative AI to study phenomena such as disease spread, ecosystem change, and norm diffusion.Source: §2.3.2 · p. 24
• Engineering applications include agents for code generation, bug detection, and system optimization, as well as multi-robot coordination toward shared goals.Multi-robot examples include coordinated search-and-rescue operations.Source: §2.3.2 · pp. 24–25

3. Simulation Environments of Multi-Agent Decision-Making

• These environments let researchers test algorithms, study agent interactions and adaptation, and refine strategies before real-world deployment.They provide predetermined scenarios and platforms in which agents collaborate to solve problems and achieve goals.Source: §3 · p. 25

3.1.1. Several Widely-used Environments on MARL

• MPE [59, 169, 170] supports cooperative and competitive tasks, while SMAC tests decentralized coordination under partial observations and SMACv2 [187] adds randomized scenarios and stochastic masking.MPE includes varied tasks through a Gym-compatible interface; SMACv2’s procedural generation and Extended Partial Observability Challenge require agents to generalize and coordinate under uncertainty.Source: §3.1.1 · pp. 26–27
• GFootball offers customizable physics-based football tasks, while Unity ML-Agents [227] supports reinforcement and imitation learning across 2D, 3D, and VR/AR simulations.Football Academy ranges from individual scoring drills to team scenarios; Unity ML-Agents also supports neuroevolution and custom multi-agent environments.Source: §3.1.1 · pp. 27–28
• Gym-µRTS targets full-game RTS research on limited hardware, while MAgent scales MARL to hundreds or millions of agents and emergent interactions.Gym-µRTS supports fine-grained actions and has produced agents reported to defeat competition bots such as CoacAI; MAgent uses GPU parallelism for large-scale simulations.Source: §3.1.1 · pp. 28–29

3.2. LLMs Reasoning-based Simulation Environments

• TDW-MAT tests visually guided object transport in realistic homes, whereas C-WAH emphasizes communication and household collaboration through symbolic or visual observations.TDW-MAT involves physics-aware coordination; C-WAH tasks include meal preparation and dishwashing.Source: §3.2 · p. 30
• CuisineWorld evaluates timed cooperative cooking and coordination, AgentScope [100] provides fault-tolerant multimodal agent workflows, and RoCoBench benchmarks communication-driven multi-robot collaboration.CuisineWorld introduces a Collaboration Score; RoCoBench uses a Collaboration Efficiency Metric that considers task time, resource allocation, and communication quality.Source: §3.2 · pp. 30–32
• Generative Agents simulate social behavior using memory, reflection, and planning; SocialAI School supports socio-cognitive research, and Welfare Diplomacy evaluates cooperation in a general-sum game.Generative Agents’ evaluation identifies memory retrieval, self-reflection, and action planning as important to coherent behavior; SocialAI School includes procedurally generated environments for studying joint attention, imitation, and scaffolding.Source: §3.2 · pp. 32–33
• Welfare Diplomacy uses welfare points to reward prioritizing welfare over military expansion, while simultaneous negotiation over supply centers balances cooperation and betrayal.Players’ final utility is determined by welfare points rather than a single winner; GPT-4 zero-shot baseline agents achieved high social welfare through mutual demilitarization but remained vulnerable to exploitation [173].Source: §3.2 · p. 34
• LLM-powered environments, including TDW-MAT, CuisineWorld, Generative Agents, and SocialAI School, support research on collaboration, social cognition, and cooperative decision-making.The survey presents these task-specific and socially driven tools as having potential for practical applications in dynamic, real-world scenarios.Source: §3.2 · p. 34

4.1. MARL-based Intelligent Applications

• Examples include energy-aware agricultural search, reinforcement-learning UAV rescue, military simulation, bandwidth-limited communication, and anti-jamming UAV swarm control.Seewald et al. [11] combine ergodic search with energy-aware coverage; Qazzaz et al. [12] optimize UAV connectivity, while Lv et al. optimize relay selection and power allocation using network topology, channel states, and shared experience.Source: §4.1 · pp. 35–36
• MARL methods also address UAV pursuit-evasion, traffic control, diverse driving styles and safety, and scalable collaborative robots.Examples include an enhanced CommNet for multi-UAV coordination, decentralized A2C traffic control, PCPO for driving safety, and MADDPG with an experience sample optimizer for Mars robot exploration.Source: §4.1 · pp. 36–37
• Other MARL contributions target scalable robotic simulation, explainable world-model learning, policy diversity, dynamic teams, and sparse-reward cooperation.LBI uses a language-guided simulator with dynamics and reward models; DPM compares trajectories and individual contributions, using LLM-collected preferences and outperforming baselines particularly in sparse-reward SMAC settings.Source: §4.1 · pp. 37–38
• MARL applications span UAVs, disaster response, military simulations, traffic, autonomous driving, and collaborative robotics, while sparse rewards and scalability remain challenges.The survey identifies communication-enhanced learning, adaptive policy optimization, and mutual-information frameworks as advances for dynamic multi-agent environments.Source: §4 · p. 35, §4.1 · p. 38

4.2. LLMs reasoning-based Intelligent Applications

• LLM-based applications range from agent conversation and task-specific teams to embodied cooperation, multimodal tools, game coordination, and complex reasoning.AgentVerse dynamically changes agent roles through recruitment, collaboration, execution, and evaluation; CoELA combines perception, memory, and execution for decentralized long-horizon tasks, while MAT reports superior performance and data efficiency on several benchmarks.Source: §4.2 · p. 39
• LLM collaboration frameworks use structured workflows, role-specific agents, replanning, and human-in-the-loop control to support complex task execution.MetaGPT applies SOPs and assembly-line subtasks; CoAct uses hierarchical planning and replanning; XAgent uses nested planning and execution loops; LangGraph supports branching, persistence, and recovery; CrewAI organizes role-based work across agent creation, task management, and execution.Source: §4.2 · pp. 40–41

5. Challenges in MARL-based and LLMs-based approaches

• Multi-agent systems add complexity through simultaneous learning, strategic interaction, coordination, scalability, and non-stationarity challenges.Compared with single-agent decision-making, they also face greater environmental stochasticity and strategy-optimization difficulty.Source: §5 · pp. 42–43

5.1. Challenges in MARL-based multi-agent systems

• MARL bottlenecks include stochastic environments, difficult strategy learning, complex rewards, coordination demands, and non-stationarity.Agents must account for others’ changing strategies and actions, while large joint action spaces and interdependent rewards make stable policy learning harder. [284]Source: §5.1 · pp. 43–44
• MARL scalability requires addressing complexity growth while balancing robustness and scalability.Knowledge reuse, parameter sharing, transfer and curriculum learning, ensembles, and adversarial training are discussed as approaches; meta-learning and graph neural networks are proposed as future directions.Source: §5.1 · p. 44

5.2. Challenges in LLMs reasoning-based multi-agent systems

• LLM-based systems need multimodal capabilities and safeguards against hallucinations spreading through agent interactions.Multimodal work must support input and output across modalities and fuse information between agents; proposed hallucination controls include reducing individual errors and limiting their propagation. [102] [113]Source: §5.2 · pp. 44–45
• LLM-based systems have yet to realize collective intelligence because they depend on reliable real-time interaction and often optimize agents individually.Knowledge sharing and behavioral coordination may improve collective performance, but maintaining interactive environments limits scalability. [286]Source: §5.2 · p. 45
• LLM-based system scalability is constrained by rapidly rising compute needs and coordination difficulty as agent numbers grow.Lightweight models, efficient communication protocols, resource optimization, and investigation of scaling laws are proposed. [286]Source: §5.2 · p. 45
• LLM-based multi-agent evaluation lacks unified benchmarks and measures of overall group performance in complex scenarios.Frequent model queries reduce interaction efficiency, while errors can accumulate across rounds; proposed remedies include comprehensive benchmarks, efficient communication, and intermediate-result correction. [286] [287]Source: §5.2 · p. 45
• LLM-based systems face privacy and security risks when sensitive information is shared with untrusted agents.Suggested safeguards include access restrictions and distributed trust management based on consensus algorithms.Source: §5.2 · pp. 45–46

6.1. Theoretical Development: From Traditional RL to LLMs-Enhanced MARL Framework

• LLMs-enhanced MARL assigns LLMs information-processing, reward-design, decision-making, and generation roles to address MARL’s task complexity, data efficiency, and generalization challenges.LLMs can translate unstructured tasks into formal semantics and design rewards for sparse-reward settings.Source: §6.1 · p. 46, Figure 27 · p. 46

6.2. Technical Integration: From Multi-Modal to Multi-Task Optimization

• LLMs can unify multimodal inputs, transfer pretrained knowledge across tasks, decompose long-horizon tasks, and provide simulated samples or reward signals to support MARL learning.These capabilities target sample efficiency and credit assignment, including through high-fidelity environment simulations and automatically designed rewards.Source: §6.2 · p. 47

6.3. Application Expansion: Driving Intelligent Collaboration in Complex Scenarios

• LLM-enhanced MARL could support autonomous driving, collaborative robots, smart grids, healthcare, and disaster relief through sensor-language processing, task allocation, and domain-informed optimization.The paper describes processing driving sensors alongside regulations and passenger instructions, coordinating robot task division, and dynamically allocating roles during disaster relief.Source: §6.3 · pp. 47–48

6.4. Human Society Coordination: Balancing Technology and Ethics

• Future work must improve robustness, reduce bias and hallucinations, increase inference efficiency and scalability, and strengthen privacy, security, accountability, transparency, and explainability.These concerns are especially relevant to dynamic real-time settings and sensitive domains such as healthcare and disaster response.Source: §6.4 · p. 48

Conclusion

• The survey reviews the evolution from rule-based and game-theory methods to MARL and LLMs, comparing their capabilities, challenges, and applications in dynamic and uncertain settings.It aims to provide a comprehensive resource for researchers and practitioners on multi-agent decision-making methodologies.Source: §7 · p. 48
• Simulation environments bridge theoretical advances and real-world implementation, influencing agent interaction, learning, and decision-making.Source: §7 · p. 48
• Applications in autonomous driving, disaster response, and robotics underscore multi-agent systems’ potential, while future research should improve inter-agent communication and adaptability.The survey also highlights leveraging the potential of DRL- and LLM-based approaches to advance cooperative decision-making.Source: §7 · p. 48

Declaration of Generative AI and AI-assisted Technologies in the Writing Process

• The authors used AI tools for proofreading and language clarity in certain sections and reviewed the content, noting AI output may be incorrect, incomplete, or biased.The tools were used during preparation of the work.Source: Declaration of Generative AI and… · p. 49

Appendix A. Technological Comparisons between Single-Agent and Multi-Agent (Under Reinforcement Learning)

• Figure A.28 contrasts fully observable MDP modeling for single-agent reinforcement learning with partially observable POMDP modeling for multi-agent reinforcement learning.The figure is identified as illustrating the two paradigms.Source: Appendix A · p. 49, Figure A.28 · p. 50
• In an MDP, a single agent chooses an action from the current state, receives a reward after a probabilistic transition, and seeks a policy maximizing cumulative reward.Because the full state is observable, the decision process is relatively straightforward.Source: Appendix A · p. 49
• In a POMDP, agents receive partial observations, choose individual actions forming a joint action, and maintain belief states to infer hidden state, complicating coordination and reward interdependencies.The observation function generates observations from the resulting state and action; the setting also involves dynamic interaction under uncertainty and limited information.Source: Appendix A · p. 49