PaperlayerView exact revision on arXiv

Source-linked AI summary

Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospects

Yuheng Cheng, Ceyao Zhang, Zhengwen Zhang, Xiangrui Meng, Sirui Hong, Wenhao Li, Zihao Wang, Zekai Wang, Feng Yin, Junhua Zhao, Xiuqiang He

arXiv:2401.03428v1cs.AIcs.MA

TL;DR

Practical deployment of LLMs raises challenges that motivate combining language models with agent decision-making and planning. This paper surveys LLM-based agents in single-agent and multi-agent systems, covering their frameworks, components, applications, evaluation, and prospects. It identifies broad research opportunities alongside persistent challenges in social simulation, computer systems, evaluation, multimodality, security, and trust.

  • Problem

    LLMs encounter obstacles in practical tasks, while objectively and quantitatively evaluating LLM-based agents remains challenging.

  • Method

    The paper comprehensively reviews LLM-based agents’ definitions, frameworks, components, multi-agent mechanisms, datasets, applications, development prospects, and challenges.

  • Results

    The review covers LLM-based agents across single-agent and multi-agent systems, including memory, environmental feedback, collaboration, message passing, applications, and emerging multimodal and security directions.

  • Takeaways & Limitations

    LLM-based agents offer a broad research and application landscape, but progress requires addressing adaptability, tool use, evaluation, social interaction, multimodal processing, security, and trust.

  • Takeaways & Limitations

    The paper highlights unresolved challenges in socially aligned interaction, personalized simulation, tool use, adaptive learning, and continuously changing environments.

Abstract

from arXiv · show

Intelligent agents stand out as a potential path toward artificial general intelligence (AGI). Thus, researchers have dedicated significant effort to diverse implementations for them. Benefiting from recent progress in large language models (LLMs), LLM-based agents that use universal natural language as an interface exhibit robust generalization capabilities across various applications -- from serving as autonomous general-purpose task assistants to applications in coding, social, and economic domains, LLM-based agents offer extensive exploration opportunities. This paper surveys current research to provide an in-depth overview of LLM-based intelligent agents within single-agent and multi-agent systems. It covers their definitions, research frameworks, and foundational components such as their composition, cognitive and planning methods, tool utilization, and responses to environmental feedback. We also delve into the mechanisms of deploying LLM-based agents in multi-agent systems, including multi-role collaboration, message passing, and strategies to alleviate communication issues between agents. The discussions also shed light on popular datasets and application scenarios. We conclude by envisioning prospects for LLM-based agents, considering the evolving landscape of AI and natural language processing.

1 Introduction

Agents perceive environments, make decisions, and act toward goals, while learning agents improve behavior from experience. LLM-based agents combine language capabilities with agentic planning and decision-making to address practical-task challenges, although RL-based approaches face training, sample-efficiency, stability, and generalization limitations.

  • Agent foundations: Agents perceive environmental information, decide how to pursue goals, and act to alter their environments.
  • Agent foundations: Learning agents improve behavior from experience, addressing limited autonomous learning and multi-step decision-making difficulties in fixed-rule or simplistic agents.
  • RL-based agents: RL-based agents learn policies through environmental interaction and feedback to maximize cumulative rewards across states and actions.
  • RL-based agents: RL-based agents can require lengthy convergence and many interaction episodes, making training time and data collection costly for complex or real-world applications.
  • RL-based agents: RL learning may become unstable in high-dimensional or non-stationary environments, while task specialization can limit generalization to new tasks and environments.
  • LLM-based agents: LLM-based agents combine LLM language comprehension and generation with agents’ decision-making and planning, supporting natural-language interaction and practical applications.

2 Overview

LLM-based agent systems are organized into single-agent and multi-agent forms, with agents defined by components such as objectives, memory, actions, and reconsideration. Multi-agent systems add coordination structures, planning arrangements, communication relationships, and reusable development templates.

  • LLM-based systems are categorized as single-agent or multi-agent systems, which differ in application domains, memory, reconsideration, data, modalities, and tools.
  • Single-Agent System: A single-agent system uses one LLM-based agent across multiple tasks and domains, including code generation, game exploration, and data management.Its tools and evaluation methods are not standardized, and it may be unimodal or multimodal.
  • Single-Agent System: Each LLM-based agent is represented as V = (L, O, M, A, R), comprising an LLM, objective, memory, action, and rethink components.The LLM supports strategizing and decision-making from observations, memory, and reward information, while the objective guides decomposition and planning.
  • Single-Agent System: Tools support agent operation through instruments such as calculators, code interpreters, and robotic arms, while environments provide observations, interactions, and feedback.
  • Multi-Agent System: Multi-agent systems contain multiple interacting agents whose coordination and information sharing support tasks spanning multiple domains.Prior taxonomies organize them by agent granularity, knowledge heterogeneity, control distribution, communication protocols, system function, and architecture.
  • Multi-Agent System: Relationships among LLM-based agents are modeled as a graph G(V, E), where nodes represent agents and edges represent message passing and relationships.The review also surveys single-agent and multi-agent templates, including tools for model training, evaluation, role customization, and system development.
  • Multi-Agent System: The paper categorizes multi-agent systems by multi-role coordination and planning type, distinguishing cooperative, competitive, mixed, and hierarchical roles from CPDE and DPDE.

3.1 LLM-based Single Agent System

An LLM-based single-agent system is organized around planning, memory, rethinking, environment, and action. These capabilities support task planning, information management, reflection, and learning through environmental feedback across varied environments.

  • The single-agent system comprises five components: Planning, Memory, Rethinking, Environment, and Action.Together, these components form the system’s unified design.
  • Planning: Planning enables agents to devise action sequences from objectives and environmental constraints while analyzing tasks, anticipating actions, and selecting among alternatives.The survey describes in-context, external-capability, and multi-stage methods for improving planning and complex problem solving.
  • Memory: Memory preserves knowledge, experience, historical information, and agent state for reference and modification during problem solving and task execution.Short-term memory handles transient context, whereas long-term memory can use external knowledge bases, databases, knowledge graphs, vector databases, and APIs.
  • Memory: Selecting suitable memory classifications and retrieval mechanisms can improve agents’ storage, administration, and rapid extraction of information while addressing tasks.Agents can use short-term and long-term memory concurrently, with retrieval supported by online learning and adaptive modification.
  • Rethinking: Rethinking evaluates prior decisions and environmental feedback, enabling agents to examine behavior, decision-making, and learning processes.Approaches are classified into In-Context Learning, Supervised Learning, Reinforcement Learning, and Modular Coordination, supporting reflection and re-planning.
  • Environment: Simulation environments provide controlled yet realistic settings where agents learn, experiment, and develop solutions applicable to real-world scenarios.The survey characterizes LLM-based agents as learning and applying knowledge through natural-language interaction and environmental feedback across environments.

3.2 LLM-based Multi-Agent System

LLM-based multi-agent systems organize agents through cooperative, competitive, mixed, or hierarchical relationships and use centralized or decentralized planning. Their communication choices trade coordination benefits against computational, communication, and global-optimality limitations.

  • Relationship of Multi-Agent Systems: LLM-based MAS coordinate multiple agents through cooperative, competitive, mixed, and hierarchical relationships.These relationships support tasks including search, optimization, decision support, resource allocation, generation, and control.
  • Cooperative Relationship: Cooperative systems emphasize role and task allocation, collaborative decision-making, and communication among specialized agents.Frameworks such as NLSOM, RoCo, and Gentopia illustrate natural-language coordination, embodied collaboration, and configurable agent integration.
  • Competitive Relationship: Competitive systems use strategies including multi-agent debate, information concealment, and adversarial behavior to improve task solving or gain advantage.ChatEval applies group collaboration, while competitive designs focus on strategic interaction.
  • Mixed Relationship: Mixed relationships require agents to balance cooperation and competition, as illustrated by agents that cooperate or betray others in Werewolf.Research in this area focuses on collaborative competition algorithms.
  • Planning and Information Exchange: CPDE centralizes planning, whereas DPDE assigns each agent independent planning and coordinates execution through local communication or negotiation.CPDE may face computational complexity, single-point failures, communication delays, and limited real-time adaptability; DPDE improves robustness and scalability but may sacrifice global optimality and incur coordination overhead.
  • Information Exchange: Communication-free execution minimizes overhead and suits limited-connectivity environments but can prevent information sharing, coordination, and conflict resolution.The resulting collaboration may be suboptimal and overall performance may deteriorate.

4 Performance Evaluation

The survey lists publicly available datasets and evaluation resources for LLM-based agents because task-specific datasets are often inaccessible. It also notes that no widely used benchmark yet exists, although benchmark development is progressing.

  • Datasets: The survey enumerates publicly available and extensively used datasets because some task-specific datasets are not publicly accessible.Most LLM-based agents do not require further LLM training.
  • Benchmark: There is currently no widely used benchmark for LLM-based agents, but researchers are proposing benchmarks as future evaluation standards.AgentSims, BMTools, and SmartPlay provide platforms or benchmarks for evaluating agents and tool utilization.
  • Datasets: ToolBench evaluates tool utilization across single-tool and multi-tool scenarios, while TE and related resources assess behavioral and social capabilities.These resources cover tool use, human-behavior emulation, social behavior, and zero-shot evaluation across CSS benchmarks.

5 Prospect Applications

The section presents the prospect of LLM-based agents.

  • Figure 12 presents the prospect of LLM-based agents.
  • The figure is titled “The Prospect of LLM-based agents.”
  • The prospect is presented as a dedicated topic for LLM-based agents.

5.1 Natural Sciences

LLM-based agents are being explored across natural-science domains, especially mathematics, chemistry, materials science, biology, and atmospheric research. Existing work demonstrates tool use, reasoning, simulation, and automation, while accuracy, reliability, interpretability, and biological complexity remain important boundaries.

  • Mathematics: Mathematics research mainly targets stronger reasoning and theoretical derivation through theorem proving, verification, hypothesis generation, and tool-integrated problem solving.Examples include Math Agents, LeanDojo, ToRA, and COPRA.
  • Mathematics: Multi-agent systems can improve mathematical computation by partitioning intricate problems into multiple sub-problems.The paper identifies symbolic and numerical computation as prospective applications.
  • Challenges: Natural-science applications face accuracy and reliability challenges, while biological agents must handle complexity across hierarchical levels, spatiotemporal scales, and interacting entities.Mathematical applications additionally require dependable and interpretable reasoning; chemistry systems need more accurate prediction and generation.
  • Chemistry and Materials Science: Chemistry and materials-science agents use search, code execution, document retrieval, and experiment automation to design, plan, and execute experiments.Coscientist and ChatMOF illustrate tool-using and evaluator-based systems.
  • Biology: Biology research remains limited, with BioPlanner evaluating protocol generation and planning and OceanGPT using multi-agent collaboration for ocean-science data generation.

5.2 Universal Autonomous Agent

LLM-based agents are being developed as general-purpose assistants across multimodal, research, creative, and domain-specific tasks. Key challenges concern efficient task decomposition, memory and knowledge integration, tool use, cross-domain skills, and originality.

  • General-purpose capabilities: Generalist Agent and related systems target multimodal, multi-task assistance across games, image naming, conversation, robotics, and task planning.HuggingGPT integrates modules and models from different domains for task planning.
  • Open challenges: General task assistants must balance task-decomposition granularity against solving efficiency, token consumption, and computational resource demands.The section also identifies memory utilization and information integration as persistent challenges.
  • Work and research assistance: LLM-based agents can support research workflows by browsing sources, summarizing literature, generating validation code, and drafting conclusions.The described workflow spans information gathering, experimentation, and report production.
  • Application breadth: Applications include multi-agent debate, research reporting, scientific experimentation, 3D structure construction, layout generation, and music production.These examples span text, science, design, and creative domains.
  • Work and research assistance: Work and research assistants require stronger memory, domain-specific tool use, cross-domain knowledge, and creativity to handle extensive materials and produce original outputs.Future directions include artistic creation and human-machine collaboration.

5.3 Social Sciences

LLM-based agents are applied across economic, financial, political, and social-science settings to simulate actors, interactions, decisions, and communication. Their promise is accompanied by challenges in action-space representation, anthropomorphism, communication, environmental complexity, and social diversity.

  • Economics and finance: Economic and financial applications simulate market actors, investor behavior, policy processes, networks, trading, and economic decision-making.Proposed uses include market, financial, macroeconomic, and socio-economic network simulation.
  • Economics and finance: LLM-based agents in economics must represent interaction action spaces and agent states accurately while addressing the credibility of LLM anthropomorphism.These design choices directly affect experimental results according to the section.
  • Political science: Political applications examine speeches, ideology, voting results, political decisions, diplomatic interactions, and conflicts among countries and domestic actors.The surveyed directions include political simulation, decision-making analysis, and international-relations research.
  • Society: Social simulations use LLM-based agents for credible human behavior, multi-party group chats, simulated social interaction, memory, planning, and opinion dynamics.The paper attributes these capabilities to simulating human communication and mimicking human thinking.
  • Society: Social-science systems must preserve turn-taking and dialogue coherence, train socially aligned models, and represent diverse, personalized human actors.Political applications additionally face communication-efficiency and complex, uncertain-environment challenges.

5.4 Engineering Systems

Engineering applications span computer systems, robotics, energy, transportation, and multi-agent control. The survey highlights practical capabilities while emphasizing unresolved requirements for tool use, adaptation, multimodal processing, integration, and standards.

  • Computer systems: Computer-science applications cover computer operation, human-computer interaction, web and spreadsheet tasks, network security, code generation, gaming, and recommendation systems.Examples include natural-language computer control, autonomous website navigation, and spreadsheet interaction.
  • Computer systems: Computer-science challenges include improving code quality and testing, exploiting agent advantages, expanding tool use, and enabling adaptation to changing problems.The section calls for adaptive learning and long-term development of agent systems.
  • Robotics: Robotics research primarily uses LLM-based agents for task planning, language-based reasoning, simulated environments, visual alignment, and sample-efficient embodied planning.The cited approaches include executable plans constrained by real-world physical scenes.
  • Robotics: Multi-robot and UAV applications can simulate task allocation, path planning, collaboration, swarm control, obstacle avoidance, and formation changes.These applications target collaborative performance, execution efficiency, stability, and safe flight.
  • Robotics: Robotics agents must adapt to complex environments, process real-time multimodal data, and make rapid decisions.These requirements follow from the complexity and real-time nature of robotic tasks.
  • Energy systems: Energy applications include smart-grid management, distributed-energy scheduling, and energy-market simulation, but large-scale integration, collaborative optimization, interoperability, and scalability remain challenging.The paper notes that relatively few LLM-based-agent studies currently address electric power and energy.

6 Discussion

The discussion identifies objective evaluation, continual evolution, multimodal processing, dynamic scaling, and permission control as central needs for deploying LLM-based agents. It also notes persistent limitations from context length and hallucinations.

  • Evaluation: Quantifiable and objective evaluation remains difficult despite agents’ capabilities in task solving, cooperation, and human interaction.The survey distinguishes broad capability from the challenge of measuring it reliably.
  • Evaluation: Evaluation should cover foundational capabilities and domain-level execution environments and tasks rather than relying only on conventional code benchmarks.AgentBench is cited as an example of task-level evaluation protocols.
  • Continual evolution: Continual evolution requires agents to adapt parameters, memory, and objectives as complex and dynamic environments change.The discussion highlights continual learning, self-training, self-evaluation, dynamic goals, and adaptability.
  • Multimodal capabilities: Multimodal models can let agents plan and reconsider directly from images instead of first converting images into text.This is presented as a way to improve information utilization and multimodal task processing.
  • LLM limitations: LLMs support planning, natural-language expression, and generalization, but context-length constraints can lose information and generation can produce invalid data or hallucinations.These limitations become relevant when processing extensive articles or complex dialogues.
  • Dynamic scaling: Dynamic scaling must adapt agent counts, memory sizes, conversion strategies, and computing resources across hardware and software environments.Resource-aware LLM reasoning and cost reduction are part of the scaling challenge.
  • Security: Permission allocation and system security are critical because excessive permissions can produce incorrect actions and create security issues.The discussion links permission control to preserving human and enterprise trust.

7 Conclusion

The paper surveys LLM-based agents across current research, applications, and prospects, while organizing their foundations, multi-agent mechanisms, challenges, and development trends. It concludes that although AGI remains distant, LLM-based agents may represent a significant step forward.

  • The survey covers LLM-based agents’ research status, applications, prospects, foundational concepts, and multi-agent systems.It discusses definitions, planning, memory, rethinking, action, external environments, multi-role relationships, and communication methods.
  • It examines development from agents through reinforcement-learning agents to LLM-based agents.
  • It identifies challenges including LLM limitations, dynamic multi-agent-system expansion, and security and trust issues.
  • The paper proposes possible solutions and discusses development prospects across various fields.
  • Although current research remains far from achieving AGI, the paper views LLM-based agents as a significant step forward.
Loading 2401.03428v1…