Source-linked AI summary
An In-depth Survey of Large Language Model-based Artificial Intelligence Agents
Pengyu Zhao, Zijian Jin, Ning Cheng
TL;DR
The paper addresses the capabilities and limitations of AI agents, including traditional agents’ generalization challenges and LLM shortcomings in some scenarios. It surveys LLM-based agents by comparing them with traditional agents and reviewing their components and applications, concluding with research issues and directions.
Problem
Traditional agents can struggle to generalize beyond their designed tasks, while LLMs also perform unsatisfactorily in some scenarios, including those involving up-to-date information and computational reasoning.
Source: §1 · p. 1, §2 · p. 2, §3.4 · p. 6Method
Results
The survey reviews LLM-agent differences, components, applications, and pressing issues, and identifies research directions for the field.
Source: §6 · p. 9Takeaways & Limitations
The review aims to help readers grasp key information and applications of AI agents and gain insights into future research directions.
Source: §6 · p. 9
Abstract
from arXiv · showhide
Due to the powerful capabilities demonstrated by large language model (LLM), there has been a recent surge in efforts to integrate them with AI agents to enhance their performance. In this paper, we have explored the core differences and characteristics between LLM-based AI agents and traditional AI agents. Specifically, we first compare the fundamental characteristics of these two types of agents, clarifying the significant advantages of LLM-based agents in handling natural language, knowledge storage, and reasoning capabilities. Subsequently, we conducted an in-depth analysis of the key components of AI agents, including planning, memory, and tool use. Particularly, for the crucial component of memory, this paper introduced an innovative classification scheme, not only departing from traditional classification methods but also providing a fresh perspective on the design of an AI agent's memory system. We firmly believe that in-depth research and understanding of these core components will lay a solid foundation for the future advancement of AI agent technology. At the end of the paper, we provide directional suggestions for further research in this field, with the hope of offering valuable insights to scholars and researchers in the field.
1. Introduction
The introduction traces the development of AI agents and highlights persistent challenges for reinforcement-learning agents, including generalization and costly training. It presents LLM-based agents as a growing research direction and frames the paper as a systematic review of agents’ components and applications.
1. Introduction
- Reinforcement-learning agents can struggle to generalize beyond training settings, require many environment interactions, and face convergence and reward-design challenges.The paper notes that these difficulties can hamper development and deployment in diverse environments.Source: §1 · p. 1
- The introduction describes LLMs’ language and generalization capabilities as a basis for exploring them as foundational models for universal agents.It also notes research exploring hybrids that combine LLM generalization with reinforcement learning.Source: §1 · pp. 1–2
- The paper offers a comprehensive, systematic review of AI agents based on their components and applications.It aims to help readers understand the research history and inspire further research.Source: §1 · p. 2
2. LLM vs. Traditional Agents
The section contrasts task-specific traditional agents with LLM-based agents’ language, generalization, and reasoning capabilities. It uses Minecraft to illustrate how those differences can appear in agent behavior.
2. LLM vs. Traditional Agents
- Traditional agents target specific problems and can struggle with generalization and reasoning outside their initial scope.LLM-based agents are described as proficient in natural language and capable of integrating with tools.Source: §2 · p. 2
- LLMs’ pre-trained knowledge and emergent reasoning abilities are presented as advantages for LLM-based agents’ decision-making and learning from faulty behavior.The section contrasts this with traditional RL agents’ reliance on interaction to learn in new environments.Source: §2 · p. 2
- In Minecraft, VOYAGER is described as showing better generalization and data efficiency than traditional RL agents.The comparison concerns decision-making without task-specific training versus learning from scratch through interaction.Source: §2 · p. 2
3. Components of AI Agents
The paper reviews planning, memory, and tool use as agent components, describing their histories, methods, and potential directions. It highlights task decomposition and feedback-based plan revision, and proposes an LLM-oriented memory classification.
3.1. Overview
- Planning, memory, and tool use are treated as crucial agent functions with established histories, while their integration in AI agents remains relatively new.The review covers each function’s research history, mainstream and combination methods, and possible future directions.Source: §3.1 · p. 2
3.2. Planning
- Planning uses a sequence of actions to reach a task goal, and LLM-based agents can decompose complex instructions and revise plans using environmental feedback.Figure 1 illustrates task decomposition and self-reflection as parts of the planning process.Source: §3.2 · p. 2, Figure 1 · p. 3
3.2.1. Task Decomposition
- Task decomposition turns complex instructions into simpler subgoals, using approaches such as chain or tree of thought and LLM-assisted PDDL planning.Tree-of-thought methods branch across candidate thoughts, while the described PDDL approach translates natural language into a planner representation and back.Source: §3.2.1 · p. 3
3.2.2. Self-Reflection
- Self-reflection uses action and environmental feedback to assess errors, store feedback, and adjust subsequent actions or plans.Examples include Reflexion’s actor-and-reflection modules, assertion-based recovery, and AdaPlanner’s refiners.Source: §3.2.2 · pp. 3–4
3.3. Memory
- The survey classifies LLM-agent memory as training, short-term, and long-term memory, excluding sensory memory from the agent memory module.Training memory is in model parameters, short-term memory holds task-time context and intermediate results, and long-term memory uses external storage.Source: §3.3 · p. 4
Memory
The memory discussion distinguishes human and AI-agent categories and describes long-term memory operations. The tool-use discussion surveys ways to extend LLM capabilities and select or combine external tools.
Memory
- The proposed memory mapping redefines AI-agent memory categories in light of LLM characteristics.The figure contrasts human-memory categories with categories for AI agents.Source: Figure 2 · p. 5
- Long-term memory stores information externally for retrieval relevant to the agent’s current context.The described workflow includes storage, retrieval, and updating; examples include summaries or key-value records and similarity-based retrieval.Source: Memory · pp. 5–6
3.4. Tool Use
- Tool use is motivated by LLM limitations with up-to-date information and computational reasoning, and includes integrations such as search, translation, and calculators.The section also describes MRKL’s adapter selecting among experts and methods for training models to invoke APIs.Source: §3.4 · p. 6
- Other surveyed methods decompose tasks into subtasks for model selection, compose tools to execute subtasks, or generate tools for an input task.Some approaches also use human feedback to improve planning and API selection.Source: §3.4 · p. 6
4. Application
The survey reviews LLM-based agent applications across seven areas, with examples spanning conversational support and game-playing agents.
4. Application
- The survey organizes LLM-based agent applications into seven areas: chatbot, game, design, research, coding, collaboration, and general purpose.Source: §4 · pp. 6–7
4.1. Chatbot
- Pi combines productivity assistance, such as searching and answering questions, with emotional companionship and conversation designed to feel natural.Source: §4.1 · p. 7
4.2. Game
- Voyager uses a learned skill library to solve new Minecraft tasks from scratch without human intervention, demonstrating in-context lifelong learning.Source: §4.2 · p. 7
4.3. Coding
The survey describes coding agents that generate codebases and clarify requirements, alongside design agents that create and refine editable interfaces.
4.3. Coding
- GPT Engineer can generate an entire codebase from a prompt, learn a developer’s coding style, and ask detailed questions to clarify missing requirements.Source: §4.3 · p. 7
4.4. Design
- Genius turns product descriptions into editable UI designs and offers design suggestions, while UI-AI uses prompting or multimodal prompts to generate design assets.Source: §4.4 · pp. 7–8
4.5. Research
The survey presents agents for scientific research and collaboration, including tool-using chemistry systems and teams of agents that divide complex work.
4.5. Research
- ChemCrow uses 17 expert-designed chemistry tools with GPT-4 for tasks including synthesis, drug discovery, and materials design; Agent combines multiple LLMs to design, plan, and execute experiments.ChemCrow follows an automated iterative chain-of-thought process; Agent uses tools for web browsing, document reading, and code execution.Source: §4.5 · p. 8
4.6. Collaboration
- Collaboration ranges from assistants working with humans on complex decisions to agent teams that divide tasks, with MetaGPT focused more on software-industry work.Examples include DialOp, MindOS, MetaGPT, and Multi-GPT.Source: §4.6 · p. 8
4.7. General purpose
The survey covers autonomous agents for broad task domains and vision-language agents that process visual and textual inputs for embodied, navigation, and conversational tasks.
4.7. General purpose
- General-purpose agents pursue diverse goals by planning tasks, executing them, and learning from results; BabyAGI builds and prioritizes tasks iteratively, while Auto-GPT is described as running GPT-4 fully autonomously.SuperAGI and AgentGPT support building and deploying autonomous agents, though the survey notes these systems have deficiencies.Source: §4.7 · p. 8
4.8. Vision-Language model-based agent application
- Vision-language agents support embodied reasoning, navigation, and multimodal conversation by combining visual and language processing.PaLM-E maps text and image inputs to embeddings for reasoning and action; Video-ChatGPT encodes video features and maps them into the language space for response generation.Source: §4.8 · pp. 8–9
5. Benchmarking
The section surveys benchmarks designed to evaluate LLM-based agents across planning, tool use, interactive tasks, and multiple environments.
5. Benchmarking
- Benchmarks assess planning, compare agent architectures using different LLMs in WebShop and HotPotQA, and evaluate tool use through API Bank.Valmeekam et al. focus on planning; Liu et al. compare architectures; Li et al. introduce API Bank.Source: §5 · p. 9
- Other evaluation approaches include a Minecraft simulator for open-ended embodied agents, chatbot answers compared with gold answers, and task completion in interactive environments.The section also describes a benchmark for comprehensive agent evaluation.Source: §5 · p. 9
- AgentBench evaluates LLM performance across multiple environments, while GentBench uses complex tasks that prompt agents to employ external tools.GentBench includes public and private sections.Source: §5 · p. 9
6. Conclusion
The paper surveys LLM-based agents by contrasting them with traditional agents and reviewing agent components and applications. It also identifies pressing issues and research directions.
6. Conclusion
- The survey compares LLM-based agents with traditional methods, then reviews research on AI-agent components and applications.It aims to help readers grasp key information and applications and provide insights into future research directions.Source: §6 · p. 9
- The paper discusses pressing issues that require solutions and presents valuable research directions.Source: §6 · p. 9