Source-linked AI summary
LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions
Xixun Lin, Yucheng Ning, Jingwen Zhang, Yan Dong, Yilong Liu, Yongxuan Wu, Xiaohua Qi, Nan Sun, Yanmin Shang, Kun Wang, Pengfei Cao, Qingyue Wang, Lixin Zou, Xu Chen, Chuan Zhou, Jia Wu, Peng Zhang, Qingsong Wen, Shirui Pan, Bin Wang, Yanan Cao, Kai Chen, Songlin Hu, Li Guo
TL;DR
LLM-based agents face hallucinations that extend beyond language errors into compound, potentially unsafe deviations across their interactive pipelines. This survey organizes those hallucinations by agent stage, reviews their causes and mitigation and detection methods, and identifies future research directions. It provides a consolidated research landscape for improving understanding of hallucinations in LLM-based agents.
Problem
LLM-based agents can produce compound hallucinations across interacting modules, but prior hallucination research has focused mainly on natural language generation.
Method
The survey distinguishes internal belief states from external behaviors, develops a stage-based taxonomy, reviews triggering causes and mitigation and detection methods, and discusses future directions.
Results
The survey identifies five hallucination types, examines eighteen triggering causes, summarizes ten general mitigation approaches with corresponding detection methods, and outlines promising research directions.
Takeaways & Limitations
The survey consolidates current knowledge on agent hallucinations to support research and real-world deployment of more robust and reliable LLM-based agents.
Abstract
from arXiv · showhide
Driven by the rapid advancements of Large Language Models (LLMs), LLM-based agents have emerged as powerful intelligent systems capable of human-like cognition, reasoning, and interaction. These agents are increasingly being deployed across diverse real-world applications, including student education, scientific research, and financial analysis. However, despite their remarkable potential, LLM-based agents remain vulnerable to hallucination issues, which can result in erroneous task execution and undermine the reliability of the overall system design. Addressing this critical challenge requires a deep understanding and a systematic consolidation of recent advances on LLM-based agents. To this end, we present the first comprehensive survey of hallucinations in LLM-based agents. By carefully analyzing the complete workflow of agents, we propose a new taxonomy that identifies different types of agent hallucinations occurring at different stages. Furthermore, we conduct an in-depth examination of eighteen triggering causes underlying the emergence of agent hallucinations. Through a detailed review of a large number of existing studies, we summarize approaches for hallucination mitigation and detection, and highlight promising directions for future research. We hope this survey will inspire further efforts toward addressing hallucinations in LLM-based agents, ultimately contributing to the development of more robust and reliable agent systems.
I. INTRODUCTION
LLM-based agents extend language-model capabilities into task automation, but their multi-module, interactive workflows create hallucinations that can propagate beyond linguistic errors. This survey addresses the gap by organizing agent hallucinations, reviewing their causes and remedies, and identifying future research directions.
- Motivation: LLM-based agents increasingly automate tasks across diverse fields by combining language generation, intent comprehension, knowledge reasoning, and emergent capabilities.Their capabilities are attributed to large-scale training data, architectural innovations, instruction tuning, and in-context learning.
- Motivation: Agent hallucinations are fabricated or misjudged human-like behaviors that can arise across reasoning, execution, perception, memorization, and communication.They differ from conventional language-model hallucinations because they emerge from interactions among multiple agent modules and can escalate to task failures or physical safety risks.
- Contributions: The paper is presented as the first survey devoted to hallucinations in LLM-based agents and covers both mitigation and detection research.It aims to provide a broad perspective on the development of LLM-based agents.
- Contributions: The survey introduces an internal–external taxonomy that distinguishes an agent’s belief state from external behaviors and classifies hallucinations by pipeline stage into five types.The belief state is treated as the fundamental unit of agent cognition, while external behaviors are procedures guided by it.
- Contributions: The survey reviews each hallucination type with definitions, examples, and representative studies, identifying eighteen triggering causes and summarizing mitigation and detection approaches.Its review is intended to provide an up-to-date overview of the research landscape.
- Contributions: The survey outlines promising future directions by reviewing existing solutions and their possible limitations for academic research and real-world deployment.The paper also provides a publicly available collection of more than 200 related papers to support community engagement and collaboration.
II. FORMAL DEFINITION OF LLM-BASED AGENTS
The paper models LLM-based agents as partially observable decision-makers that repeatedly update an internal belief state while acting through content generation or external tools. The formalization specifies states, actions, transitions, goals, observations, rewards, and discounting for this interaction loop.
- A. Interaction Dynamics: LLM-based agent interaction is formulated as a POMDP in which the agent cannot directly observe the true environment state.The formalization uses an 8-tuple E = (S, A, T, G, O, Z, R, γ).
- A. Interaction Dynamics: The state space S contains true environment states, while the agent maintains a belief state representing its subjective understanding under partial observability.The belief state supports decision-making because the underlying state is not directly available to the agent.
- A. Interaction Dynamics: The action space A includes both content generation and external-tool use, such as scheduling events through calendar APIs or issuing smart-home commands.Actions therefore extend beyond textual responses to operations in the environment.
- A. Interaction Dynamics: The transition function T maps each state–action pair to a probability distribution over subsequent states, allowing execution uncertainty such as network latency or system errors.The paper illustrates this with a “turn off lights” action that reaches the desired state with 90% probability and remains unchanged with 10% probability.
- A. Interaction Dynamics: The goal space G represents user objectives, and the observation space O contains partial multimodal views of the underlying state.Partial observability may result from environmental complexity or limits in the agent’s perceptual capabilities.
- A. Interaction Dynamics: The observation function Z maps state–action pairs to possible observations, while the reward function R supplies numerical feedback for those pairs.Together, these functions describe how the agent receives information and evaluates interaction outcomes.
- A. Interaction Dynamics: The discount factor γ ∈ [0, 1) balances immediate and future rewards, with lower significance assigned to events farther in the future because they are harder to predict accurately.This parameter captures the formalization’s treatment of long-term reward uncertainty.
B. LLM-based Agent Loop
The agent operates through repeated loops that update an internal belief state while coordinating reasoning, execution, perception, memory, feedback, and environment transitions. Multi-agent systems add broadcasting and communication-structure evolution to this loop, and hallucinations can arise as endogenous, cross-module deviations with serious safety consequences.
- Agent Loop: The agent repeatedly refines its belief state because the true environment state is not directly observable, using it to drive subsequent operations.The belief state is dynamically updated across contexts and extended time spans.
- Agent Loop: Each loop connects reasoning, execution, feedback, environment transition, perception, memorization, and belief update.Reasoning generates a plan, execution turns it into an action, and later observations and feedback support memory and belief refinement.
- Multi-Agent Extension: In multi-agent systems, agents add broadcasting and time-varying communication-structure evolution to the standard interaction loop.The multi-agent loop is described as Reasoning–Execution–Broadcasting–Feedback–Environment Transition–Perception–Memorization–Belief Update–Structure Evolution.
- Hallucinations in the Loop: Agent hallucinations are fictitious or erroneous operations driven by overconfident human-like behavior across reasoning, execution, perception, memorization, and communication.Their cross-module propagation and hidden cognitive nature can escalate to task failures and physical safety risks.
III. TAXONOMY OF AGENT HALLUCINATIONS
The paper defines agent hallucinations formally and organizes them into five types according to where they arise in the agent workflow. The taxonomy covers reasoning, execution, perception, memorization, and communication hallucinations.
- III. TAXONOMY OF AGENT HALLUCINATIONS: The taxonomy identifies five agent-hallucination types: reasoning, execution, perception, memorization, and communication hallucinations.The paper presents the formal definition before introducing these workflow-based categories.
A. Reasoning Hallucinations
Reasoning hallucinations are plausible-looking but logically flawed or unsupported plans arising during goal understanding, intention decomposition, or planning generation. The paper links them to ambiguous objectives, comprehension limits, deficient dependency modeling, and misinterpreted planning information.
- A. Reasoning Hallucinations: Reasoning hallucinations are seemingly plausible plans that are logically flawed or unsupported.They can occur during goal understanding, intention decomposition, or planning generation.
- A. Reasoning Hallucinations: The reasoning taxonomy contains Goal Understanding Hallucinations, Intention Decomposition Hallucinations, and Planning Generation Hallucinations.These correspond respectively to misinterpreted goals, flawed sub-intention decomposition, and errors in generating plans for sub-intentions.
- Goal Understanding: Semantic vagueness in user goals can cause erroneous intention parsing, while active clarification is proposed to improve understanding of user needs.The paper attributes vagueness to incomplete goal specification and ambiguous content.
- Intention Decomposition: Deficient dependency modeling produces omitted, redundant, or disordered sub-intentions, compromising reasoning integrity and efficiency.These errors respectively remove critical steps, add task-irrelevant steps, or arrange sequentially dependent steps incorrectly.
- Planning Generation: Planning-generation hallucinations can arise when agents misinterpret information about operable objects or plan beyond the bounds of their self-knowledge.Incorrect assumptions about object relationships can induce flawed planning generation.
B. Execution Hallucinations
Execution hallucinations arise when agents translate plans into tool-based actions but select unsuitable tools, populate parameters incorrectly, or assume an infeasible plan is executable.
- B. Execution Hallucinations: Execution translates a deliberated plan into an executable action, typically by invoking one or more external tools.The process comprises tool selection followed by tool calling.
- B. Execution Hallucinations: Agents first select an appropriate tool from a retrieved candidate set, which narrows selection from the full tool inventory.The candidate set is retrieved from the full tool set because many tools may be available.
- B. Execution Hallucinations: After selecting a tool, agents derive parameters from the behavior representation and plan to form the final executable action.
- B. Execution Hallucinations: Tool-documentation limitations can cause hallucinations when internal beliefs about tool behavior diverge from actual functionality because documentation is redundant, incomplete, or inaccurate.
- B. Execution Hallucinations: Lack of solvability awareness leads agents to proceed with unjustified confidence when suitable tools are unavailable or plans are unclear and incomplete.Tool solvability depends on tool availability and the clarity and completeness of the plan.
C. Perception Hallucinations
Perception hallucinations occur when agents transform multimodal environmental information into internal observations that deviate from reality, disrupting subsequent decisions.
- C. Perception Hallucinations: The perception module extends agents into textual, auditory, and visual modalities, serving as their interface with the learning environment.
- C. Perception Hallucinations: Perception hallucinations are internal observations that significantly deviate from or contain factual errors about the actual learning environment.They arise during the reception and transformation of external information.
- C. Perception Hallucinations: Environmental sensor malfunction can distort external data before agents convert it into digital signals for processing.Examples of sensors include cameras, microphones, tactile sensors, and inertial measurement units.
- C. Perception Hallucinations: Insufficient unimodal representation and weak cross-modal collaboration limit extraction and integration of information across modalities.The stated contributors include training-data quality, local-detail loss in Transformers, and conflicts between pre-training and fine-tuning.
D. Memorization Hallucinations
Memorization hallucinations arise when agents trust stored information without validation, while communication hallucinations extend misinformation and coordination failures across multiple agents.
- D. Memorization Hallucinations: The memory module retrieves relevant information and updates stored content to support subsequent decision-making.
- D. Memorization Hallucinations: Biased or incomplete initial memory can introduce memorization hallucinations before deployment and directly influence later memory reliability.The passage specifically notes gender- and nationality-related biases as relevant examples.
- D. Memorization Hallucinations: Poor ranking strategies and inadequate indexing can retrieve superficially similar but irrelevant memories, causing information loss and retrieval errors.Delayed index updates and inappropriate index granularity are identified as indexing deficiencies.
- D. Memorization Hallucinations: Communication hallucinations involve inaccurate or fabricated inter-agent information that undermines collaboration through erroneous propagation and uncoordinated protocols.
- D. Memorization Hallucinations: Delayed updates and poor coordination can route messages to inappropriate recipients, producing misunderstandings or redundant reasoning in multi-agent systems.
A. Agent Hallucination Mitigation
The survey organizes hallucination mitigation into knowledge utilization, paradigm improvement, and post-hoc verification, while detection methods are separately typologized across hallucination types.
- A. Agent Hallucination Mitigation: Knowledge utilization supplies reliable information through external knowledge, expert knowledge, world models, and internal knowledge rectification.These resources regulate reasoning, execution, perception, and memory while addressing knowledge gaps and biases.
- A. Agent Hallucination Mitigation: Mitigation methods are grouped into three branches: knowledge utilization, paradigm improvement, and post-hoc verification.Figure 3 presents ten representative methods across these branches.
- A. Agent Hallucination Mitigation: Paradigm improvement includes learning and decoding strategies intended to improve representations, decision-making, structured task management, and factual alignment.Decoding optimization is a test-time paradigm that adjusts probability distributions or attention patterns to reduce reasoning errors.
- A. Agent Hallucination Mitigation: The survey’s overview table marks whether related mitigation work exists for each hallucination and mitigation-method combination.A check mark indicates existing related work, while a dash indicates that the area has not yet been explored.
- A. Agent Hallucination Mitigation: Post-hoc verification checks intermediate decisions and actions after execution to prevent hallucinations from accumulating and propagating during long-horizon tasks.It includes self-verification and validator assistance, with validators based on language, retrieval, execution, or simulation.
- A. Agent Hallucination Mitigation: Detection research remains relatively limited compared with mitigation research, with more methods targeting perception hallucinations than memorization hallucinations.
B. Agent Hallucination Detection
Agent hallucination detection remains uneven and difficult: perception errors attract more methods, while deeper memory and communication errors are harder to localize across interdependent modules. Future work should address accumulation over sequential decisions, accurate source localization, dynamic environments, and architectural bottlenecks.
- Detection landscape: Detection methods are relatively numerous for perception hallucinations but comparatively limited for memorization and communication hallucinations.The survey attributes this imbalance to perception being shallower and easier to identify, while memory and communication depend on many coupled intermediate computations.
- Future directions: Hallucinations can accumulate and amplify across multi-step agent loops, turning initially minor issues into severe consequences.The survey contrasts this sequential setting with studies focused on single agent loops.
- Future directions: Accurately locating hallucination sources remains difficult because errors can arise anywhere in the pipeline and propagate across interdependent components.The survey links this localization challenge to the limited amount and difficulty of existing detection research.
- Future directions: Current research often assumes fixed user goals and static environments, motivating continual adaptation to changing demands and conditions.The survey connects this setting mismatch with hallucinations caused by outdated knowledge or delayed updates.
- Survey scope: The survey consolidates hallucination types, causes, mitigation and detection methods, and future research directions for LLM-based agents.Its stated purpose is to consolidate progress, clarify challenges, and guide further exploration.
APPENDIX A LOOP OF LLM-BASED MULTI-AGENT SYSTEM
The LLM-based multi-agent system loop extends the single-agent workflow with inter-agent broadcasting and evolving communication structure. Agents repeatedly reason, execute, communicate, receive feedback, update the environment, perceive observations, and revise internal states.
- Multi-agent extensions: Multi-agent systems add broadcasting and communication-structure evolution to the agent loop.The structure Gt connects N agents and changes over time to align with dynamic adjustments.
- Agent cycle: Each agent generates a plan from its belief state and translates that plan into an executable action.These stages are labeled reasoning and execution in the loop.
- Communication: Agents broadcast messages to neighbors in Gt, and received messages support the multi-agent communication process.The passages define sent messages through neighbor broadcasting and received messages from neighboring agents.
- Environment interaction: The environment returns rewards, transitions to a new state, and provides new observations for agents to perceive.Feedback depends on the current state and actions, while transition produces the next state used for perception.
- State updates: After perception, agents update memorization and belief states, then evolve the communication structure for the next step.These procedures complete the multi-agent loop after environment interaction.
APPENDIX B HALLUCINATION EXAMPLE EXPLANATION
The appendix illustrates agent hallucinations through representative failures spanning goal interpretation, planning, perception, memory, tool use, and communication. These examples show how errors can arise at distinct stages and produce incorrect actions or progressively distorted information.
- Reasoning Hallucinations: Goal-understanding hallucinations ignore user constraints, such as recommending a spicy restaurant for dining with elders.The recommendation conflicts with the implied need for light, digestible cuisine and a family-appropriate setting.
- Reasoning Hallucinations: Intention-decomposition hallucinations omit necessary sub-intentions, such as calculating food costs under a stated budget.The picnic plan includes choosing a venue and buying food but omits cost verification.
- Reasoning Hallucinations: Planning hallucinations conflate venue opening hours with permission to hold a picnic.The example selects an all-day-open lawn without establishing that picnicking is allowed.
- Perception and Memorization Hallucinations: Perception hallucinations reverse the spatial relation between a chair and bookshelf, while memory errors misdate or over-delete events.The examples include placing the chair left instead of right, treating today’s meeting as tomorrow’s, and clearing an afternoon meeting when only morning meetings should be canceled.
- Execution Hallucinations: A tool-selection hallucination invents a plausible but nonexistent API instead of calling the available tourist-recommendation tool.The example contrasts “get kyoto travel info” with the correct “recommend tourist spots” tool.
- Execution Hallucinations: Tool-calling hallucinations may append an unsupported parameter even when the selected tool is correct.The example adds language=“English” to a tourist-recommendation call, potentially causing failure or unintended behavior.
- Communication Hallucinations: A communication hallucination can progressively distort “code review” into “old review” through repeated multi-agent transmission.The example describes successive reinterpretations as “cold review,” “gold review,” and finally “old review.”