PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey of Large Language Model Agents for Question Answering

Murong Yue

arXiv:2503.19213v1cs.CLcs.AIcs.HC

TL;DR

QA research must address hallucinations, limited calibration, and the inability of standalone LLMs to consult dynamic external sources. This survey organizes LLM-agent QA methods by planning and QA stages, reviews retrieval and reasoning designs, and identifies challenges and future directions. It presents LLM agents as a way to combine LLM reasoning with interaction across external environments.

  • Problem

    Standalone LLM QA can hallucinate, lacks calibrated uncertainty, and cannot consult external databases, APIs, or other dynamic sources during inference.

  • Method

    The paper surveys LLM-agent QA design across planning, question understanding, information retrieval, answer generation, and follow-up interaction.

  • Results

    The survey identifies LLM-based agents’ external-environment interaction, retrieval, reasoning, and calibration designs as central approaches for advancing QA systems.

  • Takeaways & Limitations

    Future progress depends on improving source-reliability assessment, reasoning-path scoring, and scalable LLM-based document indexing.

Abstract

from arXiv · show

This paper surveys the development of large language model (LLM)-based agents for question answering (QA). Traditional agents face significant limitations, including substantial data requirements and difficulty in generalizing to new environments. LLM-based agents address these challenges by leveraging LLMs as their core reasoning engine. These agents achieve superior QA results compared to traditional QA pipelines and naive LLM QA systems by enabling interaction with external environments. We systematically review the design of LLM agents in the context of QA tasks, organizing our discussion across key stages: planning, question understanding, information retrieval, and answer generation. Additionally, this paper identifies ongoing challenges and explores future research directions to enhance the performance of LLM agent QA systems.

I. INTRODUCTION

The introduction frames QA agents as systems that combine LLM reasoning with modular planning and environmental interaction. It surveys their design across the QA process while distinguishing them from earlier rule-based and reinforcement-learning agents.

  • Motivation: QA addresses users’ information needs by providing answers from context or knowledge, while direct LLM QA can hallucinate and lacks external interaction.These limitations are especially problematic for nuanced, domain-specific questions.
  • LLM-based agents: LLM-based agents use heuristic modules to guide actions such as planning and interact with databases, tools, models, or humans.The modules distinguish agent-based QA from standalone LLM answering.
  • Survey scope: The survey organizes LLM-agent QA research around planning, question understanding, information retrieval, answer generation, and follow-up interaction.It discusses the motivation and design of each stage, plus challenges and future directions.
  • Agent foundations: An agent interacts with its environment through states, observations, and actions, using a planning policy and transition function to update its state.The formalization describes observation through sensory processing and state changes through transition dynamics.
  • Agent foundations: Naive agents handle explicitly programmed scenarios, whereas reinforcement-learning agents learn from environmental interaction, trial, error, and rewards.The comparison highlights the historical shift from fixed heuristics toward learning through feedback.

B. Question Answering

Question answering automatically provides answers to user questions and is widely used across domains. The section characterizes QA probabilistically using a question, context, and generated answer sequence.

  • Definition: Question answering is the task of automatically providing answers to questions posed by users.The paper identifies search engines, customer support, and education as application domains.
  • Formalization: A QA instance consists of an input question Q, context C, and answer sequence A composed of T tokens.P(A|Q) denotes the probability of generating the answer given the question.
  • Approaches: QA systems are categorized into three main approaches according to the format of their answers.The subsequent categories include classification, span extraction, and sequence generation.

1) Classification-based Methods:

The section describes answer prediction through classification, span extraction, and sequence generation. These methods range from selecting among candidate answers to generating tokens conditioned on the question, context, and prior output.

  • Classification-based Methods: Classification-based QA treats multiple possible answers A1 through An as classes and predicts one answer from that set.Ai denotes the predicted answer among the candidate answers.
  • Span Extraction: Span-extraction QA identifies the text span containing the answer within a document or context.The model predicts start and end positions using probabilities Ps and Pe.
  • Sequence Generation: Sequence-generation QA generates each answer token conditioned on the question, context, and previously generated tokens.The formulation models P(at | Q, C, a1, a2, . . . , at−1; θ).
  • Sequence Generation: Early text-generation approaches used neural sequence-to-sequence models with an encoder for the question and a decoder for the answer.The encoder maps the question to a latent representation before decoding the answer.

C. Traditional QA Pipeline

Traditional QA uses a fixed pipeline, while LLM-based QA uses pretrained models to comprehend and generate answers. LLMs improve coherence and open-domain answering but remain vulnerable to hallucination and stale internal knowledge.

  • Traditional QA Pipeline: Traditional QA decomposes answering into fixed stages beginning with query understanding and continuing with retrieval from a pre-established database.Query understanding includes syntactic analysis, semantic understanding, intent classification, and slot extraction.
  • Traditional QA Pipeline: Traditional pipelines require specialized models, lack world knowledge for out-of-domain questions, cannot dynamically plan, and depend on large corpora and effective retrievers.These limitations constrain both generalization and open-domain retrieval.
  • LLM-based QA: LLM-based QA uses pretrained LLMs through fine-tuning or few-shot prompting to comprehend and generate answers.The survey focuses primarily on autoregressive LLMs for text generation.
  • LLM-based QA: LLMs can generate coherent, contextually relevant answers for questions not seen during training and perform open-domain QA directly.The paper attributes this capability to knowledge encoded through large-scale pretraining.
  • LLM-based QA: LLM-based QA can hallucinate plausible but factually incorrect answers and cannot consult external databases, APIs, or other dynamic sources during inference.After training, fixed parameters force reliance on internalized knowledge.

E. LLM-based QA Agent

LLM-based QA agents combine memory, planning, inner thinking, and environmental interaction to answer questions dynamically. Their process alternates between external actions that gather observations and internal actions that update the agent’s memory.

  • Architecture: The agent architecture includes memory, a planning module, and an inner-thinking module.Memory aggregates the question, intermediate understanding, and retrieved information; planning selects actions, while inner thinking executes reasoning actions.
  • Action cycle: At each step, the planner selects an action from the current memory state.The action may involve interacting with the external environment or performing internal thinking.
  • External actions: External actions obtain environment observations and store the action-observation pair in memory.This enables the agent to gather information from external sources during question answering.
  • Internal actions: Internal actions generate thoughts and update memory without interacting with the external environment.The thinking result is concatenated with the current memory state.
  • LLM specialization: LLM-based QA agents specialize the conceptual agent by using the LLM as the planning policy and inner thinker.Their action space includes both environmental interaction and internal thinking processes.

III. AREA TAXONOMY

The survey organizes LLM-based QA research around planning, question understanding, information retrieval, answer generation, and follow-up interaction. It also distinguishes QA datasets by knowledge scope and reasoning demands.

  • Survey organization: The survey’s taxonomy follows the QA stages of planning, question understanding, information retrieval, answer generation, and follow-up interaction.Planning is treated as the distinctive capability through which an agent chooses how to answer a question.
  • Dataset scope: QA datasets are broadly divided into closed-domain/context-based and open-domain settings according to whether knowledge is restricted to specified documents.Closed-domain tasks use predefined contexts, whereas open-domain tasks draw on broader knowledge sources.
  • Open-domain QA: Open-domain datasets evaluate varied capabilities including factual, ambiguous, long-form, and multi-step question answering.Examples include StrategyQA, ASQA, and ELI5, which target different reasoning and answer-format requirements.
  • Reasoning abilities: The taxonomy includes mathematical, symbolic, and broad knowledge-intensive reasoning datasets.GSM8k, MATH, TheoremQA, BBH, Folio, MMLU, GPQA, and WikiQA cover these evaluation dimensions.
  • Conditional reasoning: Conditional reasoning datasets require models to reason over hypothetical or counterfactual changes in context.IFQA is presented as an example of this setting.

B. Planning

Planning-based QA agents use prompting or tuning to select intermediate actions and strategies. Prompting methods depend heavily on task-specific instructions, while tuning methods face search and scalability constraints.

  • Overview: Planning formulates intermediate actions or steps that lead toward a final goal or answer.The survey identifies prompting-based and tuning-based planning as two primary paradigms.
  • Prompting-based Planning: Prompting-based planning guides the LLM with instructions to choose actions and reasoning steps.Methods such as ReAct and Active Retriever support iterative information gathering and decisions about when to stop searching.
  • Prompting-based Planning: Prompting-based planning depends strongly on carefully designed prompts and in-context demonstrations.Because these are often tailored to particular tasks or domains, transferring planning ability to unseen contexts is challenging.
  • Tuning-based Planning: Tuning-based planning learns from successful and failed action trajectories through trial and error.FireAct fine-tunes LLMs on action trajectories from multihop QA tasks using positive and negative examples.
  • Tuning-based Planning: Tuning-based planning can be limited by the need to search multiple potential trajectories and by the demands of fine-tuning.These requirements can constrain scalability.

C. Question Understanding

Question understanding transforms ambiguous or complex user queries into forms that support downstream QA, while information retrieval finds, ranks, and compresses relevant evidence. LLMs contribute through query processing, semantic ranking, and prompt compression.

  • Question Understanding: Question understanding extracts and interprets query information so machines can process questions more effectively.The survey discusses slot identification, query expansion, and query reformulation.
  • Identifying Slots: Slot identification maps entities, variables, and attributes in natural-language questions to predefined categories.For example, a location such as California can be extracted from a legal question.
  • Query Expansion: Query expansion augments the original query with related terms, hypothetical documents, or decomposed sub-questions.HyQE generates hypothetical documents, while Query2CoT breaks complex queries into step-by-step sub-questions.
  • Query Reformulation: Query reformulation rewrites ambiguous or vague questions into clearer forms and can select rewrites using downstream answer correctness.Rephrase and Response generates reformulations, evaluates them, and can use DPO to improve future rewrites.
  • Information Retrieval: Information retrieval identifies and ranks documents, passages, or snippets that may contain evidence for knowledge-intensive questions.The process typically follows retrieval and ranking stages.
  • Retrieval: Dense retrieval embeds queries and documents in a shared semantic space, computes similarity, and returns top-K documents.Its objective is formulated using contrastive learning with positive and negative document examples.
  • Ranking: LLMs can be effective re-rankers because they analyze semantic meaning and assign relevance scores aligned with user intent.Haystack and Self-RAG use LLM-based relevance assessment after documents are retrieved.
  • Compression and Selection: Compression methods reduce document or prompt length while preserving essential semantic content.LLMLingua applies coarse-to-fine compression to remove low-importance information and refine the remaining input.

E. Answer Generation

Answer generation methods enhance LLM agents through external tools and structured prompting. These approaches support computation, specialized processing, decomposition, verification, and learned reasoning strategies.

  • Tool-Augmented Generation: Tool-augmented generation lets LLMs use calculators, code interpreters, and specialized models to extend reasoning beyond internal capabilities.Program-of-Thought generates executable Python whose results become the final answer, while systems such as HuggingGPT invoke domain-specific models.
  • Tool-Augmented Generation: Binding combines a natural-language solution skeleton with external computation by leaving selected answer components as placeholders.
  • Prompt-Enhanced Generation: Prompt-enhanced generation uses Chain-of-Thought, question decomposition, and self-verification to structure and check multi-step reasoning.Least-to-Most prompting decomposes complex questions into sequential simpler sub-questions.
  • Prompt-Enhanced Generation: Self-Discovery and PromptAgent optimize reasoning prompts by exploring alternative strategies and automating their composition.Self-Discovery trains the LLM to identify strategies associated with more successful outcomes.

F. Follow-up Interaction

Follow-up interaction supports multi-turn QA by refining answers through feedback and maintaining context across sequential questions. Although difficult tasks have become more manageable, LLM QA agents remain imperfect and face new challenges.

  • Follow-up interaction enables LLM agents to refine answers based on user feedback during multi-turn conversations.
  • Error Resolution: Human feedback can help resolve misunderstandings, request missing details, and revise outputs such as creative writing or semantic parses.
  • Sequential Question Answering: Sequential question answering maintains context across multiple QA turns through instruction tuning, reinforcement learning from human feedback, and context-enhanced tuning.ChatQA is cited as an example of context-enhanced instruction tuning.
  • Despite improved handling of coherent conversations and complex mathematics, LLM QA agents remain far from perfect and introduce new challenges.

A. Challenging and Practical Benchmarking

Benchmarking LLM QA agents remains challenging because evaluations must assess reasoning processes, difficult-to-judge questions, currentness, hallucination, and reasoning quality. The survey identifies methodological and capability gaps requiring better evaluation and training approaches.

  • Fine-Grained Answer Generation Process Evaluation: Current benchmarks often evaluate only final free-form answers with multiple-choice formats or coarse metrics, overlooking the reasoning process.The survey argues that CoT adoption increases the need for finer-grained process evaluation.
  • Hard-to-Judge Question Evaluation: Some questions are difficult to evaluate because correctness is hard to verify for complex mathematics or undefined for open-ended questions.
  • Up-to-Date Evaluation: Benchmarks must remain current and fair by preventing data leakage, protecting dataset integrity and structure, and resisting leaderboard manipulation.Dynamic benchmark updates are proposed, but few effective mechanisms currently prevent overfitting through synthetic or hand-labeled data.
  • Hallucination and Calibration: LLM hallucinations reflect false or fabricated information produced with confidence despite poorly calibrated judgments of correctness.Reducing hallucinations requires models to assess the reliability of external sources rather than being overly influenced by them.
  • Hallucination and Calibration: Uncertainty-aware training has not yet significantly surpassed traditional logit-based or voting-based calibration methods.Developing stronger internal calibration mechanisms remains an open problem.
  • Reasoning Ability Improvement: Improving reasoning requires reliable scoring for explored reasoning paths because existing search methods often depend on domain-specific signals that limit generalization.
  • Reasoning Ability Improvement: LLMs need to retain foundational knowledge from past interactions rather than treating every session as new, enabling adaptation to new scenarios.
  • Reasoning Ability Improvement: Causal reasoning is proposed to make generated logical chains more rigorous and coherent than responses based primarily on statistical correlations.

D. Autonomous Tool Selection and Creation

The survey identifies autonomous tool selection and creation as a limitation of current LLMs and proposes planning capabilities that dynamically choose or develop tools. It also highlights LLM-based document indexing as a promising retrieval direction constrained by large-scale computational cost.

  • Autonomous Tool Selection and Creation: Current LLMs do not inherently choose external resources, plan tool use, or create tools for repeated question patterns as humans do.
  • Autonomous Tool Selection and Creation: Tool-use and tool-creation planning could let models decide dynamically when to leverage existing tools or create new ones.The survey connects these capabilities with effectiveness in open-ended and collaborative tasks.
  • LLMs in Building Document Indexing: LLM-based document indexing is proposed as a promising retrieval direction because LLMs can represent documents and queries semantically.The main obstacle is the cost of indexing millions or billions of documents with LLMs.
  • The survey synthesizes LLM-agent QA development by decomposing answering into subtasks, reviewing methods, and identifying challenges and future research directions.
Loading 2503.19213v1…