PaperlayerView exact revision on arXiv

Source-linked AI summary

Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey

Shengyue Guan, Jindong Wang, Jiang Bian, Bin Zhu, Jian-guang Lou, Haoyi Xiong

arXiv:2503.22458v2cs.CLcs.AI

TL;DR

Evaluating LLM-based agents in multi-turn conversations requires automated, reproducible methods aligned with human judgments. This survey provides a comprehensive overview and structured taxonomy of evaluation dimensions and methodologies, while identifying inadequate long-term-memory benchmarks and the need for scalable, annotation-free pipelines.

  • Problem

    Evaluating conversational agents requires methods that are automated, reproducible, and aligned with human judgments for real-world reliability.

  • Method

    The paper develops a structured taxonomy defining critical evaluation dimensions and the methodological steps for assessing them reliably.

  • Results

    The framework clarifies what elements of multi-turn conversational performance should be evaluated and how to assess them reliably.

  • Takeaways & Limitations

    Comprehensive evaluation spans conversational performance and can ultimately support improved user experience.

  • Takeaways & Limitations

    The field has inadequate benchmarks for long-term memory retention and needs scalable, annotation-free evaluation pipelines.

Abstract

from arXiv · show

This survey examines evaluation methods for large language model (LLM)-based agents in multi-turn conversational settings. Using a PRISMA-inspired framework, we systematically reviewed nearly 250 scholarly sources, capturing the state of the art from various venues of publication, and establishing a solid foundation for our analysis. Our study offers a structured approach by developing two interrelated taxonomy systems: one that defines \emph{what to evaluate} and another that explains \emph{how to evaluate}. The first taxonomy identifies key components of LLM-based agents for multi-turn conversations and their evaluation dimensions, including task completion, response quality, user experience, memory and context retention, as well as planning and tool integration. These components ensure that the performance of conversational agents is assessed in a holistic and meaningful manner. The second taxonomy system focuses on the evaluation methodologies. It categorizes approaches into annotation-based evaluations, automated metrics, hybrid strategies that combine human assessments with quantitative measures, and self-judging methods utilizing LLMs. This framework not only captures traditional metrics derived from language understanding, such as BLEU and ROUGE scores, but also incorporates advanced techniques that reflect the dynamic, interactive nature of multi-turn dialogues.

1 Introduction

This survey addresses the need for reliable evaluation of LLM-based agents in multi-turn conversations by organizing both evaluation targets and methodologies. It synthesizes prior work into a taxonomy covering agent capabilities, evaluation goals, and assessment approaches.

  • Scope and motivation: Multi-turn evaluation must be automated, reproducible, and aligned with human judgments to support real-world reliability.
  • Scope and motivation: LLM-based multi-turn agents extend language models with dynamic tool use, persistent memory, and sequential planning for complex requests.These capabilities support decomposing requests, invoking external services, and retaining recent dialogue and long-term preferences.
  • Survey framework: The survey distinguishes what to evaluate—including task success, response quality, user engagement, and overall user experience—from how to evaluate it.
  • Survey framework: Its methodology combines data-driven annotations, automatic metrics, and human–machine assessment tools within a structured evaluation framework.
  • Survey methodology: Using a PRISMA-inspired process, the review screened 1,123 unique records and retained 351 papers for qualitative synthesis, including 276 with targeted evaluation insights.
  • Contribution: The taxonomy responds to gaps in prior surveys by jointly examining evaluation targets, methodologies, scalability, and real-world applicability for multi-turn agents.

2 What to evaluate? The evaluation goals and target

Evaluating multi-turn agents requires assessing ongoing dialogue, context maintenance, user adaptation, and complex task execution rather than isolated responses. The survey therefore organizes evaluation around end-to-end experience and underlying functions such as tools, memory, and planning.

  • Evaluation scope: Multi-turn agents must maintain context across interactions, adapt to user inputs, and execute complex tasks.
  • Evaluation scope: Evaluation should assess the interplay of components that drives overall performance, reliability, and user satisfaction.
  • Evaluation workflow: The proposed workflow begins with end-to-end experience to capture real-world usability before examining specific functions.
  • Evaluation workflow: Specific evaluation targets include actions and tools, memory, and planning in multi-turn interactions.

2.1 Evaluating the End-to-end Experience of LLM-based Agents For Multi-Turn Conversation

End-to-end evaluation treats multi-turn agents as dialogue-level systems, assessing task fulfillment alongside coherence, interaction quality, user experience, and safety. The section organizes evaluation around task completion, multitask ability, interaction patterns, and user experience and safety.

  • End-to-end evaluation: Multi-turn evaluation must assess dialogue-level performance across sequential or interleaved tasks, not only individual exchanges.It examines both whether tasks are completed and how smoothly the agent transitions, maintains context, and supports the user.
  • Evaluation dimensions: The end-to-end taxonomy covers task completion, multitask capabilities, interaction patterns, and user experience and safety.These dimensions address fulfillment, cross-domain expertise, dialogue structure, satisfaction, engagement, and safeguards against harmful content.
  • Task completion: Task-completion evaluation can combine completion rates, unrecognized-utterance tracking, user feedback, system logs, and multi-level interaction-quality scores.The Cookpanion evaluation used twelve conversation patterns and assessed usability, likability, conversation quality, and interaction metrics.
  • Open limitations: Current frameworks generally lack detailed failure-mode analysis and user-centric feedback loops.The identified gap motivates richer interaction metrics and error diagnostics for multi-turn evaluation.
  • Interaction patterns: Interaction-pattern evaluation examines recollection, expansion, refinement, and follow-up as mechanisms for building and adapting conversational context.A related three-tier taxonomy uses Perceptivity, Adaptability, and Interactivity to analyze dialogue behavior.
  • User experience and safety: Safety evaluation should test resilience to coreference attacks, prompt leakage, backdoor insertion, and stealthy toxic triggers.These threats are especially important for deployments in high-stakes domains such as medicine and education.

2.2 Evaluating The Action and Tool-use Components of LLM-based Agents for Multi-turn Conversation

Action and tool-use evaluation spans basic command execution, coherent multi-step orchestration across turns, and reliability of resulting actions. The survey groups methods around API interaction, multi-step selection and reasoning, and hallucination and reliability.

  • Evaluation scope: Tool-use evaluation requires three layers: discrete command execution, cross-turn chains of execution, and reliable outcomes.Outputs from one turn must inform inputs to the next, while action hallucination makes reliability a safety constraint.
  • API interaction and dynamic tool-use: API and dynamic tool-use evaluation measures intent recognition, response selection, context preservation, consistent invocation, data integration, retrieval, and API chaining.These criteria cover both interaction precision and adaptive orchestration.
  • Multi-step tool selection and reasoning: Benchmarks such as ToolE expose gaps in tool selection, while MTU-Bench evaluates multi-tool and multi-step interactions across single- and multi-turn tasks.Other benchmarks target program synthesis, Python-based tool use, and mathematical reasoning in dialogue contexts.
  • Multi-step tool selection and reasoning: Multi-step tool-use evaluation should assess invocation precision, operational efficiency, error recovery, tool awareness, planning, execution accuracy, and multi-turn coherence.The survey frames these criteria as components of unified evaluation for sequential tool use.
  • Reliability and hallucination: Reliability evaluation must address functional failures, tool limitations, reasoning calibration, and multi-level diagnoses rather than merely count errors.Suggested protocols include identifying failure causes and enforcing confidence thresholds before action execution.
  • Reliability and hallucination: Tool-use hallucination evaluation includes solvability detection, missing-tool analysis, toolset limitations, and thresholded response generation.One hard evaluation approach compares retrieved-knowledge similarity with confidence levels and responds only when the threshold is met.

2.3 Evaluating the Memory of LLM-based Agents For Multi-turn Conversation

Memory evaluation considers both how long information persists and how it is represented and retrieved. The survey distinguishes turn, conversational, and permanent spans alongside textual and parametric memory forms.

  • Memory span: Agents face a trade-off between retaining everything, which creates cognitive overload, and discarding information, which breaks continuity.The survey presents memory-span categories from immediate turn-level buffers to permanent knowledge stores.
  • Memory taxonomy: Memory evaluation uses two dimensions: memory span, covering persistence over time, and memory form, covering representation and implementation.These dimensions jointly address retention duration and access efficiency.
  • Memory span: Turn, conversational, and permanent memory require complementary evaluation of immediate context, extended dialogue retention, and cross-session persistence.Relevant criteria include capacity, extended retention, cross-session consistency, and recall and application of facts, preferences, and events.
  • Memory span: Benchmarks evaluate memory across varied timescales, including dialogues exceeding 40 utterances, conversations of 600 turns and 16K tokens, and intervals from hours to years.These settings cover factual recall, temporal reasoning, multimodal generation, and persona consistency.
  • Memory forms: Textual memory forms include complete, recent, retrieved, and external interactions, each balancing retention, efficiency, relevance, or tool-based data access.Evaluation examines history coverage, recent-window efficiency, selective retrieval precision and recall, and external-data correctness and latency.
  • Memory forms: Parametric memory encodes information in model parameters, including fine-tuning and targeted memory editing, thereby avoiding context-length constraints.Across memory forms, the survey links evaluation to coherence, personalization, and multi-turn performance.

2.4 Evaluating the Planner of LLM-based Agents For Multi-turn Conversation

Planner evaluation models planning as a continuous control loop rather than a linear pipeline. Its four dimensions are task modeling, task decomposition, adaptation and control, and reflection through plan verification and selection.

  • Planner taxonomy: The planner taxonomy contains task modeling, task decomposition, adaptation and control, and reflection.Together, these dimensions cover environmental understanding, executable sequencing, real-time adjustment, and metacognitive validation.
  • Task modeling: Task modeling evaluates task representation and context modeling, including structured objectives, actions, expected outcomes, and evolving conversational details.Robust assessment tracks multi-turn coherence, context utilization, and success across diverse scenarios.
  • Task decomposition: Task decomposition evaluates granularity, interdependency, dynamism, adaptive subdivision, coordination, and feedback utilization.These criteria address how complex goals are divided, logically coupled, and restructured during execution.
  • Adaptation and control: Adaptation and control measures whether planners detect changing environments and user goals, re-plan dynamically, and preserve coherence.Assessment focuses on interactive adjustment and goal alignment under uncertainty or context change.
  • Reflection: Reflection operates through plan verification and plan selection to evaluate action feasibility, outcomes, goal alignment, constraints, and strategy quality.Verification identifies pitfalls before execution, while selection can use in-generation filtering or post-generation methods.

3 How to evaluate? The evaluation methodologies and data

The survey organizes evaluation around data generation and annotation, then contrasts annotation-based and annotation-free methods for measuring multi-turn agent performance.

  • Conversation Data Generation: Evaluation data covers next-turn responses, tool use, query rewriting, and fact checking.These data types target contextual response generation, API use, intent adaptation, and factual accuracy.
  • Conversation Data Annotation: Annotation processes label expected responses, tool calls, rewritten queries, retrieved items, and factual accuracy.The annotations support alignment, relevance, context preservation, and verification across dialogue turns.
  • Data Construction: Automated dialogue-data generation reduces manual annotation while supporting realistic scenarios and precise evaluation labels.The survey presents automation as improving scalability and reducing evaluation costs.
  • Evaluation Methodologies: Annotation-based evaluation uses pre-labeled benchmarks, whereas annotation-free evaluation scores responses without manual labels.Annotation-based methods provide detailed human-curated insights but require substantial resources; annotation-free methods emphasize scalability and adaptation.
  • Metrics and Judging: The survey highlights both traditional metrics such as BLEU and ROUGE and broader agent evaluations covering intermediate milestones, dependencies, and interaction quality.It also discusses agent-as-judge approaches and the need for metrics that capture multi-turn dynamics.

4 Summary, Challenges, and Future Works

The survey traces evaluation from rule-based and qualitative assessments to multi-dimensional benchmarks for modern LLM-based agents, while identifying unresolved challenges in holistic, adaptive measurement.

  • Trends and Status-quo: Evaluation evolved from rule-based and qualitative assessments toward neural-model metrics and multi-faceted benchmarks.The roadmap covers changes in conversational systems alongside increasingly complex evaluation practices.
  • Benchmarks and Datasets: Modern benchmarks assess dialogue modeling, tool and action integration, reasoning, user intent, and factual veracity.Examples include HUMOD, BotChat, AgentBench, API-Bank, MMDU, MTU-Bench, and Factcheck-Bench.
  • Self-Judging Evaluation: LLM-based evaluators increasingly support point-wise scoring, side-by-side comparison, ranking explanations, and rapid iterative testing.Learned reward models provide approximate quality scores and have correlated with aggregate human judgments in many settings.
  • Challenges: Current evaluation struggles to capture multi-turn understanding comprehensively and often assesses turns in isolation rather than holistically.The survey calls for frameworks integrating turn-level and overall conversation assessments.
  • Future Work: Future work should develop adaptive metrics and specialized benchmarks for temporary and persistent memory retention over extended dialogues.The survey also identifies test-time self-assessment, scalability, privacy preservation, and unified evaluation as priorities.

5 Conclusion

The paper concludes by offering a structured, holistic framework for evaluating LLM-based multi-turn conversational agents, covering both evaluation targets and methodologies. It also identifies current limitations and future directions for more reliable, scalable, and interpretable assessment.

  • The evaluation scope includes dialogue coherence, effective tool use, memory retention, and diverse methods such as annotation-based assessments and automated metrics.
  • The synthesis spans the progression from rule-based systems to transformer-based agents and integrates insights from nearly 200 scholarly sources.
  • The survey develops a structured taxonomy that clarifies what aspects of multi-turn conversational performance should be evaluated and how to assess them reliably.The framework addresses both critical performance dimensions and methodological steps.
  • The survey highlights inadequate long-term-memory benchmarks and the need for scalable, annotation-free evaluation pipelines as current challenges.
  • Future research should develop automated evaluation techniques and tools that improve the interpretability and reliability of assessment processes.
  • The review and its critical analysis are presented as a foundation for more robust evaluation tools and continued advancement in conversational AI.
Loading 2503.22458v2…