PaperlayerView exact revision on arXiv

Source-linked AI summary

A Survey on Recent Advances in LLM-Based Multi-turn Dialogue Systems

Zihao Yi, Jiarui Ouyang, Zhe Xu, Yuwen Liu, Tianhao Liao, Haohao Luo, Ying Shen

arXiv:2402.18013v2cs.CLcs.AI

TL;DR

Multi-turn dialogue systems face challenges in maintaining long-range context and consistency across turns. This survey synthesizes LLM architectures, adaptation methods, TOD and ODD systems, datasets, metrics, and research directions. It identifies external tool calls as supporting more accurate and comprehensive knowledge and better accommodation of user needs, while computational cost remains a bottleneck to scalable deployment.

  • Problem

    Multi-turn dialogue systems must maintain long-range dependencies and consistent dialogue states across turns.

  • Method

    The survey categorizes LLMs, reviews fine-tuning and prompt-engineering approaches, and synthesizes LLM-based TOD and ODD methods, datasets, and evaluation metrics.

  • Results

    External tool calls support more accurate and comprehensive knowledge, better user-need fulfillment, and accommodation of personalized preferences during conversations.

  • Takeaways & Limitations

    The survey frames tool integration as a way for multi-turn dialogue systems to engage with the real world and fulfill more complex user requests.

  • Takeaways & Limitations

    Computational cost remains a critical bottleneck to scalable deployment of LLM-based dialogue systems.

Abstract

from arXiv · show

This survey provides a comprehensive review of research on multi-turn dialogue systems, with a particular focus on multi-turn dialogue systems based on large language models (LLMs). This paper aims to (a) give a summary of existing LLMs and approaches for adapting LLMs to downstream tasks; (b) elaborate recent advances in multi-turn dialogue systems, covering both LLM-based open-domain dialogue (ODD) and task-oriented dialogue (TOD) systems, along with datasets and evaluation metrics; (c) discuss some future emphasis and recent research problems arising from the development of LLMs and the increasing demands on multi-turn dialogue systems.

1 INTRODUCTION

Multi-turn dialogue systems must maintain context across several turns while handling long-range dependencies, consistency, and ambiguity. This survey reviews LLM-based approaches across task-oriented and open-domain dialogue, including adaptation methods, datasets, metrics, limitations, and future directions.

  • Multi-turn dialogue systems: Compared with single-turn systems, multi-turn dialogue introduces long-range dependencies, cross-turn consistency, and ambiguous or incomplete information.
  • Multi-turn dialogue systems: Multi-turn systems incorporate current user messages and dialogue history to generate coherent, contextually relevant responses.
  • Traditional approaches: Traditional systems rely on predefined rules or retrieval templates, constraining application scope and flexibility across domains and unexpected scenarios.
  • LLM-based approaches: LLMs offer strong context understanding, generalization, few-shot learning, and reduced dependence on extensive task-specific training data.
  • Survey scope and contributions: The survey systematically covers LLM adaptation methods, TOD and ODD advances, datasets, evaluation metrics, current limitations, and future research directions.

2 GENERAL METHODS

LLMs are built on Transformer architectures whose attention masking determines how context is processed. The survey distinguishes encoder-decoder, decoder-only, and encoder-only structures.

  • LLM architectures: Scaling LLMs improves language representations and performance across diverse downstream NLP tasks, especially natural language generation.
  • Transformer foundations: The vanilla Transformer uses encoder and decoder components with self-attention, providing parallelism and representational capacity.
  • LLM architectures: LLMs are categorized as encoder-decoder, decoder-only, or encoder-only according to their attention masking mechanisms.

2.1 Decoder-only Transformer Architecture

Decoder-only Transformers process tokens autoregressively with causal or prefix attention and underpin major GPT, LLaMA, and GLM-family models. These models support increasingly capable dialogue and task applications.

  • Decoder-only mechanisms: Causal decoders use unidirectional attention, allowing each token to attend only to preceding tokens and itself.
  • GPT series: GPT-based systems support conversational use, with ChatGPT powered by GPT-3.5-turbo and GPT-4 supporting text and image inputs.
  • LLaMA series: LLaMA2-Chat is optimized for dialogue, outperforming other open-source chat models in benchmarks and providing an alternative to closed-source models.
  • Prefix decoders: Prefix decoders combine bidirectional attention for prefix tokens with unidirectional attention for generated tokens.
  • GLM series: ChatGLM2-6B achieves a 42% boost in inference speed while reducing GPU memory usage, and ChatGLM3-6B adds tool invocation, code execution, and agent operations.

2.2 Encoder-only Transformer Architecture

Encoder-only Transformers focus on understanding input content and producing task-specific outputs. BERT uses bidirectional context, while related models modify objectives and masking for broader language tasks.

  • Encoder-only architecture: Encoder-only LLMs focus on understanding input content and generating task-specific outputs rather than autoregressive regression.
  • BERT series: BERT uses masked language modeling to capture bidirectional context, but autoregressive prediction limits its effectiveness for generation tasks.
  • BERT series: BERT performance has been enhanced through longer training, parameter tying across layers, and span masking.
  • BERT series: RoBERTa removes next sentence prediction and adds longer training, larger batches, dynamic masking, and an expanded byte-level BPE vocabulary.
  • Unified pre-training: UNiLM supports bidirectional, unidirectional, and sequence-to-sequence objectives through shared parameters and task-specific self-attention masks.

2.3 Encoder-decoder Transformer Architecture

Encoder-decoder Transformers separate input representation from autoregressive generation, while BART and T5 adapt this architecture for versatile sequence-to-sequence and text-to-text tasks.

  • The Transformer encoder uses multi-head self-attention to represent inputs, while the decoder generates target sequences autoregressively through cross-attention.
  • BART combines bidirectional and autoregressive Transformers in a denoising autoencoder trained through text corruption and reconstruction.
  • BART generalizes BERT, GPT, and other pre-training approaches for sequence-to-sequence tasks.
  • T5 treats all NLP tasks as text-to-text problems using span corruption and multi-task pre-training.
  • T5 unifies the model, training procedure, and decoding process across a wide range of tasks.

2.4 Fine-Tuning

Fine-tuning adapts pretrained models to task-specific data, ranging from updating all parameters to parameter-efficient methods that preserve the original model while reducing resource demands. The survey also reviews instruction tuning and stabilization techniques for improving robustness and performance.

  • Full Fine-Tuning: Full Fine-Tuning optimizes all pretrained-model parameters on task-specific data to minimize task loss.The objective selects parameters θ* that minimize L(M(θ), D), where the loss measures prediction error.
  • Parameter-Efficient Fine-Tuning: Parameter-Efficient Fine-Tuning adjusts only part of a model’s parameters as a more resource-efficient alternative to full fine-tuning.
  • Adapters: Adapters insert compact down-projection, nonlinear, and up-projection modules while freezing the remaining model parameters.Adapter outputs add processed adapter outputs to the original layer outputs, and training minimizes a task-specific loss over adapter parameters.
  • Adapters: Adapters train significantly fewer parameters, support modular insertion and removal, and preserve pretrained representations by keeping the original parameters frozen.
  • LoRA: LoRA applies low-rank updates to selected weight matrices, reducing computational and memory requirements while retaining pretrained representations.Its update uses low-rank matrices with rank r much smaller than the matrix dimensions.
  • LoRA: QLoRA combines LoRA with 4-bit quantization to reduce memory footprint and enable fine-tuning extremely large models on limited hardware while maintaining task performance.
  • Instruction Fine-Tuning: Instruction Fine-Tuning trains pretrained language models on instructions paired with inputs and outputs so they better understand and execute task directions.It preserves the original architecture while extending applicability to new tasks and domains.
  • Mitigating Fine-Tuning Instabilities: Fine-tuning instabilities include erratic loss landscapes, convergence problems, representational collapse, overfitting, and hyperparameter sensitivity.R3F, R4F, SMART, and FreeLB use regularization, reparameterization, adversarial perturbations, or proximal optimization to stabilize training and improve robustness.

2.5 Prompt Engineering

Prompt engineering adapts language models without changing their parameters through in-context demonstrations, step-by-step reasoning prompts, and retrieved external knowledge. Prompt selection and organization affect performance, while retrieval addresses the limits of static pretrained knowledge.

  • 2.5 Prompt Engineering: Tuning-free prompting generates answers without modifying pretrained language-model parameters and includes ICL, CoT, and RAG.
  • 2.5.1 In-Context Learning.: In-context learning uses multiple input-output demonstrations to generate the desired response.In-context examples can convey the target label space, input distribution, and input-label correspondence.
  • 2.5.1 In-Context Learning.: ICL performance generally improves when prompts are closer to test samples in embedding space, and prompt arrangement matters especially for smaller models.
  • 2.5.2 Chain-of-Thought.: Chain-of-thought prompting guides reasoning through step-by-step thought processes, including zero-shot prompts and decomposition into simpler sub-problems.
  • 2.5.2 Chain-of-Thought.: CoT prompt selection significantly influences generated results, motivating voting and automatic prompt-generation methods.
  • 2.5.2 Chain-of-Thought.: CoT research also examines multimodal and multilingual scenarios.
  • 2.5.3 Retrieval Augmented Generation.: RAG addresses LLM knowledge-boundary problems by retrieving relevant documents from external knowledge bases during generation.This supports information that is up-to-date and contextually appropriate for recent events or niche domains.
  • 2.5.3 Retrieval Augmented Generation.: RAG progresses from similarity-based Naive RAG to Advanced RAG with pre- and post-retrieval strategies and Modular RAG with replaceable retrieval and reranking modules.

3 LLM BASED TASK-ORIENTED DIALOGUE SYSTEMS

LLM-based task-oriented dialogue systems support domain-specific goals through pipeline, modularly end-to-end, and fully end-to-end designs. The survey describes LLM applications across NLU and dialogue-state components, emphasizing improved intent, slot, and few-shot capabilities alongside pipeline error-propagation trade-offs.

  • 3 LLM BASED TASK-ORIENTED DIALOGUE SYSTEMS: TOD systems help users achieve domain-specific goals through interactive conversations such as hotel reservations and restaurant queries.
  • 3 LLM BASED TASK-ORIENTED DIALOGUE SYSTEMS: LLM-powered TOD systems recognize intent, manage task details, maintain multi-turn coherence, and switch across domains such as flights and local recommendations.
  • 3.1 Pipeline-based Methods: Pipeline-based TOD systems divide processing into NLU, dialogue state tracking, policy learning, and natural language generation modules.Dialogue state tracking and policy learning form the dialogue manager.
  • 3.1 Pipeline-based Methods: Pipeline systems suffer error propagation because independently trained module failures can degrade entire-system performance.
  • 3.1 Pipeline-based Methods: Pipeline modularity enables individual components to be interchanged and replaced with separately fine-tuned large-scale modules for target-domain adaptation.
  • 3.1.1 Natural Language Understanding: LLM-based NLU methods address intent detection and slot filling through prompting, PEFT, contextual representations, and joint modeling.
  • 3.1.1 Natural Language Understanding: Pointwise V-information improves LLM-based intent detection where in-context prompting alone does not improve performance.
  • 3.1.1 Natural Language Understanding: Joint NLU based on fine-tuned LLMs outperforms separated NLU and NLU based on untuned LLMs.

4 LLM BASED OPEN-DOMAIN DIALOGUE SYSTEMS

LLM-based open-domain dialogue systems handle broad topics without predefined goals, using retrieval, generation, or hybrid methods to produce contextually relevant responses. Recent approaches improve context modeling and knowledge integration, while retrieval systems still face scalability, diversity, and context-dependency challenges.

  • Open-Domain Dialogue: ODD systems support conversations across diverse topics without a specific task or predefined goal, aiming for coherent and contextually relevant responses.They must handle broader topic diversity and maintain context throughout interaction than task-oriented systems.
  • LLM Benefits: LLMs enhance ODD by supporting accurate responses across topics, emotion detection for empathetic replies, and personalized recommendations.These capabilities contribute to more natural, engaging, and individualized conversations.
  • Open-Domain Dialogue: LLM-based ODD systems are commonly categorized as retrieval-based, generation-based, or hybrid methods.Retrieval selects existing responses, generation produces new text, and hybrid methods combine both techniques.
  • Retrieval-based Methods: Neural retrieval models improve semantic matching through dense representations, interaction layers, self-attention, and hierarchical aggregation of local and global dependencies.DPR uses separate BERT encoders for queries and passages, while IoI combines matching signals across layers for response selection.
  • Retrieval-based Methods: Retrieval-based methods use dialogue corpora without explicit state or intent annotation and can separately optimize retrieval and ranking components.Their main technical challenges are context dependency modeling, response diversity, and retrieval efficiency at scale.
  • Generation-based Methods: Knowledge-enhanced generation selects relevant external information and can use reflection or dynamic integration during response generation.Self-RAG combines retrieval with self-reflection, while related approaches evaluate retrieved information and critique generated outputs.

5 EVALUATION APPROACHES

The survey reviews automatic and human evaluation for task-oriented and open-domain dialogue systems. It emphasizes that individual metrics capture specific properties but can overlook dialogue context, semantic coherence, consistency, or the balance between diversity and relevance.

  • Automatic Evaluation: TOD evaluation uses joint goal accuracy, slot accuracy, average goal accuracy, requested slot F1, F1, BLEU, and Entity F1.These metrics assess dialogue-state correctness, slot prediction, requested-slot identification, lexical overlap, and entity generation.
  • Automatic Evaluation: Joint goal accuracy requires all predicted dialogue-state values to match the ground truth at each turn, making it a stringent state-tracking measure.It may overlook natural dialogue variations and subtle shifts in user intentions.
  • Automatic Evaluation: Slot accuracy evaluates individual slot predictions but disregards previous turns and long-range dependencies in complex dialogues.Average goal accuracy focuses on correctly predicting active slots, yet it also overlooks broader dialogue context needed for long-term understanding.
  • Automatic Evaluation: BLEU measures weighted n-gram overlap, while Entity F1 measures relevant entity generation; neither fully captures semantic meaning, dialogue consistency, or coherence.Entity F1 emphasizes precision over fluency, whereas BLEU is limited in assessing dialogue depth and naturalness.
  • Automatic Evaluation: ODD evaluation commonly includes perplexity, BLEU, DIST-n, and Recall@K, covering sequence prediction, lexical overlap, response diversity, and retrieval relevance.Perplexity and Recall@K assess narrower properties and do not directly measure overall dialogue quality or individual response quality.
  • Human Evaluation: Human evaluation examines grammar, faithfulness, coherence, consistency, diversity, and creativity across dialogue responses.Overemphasizing grammar, faithfulness, coherence, or consistency can respectively constrain creativity, flexibility, or response variability.

6 DATASETS

The survey catalogs widely used task-oriented and open-domain multi-turn dialogue datasets with varied domains, languages, annotation schemes, grounding sources, and scales. These resources support research on state tracking, response selection, persuasion, personalization, and multimodal conversation.

  • Task-oriented Dialogue Datasets: MultiWOZ has successive versions that correct annotations and add descriptions for slots, domains, and categorical values.MultiWOZ 2.4 is described as fixing incorrect and inconsistent annotations.
  • Task-oriented Dialogue Datasets: CrossWOZ contains 6 thousand dialogue sessions and 102 thousand utterances, with about 60% of dialogues involving cross-domain user goals.Its annotations include dialogue states and acts on both user and system sides.
  • Open-Domain Dialogue Datasets: Open-domain datasets represent human-human, scraped, personalized, emotional, and multimodal conversations across English and Chinese resources.PersonaChat labels dialogue acts and emotions, while PersonalDialogue associates utterances with speaker traits for personalized generation.
  • Open-Domain Dialogue Datasets: Large-scale open-domain resources include Pchatbot with almost 200 million dialogue pairs and MMdialog with 1.08 million dialogues, 1.53 million images, and 4148 topics.Pchatbot provides anonymized user IDs and timestamps, while MMdialog targets multimodal open-domain conversation.
  • Open-Domain Dialogue Datasets: Douban is a human-labeled multi-turn response-selection dataset built from open-domain conversations longer than two turns.It includes sampled training, validation, and test splits, with response candidates collected from a retrieval engine.

7 DISCUSSION

The discussion identifies persistent challenges for LLM-based multi-turn dialogue—including long-context consistency, domain robustness, agentic tool use, privacy, and computational cost—and proposes dialogue-specific, efficient architectures as future directions.

  • Challenges: LLM-based multi-turn dialogue systems still face sustained challenges in long-context modeling, domain adaptation, robustness, agentic interaction, privacy, and computational cost.These constraints limit applicability in real-world, sustained dialogue scenarios.
  • High Computational Resource Requirements: Computational cost remains a critical bottleneck because LLM training and inference demand substantial computing power, limiting deployment in resource-constrained environments.Compression methods such as quantization and knowledge distillation are proposed but frequently lead to performance degradation.
  • Future Directions: Future systems may balance contextual reasoning with efficiency through dialogue-specific architectures, hybrid model selection, and continual learning with efficient parameter updates.Suggested strategies include distilled models for routine turns, powerful LLMs for complex reasoning, and LoRA or adapters for low-resource deployment.
  • Long-Context Modeling: Long dialogue histories cause models to forget, overlook, or misrepresent earlier turns, producing incoherent, contradictory, or repetitive responses.The discussion connects this difficulty to limitations of Transformer architectures in modeling long-range dependencies.
  • Domain Adaptation and Robustness: Domain adaptation remains difficult because training can reduce robustness, generalization, and security, while few-shot ICL produces unstable performance and complicates dialogue history.The paper calls for secure and robust adaptation without degrading overall performance or safety.
  • Multi-turn Interactions of Agentic AI: External tool calls can ground dialogue turns in up-to-date information, improving accuracy, comprehensiveness, and accommodation of personalized preferences.Examples include flight search, schedule queries, seat availability, fare comparison, and reservations.

8 CONCLUSION

The survey reviews LLM-based multi-turn dialogue systems by covering LLM adaptation methods, TOD and ODD systems, evaluation resources, and open research problems. It identifies long-context modeling as a major challenge and frames it as a representation-learning problem requiring selective attention to salient dialogue history.

  • The survey categorizes common LLMs by model structure and reviews adaptation methods for downstream subtasks, including fine-tuning and prompt engineering.
  • It surveys two main categories of LLM-based multi-turn dialogue systems: task-oriented dialogue and open-domain dialogue.
  • The paper outlines evaluation metrics and widely used datasets for assessing and understanding the conversational abilities of LLM-based dialogue systems.
  • It identifies open problems and future research directions arising from the development of LLMs and the demands of multi-turn dialogue systems.
Loading 2402.18013v2…