PaperlayerView exact revision on arXiv

Source-linked AI summary

Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey

Jinjie Ni, Tom Young, Vlad Pandelea, Fuzhao Xue, Erik Cambria

arXiv:2105.04387v5cs.CLcs.AIcs.IR

TL;DR

Existing dialogue-system surveys were outdated, incomplete, and lacked multi-angle analysis. This survey systematically reviews recent deep learning-based work by model and system type, summarizes datasets and evaluation, and identifies research trends to provide a comprehensive state-of-the-art picture.

  • Problem

    Existing dialogue-system surveys were either outdated or not comprehensive and often lacked multi-angle analysis.

  • Method

    The survey systematically categorizes recent dialogue-system research by model type and system type, while reviewing datasets, evaluation methods, and research trends.

  • Results

    The survey provides a comprehensive picture of state-of-the-art dialogue systems and dialogue-related tasks.

  • Takeaways & Limitations

    The synthesis is intended to provide insights that pave the way for further dialogue-system research.

  • Takeaways & Limitations

    CNNs have fixed input length and limited convolution span, making them less suitable as direct sequential encoders in many state-of-the-art dialogue systems.

Abstract

from arXiv · show

Dialogue systems are a popular natural language processing (NLP) task as it is promising in real-life applications. It is also a complicated task since many NLP tasks deserving study are involved. As a result, a multitude of novel works on this task are carried out, and most of them are deep learning based due to the outstanding performance. In this survey, we mainly focus on the deep learning based dialogue systems. We comprehensively review state-of-the-art research outcomes in dialogue systems and analyze them from two angles: model type and system type. Specifically, from the angle of model type, we discuss the principles, characteristics, and applications of different models that are widely used in dialogue systems. This will help researchers acquaint these models and see how they are applied in state-of-the-art frameworks, which is rather helpful when designing a new dialogue system. From the angle of system type, we discuss task-oriented and open-domain dialogue systems as two streams of research, providing insight into the hot topics related. Furthermore, we comprehensively review the evaluation methods and datasets for dialogue systems to pave the way for future research. Finally, some possible research trends are identified based on the recent research outcomes. To the best of our knowledge, this survey is the most comprehensive and up-to-date one at present for deep learning based dialogue systems, extensively covering the popular techniques. We speculate that this work is a good starting point for academics who are new to the dialogue systems or those who want to quickly grasp up-to-date techniques in this area.

1 Introduction

Dialogue systems support both open-ended conversation and domain-specific assistance, motivating extensive deep-learning research. This survey addresses gaps in earlier reviews by organizing recent work across models, system types, evaluation, datasets, and trends.

  • System types: Dialogue systems are commonly divided into task-oriented systems for specific domains and open-domain systems without task or domain restrictions.Both types map user messages to agent responses, sometimes using external knowledge or databases.
  • System types: Traditional task-oriented systems use four pipeline modules, while open-domain systems include generative, retrieval-based, and ensemble categories.The supplied passage identifies natural language understanding, dialogue state tracking, policy learning, and natural language generation as the traditional task-oriented modules.
  • System types: Retrieval-based systems can improve surface-level coherence but remain limited by finite response sets and weak context correlation.These limitations distinguish retrieval systems from approaches that generate responses more flexibly.
  • Survey scope: Earlier dialogue-system surveys were outdated, incomplete, or lacked multi-angle analysis, prompting this survey’s focus on recent deep-learning research.The survey updates definitions and examines the field from both model and system perspectives.
  • Survey scope: The survey also covers dialogue-related NLP tasks, evaluation methods, datasets, and possible research trends.Its structure includes neural models, task-oriented and open-domain systems, and supporting resources for research.

2 Neural Models in Dialogue Systems

The survey introduces neural models used in state-of-the-art dialogue systems and related subtasks. It emphasizes their principles, applications, and relevance when designing new frameworks.

  • Neural models: This section reviews popular neural models and their variants from the model perspective of modern dialogue-system research.The goal is to connect model principles and characteristics with applications in state-of-the-art frameworks.
  • Neural models: The discussion is intended to help researchers understand how these models are applied when designing new dialogue systems.The section links model knowledge to framework design rather than presenting models in isolation.
  • Neural models: The listed model coverage includes CNNs, RNNs, vanilla sequence-to-sequence models, and additional neural architectures.The supplied passage begins the model taxonomy and indicates broader coverage beyond the examples shown.

2.1 Convolutional Neural Networks

CNNs extract local and hierarchical textual features efficiently, but their fixed input length and limited convolution span make them less suitable as direct sequential encoders. Consequently, dialogue systems often use CNNs after text encoding.

  • CNN architecture: CNNs combine convolutional, pooling, and feed-forward layers to transform text representations into class-score probabilities.Convolution applies kernels, pooling downsamples the resulting features, and feed-forward layers produce the prediction distribution.
  • CNN properties: Sliding windows capture local features, pooling builds hierarchical features, and parameter sharing reduces model complexity and overfitting.These mechanisms provide local and global perception while using fewer trainable parameters.
  • CNNs in NLP: CNNs are useful for hierarchical language modeling because phrases, sentences, and paragraphs provide semantic information beyond individual words.The survey notes a substantial increase in CNN applications across NLP tasks.
  • CNNs in dialogue systems: CNNs are often used as hierarchical feature extractors after encoding text rather than as direct dialogue encoders.The survey attributes this pattern to fixed input length and limited convolution span.
  • CNN limitations: CNNs may not continuously and flexibly capture information across temporal sequence steps, limiting their suitability for sequential encoding.This limitation helps explain why later dialogue systems often favor other encoder designs.

2.2 Recurrent Neural Networks and Vanilla Sequence-to-sequence Models

Recurrent neural networks address sequential-data limitations by modeling information flow across time, while sequence-to-sequence models map variable-length input sequences to variable-length outputs. The section surveys RNN architectures, gated variants, bidirectional networks, and encoder-decoder applications in dialogue systems.

  • Recurrent Neural Networks: RNNs represent sequential information flow, addressing standard neural networks’ independence and fixed-length-input assumptions.Markov models are also limited by inference complexity and transition-matrix growth for large hidden state spaces.
  • Jordan-Type and Elman-Type RNNs: Modern RNNs include Jordan-type architectures, whose hidden states depend on current inputs and previous outputs, and Elman-type architectures, whose hidden states depend on current inputs and previous hidden states.The surveyed formulations distinguish these architectures by the recurrent connection used to update the hidden state.
  • Recurrent Neural Networks: Simple RNNs can theoretically model long-term dependencies, but training across many time steps is hindered by vanishing and exploding gradients.These limitations motivated recurrent variants with mechanisms designed to improve long-range learning.
  • Gated Recurrent Units: LSTM and GRU gating mechanisms preserve selected past information while incorporating new content, helping recurrent units remember information from earlier steps and avoid gradient problems.LSTM uses input, forget, and output gates; GRU uses update and reset gates, with fewer parameters reported for GRU.
  • Bidirectional Recurrent Neural Networks: Bidirectional recurrent neural networks encode sequence information in both the forward and reversed directions using two hidden layers.This design incorporates past and future information into sequence inference.
  • Vanilla Sequence-to-sequence Models: Sequence-to-sequence models use an encoder to map an input sequence to an intermediate vector and a decoder to generate an output sequence from that representation and its history.Because source and target lengths can differ, the framework supports variable-length inputs and outputs and underlies many advanced models.
  • Applications in Dialogue Systems: RNN-based dialogue representation models trained on dialogue tasks consistently outperformed unfine-tuned BERT, ELMo, and GPT on some dialogue tasks.These models train dialogue embeddings that capture more dialogue features for downstream dialogue-related tasks.

2.3 Hierarchical Recurrent Encoder-Decoder (HRED)

HRED extends hierarchical sequence-to-sequence modeling to dialogue by representing both utterance-level tokens and turn-level context. Later variants add latent variables or self-attention to model complex dependencies and selectively process dialogue history.

  • 2.3 Hierarchical Recurrent Encoder-Decoder (HRED): HRED is a context-aware sequence-to-sequence model extended from online query suggestion to end-to-end dialogue systems.The dialogue version achieved noticeable improvements in dialogue and end-to-end question answering.
  • 2.3 Hierarchical Recurrent Encoder-Decoder (HRED): Two RNN levels encode each utterance token by token and iteratively track the history of turns.The context RNN summarizes utterances up to the current turn and initializes the decoder.
  • 2.3 Hierarchical Recurrent Encoder-Decoder (HRED): VHRED adds a latent variable to the decoder, sampling it before conditionally generating the response.It is trained with a variational lower bound on log-likelihood and showed promising improvement in diversity.
  • 2.3 Hierarchical Recurrent Encoder-Decoder (HRED): ReCoSa uses self-attention over hierarchical context representations because standard HRED processes dialogue history indiscriminately.It first encodes token-level information with LSTM and then calculates self-attention over context representations.

2.4 Memory Networks

Memory networks add explicit memory representations for reusing knowledge facts and reasoning over past information. End-to-end variants calculate query-dependent weights, softly select memory content, and support differentiable training across dialogue and question-answering applications.

  • 2.4 Memory Networks: Explicit memory addresses the limited capacity of recurrent models to discriminate, compress, and reuse specific knowledge facts.Memory networks are applied especially to task-oriented, knowledge-grounded, and question-answering dialogue tasks.
  • 2.4 Memory Networks: Original memory networks use separate modules for storing facts, embedding inputs, updating memory, producing outputs, and organizing responses.Training requires strong supervision for each module, making end-to-end training impractical.
  • 2.4 Memory Networks: End-to-end memory networks comprise weight calculation, memory selection, and final prediction stages.Sukhbaatar et al. extended prior memory-network work with a version described as easy to train and apply.
  • 2.4 Memory Networks: Query-dependent weights are computed by applying Softmax(u^T m_i) to similarities between the query embedding and memory representations.Each p_i is the weight assigned to memory x_i conditioned on the query.
  • 2.4 Memory Networks: The model forms a weighted sum of embedded memories, producing a selected memory vector that supports differentiable end-to-end training.The selected vector is combined with the embedded query for final prediction.
  • 2.4 Memory Networks: Memory-network applications include separating dialogue history and knowledge-base entries into long-term memories plus a working memory for task-oriented response prediction.Other systems store query-response pairs and condition open-domain generation on retrieved memory information.

2.5 Attention and Transformer

Attention lets decoders use different parts of the encoded source at each step, addressing fixed-vector limitations. Transformers replace recurrent units with attention to improve parallelization and support broad NLP and dialogue applications.

  • Attention: Fixed-length sequence-to-sequence representations lose information on long inputs and make early decoding errors affect the whole output.Experiments reported worse performance as input sequences became longer.
  • Attention: Attention conditions each decoding state on correlated parts of the encoded source rather than only the immediately preceding output.The mechanism computes normalized importance weights over encoder hidden states.
  • Transformer: Transformers use attention without recurrent units, enabling more parallelized training than recurrent models.Their encoder maps input sequences to hidden states and the decoder generates outputs from those states.
  • Transformer: Self-attention computes token interactions through query, key, and value representations, while multi-head attention processes multiple embedding subspaces in parallel.The parallel head outputs are concatenated and projected.
  • Transformer: Positional encoding adds sequence-order information because the Transformer contains no recurrent units.The architecture uses cosine functions with different wavelengths across dimensions.
  • Transformer: Transformer pretraining models such as GPT-2 and BERT adapt to new tasks after pretraining, while variants target efficiency or performance improvements.The survey also describes Transformer applications to response retrieval and generation in dialogue systems.

2.6 Pointer Net and CopyNet

Pointer Net converts copying from an input sequence into position prediction, while CopyNet combines copying with generating new tokens. These mechanisms support dialogue, knowledge-grounded, and task-oriented applications.

  • Pointer Net: Traditional sequence-to-sequence models encode source sequences and generate target tokens step by step using greedy or beam search.Attention-based variants use all encoder hidden states at each decoding step.
  • Pointer Net: Pointer Net predicts positions in the input sequence instead of tokens from a fixed dictionary, enabling direct copying.Its attention distribution has a dictionary size equal to the input length.
  • CopyNet: CopyNet combines generate-mode and copy-mode probabilities so the decoder can either copy a source token or generate a new one.Its probability is the sum of the two mode-specific distributions.
  • Applications: Copy mechanisms are useful for dialogue involving terminology or external knowledge and are popular in knowledge-grounded and task-oriented systems.Applications include copying from external knowledge and generating slot values in multi-domain dialogue state tracking.
  • Applications: Pointer networks and CopyNet also support conversation disentanglement and dialogue parsing tasks.Pointer modules can identify message ancestors or construct discourse trees.

2.7 Deep Reinforcement Learning Models and Generative Adversarial Networks

Deep reinforcement learning models train dialogue agents through state–action interaction, while GANs use generator–discriminator competition. The survey covers value-based, policy-based, and combined reinforcement approaches alongside dialogue applications.

  • Reinforcement learning: Dialogue systems fit reinforcement learning because an agent interacts with an environment represented by a user or user simulator.The framework is modeled as a Markov Decision Process with states, actions, transitions, rewards, and a discount factor.
  • Reinforcement learning: Deep Q-Networks model a value function with a deep neural network, whereas REINFORCE directly optimizes a policy network.They represent value-based and policy-gradient reinforcement learning families, respectively.
  • Reinforcement learning: Experience replay improves learning stability and efficiency by reusing buffered samples, while a frozen target network stabilizes temporal-difference targets.The target network periodically aligns with the Q-network and facilitates convergence.
  • Reinforcement learning: Deep Q-Networks are more sample efficient, while REINFORCE is more stable; policy-gradient methods are favored when action spaces are large.The survey notes that value-based methods require discrete, moderately sized action sets.
  • Reinforcement learning: Actor-critic methods combine value and policy learning to alleviate severe variance in policy-gradient estimation.They estimate a value function and incorporate it into policy optimization.
  • Generative adversarial networks: GAN training pits a generator against a discriminator: the generator tries to fool the discriminator, which distinguishes real from generated data.Dialogue research also combines GAN training with reinforcement-learning frameworks.

2.8 Knowledge Graph Augmented Neural Networks

Knowledge graphs provide structured entities and relationships that neural systems can retrieve and reason over. The survey presents graph-augmented dialogue systems for more accurate and informative knowledge-grounded responses.

  • Knowledge graphs: A knowledge graph represents entities and their relationships as structured facts, commonly stored as RDF triples.An example triple links Albert Einstein, the University of Zurich, and their graduate relationship.
  • Knowledge graphs: Knowledge-graph augmented neural networks embed entities and relations in lower-dimensional spaces before retrieving relevant facts with a neural model.The survey distinguishes structure-based and semantically enriched representations.
  • Dialogue systems: Knowledge-grounded dialogue systems use graph structure to reason over interconnected facts and incorporate commonsense knowledge into response generation.The survey characterizes this as supporting accurate and informative responses.
  • Dialogue systems: Graph-based dialogue agents explore knowledge graphs to retrieve knowledge or guide concept-level conversation flows.Examples include bidirectional graph exploration and gated generation from vocabulary and concept words.

3 Task-oriented Dialogue Systems

Task-oriented dialogue systems solve domain-specific problems, and this section reviews their modular architecture and deep learning-based techniques for its components. It covers NLU, DST, policy learning, and NLG, including approaches that reduce pipeline error propagation and use direct audio input.

  • Task-oriented systems address specific problems in domains such as movie ticket booking and restaurant table reservation.
  • Modular systems contain NLU, DST, policy learning, and NLG modules that transform user input into a system response.NLU extracts semantic information, DST tracks dialogue state, policy learning selects actions, and NLG verbalizes them.
  • Natural Language Generation: NLG converts selected dialogue actions into surface-level language, with recent end-to-end methods collapsing traditional pipeline components into one data-driven module.Traditional NLG includes content determination, sentence planning, and surface realization, while neural methods learn response generation directly.
  • Natural Language Understanding: NLU performs domain classification, intent detection, and slot filling to convert user messages into semantic representations.
  • Natural Language Understanding: Deep learning approaches for NLU include CNNs, RNNs, CNN-RNN combinations, DBN-initialized networks, and multitask architectures.These methods address classification, contextual intent detection, slot filling, and joint optimization of NLU subtasks.
  • Natural Language Understanding: Some recent systems bypass ASR and use audio signals directly as NLU input, reducing pipeline modules and making predictions more robust.

3.2 Dialogue State Tracking

Dialogue State Tracking maintains the user’s evolving goals and requested information across turns so policy learning can choose the next action. The section describes state representations, DST benchmarks, neural tracker families, and persistent efficiency and adaptability challenges.

  • DST tracks user goals and related details from the full dialogue history, providing the information used by policy learning to select actions.
  • Dialogue State Representation: A dialogue state includes informable slots that constrain database search and requestable slots whose values users ask the system to provide.
  • Dialogue State Representation: DST state representations also record goal constraints, requested slots, and the current turn’s search method, including constraints, alternatives, and finished.
  • Challenges: DST research continues to address tracking efficiency, accuracy, domain adaptability, and end-to-end training through alternative state representations.
  • Neural DST: Neural trackers use either predefined slot names and values or flexible mechanisms that extract or generate values from dialogue context.
  • Neural DST: Predefined-value trackers formulate DST as multi-class or multi-hop classification, while large value sets increase multi-class complexity.
  • Neural DST: Unfixed-value trackers reduce model and time complexity, support end-to-end training, and adapt more flexibly when the target domain changes.

3.3 Policy Learning

Policy learning maps dialogue states to system actions and is a central dialogue-manager component. The section reviews supervised and reinforcement learning, including warm starts and methods for reducing the cost of online interaction.

  • Policy learning selects the next system action by mapping the current dialogue state to an action in the available action set.
  • Supervised learning and reinforcement learning are the mainstream training methods for dialogue policies.
  • Supervised Learning: Supervised policies can make precise decisions in specific tasks but depend strongly on the quality of training data and intensive annotation.
  • Reinforcement Learning: Online reinforcement learning commonly uses a user or simulated user as the environment, but learning from scratch is costly in data and time.
  • Reinforcement Learning: Warm-start methods accelerate reinforcement learning by restricting initial exploration or transferring knowledge from expert teachers.
  • Reinforcement Learning: Budget-Conscious Scheduling treats user interaction as a limited budget and allocates it with a probability scheduler during policy learning.

3.5 End-to-end Methods

End-to-end task-oriented systems address limitations of modular pipelines by enabling joint optimization or integrating multiple functions in one model. The section covers differentiable pipelines, single-module systems, domain transfer, efficiency, consistency, and limited-data response quality.

  • Modular pipelines suffer when nondifferentiable modules prevent end-to-end error propagation and when intermediate supervision is unavailable.
  • End-to-end training either makes pipeline modules differentiable for back-propagation or uses one model for knowledge-base retrieval and response generation.
  • End-to-end Trainable Pipelines: Differentiable neural pipelines preserve explicit intermediate results, enabling interaction with external systems and improving interpretability.
  • Single-module Systems: Single-module systems implicitly represent key functions and integrate task-oriented dialogue components into one complex neural model.
  • Research Challenges: Domain transfer remains challenging because changing domains alters slot values, slot counts, and the usefulness of hand-crafted lexicons.
  • Research Challenges: End-to-end systems seek better response quality with limited annotated data using semantic constraints, templates, pretrained language models, and reinforcement learning.
  • Research Challenges: Retrieval-based methods can be more precise and effective for responses that do not depend on specific external knowledge facts.

4 Open-Domain Dialogue Systems

Open-domain dialogue systems support unrestricted chit-chat and are organized into generative, retrieval-based, and ensemble approaches. Research addresses context awareness, coherence, response specificity, and multimodal reasoning.

  • Open-domain systems perform chit-chat without task or domain restrictions and are usually fully data-driven.
  • Generative systems map messages and dialogue history to new responses, whereas retrieval systems select responses from predefined sets.
  • Context awareness: Context modeling uses dialogue history and increasingly structured representations, although one study found little or no learned discourse structure from structured attention.Approaches include multilevel context modeling, structured attention, future-context transfer, conversation threads, and pretrained Transformer encoders.
  • Response quality: Coherence methods modify generation objectives, add evaluation feedback, annotate sentence functions, or weight training samples by conversational quality.These strategies target logical consistency, response attributes, sentence organization, and data effectiveness.
  • Response quality: Generative systems tend to produce bland responses because frequent generic utterances and beam search favor universally acceptable outputs.Mutual-information objectives were proposed to trade off response safety and relevance to dialogue context.

5 Evaluation Approaches

Dialogue evaluation combines task-specific metrics, human assessment, simulators, automatic metrics, and ranking methods. Task-oriented systems have clearer success measures, while open-domain evaluation remains difficult because no single equivalent metric exists.

  • Task-oriented evaluation: Task-oriented evaluation directly measures task completion and cost, sometimes adding BLEU, human assessment, or user simulation.
  • Task-oriented evaluation: Task Completion Rate measures successful task attempts, such as dialogues satisfying all requirements in a movie-booking task.
  • Task-oriented evaluation: Task Completion Cost measures resources required, with conversation-turn count commonly representing time efficiency.
  • Task-oriented evaluation: Human evaluation uses crowdsourced workers or real user interactions to obtain conversations, satisfaction scores, and task metrics.
  • Task-oriented evaluation: User simulators provide lower-cost task-oriented dialogues through agenda-based or model-based simulated users.
  • Open-domain evaluation: Open-domain evaluation lacks clear task-completion metrics and therefore combines human ratings with word-overlap, neural, custom-quality, denoising, and ranking approaches.Human evaluation is widely used but is not ideal because it requires human labor.

6 Datasets

The survey reviews datasets for task-oriented and open-domain dialogue research, emphasizing that available data remain insufficient, especially when additional annotations are required. Its dataset tables organize the recent resources by application and annotation characteristics.

  • Dialogue datasets remain insufficient for both task-oriented and open-domain systems, particularly for tasks requiring additional annotations.
  • The survey reviews and categorizes datasets and summarizes nearly all available resources used in recent task-oriented and open-domain dialogue systems.
  • Task-oriented datasets: Task-oriented resources include datasets for dialogue state tracking, intent detection, flight reservations, multi-domain conversations, voice assistants, and restaurant or booking tasks.Examples span crowdsourced, simulated, human-human, customer-service, and machine-machine data.
  • Open-domain datasets: Open-domain resources cover knowledge grounding, emotion, social media, customer service, multi-party interaction, media, role-playing, and conversational psychotherapy.Several datasets add annotations such as emotion, discourse relations, localization, politeness, intention, or user knowledge.
  • Open-domain datasets: The dataset collection spans diverse domains and formats, including Wikipedia-grounded conversations, phone calls, movie dialogue, QA dialogues, emails, speech, and multimodal sales interactions.

7 Conclusions and Trends

The survey synthesizes recent dialogue-system research and identifies trends including multimodal, multitask, user-modeled, long-term-goal, and Internet-corpus-based systems. It positions neural methods as central while noting continuing data challenges.

  • Dialogue research is popular because chatbots are increasingly demanded in industry and daily life, while dialogue data are abundant in natural language use.
  • Recent dialogue systems favor neural methods, which the survey describes as outperforming traditional approaches through stronger fitting ability and less handcrafted feature engineering.
  • The survey systematically summarizes and categorizes recent dialogue-system and dialogue-related-task research to provide a comprehensive state-of-the-art picture.
  • Trends: Multimodal systems are motivated by human use of multiple senses and aim to blend information from modalities such as vision and hearing.
  • Trends: End-to-end task-oriented and knowledge-grounded open-domain work may enable blended systems that function as assistants and chatbots simultaneously.
  • Trends: Future directions include real-time Internet-corpus exploration, user modeling for coherent interaction, and reinforcement-learning approaches for long-term conversational goals.The survey notes that existing data remain insufficient and online corpora require standardized access and legal terms.
Loading 2105.04387v5…