Source-linked AI summary
Hierarchical Recurrent Attention Network for Response Generation
Chen Xing, Wei Wu, Yu Wu, Ming Zhou, Yalou Huang, Wei-Ying Ma
TL;DR
Existing multi-turn response-generation models represent conversation hierarchy but give insufficient attention to the unequal importance of words and utterances. HRAN addresses both through hierarchical word- and utterance-level attention, and experiments report significant gains over state-of-the-art models in automatic and human evaluations.
Problem
Existing models capture conversation-context hierarchy but insufficiently model the differential importance of words and utterances, risking lost context information and irrelevant responses.
Method
HRAN unifies hierarchical context modeling with word-level attention within utterances and utterance-level attention across the context for response decoding.
Results
HRAN significantly outperforms existing models for multi-turn response generation in automatic evaluation and side-by-side human comparison.
Takeaways & Limitations
Hierarchical attention provides a unified framework that models context hierarchy and highlights the words and utterances contributing to response generation.
Takeaways & Limitations
Error analysis reports that HRAN does not explicitly model universal replies and logical contradictions, which remain major error sources.
Abstract
from arXiv · showhide
We study multi-turn response generation in chatbots where a response is generated according to a conversation context. Existing work has modeled the hierarchy of the context, but does not pay enough attention to the fact that words and utterances in the context are differentially important. As a result, they may lose important information in context and generate irrelevant responses. We propose a hierarchical recurrent attention network (HRAN) to model both aspects in a unified framework. In HRAN, a hierarchical attention mechanism attends to important parts within and among utterances with word level attention and utterance level attention respectively. With the word level attention, hidden vectors of a word level encoder are synthesized as utterance vectors and fed to an utterance level encoder to construct hidden representations of the context. The hidden vectors of the context are then processed by the utterance level attention and formed as context vectors for decoding the response. Empirical studies on both automatic evaluation and human judgment show that HRAN can significantly outperform state-of-the-art models for multi-turn response generation.
1 Introduction
Multi-turn response generation must account for both the hierarchical structure of conversation contexts and the unequal importance of their words and utterances. HRAN unifies these requirements with hierarchical attention and outperforms existing models in automatic and human evaluations.
- Multi-turn response generation requires responses relevant to contexts composed of messages and previous-turn utterances.
- Words and utterances contribute unequally to response generation, but prior hierarchical models focused mainly on context structure.Ignoring importance can lose key context information and produce irrelevant responses.
- HRAN uses hierarchical attention to highlight important words within utterances and important utterances within the context.Its word-level attention dynamically selects parts of word sequences, while utterance-level attention selects parts of the utterance sequence.
- Word-level attention combines hidden vectors into utterance vectors, which an utterance-level RNN uses to construct context representations for decoding.The word-level mechanism assigns larger contributions to hidden vectors corresponding to important utterance content.
- HRAN is empirically evaluated on large-scale open-domain conversations using automatic metrics and side-by-side human comparison.The authors report significant outperformance of existing models on both evaluation types.
- The paper contributes a unified model of context hierarchy, word importance, and utterance importance, with effectiveness verified by automatic and human evaluation.
2 Related Work
Response-generation research began largely with single-turn encoder-decoder systems and later expanded to multi-turn models that incorporate conversation history. Attention mechanisms and hierarchical attention provide relevant foundations for modeling important conversational content.
- Early response-generation work primarily addressed single-turn conversation with encoder-decoder models targeting relevance, diversity, and personality.
- Multi-turn response generation emerged to incorporate context information that single-turn systems omitted.
- Attention mechanisms originated in machine translation and were subsequently applied to single-turn response generation.
- Hierarchical attention networks modeled contributions from words and sentences, motivating hierarchical approaches to conversational context.
3 Problem Formalization
The paper formalizes a dataset of responses paired with multi-turn contexts and models response generation as estimating the conditional probability of a response given its context.
- Each dataset example pairs a response Y_i with a context U_i containing a message and utterances from previous turns.The formulation requires each context to contain at least two utterances or messages.
- The model estimates p(y1, ..., yT |U) from the dataset and generates a response Y conditioned on a new conversation context U.
- The formalization defines w_i,j,k as the k-th word in an utterance.
4 Hierarchical Recurrent Attention Network
HRAN combines hierarchical context encoding with word- and utterance-level attention to emphasize information relevant to response generation. It encodes utterances, constructs context representations, attends over them, and decodes responses conditionally.
- Architecture: HRAN uses a word-level encoder, utterance-level encoder, and decoder to model response generation from multi-turn context.Each utterance is encoded into hidden vectors; utterance representations are transformed into context hidden vectors before decoding.
- Word Level Encoder: A bidirectional GRU encodes each utterance into word-position hidden vectors, combining forward and backward states.The backward GRU processes each utterance in reverse order, while the forward GRU reads it in its original order.
- Hierarchical Attention: Word-level attention synthesizes each utterance from hidden vectors, assigning larger weights to more important word representations.The resulting utterance vectors are passed to the utterance-level encoder.
- Hierarchical Attention: Utterance-level attention weights context hidden vectors and forms context vectors for the decoder.The decoder is conditioned on these context vectors at each generation step.
- Hierarchical Attention: HRAN’s word-level attention depends on both decoder hidden states and utterance-level encoder hidden states, unlike classic attention.The two mechanisms are conducted in reverse order across the utterance sequence, beginning with the latest utterance.
- Decoder and Learning: The decoder is an RNN language model conditioned on context vectors and can generate n-best responses with beam search.HRAN parameters are estimated by minimizing the training objective over context-response pairs.
5 Experiments
Experiments compare HRAN with established baselines using automatic and human evaluation, then examine attention behavior, component ablations, and remaining errors.
- Experimental Setup: HRAN was compared with S2SA, HRED, and VHRED on large-scale open-domain conversation data using automatic and side-by-side human evaluation.The dataset contains 20 million two-person conversations averaging 6.32 turns.
- Evaluation Metrics: Perplexity evaluates response prediction, while three native-speaking annotators judged paired HRAN and baseline responses for relevance, logic, and fluency.BLEU was excluded because prior work found weak correlation with human judgment and test contexts generally have a single reference.
- Evaluation Results: HRAN achieved the lowest perplexity on both validation and test, with statistically significant improvement over every baseline on test perplexity (p-value < 0.01).The result is reported in Table 1 and supported by a t-test.
- Evaluation Results: HRAN exceeded all baselines in human preference, gaining 6.7%, 6%, and 4.8% over S2SA, HRED, and VHRED, respectively.Sign tests found significance at p-value < 0.01 against S2SA and HRED and p-value < 0.05 against VHRED.
- Discussions: Case studies show HRAN can answer the latest turn using context and introduce a context-based topic when the current topic ends.Examples include inferring a dinner invitation’s implied question and asking about lotion brand after discussing skin exfoliation.
- Discussions: Attention visualizations show HRAN selecting important words and utterances, while ablations show every component matters and word-level attention is most important.Removing any component reduced performance; HRAN gained 4.6% preference over the model without word-level attention.
- Discussions: Among cases where HRAN loses to VHRED, 51.81% involve logic contradiction and 26.95% involve universal replies.The analysis identifies response diversity, context logic, and attention as future improvement directions.
6 Conclusion
The paper proposes HRAN for multi-turn response generation in chatbots and reports that it significantly outperforms state-of-the-art models on large-scale conversation data.
- HRAN is proposed for multi-turn response generation in chatbots.
- The model is evaluated empirically on large-scale conversation data.
- HRAN significantly outperforms state-of-the-art models.