Source-linked AI summary
Dial2Desc: End-to-end Dialogue Description Generation
Haojie Pan, Junpei Zhou, Zhou Zhao, Yan Liu, Deng Cai, Min Yang
TL;DR
Dialogue summarization has received little attention, and describing the objects or actions speakers discuss requires connecting their statements. Dial2Desc builds dialogue-description pairs and uses an interaction-aware encoder with a Transformer-pointer generator; its model reports CIDEr gains over two baselines.
Problem
Dialogue summarization has received little attention, while descriptions of discussed objects or actions require connecting statements across speakers.
Source: introduction · p. 1Method
Dial2Desc pairs dialogues with descriptions and uses an encoder that models speaker interaction with a Transformer-pointer generator.
Source: introduction · pp. 1–2, Conclusion · p. 7Results
CIDEr improves 9.8% over PGN and 10.8% over Onmt-transformer, and the encoder plus Transformer decoder achieves the highest performance over other summarization baselines.
Source: introduction · p. 2, Quantitative Results · p. 7Takeaways & Limitations
Dial2Desc frames dialogue summarization as describing the discussed object or action rather than preserving the conversation’s natural flow.
Source: introduction · p. 1
Abstract
from arXiv · showhide
We first propose a new task named Dialogue Description (Dial2Desc). Unlike other existing dialogue summarization tasks such as meeting summarization, we do not maintain the natural flow of a conversation but describe an object or an action of what people are talking about. The Dial2Desc system takes a dialogue text as input, then outputs a concise description of the object or the action involved in this conversation. After reading this short description, one can quickly extract the main topic of a conversation and build a clear picture in his mind, without reading or listening to the whole conversation. Based on the existing dialogue dataset, we build a new dataset, which has more than one hundred thousand dialogue-description pairs. As a step forward, we demonstrate that one can get more accurate and descriptive results using a new neural attentive model that exploits the interaction between utterances from different speakers, compared with other baselines.
introduction
Dialogue summarization has received little attention despite the volume of interactional information, and conventional conversation summaries emphasize event flow rather than the objects or actions speakers discuss. Dial2Desc instead generates higher-level descriptions from dialogues, supported by a large paired dataset and an interaction-aware neural model.
introduction
- Automatic dialogue summarization could help people consume the large volume of dialogue records generated in daily life.The paper notes that dialogue summarization has received little attention compared with summarization of reviews, news, and scientific articles.Source: introduction · p. 1
- Dial2Desc describes the object or action speakers discuss instead of maintaining the conversation’s natural flow.The task addresses descriptions that require combining speakers’ statements across a dialogue.Source: introduction · p. 1
- The dataset contains 122,621 VisualDialog dialogues paired with corresponding COCO captions.The image links each dialogue to a description of what the speakers are talking about.Source: introduction · p. 2
- The proposed encoder uses co-attention and dense connections for speaker interaction, with a Transformer framework for message passing across dialogue turns.The Transformer is also used as the decoder.Source: introduction · p. 2
- The encoder plus Transformer decoder achieves the highest performance over other summarization baselines.The paper presents the task and dataset as contributions alongside its neural attentive model.Source: introduction · p. 2
Related Work
The related work connects image captioning and Visual Dialog to the dataset design, distinguishes Dial2Desc from conversation summaries that preserve event flow, and motivates adapting neural attention to speaker interaction.
Image caption and visual Dialogue
- Visual Dialog uses an image, dialogue history, and a question to produce an image-grounded answer; the paper links this task with image captioning to build its dataset.Both source datasets evolved from MSCOCO.Source: Image caption and visual Dialogue · p. 2
Conversation summarization
- Prior conversation summarization targets speech, mail threads, and meetings, while Dial2Desc describes what people discuss rather than stating conversation events.Earlier approaches include extractive and abstractive meeting summaries and methods for speech and email threads.Source: Conversation summarization · p. 2
Neural attentive models and summarization
- The paper adapts neural attentive models to conversation summarization and exploits interaction between utterances from different speakers.Earlier attention-based summarization work includes pointer functions, temporal-coverage loss, and intra-attention with reinforcement learning.Source: Neural attentive models and… · p. 3
Our Approach
The approach encodes speaker-paired dialogue through utterance, interaction, recurrent, and memory layers, then uses a transformer-pointer decoder to generate descriptions.
Position Encoding and Multi-head Attention
- Sinusoidal position encodings supply sequence order, while multi-head attention combines parallel scaled dot-product heads over query-key-weighted values.Source: Position Encoding and Multi-head… · p. 3
utterance encoding layer
- Each speaker’s utterance is embedded and encoded with a biLSTM into token-level representations.Source: utterance encoding layer · p. 4
utterance interaction layer
- Cross-speaker attention re-encodes each utterance from interaction context, then concatenates differences and element-wise products with original and context vectors.Source: utterance interaction layer · p. 4
densely connected recurrent layer
- A further biLSTM builds higher-level representations while concatenating earlier hidden features with word embeddings so prior features persist to the top layer.Source: densely connected recurrent layer · p. 4
memory output layer
- Turn representations are assembled into memory, position-encoded, and passed through transformer self-attention and position-wise feed-forward sublayers with residual connections and layer normalization.Source: densely connected recurrent layer · p. 4, memory output layer · p. 4
Transformer-pointer Generator
- The decoder attends to encoder memory, produces vocabulary probabilities, and uses a pointer generator to choose between vocabulary generation and copying dialogue words, including OOV words.Source: Transformer-pointer Generator · p. 5
Experiments
Experiments evaluate Dial2Desc on dialogue-caption pairs built from VisDial and MSCOCO, comparing neural generative baselines with an interaction-aware model. The proposed model leads all metrics, while qualitative analyses examine beam search, co-attention, and generated descriptions.
Dataset
- The dataset contains 122,621 training pairs, created by matching VisDial dialogues with MSCOCO captions and split into train, development, and test sets.Each test dialogue has five descriptions.Source: Dataset · p. 5
Baselines
- Comparisons include Attn-Seq2seq, PGN, Onmt-brnn, and Onmt-transformer; MMR was excluded after poor results on abstractive descriptions.Source: Baselines · pp. 5–6
Quantitative Results
- Our model performs best on all metrics, with CIDEr improvements of 9.8% over PGN and 10.8% over Onmt-transformer.Source: Table 3 · p. 6, Quantitative Results · pp. 6–7
Experimental setup
- Models share a 20k vocabulary; turns are truncated to 20 tokens and descriptions to 5–15 tokens, while transformer models use 256-dimensional hidden states and embeddings.Source: Experimental setup · p. 6
Evaluation Metrics
- Evaluation uses BLEU, ROUGE-L, METEOR, and CIDEr, with CIDEr IDF values computed from the reference descriptions.Source: Evaluation Metrics · p. 6
Quantitative Results
- Our model requires a larger beam than Onmt-brnn to reach optimal results and is more stable across the tested beam sizes.Source: Figure 3 · p. 6, Quantitative Results · p. 7
Qualitative Analysis
- In the co-attention example, “many” attends to numerical answer terms, while “many” also draws attention as an informative query word; each row sums to one.Source: Figure 4 · p. 6, Qualitative Analysis · p. 7
- In the 10-turn case study, all systems omit grapes; the proposed model’s output is nevertheless judged more accurate and comprehensive overall.Source: Figure 5 · p. 7, Qualitative Analysis · p. 7
Conclusion
The conclusion frames Dial2Desc as generating high-abstraction descriptions of dialogue objects or actions and introduces a dataset linked from two open-source datasets. It also presents an attentive encoder with a transformer-pointer generator, reporting effectiveness on Dial2Desc.
Conclusion
- Dial2Desc encodes an input dialogue and decodes a high-abstraction description of the object or action speakers discuss, rather than preserving conversational flow.Source: Conclusion · p. 7
- The authors link two open-source datasets to create a dialogue-to-description dataset for Dial2Desc.Source: Conclusion · p. 7
- The proposed model combines an enhanced interaction dialogue encoder with a transformer-pointer generator, and results on Dial2Desc demonstrate its effectiveness.Source: Conclusion · pp. 7–8