PaperlayerView exact revision on arXiv

Source-linked AI summary

A Deep Sequential Model for Discourse Parsing on Multi-Party Dialogues

Zhouxing Shi, Minlie Huang

arXiv:1812.00176v1cs.CLcs.AI

TL;DR

Multi-party dialogue parsing must account for non-adjacent discourse relations and shortcomings of pipelines that separate dependency prediction from structure construction. This paper scans EDUs sequentially, jointly predicting links and relations while building the discourse structure, and reports that its model significantly outperforms state-of-the-art baselines.

  • Problem

    Multi-party dialogues can have non-adjacent discourse relations, while prior pipeline parsers rely on local EDU-pair information and separate relation prediction from structure construction.

    Source: Introduction · pp. 1–2

  • Method

    The model scans EDUs sequentially, predicting each dependency link and relation while using local and global information to build the discourse structure incrementally.

    Source: Introduction · p. 2

  • Results

    The sequential model significantly outperforms all state-of-the-art baselines.

    Source: Conclusion and Future Work · p. 7

  • Takeaways & Limitations

    The paper reports that structured representations can improve dependency prediction and presents discourse structures as useful representations for other multi-party dialogue NLP tasks.

    Source: Conclusion and Future Work · p. 7

    The parser assumes EDU segmentations are preprocessed.

    Source: Introduction · p. 1

Abstract

from arXiv · show

Discourse structures are beneficial for various NLP tasks such as dialogue understanding, question answering, sentiment analysis, and so on. This paper presents a deep sequential model for parsing discourse dependency structures of multi-party dialogues. The proposed model aims to construct a discourse dependency tree by predicting dependency relations and constructing the discourse structure jointly and alternately. It makes a sequential scan of the Elementary Discourse Units (EDUs) in a dialogue. For each EDU, the model decides to which previous EDU the current one should link and what the corresponding relation type is. The predicted link and relation type are then used to build the discourse structure incrementally with a structured encoder. During link prediction and relation classification, the model utilizes not only local information that represents the concerned EDUs, but also global information that encodes the EDU sequence and the discourse structure that is already built at the current step. Experiments show that the proposed model outperforms all the state-of-the-art baselines.

Introduction

Multi-party dialogues can contain non-adjacent, non-projective discourse relations that adjacent-unit RST structures do not represent. The paper proposes a sequential model that predicts dependency links and relations while incrementally building the discourse structure, using local and global information.

Introduction

  • EDU segmentation is assumed to be preprocessed rather than performed by the proposed parser.The paper follows prior work in treating clause-level Elementary Discourse Units as the basic units for parsing.Source: Introduction · p. 1
  • Dependency-based parsing better fits multi-party dialogues, whose non-adjacent relations can form non-projective structures that RST does not allow.RST is designed for written text and allows relations only between adjacent discourse units.Source: Introduction · p. 1
  • Prior pipeline approaches score each EDU pair locally, then build the structure separately, so predictions cannot use the structure already constructed.Errors in dependency prediction can also degrade structure construction.Source: Introduction · pp. 1–2
  • The proposed model scans EDUs sequentially, predicting each link and relation while alternately constructing the discourse structure.Predictions use local information about the concerned EDUs and global information about the EDU sequence and the structure built so far.Source: Introduction · p. 2
  • The model’s prediction module uses local and global information, while its structured encoder and speaker-highlighting mechanism represent structure and speaker information.Global information includes the EDU sequence and the currently constructed structure.Source: Introduction · p. 2

Related Work

Prior discourse parsing work includes shallow-relation approaches and constituency-based RST methods, while this paper targets discourse structures with non-adjacent dependency relations. Earlier dependency parsers use separate scoring and decoding stages or incremental methods with structural restrictions.

Related Work

  • PDTB focuses on shallow discourse relations and ignores overall discourse structure, while RST constituency methods cannot represent non-adjacent relations in multi-party dialogue.The paper instead targets non-projective dependency structures in which relations may link non-adjacent EDUs.Source: Related Work · p. 2
  • Earlier dependency parsers estimated local EDU-pair relation probabilities before constructing structures with MST or ILP decoding.Muller et al. (2012) also tried A∗ without outperforming MST; the cited approaches did not involve the constructed structure in relation prediction.Source: Related Work · p. 2
  • Transition-based dependency parsers still underperformed Li et al. (2014)'s MST approach and were limited to projective structures.The paper contrasts these methods with a model that compares all preceding EDUs when selecting a parent and can predict non-projective structures.Source: Related Work · p. 2
  • Prior state-of-the-art parsers relied on hand-crafted features or external parsers, while neural networks had also been applied to discourse parsing and dialogue-act recognition.This paper uses hierarchical GRU encoders to compute discourse representations.Source: Related Work · p. 2

Methodology

The method incrementally parses dialogue by alternating parent selection, relation labeling, and structured-representation updates. Predictions use local and global representations, including speaker-conditioned discourse history.

Problem Definition

  • The task predicts links and relation types among dialogue EDUs, with a dummy root and a DAG constraint; the model outputs a dependency tree, a special type of DAG.The model assumes a tree despite fewer than 6.4% of dataset EDUs having multiple incoming relations, and can be extended to predict a more general DAG.Source: Problem Definition · pp. 2–3, §1 · p. 3

Model Overview

  • The model scans EDUs sequentially, alternating parent-link prediction, relation classification, and structured-state updates to build the discourse tree incrementally.For each EDU, it selects a preceding parent and relation type before moving to the next EDU.Source: §1 · p. 3, §2 · p. 3

Discourse Representations

Link Prediction and Relation Classification

  • Candidate-parent prediction uses local and global features, while a relation classifier labels the predicted link; softmax makes each parent's probability depend on all candidates.The candidate features include both EDUs' non-structured global representations and the parent's structured representation highlighting the current EDU's speaker.Source: Link Prediction and Relation… · pp. 4–5

Loss Function

  • Training minimizes negative log-likelihood for links and relations, using the gold parent rather than the predicted parent to score the gold relation type.The training parent is the earliest EDU linked to the current EDU, or the dummy root when none is linked.Source: Loss Function · p. 5

Experiments

The experiments evaluate the model on STAC dialogue data and compare it with decoding and neural baselines. The deep sequential model significantly outperforms all baselines, while structured representations and speaker highlighting improve its predictions.

Data Preparation

  • The STAC Corpus contains multi-party dialogue from an online game; after CDU elimination, training has 1,062 dialogues and testing has 111.The preprocessing replaces each CDU recursively with its earliest unit without incoming relations, rather than cloning links to every CDU unit.Source: Data Preparation · p. 5, Figure 4 · p. 5

Baselines

  • Baselines include MST and ILP decoders with local classifiers, neural variants of those decoders, and a neural greedy decoder.The neural variants use non-structured GRU representations.Source: Baselines · pp. 5–6

Implementation Details

  • The model uses 100-dimensional GloVe word vectors, 256-dimensional discourse representations, and SGD; the final setting uses separate networks for link prediction and relation classification.Dropout is 0.5, the mini-batch size is 4, and the initial learning rate is 0.1.Source: Implementation Details · p. 6

Results

  • The deep sequential model significantly outperforms all baselines; deep sequential parsing also beats neural baselines using MST or ILP decoding without complex decoding of its own.Sharing representations between link prediction and relation classification lowers performance.Source: Results · p. 6, Table 1 · p. 6

Effectiveness of the Structured Representations

  • The full model outperforms versions without structured representations and with random structures; disabling speaker highlighting causes a remarkable performance drop.Random structures slightly improve link prediction over the non-structured variant (71.8 vs. 71.0), but the full model performs better.Source: Effectiveness of the Structured… · pp. 6–7, Table 2 · p. 7

Case Study

  • For u4, the non-structured model predicts u1 instead of the gold parent u3, while the model with structured representations predicts u3 correctly.For u6, only the full model predicts the parent correctly; speaker highlighting surfaces a previous EDU from u6’s speaker.Source: Figure 5 · p. 7, Case Study · p. 7

Conclusion and Future Work

The paper presents a sequential model that predicts discourse links and relations while incrementally building multi-party dialogue structures. It reports that the model outperforms state-of-the-art baselines and that structured representations improve dependency prediction.

Conclusion and Future Work

  • The sequential model significantly outperforms all state-of-the-art baselines on multi-party dialogue discourse parsing.Source: Conclusion and Future Work · p. 7
  • The model predicts EDU links and relation types sequentially, using structured representations while incrementally constructing the discourse structure.Predicted relations are used to build the structure, whose structured representations are used in dependency prediction.Source: Conclusion and Future Work · p. 7
  • Structured representations are reported to improve dependency prediction, while the paper also demonstrates the benefit of discourse structures.Source: Conclusion and Future Work · p. 7
  • The authors identify applying the method to improve other NLP tasks for multi-party dialogues as future work.Source: Conclusion and Future Work · p. 7

Related papers

Loading 1812.00176v1…