Implicit Discourse Relation Classification via Multi-Task Neural Networks

• Identifying discourse relations can support downstream NLP tasks such as question answering and automatic summarization.Source: Introduction · p. 1
• Without explicit markers, implicit discourse relations are difficult to classify, despite outnumbering explicit relations in naturally occurring text.Source: Introduction · p. 1
• PDTB and RST-DT use different annotation frameworks, but their relations can be connected, while separate, limited corpora contribute to sparsity.For example, RST-DT Elaboration and Joint have similar senses to PDTB Expansion.Source: Introduction · p. 1
• The proposed system uses CNNs to represent sentence pairs in a low-dimensional latent space and combines corpus-specific tasks through shared and task-specific representations.The CNN representations replace sparse lexical features, and the tasks are trained simultaneously.Source: Introduction · pp. 1–2

Prerequisite

• The study combines PDTB, RST-DT, and raw text with connective words as discourse corpora for implicit relation classification.Source: Prerequisite · p. 2

PDTB

• PDTB contains 2,159 Wall Street Journal articles and organizes relations into four top-level classes, with further types and subtypes.The experiments focus on the four top-level classes.Source: PDTB · p. 2

RST-DT

• RST-DT represents texts as discourse trees; the study converts them to discourse dependencies and selects the 12 most frequent of 18 relation classes.This yields 19,681 relations.Source: RST-DT · p. 2

Raw Text with Connective Words

• The NYT corpus provides 40,000 sentence-pair relations extracted around 35 common connective words, but the resulting labels are noisy and not human-verified.The source contains over 1.8 million news articles.Source: Raw Text with Connective Words · p. 2

Motivation and Overview

• Related corpora contain similarly meaningful relations despite different annotation theories, motivating four related tasks: implicit and explicit PDTB classification, RST-DT classification, and connective prediction.Implicit PDTB classification is the main task; the other three are auxiliary tasks.Source: Motivation and Overview · p. 2

CNNs: Modeling Argument Pairs

• The CNN applies filters to paired word windows from both arguments, then dynamic pooling selects maxima from regions of the feature map to form a fixed-size representation.The pooled representation retains global structure, including positional information about relationships between argument beginnings.Source: CNNs: Modeling Argument Pairs · p. 3

Multi-Task Neural Networks: Classifying Discourse Relations

• Each task receives a unique argument-pair representation and a shared representation; concatenation and nonlinear transformation produce a task-specific representation for classification.The shared representation connects tasks, while separate CNNs allow task-specific hyperparameters such as different window sizes.Source: Multi-Task Neural Networks:… · pp. 3–4

Additional Features

• Task-specific surface features supplement the learned representations, including argument boundary words, parse-tree production rules, and whether two EDUs share a sentence.The listed features vary by task: boundary words are for Task 1, production rules for Tasks 1, 2, and 4, and same-sentence status for Task 3.Source: Additional Features · p. 4

Model Training

• The model trains classification tasks with cross-entropy loss and mini-batch SGD, selecting one task per epoch for task-specific updates.Different regulative ratios adjust task learning rates; auxiliary tasks use very small embedding ratios to limit changes to the word embeddings.Source: Model Training · p. 4

Datasets

• The experiments combine implicit and explicit PDTB relations, converted RST-DT relations, and NYT connective-word instances; the implicit PDTB task uses Sections 2–20 for training and 21–22 for testing.The RST-DT corpus uses 12 frequent coarse-grained relations and 19,681 instances; the NYT corpus has 40,000 sampled instances.Source: Datasets · pp. 4–5

Model Configuration

• Each task has task-specific CNN settings, while window size, pooling size, and filter count are shared for learning shared representations.The model uses 50-dimensional GloVe embeddings and is tuned based on main-task development performance.Source: Model Configuration · p. 5, Table 6 · p. 5

Evaluation and Analysis

• The main implicit PDTB evaluation is four-way classification, measured with per-class Precision, Recall, and F1, plus overall Accuracy and macro-averaged F1.Source: Evaluation and Analysis · p. 5

Analysis of Our Model

• Across all four tasks, the MTL system generally improved relation classification, with slight deterioration only for Comparison.Explicit-relation Task 2 had slight or negative effects except for Temporal; connective-word Task 4 helped Contingency, while RST-DT Task 3 improved Comparison.Source: Analysis of Our Model · pp. 5–6, Table 7 · p. 5

Comparison with Other Systems

• Accuracy rose from 52.82 to 57.27 and macro-averaged F1 from 37.65 to 44.98 with MTL versus STL, with both gains significant (p < 0.05).The system also achieved higher accuracy and macro-averaged F1 than the reported state-of-the-art system.Source: Comparison with Other Systems · p. 6, Table 8 · p. 6
• MTL was especially helpful for Contingency and Temporal, raising Temporary performance from 33.30 to 37.17; the system reached state-of-the-art performance on Expansion, Contingency, and Temporary.The authors suggest limited STL training data may explain the Temporary gain; Comparison benefited least.Source: Comparison with Other Systems · p. 6, Table 9 · p. 6

• Supervised approaches use surface, contextual, syntactic, dependency, and lexical features, with word pairs a major contributor in early PDTB classifiers.Later work addressed sparsity with feature aggregation or Brown clusters, and recursive neural networks represented arguments and entity spans.Source: Related Work · p. 6
• Semi-supervised methods exploit unlabeled data through feature cooccurrence or predicted connectives, while Hernault et al. (2010) report particular gains for infrequent relations.Source: Related Work · p. 6
• Prior multi-task work leveraged implicit and explicit discourse data, while neural multi-task systems also combined sequence-labeling tasks and query classification with web search.Source: Related Work · p. 7

• The paper identifies sparsity and argument representation as persistent problems in implicit discourse relation classification.Source: Conclusion · p. 7
• Its MTNN uses CNNs with dynamic pooling to model argument pairs and learns task-specific and shared representations across corpus-specific classification tasks.The authors describe these task connections as allowing tasks to optimize one another without bringing useless noise.Source: Conclusion · p. 7
• The authors report state-of-the-art performance and propose a future multi-task system based on syntactic trees to share structural information.Source: Conclusion · p. 7