Source-linked AI summary
Simpler but More Accurate Semantic Dependency Parsing
Timothy Dozat, Christopher D. Manning
TL;DR
Semantic dependency parsing seeks richer graph-structured representations than syntactic trees, but producing these graphs is more difficult. This paper minimally extends a simple syntactic parser to generate semantic dependency graphs and achieves state-of-the-art performance, with further gains from richer input representations.
Problem
Semantic dependency graphs capture more linguistic relationships than tree-structured syntactic representations, but are harder to extract automatically.
Method
The paper extends a simple syntactic dependency parser with edge prediction and labeling modules to train on and produce graph-structured dependencies.
Results
The minimally extended parser achieves state-of-the-art performance, while lemma and character-level representations further improve accuracy.
Takeaways & Limitations
A simple graph-based parser can handle dependency graphs with different structural restrictions with only small changes, without obviously hurting accuracy.
Takeaways & Limitations
The analysis suggests performance may be affected by frequent semantically vacuous function words receiving disproportionate representation in the loss or LF1 score.
Abstract
from arXiv · showhide
While syntactic dependency annotations concentrate on the surface or functional structure of a sentence, semantic dependency annotations aim to capture between-word relationships that are more closely related to the meaning of a sentence, using graph-structured representations. We extend the LSTM-based syntactic parser of Dozat and Manning (2017) to train on and generate these graph structures. The resulting system on its own achieves state-of-the-art performance, beating the previous, substantially more complex state-of-the-art system by 0.6% labeled F1. Adding linguistically richer input representations pushes the margin even higher, allowing us to beat it by 1.9% labeled F1.
1 Introduction
Semantic dependency parsing uses richer graph structures to capture relationships that tree-based syntactic dependencies miss. This paper extends a simple syntactic parser to generate such graphs and evaluates richer input representations.
- Semantic dependency schemes use graph-structured representations to capture more linguistic information than strict syntactic trees.This can provide more useful information to downstream tasks, while making automatic extraction harder.
- Syntactic dependency parsing is widely used to extract low-level word relationships for natural language understanding.
- Tree-structured syntactic dependencies may omit useful relationships, such as Mary being the subject of both want and buy.A tree representation can encode only one of those relationships.
- The paper extends Dozat and Manning’s syntactic parser to train on and produce graph-structured semantic dependencies.The authors also examine lemma embeddings, character-level word embeddings, and architectural design choices.
2 Background
Semantic dependency formalisms represent meaning-oriented relationships as directed acyclic graphs rather than ordinary dependency trees. Prior systems differ in how they score and constrain graph structures, motivating simpler graph-based approaches.
- Semantic dependency formalisms: The SemEval task introduced DM, PAS, and PSD semantic dependency formalisms for graph-structured annotation.These formalisms were applied to the Penn Treebank.
- Semantic dependency formalisms: Semantic dependencies target relationships such as agent and patient, while syntactic dependencies generally encode functional relationships such as subject and object.
- Semantic dependency formalisms: The SemEval schemes use directed acyclic graphs, allowing function words to serve as heads without lengthening paths between content words.
- Previous parsing systems: Peng et al.’s system scores predicates, ordered arcs, and labeled arcs, then applies hard constraints because independently scored decisions can form invalid graphs.
- Previous parsing systems: Dozat and Manning’s factorized parser makes edge decisions before labeling and uses an MST algorithm to avoid invalid trees.
- Previous parsing systems: Wang et al. adapt a transition system and stack-LSTM architecture to generate arbitrary directed acyclic graphs.
3 Approach
The approach factorizes semantic dependency parsing into edge detection and edge labeling over BiLSTM representations. It requires only small changes to the original syntactic parser while supporting graph-structured outputs.
- Basic architecture: The parser predicts whether each directed word pair has an edge and then assigns the best label to each predicted edge.This factorization separates graph structure prediction from label prediction.
- Basic architecture: Word and POS tag embeddings are concatenated and processed by a multilayer bidirectional LSTM to produce richer word representations.The datasets provide the POS tags used by the system.
- Basic architecture: Feedforward networks split recurrent states into head and dependent representations before bilinear or biaffine classifiers score edges and labels.The split reduces recurrent size to help avoid classifier overfitting without weakening LSTM capacity.
- Training and decoding: Sigmoid cross-entropy scores candidate edges independently, while softmax cross-entropy selects the highest-scoring label for each predicted edge.Training sums the module losses and backpropagates to the labeler through gold edges.
- Training and decoding: A tunable interpolation constant λ balances the edge and label losses when one module’s loss overwhelms the other.
- Graph generality: Removing the maximum spanning tree algorithm and replacing softmax with sigmoid cross-entropy are the only changes needed for fully graph-structured semantic output.
- Graph generality: The system can support graph-structured dependency schemes, including Universal Dependencies enhanced dependencies that allow cyclic graphs.
- Richer input representations: Character-level word embeddings and lemma information are explored as additional low-level lexical representations for improving semantic dependency parsing.
4 Results
The parser achieves strong performance with extensive hyperparameter tuning and benefits from richer lexical representations, while architectural variants show mostly modest effects. Results also reveal dataset-specific behavior, especially limited gains from lemma and character embeddings on PAS.
- Hyperparameters: The final system uses extensively tuned hyperparameters, with a three-layer BiLSTM and embeddings including word, GloVe, POS, and character-level representations.The basic system excludes character embeddings and lemmas during tuning; the final configuration includes richer inputs.
- Performance: About 300 sequences/second are used for training and about 1,000 sequences/second for parsing on an Nvidia Titan X.
- Performance: The fully factorized basic system outperforms Peng et al.’s single-task baseline and their more complex multitask approach.Adding either character-level word embeddings or lemma embeddings improves performance further, and using both generally improves it more.
- Performance: PAS shows no substantial benefit from lemma or character embeddings, possibly because 94% LF1 is near the dataset’s performance ceiling.The passage presents this as a possible explanation rather than a confirmed cause.
- Variations: The basic system significantly outperforms variants without hidden layers and a parser with a diagonal tensor, while ReLU offers no clear advantage.The basic system beats the no-hidden-layer edge and label classifiers and the diagonal-tensor parser in rank-sum tests.
5 Discussion
A simple extension of a syntactic dependency parser can produce graph-structured semantic dependencies while achieving state-of-the-art performance. Richer lexical inputs improve accuracy without requiring comparable increases in system complexity, and the graph-based design adapts across dependency graph types.
- Discussion: A minimally extended syntactic parser achieves state-of-the-art semantic dependency parsing without a complex multitask decoding algorithm.The authors note that such decoding could potentially push performance even higher.
- Discussion: Lemma or subtoken information provides an accuracy boost comparable to that from drastically increasing system complexity.
- Discussion: A high-performing graph-based parser adapts to projective trees, non-projective trees, and directed graphs with only small changes and no obvious accuracy loss.The authors contrast this with transition-based parsers, which require new transition sets or data structures for arbitrary graphs.