Source-linked AI summary
AMR Parsing as Sequence-to-Graph Transduction
Sheng Zhang, Xutai Ma, Kevin Duh, Benjamin Van Durme
TL;DR
AMR parsing must handle graph reentrancy, absent gold alignments, and limited labeled data. The paper proposes an aligner-free attention-based sequence-to-graph parser, which achieves the best reported SMATCH scores on AMR 2.0 and AMR 1.0. The approach uses indexed-tree prediction to represent reentrancy and recovers graphs by merging indexed nodes.
Problem
AMR parsing is challenged by reentrancy, missing gold alignments, and relatively limited labeled data.
Method
The paper proposes an aligner-free attention-based model that treats AMR parsing as sequence-to-graph transduction using indexed-tree prediction and graph reconstruction.
Results
The parser achieves the best reported SMATCH scores: 76.3% F1 on LDC2017T10 and 70.2% F1 on LDC2014T12.
Takeaways & Limitations
The model achieves the best performance on two AMR corpora and is effectively trained with limited labeled AMR data.
Takeaways & Limitations
BERT parameters are fixed during training because of limited GPU memory, leaving BERT fine-tuning for future work.
Abstract
from arXiv · showhide
We propose an attention-based model that treats AMR parsing as sequence-to-graph transduction. Unlike most AMR parsers that rely on pre-trained aligners, external semantic resources, or data augmentation, our proposed parser is aligner-free, and it can be effectively trained with limited amounts of labeled AMR data. Our experimental results outperform all previously reported SMATCH scores, on both AMR 2.0 (76.3% F1 on LDC2017T10) and AMR 1.0 (70.2% F1 on LDC2014T12).
1 Introduction
AMR parsing converts text into graph-based sentence semantics, but reentrancy, missing gold alignments, and limited labeled data complicate learning. The proposed aligner-free sequence-to-graph parser addresses these challenges and achieves the best reported SMATCH scores on two benchmarks.
- AMR parsing transduces natural language text into graph-based sentence-level semantics.
- Reentrancy allows one concept to participate in multiple relations, producing graphs rather than trees.
- Missing gold alignments between graph concepts and input words limit reliance on explicit alignments for training-data generation.
- The proposed attention-based parser treats AMR parsing as sequence-to-graph transduction and is aligner-free.
- 76.3% F1 on LDC2017T10 and 70.2% F1 on LDC2014T12 are the best reported SMATCH scores on the two benchmarks.
2 Another View of Reentrancy
The paper converts reentrant AMR graphs into indexed trees, enabling tree-based prediction while preserving enough information to reconstruct the original graph.
- AMR uses graphs because a concept can participate in multiple semantic relations, such as “victim” serving both ARG0 and ARG1 of “help-01.”
- This representation allows the parser to leverage efficient tree-based algorithms and structured prediction methods.
- Reentrant nodes are duplicated to convert an AMR graph into a tree, and each duplicate receives the original node’s index.
- Merging identically indexed nodes and unioning their edges recovers the original AMR graph.
3 Task Formalization
The parser predicts an indexed AMR tree in two stages: sequential node prediction followed by constrained edge prediction. It then merges identically indexed nodes to recover the standard AMR graph.
- The two-stage process consists of node prediction and edge prediction over an indexed AMR tree.
- Node Prediction: Given sentence words, node prediction sequentially decodes a node list and deterministically assigns node indices.
- Node Prediction: Multiple occurrences of a node are allowed in the list and receive the same index.
- Node Prediction: Training obtains reference nodes and indices through a pre-order traversal of the reference AMR tree.
- Edge Prediction: Edge prediction searches for the highest-scoring valid tree over the predicted nodes while respecting their index constraints.
- Edge Prediction: The resulting parse tree is converted back into a standard AMR graph by merging identically indexed nodes.
4 Model
The model parses AMR in two jointly learned stages: an extended pointer-generator predicts indexed nodes, then a deep biaffine classifier predicts edges and labels. It combines source and target copying, attention, contextual features, and constrained graph decoding.
- Model architecture: The parser jointly learns node prediction with an extended pointer-generator and edge prediction with a deep biaffine classifier.These modules correspond to the model’s two-stage AMR parsing process.
- Node prediction: The pointer-generator can generate vocabulary nodes, copy words from the input, or copy previously generated target-side nodes.A soft switch uses source-copy, target-copy, and generation probabilities.
- Node prediction: Encoder word representations concatenate GloVe, BERT, POS, anonymization, and CharCNN features, with BERT subword states average-pooled into word embeddings.Named-entity spans are anonymized during preprocessing to reduce sparsity.
- Node prediction: The decoder combines previous-node embeddings with prior attention information, using attention to compute vocabulary, target-copy, source-copy, and generation distributions.Node indices distinguish copied or repeated surface forms and are assigned deterministically.
- Edge prediction: The deep biaffine classifier uses pointer-generator decoder states to score directed edge existence and edge labels without re-encoding AMR nodes.The classifier factorizes edge prediction into head selection and label prediction.
- Training: Training jointly minimizes negative log likelihood for nodes, edge heads, and edge labels, with coverage loss penalizing repetitive nodes.Coverage is computed from accumulated source-attention distributions.
- Prediction: At prediction time, node lists use greedy or beam search, while edges use an adapted Chu-Liu-Edmonds algorithm to construct a maximum spanning tree.A dummy root ensures every node has a head, and same-index edges are excluded before recovering the AMR graph.
5 Related Work
AMR parsing approaches span alignment-based, transition-based, grammar-based, and attention-based methods, with differing dependencies on alignments, external resources, and additional data. This work’s attention-based approach avoids explicit alignments and data augmentation while training effectively on limited labeled AMR data.
- AMR parsing approaches include alignment-based, transition-based, grammar-based, and attention-based methods.
- Alignment-based parsers use alignment-derived features, while many transition-based parsers require pretrained aligners to generate training data.
- The proposed approach requires no explicit alignments and implicitly learns a source-side copy mechanism through attention.
- Grammar-based approaches leverage external semantic resources and induce CCG-based or SHRG-based grammars for converting logical forms into AMRs.
- Attention-based Seq2Seq parsers face limited labeled AMR data, whereas this approach trains effectively with limited labeled data and no data augmentation.
6 AMR Pre- and Post-processing
The preprocessing pipeline anonymizes entities, removes senses, lemmatizes input, and adds POS tags, while post-processing restores senses and wiki links and adds polarity attributes.
- Named entities and other entities are anonymized to reduce sparsity.
- Input sentences are lemmatized and augmented with POS tags, while senses are removed before parsing.
- Post-processing assigns the most frequent node sense, using -01 when a sense is unseen.
- An entity linker restores wiki links through the DBpedia Spotlight API, and training-derived rules add polarity attributes.
7 Experiments
Experiments on AMR 2.0 and AMR 1.0 evaluate the parser's benchmark performance, components, node sources, and design choices. The model achieves strong SMATCH results, while ablations identify copying and node prediction as important factors.
- Setup: 19 hours on AMR 2.0 and 7 hours on AMR 1.0 were required for full-model training, using two GeForce GTX TI-TAN X GPUs.
- Ablation Study: Target-side copy improves reentrancy performance, while removing source-side copy causes the largest ablation drop.Disabling target-side copy reduces the reentrancy subtask score to 38.4%; source-side copy reduces sparsity from open-class vocabulary entries.
- Ablation Study: Node prediction is the key target for future improvement because perfect node prediction would substantially raise the final SMATCH score.
- Node Sources: 47.6% of reference nodes come from source-side copy, which has the highest precision and recall among the three node sources.Vocabulary generation accounts for 43.8% and target-side copy for 8.6%; target-side copy has relatively low recall.
- Design Choices: Pre-order traversal with alphanumerical child sorting outperforms alignment-based and other tested traversal strategies.Average pooling also performs slightly better than max pooling for BERT word-level representations.
8 Conclusion
The paper introduces an attention-based AMR parser with novel components in a transductive setting and reports the best performance on two AMR corpora.
- The model combines attention-based AMR parsing with novel components in a transductive setting beyond typical NMT systems.
- The model achieves the best performance on two AMR corpora.
- Future work will extend the model to other semantic parsing tasks and cross-lingual semantic parsing.
A.1 AMR Pre- and Post-processing
The preprocessing pipeline removes selected AMR attributes, anonymizes entity subgraphs and corresponding text spans, and restores graph information during post-processing.
- A.1 AMR Pre- and Post-processing: Preprocessing removes senses, wiki links, and polarity attributes from AMR graphs.
- A.1 AMR Pre- and Post-processing: Named-entity and other entity subgraphs are replaced with indexed tokens such as HIGHWAY 0 and DATE 0.
- A.1 AMR Pre- and Post-processing: Training maps anonymized subgraphs to text spans and maps Stanford CoreNLP NER tags to AMR entity types for prediction.
- A.1 AMR Pre- and Post-processing: At test time, entity spans are normalized using learned fine-grained mappings or fallback coarse-grained NER tags.
- A.1 AMR Pre- and Post-processing: Post-processing deterministically regenerates anonymized subgraphs and adds senses, wiki links, and polarity attributes.
A.2 Side-by-Side Examples
Side-by-side ablations compare the full model with versions lacking source-side copy, target-side copy, coverage loss, or BERT embeddings.
- A.2 Side-by-Side Examples: Without source-side copy, the prediction becomes totally different and inaccurate in the example.
- A.2 Side-by-Side Examples: Without target-side copy, the model generates a new indexed country node instead of copying the existing node.
- A.2 Side-by-Side Examples: Without coverage loss, the model generates the repetitive modifier “magnificent” instead of correctly predicting “solemn”.
- A.2 Side-by-Side Examples: The examples also include a full-model comparison without BERT embeddings.