Source-linked AI summary
An Incremental Parser for Abstract Meaning Representation
Marco Damonte, Shay B. Cohen, Giorgio Satta
TL;DR
AMR parsing combines several semantic subtasks, motivating parsers and evaluations that expose their component abilities. This paper develops a left-to-right, linear-time transition-based parser and proposes fine-grained metrics; it finds competitive overall performance, with stronger results on several component metrics. The approach has boundaries around reentrancy handling and the treatment of distinct reentrancy types.
Problem
AMR parsing includes multiple subtasks, but Smatch reduces their evaluation to one score that does not assess each subtask separately or weight error types for specific applications.
Method
The paper develops a transition-based AMR parser inspired by ARCEAGER and proposes metrics for evaluating AMR parsing subtasks separately.
Results
The parser is competitive with publicly available state-of-the-art systems, achieving on-par or better results on several proposed metrics despite not having the best Smatch score.
Takeaways & Limitations
Left-to-right transition-based techniques can perform AMR parsing in linear time, while fine-grained evaluation provides a more informative comparison across parsing subtasks.
Takeaways & Limitations
The transition strategy can make some reentrant edges unrecoverable, and AMR annotation collapses reentrancies that may be fundamentally different types.
Abstract
from arXiv · showhide
Meaning Representation (AMR) is a semantic representation for natural language that embeds annotations related to traditional tasks such as named entity recognition, semantic role labeling, word sense disambiguation and co-reference resolution. We describe a transition-based parser for AMR that parses sentences left-to-right, in linear time. We further propose a test-suite that assesses specific subtasks that are helpful in comparing AMR parsers, and show that our parser is competitive with the state of the art on the LDC2015E86 dataset and that it outperforms state-of-the-art parsers for recovering named entities and handling polarity.
1 Introduction
The paper introduces an incremental AMR parser inspired by dependency parsing and evaluates it with fine-grained metrics alongside Smatch. The system processes sentences left-to-right in linear time and is competitive with publicly available state-of-the-art parsers.
- AMR unifies shallow-semantic tasks such as named entity recognition, semantic role labeling, and co-reference resolution in a semantic representation.
- The paper positions greedy transition-based parsing as a promising AMR approach because AMR structures and dependency trees both use lexically anchored nodes and linguistic-relation edges.
- The proposed parser adapts the ARCEAGER dependency transition system to AMR while handling token-to-node mappings, non-projectivity, and reentrant nodes.
- The transition system processes sentences left-to-right with linear complexity, supporting incremental AMR construction.
- The authors propose metrics beyond Smatch to compare the distinct subtasks involved in AMR parsing.
- The parser is competitive with publicly available state-of-the-art parsers on several metrics.
2 Background and Notation
AMRs represent sentence meaning as aligned, labeled directed graphs that may include reentrancies and non-projective edges. The paper adapts transition-based parsing to these structures by aligning tokens with AMR subgraphs and relaxing tree-specific constraints.
- Notation: Most dataset AMRs are treated as directed acyclic graphs, although a few specific cases permit cycles.
- AMR Structures: AMRs are rooted, directed graphs with labeled nodes and edges, defined together with a sentence and a total alignment from graph nodes to sentence-token indices.Nodes represent concepts, while directed labeled edges represent relations; multi-word named entities are collapsed into single tokens during training and parsing.
- Alignment: The pre-image of a token under the alignment maps that token to a set of AMR nodes, inducing the AMR subgraph aligned with the token.The induced subgraph inherits its node and edge labeling functions from the full AMR graph.
- Transition-Based Parsing: Transition-based AMR parsing is motivated by dependency parsing’s linear-time left-to-right scan and intermediate-structure updates.
- Transition-Based Parsing: AMREAGER adapts ARCEAGER to parse AMR structures left-to-right, constructing edges when their head and dependent are available in the stack.The design targets incremental parsing while addressing AMR-specific structural differences from dependency trees.
- AMR-Specific Properties: AMR parsing must accommodate non-projectivity and reentrancy, unlike ordinary dependency trees, because AMR edges may cross and nodes may have multiple parents.The transition system therefore drops the dependency-parser constraint that prevents a dependent from acquiring additional parents.
- AMR-Specific Properties: 35% of the sentences contain at least one non-projective edge, despite non-projective edges forming a relatively small percentage of all AMR edges.
- Alignment: A sentence token may align to no nodes, one node, or multiple nodes, so concept identification is required to map tokens to AMR concepts.For example, “teacher” can yield teach-01 and person connected by an :ARG0 edge.
3 Transition system for AMR Parsing
AMREAGER adapts an arc-eager transition system to construct AMR graphs incrementally from left to right. Its transitions handle token-to-graph mappings, reentrancies, and non-projective structures while maintaining linear-time parsing.
- Configurations: The parser configuration consists of a stack of partially constructed AMR nodes, a buffer of unseen token indices, and the constructed edge set A.
- Token-to-graph mapping: The mapping a(i) assigns each token a graph fragment that may be empty, contain one concept node, or contain multiple nodes for a complex concept.
- Transitions: Shift consumes a buffer token, merges its fragment into the current graph, and pushes its root; LArc and RArc create labeled edges, with only LArc popping the dependent.
- Transitions: Reduce pops the top node and can recover a reentrant edge between sibling nodes, including patterns associated with control verbs.
- Design trade-offs: The design inherits arc-eager incrementality, constructing edges once their head and dependent are available, but LArc can prevent later recovery of some reentrant edges.
- Complexity: Linear time follows because Shift creates O(n) nodes, each node has at most three parents, and the total number of constant-time transitions is O(n).
4 Training the System
Training combines oracle-generated transition sequences with classifiers for transitions, reentrancies, concepts, and edge labels. Concept identification uses alignment-based phrase-table heuristics supplemented by deterministic hooks, while labeling is constrained by AMR rules.
- Training components: The system learns classifiers for the next transition, reentrancy creation after Reduce, concept identification during Shift, and edge labeling after arc transitions.
- Oracle: Training requires an oracle that maps each sentence and gold AMR graph to transition sequences maximizing graph overlap.
- Oracle: The shortest-stack static oracle avoids introducing mistakes when the configuration is correct and prefers transitions that minimize stack size.
- Oracle: The oracle selects Reduce after checking that no gold edge involving the top stack node remains constructible from the buffer.
- Transition classifier: The transition controller predicts among Shift, LArc, RArc, and Reduce using features extracted from oracle transition sequences and the current configuration.
- Transition classifier: The transition classifier uses word, part-of-speech, dependency, named-entity, and configuration features in a two-hidden-layer feed-forward neural network.
- Concept identification: Concept identification approximates each token's aligned AMR fragment with its most frequent training phrase-table mapping, while named entities and quantities use deterministic hooks.
- Edge labeling: Edge labeling uses a feed-forward neural network, whose development-set accuracy is 77%, followed by rules restricting labels to valid AMR choices and frame arguments.
5 Fine-grained Evaluation
The paper argues that Smatch alone cannot reveal how AMR parsers perform on distinct subtasks, so it proposes a fine-grained evaluation suite covering concepts, entities, polarity, roles, and graph properties.
- Evaluation motivation: Smatch reduces AMR evaluation to one score, although parsing quality spans multiple subtasks with different importance.The paper notes that concept detection, edge detection, word-sense errors, and verb errors may matter differently for particular NLP applications.
- Additional metrics: The suite also reports Unlabeled, No WSD, NP-only, Reentrancy, and :ARG-only Smatch scores for graph structure, sense, noun phrases, reentrant edges, and semantic roles.Unlabeled removes edge labels; No WSD strips PropBank frame suffixes; NP-only evaluates extracted simple noun phrases; Reentrancy isolates reentrant edges; :ARG-only targets SRL.
- Subtask metrics: Concepts measures predicted concept identification with an F-score because missing concepts prevent recovery of their incident edges.The authors describe concept identification as a critical component of parsing accuracy.
- Subtask metrics: Named Ent. and Wikification score :name and :wiki edges, while Negations scores concepts connected by the :polarity role.These metrics use simple F-scores because their edges do not involve variable names.
- Illustrative comparison: The two example parses expose different strengths: Parse 1 is stronger for semantic role labeling, whereas Parse 2 is optimal for named entity recognition and unlabeled scoring.Table 6 evaluates the two parses from Figure 5 using the proposed suite.
6 Experiments
The experiments compare AMREAGER with JAMR and CAMR on LDC2015E86. Although AMREAGER does not lead on Smatch, it achieves the best results on several fine-grained metrics and supports incremental linear-time parsing.
- Experimental setup: AMREAGER is evaluated against JAMR and CAMR on the LDC2015E86 dataset.CAMR has quadratic worst-case complexity, though it is linear in practice.
- Results: AMREAGER achieves the best results for Unlabeled and Concept and outperforms the other parsers for Named Ent. and Negations.Its Reentrancy score is also close to the best system, and the Reduce transition is critical for reentrancies caused by control verbs.
- Results: AMREAGER does not obtain state-of-the-art Smatch, while its relatively high Unlabeled score suggests difficulty labeling arcs.Its concept-identification score is on par with the best competing result.
- Error analysis: Named-entity and wikification scores depend heavily on named-entity recognizer predictions, while polarity alignment uses post-processing for negative words.The post-processing aligns the AMR minus constant with words such as “not,” “illegitimate,” and “asymmetry.”
- Conclusion: No parser conclusively outperforms all others on every metric, but AMREAGER offers worst-case linear complexity and incremental left-to-right parsing.The authors connect these properties to real-time applications and investigating how sentence meaning is built incrementally.
7 Related Work
Prior AMR work includes staged, logical-form, learning-to-search, and transition-based approaches. Transition-based methods are attractive because graph-based AMR parsing can be computationally expensive, but AMR remains complex because it combines many subtasks.
- Earlier approaches: Early AMR parsers separated concept identification from relation identification or converted other logical representations into AMR graphs.Approaches included sequence labeling with constrained optimization, first-order logic treatment, and conversion from a logical-form parser.
- Transition-based approaches: Wang et al. use two transition-based stages: dependency parsing followed by AMR generation.This framework illustrates how transition-based parsing can be applied to AMR through an intermediate dependency structure.
- Restricted-scope approaches: Sawai et al. restrict evaluation to simple noun phrases, producing mostly tree-shaped AMRs that are easier to handle than full-sentence graphs.Their restrictions exclude nested noun phrases, pronouns, and named entities.
- Task scope: AMR parsing is complex because it includes subtasks such as named entity recognition, co-reference resolution, and semantic role labeling.These subtasks are part of the broader challenge of constructing AMR graphs for complete sentences.
- Semantic representations: AMR is one of several semantic representations, alongside Boxer, CCG, and UCCA.The paper situates AMR within a broader landscape of meaning representations for natural-language sentences.
8 Conclusion
The paper presents a left-to-right, linear-time AMR transition system trained with feed-forward neural networks and argues that fine-grained evaluation is more informative than Smatch alone. Its parser is competitive with public state-of-the-art systems on several metrics, despite not leading on Smatch.
- Contribution: The parser builds AMR graphs in linear time while processing sentences left-to-right with feed-forward neural networks.The approach is inspired by dependency-parsing techniques.
- Evaluation: A collection of subtask metrics is less informative than Smatch alone for evaluating the entire parsing process.The authors argue that moving beyond a single metric can help identify parser strengths and weaknesses.
- Results: The left-to-right transition system is competitive with publicly available state-of-the-art parsers and is on par or better on several proposed metrics.The parser does not outperform the best baseline in Smatch score.