Source-linked AI summary
Toward Abstractive Summarization Using Semantic Representations
Fei Liu, Jeffrey Flanigan, Sam Thomson, Norman Sadeh, Noah A. Smith
TL;DR
Abstractive summarization requires generating new textual summaries, motivating semantic approaches beyond extractive or compressive methods. The paper parses text into AMR graphs, transforms them into a summary graph, and evaluates the graph-transformation framework with promising results. Its experiments also expose parsing and graph-decoding constraints that guide future work.
Problem
Abstractive summarization must generate textual summaries de novo, but extractive and compressive techniques do not suffice for some textual and non-textual content.
Method
The framework parses sentences into AMR graphs, transforms them into a single summary graph through structured prediction, and uses a simple bag-of-words generator.
Results
The approach produces promising results; ramp loss yields substantial gains, while hinge versus perceptron has little effect across evaluations.
Takeaways & Limitations
The results suggest that semantic graph transformation is a viable direction for abstractive summarization and that better AMR parsing may benefit graph summarization.
Takeaways & Limitations
The framework assumes text input, uses a simple bag-of-words generator rather than full AMR-to-text generation, and faces inefficient decoding on large source graphs.
Abstract
from arXiv · showhide
We present a novel abstractive summarization framework that draws on the recent development of a treebank for the Abstract Meaning Representation (AMR). In this framework, the source text is parsed to a set of AMR graphs, the graphs are transformed into a summary graph, and then text is generated from the summary graph. We focus on the graph-to-graph transformation that reduces the source semantic graph into a summary graph, making use of an existing AMR parser and assuming the eventual availability of an AMR-to-text generator. The framework is data-driven, trainable, and not specifically designed for a particular domain. Experiments on gold-standard AMR annotations and system parses show promising results. Code is available at: https://github.com/summarization
1 Introduction
The paper proposes an abstractive summarization framework that transforms semantic representations, specifically AMR graphs, into summary graphs before text generation. Its central study focuses on learning the graph transformation stage within a three-step pipeline.
- Contribution: The paper presents the first feasibility study of abstractive summarization based on transformations of semantic representations such as AMR.The approach builds on an AMR treebank and the automatic JAMR parser.
- Framework: The framework parses input sentences into AMR graphs, transforms them into one summary AMR graph, and generates text from that graph.The paper focuses on the graph-transformation stage and uses a simple bag-of-words generator for evaluation.
- Graph transformation: The graph summarizer merges coreferent concepts, connects sentence roots through a dummy ROOT node, and optionally expands the graph.These operations produce a connected source graph from which summary nodes and arcs are selected.
- Graph transformation: The selected summary subgraph is intended to condense the most salient semantic content of the source graph.The source graph is assembled before selecting nodes and arcs for the summary graph.
- Evaluation: Experiments evaluate graph transformation quality, selected summary concepts, learning variations, and oracle upper bounds using ROUGE-1.The evaluation uses gold-standard AMR annotations and system parses.
2 Background: Abstract Meaning Representation and JAMR
AMR represents sentence meaning as a rooted, directed, acyclic graph whose labeled nodes encode concepts and whose labeled edges encode relations. JAMR is the statistical parser used to convert document sentences into AMR graphs for the framework.
- AMR: AMR represents a whole sentence as a rooted, directed, acyclic graph.This representation provides the semantic structure used by the summarization framework.
- AMR: AMR nodes are labeled with concepts and edges are labeled with relations.Concept labels may include English words, PropBank predicates, and special keywords.
- AMR: AMR uses PropBank rolesets and core semantic relations alongside relations such as location, mode, name, time, and topic.The representation contains approximately 100 relations.
- JAMR: JAMR converts input document sentences into AMR graphs and achieved 63% F-score on the test dataset.The experiments compare JAMR outputs with gold-standard annotations to assess parsing-error effects.
- JAMR: JAMR also aligns source-word spans with fragments of predicted graphs, supporting the framework’s text-generation step.These alignments connect concept nodes to word spans used by the simple generator.
3 Dataset
The dataset uses proxy reports derived from single newswire articles, with gold-standard AMR annotations for both inputs and summaries. This supports statistical graph summarization and separates graph-transformation errors from parsing errors.
- Dataset purpose: Gold-standard AMR annotations allow the study to separate graph-summarization errors from errors introduced during AMR parsing.This separation helps assess whether the approach merits further investment.
- Dataset construction: The dataset is drawn from the AMR Bank’s proxy-report section and consists of single-document summarization examples.Each proxy report is based on one English Gigaword newswire article and rewritten to approximate analyst-report style.
- Dataset scope: Traditional DUC and TAC multi-document summarization datasets lack gold-standard AMR annotations.This motivates using the proxy-report section of the AMR Bank for the study.
- Dataset statistics: Table 1 reports averaged dataset statistics across the official train, development, and test splits, with one training document removed.The “Expand” statistic counts edges after graph expansion.
- Annotations: All source and summary sentences in the dataset have gold-standard AMR annotations.These annotations enable training and evaluation of the graph-summarization stage.
4 Graph Summarization
The framework constructs a connected source graph from sentence-level AMR graphs, then predicts a concise summary subgraph using a structured model with graph and linguistic constraints.
- 4.1 Source Graph Construction: Sentence AMR graphs are merged by collapsing selected fragments, merging identical concepts, and connecting sentence roots to a dummy ROOT node.Flat date-entity and named-entity fragments receive concatenated labels; multiple edges between concept pairs are merged into one unlabeled edge.
- 4.1 Source Graph Construction: The resulting source graph contains 144 nodes and 221 edges on average and is intended to cover the edges of gold-standard summary graphs.Coverage is evaluated for labeled and unlabeled edges, with unlabeled matching based on endpoint concept labels.
- 4.1 Source Graph Construction: Sentence-level graph expansion increases average edges by a factor of 15 to 3,292, while document-level expansion is computationally prohibitive.Sentence-level expansion is used experimentally because it can add semantically useful edges without the infeasibility reported for document-level expansion.
- 4.2 Subgraph Prediction: Subgraph prediction selects nodes and edges that balance important information, meaning preservation, brevity, and fluent language under a factorized scoring model.The model uses node and edge features with empirically estimated coefficients and decodes a connected subgraph through integer linear programming.
- 4.2.1 Decoding: The ILP enforces endpoint inclusion, connectivity through single-commodity flow, and a preliminary tree structure that permits at most one incoming edge per node.The tree restriction excludes most AMR reentrancies, which account for about 5% of dataset edges.
- 4.2.2 Parameter Estimation: Parameter estimation compares structured perceptron, hinge, and ramp losses because gold-standard summary graphs may fall outside the model’s source-graph hypothesis space.The framework uses paired source and summary graphs to learn node and edge model coefficients.
5 Generation
The framework generates summaries from predicted AMR subgraphs by mapping each concept node to its most frequently aligned word span. The resulting words are unordered, making the output suitable for ROUGE-1 but not natural-language summary evaluation.
- 5 Generation: The generator maps each predicted subgraph concept node to its most frequently aligned word span and concatenates the resulting words.JAMR supplies the word-to-graph alignments used by this heuristic.
- 5 Generation: The generated words appear in no particular order, so the output is a bag of words rather than a natural-language summary.
- 5 Generation: The bag-of-words output supports evaluation with unigram-based ROUGE-1 despite lacking natural-language fluency.
6 Experiments
The experiments evaluate subgraph prediction and bag-of-words summarization under gold-standard and JAMR parses, with oracle comparisons and graph-expansion variants. Ramp loss yields substantial gains, while parsing errors degrade graph prediction and expansion has limited system impact.
- 6 Experiments: The study evaluates node and edge prediction against gold-standard summary AMR graphs, reporting node precision, recall, and F1 plus edge F1.
- 6 Experiments: Oracle node prediction reaches about 80% with gold-standard AMR annotations and 70% with JAMR output, while edge prediction reaches 52.2% and 31.1%, respectively.With graph expansion, edge oracle performance increases to 64% and 46.7%, respectively.
- 6 Experiments: System summaries are evaluated against references with ROUGE-1 after converting predicted subgraphs into bags of aligned word spans.
- 6 Experiments: JAMR parses substantially degrade edge prediction and significantly degrade concept prediction, yet slightly improve ROUGE-1 under the bag-of-words generator.ROUGE-1 depends only on concept prediction in this evaluation and is unaffected by edge prediction.
- 6 Experiments: Ramp loss produces substantial gains across conditions and evaluations, whereas the cost-aware hinge loss has little effect relative to the perceptron.
- 6 Experiments: Graph expansion marginally affects system performance and slightly hurts edge prediction; with JAMR and ramp loss, node and edge scores are 50.7% and 19.0% expanded versus 51.5% and 20.0% unexpanded.
7 Related and Future Work
The paper situates its AMR-based approach among extractive, abstractive, and graph-based summarization methods. It identifies scalability, full text generation, richer prediction, and evaluation as future work.
- 7 Related Work: Most competitive summarization systems are extractive, selecting representative input sentences and often compressing them to fit a length budget.
- 7 Related Work: Prior abstractive approaches construct summaries from graph paths, discourse parse trees, or recombined dependency parse trees.
- 7 Related Work: Earlier graph-based methods build graphs from logical forms or surface-term co-occurrence, whereas this work uses semantic representations.
- 7 Future Work: The source graph can reach ten times or more the size of a sentence dependency graph, motivating more efficient or approximate decoding algorithms.
- 7 Future Work: Future directions include joint subgraph and edge-label prediction, a full AMR parsing-to-generation pipeline, and an evaluation metric suited to abstractive summarization.
- 7 Future Work: The framework is intended to benefit domains including books, audio/video segments, and legal texts.
8 Conclusion
The paper introduces a statistical abstractive summarization framework driven by AMR, centered on transforming input semantic graphs into a single summary semantic graph.
- The framework uses structured prediction to transform semantic graphs of the input into one summary semantic graph.