Source-linked AI summary
Abstract Meaning Representation for Multi-Document Summarization
Kexin Liao, Logan Lebanoff, Fei Liu
TL;DR
Multi-document abstractive summarization remains underexplored, and obtaining parallel training data can be costly. This paper uses AMR summary graphs in a full multi-document summarization pipeline and reports that its framework outperforms other abstractive systems on selected ROUGE scores across two datasets.
Problem
Multi-document abstractive summarization remains challenging, and parallel training data for the task can be costly to obtain.
Source: §1 · p. 1Method
The framework selects related source sentences, plans AMR summary graphs from them, and realizes each graph as a natural-language sentence.
Source: §1 · p. 2, §3.2 · p. 5Results
The framework outperforms all other abstractive systems on R-1 and R-SU4 F-scores on both DUC-04 and TAC-11, while AMRSumm-VSM has the highest R-2 F-scores.
Source: §5.2 · p. 8Takeaways & Limitations
The authors conclude that AMR is a powerful semantic formalism with potential for abstractive summarization.
Source: §6 · p. 9The authors suggest that comprehensive AMR graphs may be too detailed for summarization and that a reduced graph representation could increase algorithm robustness.
Source: §5.3 · p. 9
Abstract
from arXiv · showhide
Generating an abstract from a collection of documents is a desirable capability for many real-world applications. However, abstractive approaches to multi-document summarization have not been thoroughly investigated. This paper studies the feasibility of using Abstract Meaning Representation (AMR), a semantic representation of natural language grounded in linguistic theory, as a form of content representation. Our approach condenses source documents to a set of summary graphs following the AMR formalism. The summary graphs are then transformed to a set of summary sentences in a surface realization step. The framework is fully data-driven and flexible. Each component can be optimized independently using small-scale, in-domain training data. We perform experiments on benchmark summarization datasets and report promising results. We also describe opportunities and challenges for advancing this line of research.
1 Introduction
Multi-document abstractive summarization remains challenging, and heuristic content representations make summary generation difficult. This paper explores AMR as a content representation for a full multi-document pipeline and reports promising benchmark results.
1 Introduction
- Multi-document abstractive summarization remains challenging amid limited training data, while heuristic semantic units can make generating summary text difficult.The authors identify a need for a principled semantic representation that supports both source condensation and summary generation.Source: §1 · p. 1
- AMR represents sentence meaning as a rooted, directed, acyclic graph of concepts and semantic relations, abstracting from surface strings and syntax.Its representation is language-neutral and not designed for a particular domain.Source: §1 · p. 1
- The framework selects related source sentences, derives summary graphs through content planning, and realizes each graph as a natural-language sentence.Its components can be optimized independently with small-scale, in-domain data, reducing reliance on large-scale parallel training data.Source: §1 · pp. 1–2
- The paper investigates AMR for multi-document summarization and presents a full pipeline, extending beyond Liu et al.'s (2015) single-document pilot.The authors also study the impact of the AMR parser and generator and compare source-sentence selection strategies.Source: §1 · p. 2
- The authors report promising benchmark results and describe the framework as flexible, with independently optimizable components and opportunities and challenges for future work.The experiments contrast the approach with state-of-the-art baselines, including pointer-generator networks (See et al., 2017).Source: §1 · p. 2
2 Related Work
Prior abstractive summarization research has focused largely on single documents, while multi-document training data can be costly and content representation remains a challenge. The paper situates AMR as a domain-flexible semantic representation amid prior AMR applications and text–graph research.
2 Related Work
- Neural abstractive summarization studies have focused primarily on short and single-document tasks (Nallapati et al., 2016), while multi-document parallel data is costly to obtain.The passage describes a need to investigate approaches that require less data.Source: §2 · p. 2
- Earlier NLG approaches select salient text units, arrange them in compact forms such as templates, and synthesize natural-language text, but lack a principled content representation.The paper studies AMR as a data-driven, domain-flexible representation for condensing documents to summary graphs and generating sentences from them.Source: §2 · pp. 2–3
- AMR has been explored in downstream applications, while research on mapping sentences to graphs and generating sentences from graphs provides groundwork for multi-document summarization.The cited applications include machine translation, entity linking, summarization, question answering, and machine comprehension.Source: §2 · p. 3
3 Our Approach
The system builds summary graphs from grouped source sentences, then realizes each graph as a natural-language sentence. Its pipeline combines graph construction and structured prediction, graph-to-PENMAN conversion, AMR-to-text generation, and source sentence selection.
3.1 Content Planning
- Sentence AMRs are merged into a connected source graph, with nodes joined by shared surface forms or coreferential mentions and named/date entities collapsed into special meganodes.A ROOT node connects the roots of the sentence graphs.Source: §3.1 · pp. 3–4
- Structured prediction selects a salient summary graph by scoring its nodes and edges, decoding a connected tree of at most L nodes, and updating parameters against a gold-standard graph.The system uses structured ramp loss to address cases where the gold graph is unreachable through decoding.Source: §3.1 · p. 4
3.2 Surface Realization
- Summary graphs are converted to PENMAN format by traversing ordered ROOT-to-leaf paths, then passed to an AMR-to-text generator to produce sentences.The JAMR generator maps an input graph to a spanning tree and decodes it with a tree-to-string transducer and language model.Source: §3.2 · p. 5
3.3 Source Sentence Selection
- Spectral clustering groups similar source sentences into topic-aspect clusters; the system selects the M largest clusters and N sentences per cluster for graph generation.Selected sentences have the highest similarity to other sentences in their cluster.Source: §3.3 · p. 6
- Training instances pair AMR-parsed reference sentences with similar source sentences, selected using LCS, vector-space cosine similarity, Smatch, or Concept Coverage.The paper compares these sentence-selection strategies for training content planning.Source: §3.3 · p. 6
4 Datasets and Baselines
The study uses DUC/TAC multi-document summarization benchmarks with short-summary constraints and compares the AMR framework against extractive and abstractive systems. Table 1 reports graph-prediction scores on DUC-04.
4 Datasets and Baselines
- Each benchmark task summarizes 10 documents on one topic in at most 100 words, with four NIST reference summaries; the study uses DUC-03/04 and TAC-09/10/11.The datasets contain 30, 50, 44, 46, and 44 document clusters, respectively.Source: §4 · p. 6
- The framework is compared with extractive and abstractive systems, including the neural pointer-generator, which had not previously been tested for multi-document summarization.The pointer-generator combines copying from source text with novel-word generation and a coverage mechanism to reduce repetition.Source: §4 · pp. 6–7
- Table 1 reports DUC-04 summary-graph prediction scores for matching reference graphs on nodes and edges; its Oracle results are upper bounds for the respective approaches.Reference graphs are parsed from human summaries with CAMR.Source: Table 1 · p. 7
5 Experimental Results
The evaluation tests summary-graph prediction and generated summaries against baselines, then analyzes system outputs and remaining challenges.
5.1 Results on Summary Graph Prediction
- Concept Cov preserves 40.3% of gold-standard summary concepts (77.5% with oracle decoding), while VSM edge prediction reaches 9.8% F-score (22.1% oracle).On DUC-04 trained on DUC-03, decoded graphs are capped at 15 nodes; spectral-clustering selections lack gold-standard graph evaluation.Source: §5.1 · p. 7
5.2 Results on Summarization
- AMRSumm outperforms all other abstractive systems on R-1 and R-SU4 F-scores on DUC-04 and TAC-11; AMRSumm-VSM has the highest R-2 F-scores.Source: §5.2 · pp. 7–8, Table 2 · p. 8
- AMRSumm-Clst exceeds Pointer-Generator in R-1 F-score on DUC-04 (30.2% vs. 25.6%) and TAC-11 (31.9% vs. 26.2%).For DUC-04, source-contained bigrams/trigrams are 31.3%/8.4% for AMRSumm-Clst, 95.2%/87.2% for Pointer-Generator-all, and 41.6%/17.1% for human summaries.Source: §5.2 · p. 8
5.3 Result Analysis
- Parser choice shifts ROUGE by 1–2 points; CAMR scores 63% F-score versus JAMR’s 58%, and oracle decoding adds 7–10% R-1 and 2–5% R-2 F-score.Inter-annotator agreement places the expected parser-performance upper bound at 83% F-score.Source: §5.3 · pp. 8–9, Table 3 · p. 9
- The authors suggest simplifying fine-grained AMR graphs, such as collapsing semantic units into mega-nodes, to improve summarization robustness.Source: §5.3 · p. 9
6 Conclusion
The paper investigates AMR for multi-document summarization and describes a full abstractive approach. On benchmark datasets, the framework performs competitively with state-of-the-art abstractive approaches, suggesting AMR has potential for abstractive summarization.
6 Conclusion
- The AMR summarization framework performs competitively with state-of-the-art abstractive approaches on benchmark summarization datasets.Source: §6 · p. 9
- The paper describes a full-fledged approach for generating abstractive summaries from multiple source documents.Source: §6 · p. 9
- The findings suggest AMR is a powerful semantic formalism with potential for abstractive summarization.Source: §6 · p. 9