Source-linked AI summary
Abstractive Multi-Document Summarization via Phrase Selection and Merging
Lidong Bing, Piji Li, Yi Liao, Wai Lam, Weiwei Guo, Rebecca J. Passonneau
TL;DR
Compression-based summaries cannot merge facts from different source sentences. This framework constructs summaries by selecting and merging salient noun and verb phrases under sentence constraints. On TAC 2011, it outperformed the top systems under automated pyramid evaluation.
Problem
Compression-based summarization cannot merge facts from different source sentences, while extraction can retain overlapping information.
Source: §1 · p. 1Method
The framework pools noun and verb phrases as concepts and facts, then jointly selects and merges them using integer linear optimization under sentence-construction constraints.
Source: §1 · pp. 1–2Results
On TAC 2011, the framework achieved the best automated-pyramid results at both thresholds and significantly outperformed System 22 (paired t-test, p < 0.01).
Source: §3.2 · p. 7Takeaways & Limitations
Phrase-level construction is intended to include more summary content units in a sentence by crossing original sentence boundaries.
Source: §1 · p. 2
Abstract
from arXiv · showhide
We propose an abstraction-based multi-document summarization framework that can construct new sentences by exploring more fine-grained syntactic units than sentences, namely, noun/verb phrases. Different from existing abstraction-based approaches, our method first constructs a pool of concepts and facts represented by phrases from the input documents. Then new sentences are generated by selecting and merging informative phrases to maximize the salience of phrases and meanwhile satisfy the sentence construction constraints. We employ integer linear optimization for conducting phrase selection and merging simultaneously in order to achieve the global optimal solution for a summary. Experimental results on the benchmark data set TAC 2011 show that our framework outperforms the state-of-the-art models under automated pyramid evaluation metric, and achieves reasonably well results on manual linguistic quality evaluation.
1 Introduction
The introduction motivates abstractive multi-document summarization as a way to combine facts across source sentences, then proposes constructing summaries from salient noun and verb phrases under global optimization.
1 Introduction
- Extraction can preserve overlapping information, while compression removes words or phrases but cannot fuse facts from separate source sentences.These limitations motivate methods that can aggregate information across sentence boundaries.Source: §1 · p. 1
- Sentence-fusion approaches cluster related sentences and combine their common information, potentially covering more content concisely.The paper contrasts their sentence-cluster salience with its finer-grained phrase units.Source: §1 · pp. 1–2
- The framework builds a pool of concepts and facts represented by noun and verb phrases, then selects and merges phrases to construct new sentences.Phrase salience is computed using document-wide redundancy, and integer linear optimization enforces sentence validity.Source: §1 · p. 2
- The optimization jointly selects and merges phrases to maximize covered summary content units, while compatibility relations and other constraints help ensure generated sentences contain correct facts.Unlike sentence-cluster salience, it globally maximizes the salience of selected phrases.Source: §1 · p. 2
2 Description of Our Framework
The framework represents concepts and facts with phrases, then selects compatible NPs and VPs to construct new sentences under summarization constraints.
2.1 Phrase Salience Calculation
- NPs and VPs extracted from constituency trees represent potential concepts and facts, and their salience scores reflect weighted concept occurrences, including paragraph position.Recursive extraction is limited to two levels because lower-level phrases may not convey a complete fact.Source: §2.1 · pp. 2–3
2.2 New Sentence Construction Model
- Each generated sentence combines one NP with at least one VP, potentially drawn from different source sentences, subject to phrase compatibility and summarization requirements.Source: §2.2 · p. 3, §2.2.1 · p. 3
2.2.1 Compatibility Relation
- The compatibility matrix permits NP–VP pairings from the same sentence and extends them through coreferent NPs or similar VPs, with VP similarity measured by Jaccard Index.Source: §2.2.1 · pp. 3–4
2.2.2 Phrase-based Content Optimization
- The objective maximizes selected phrase salience while penalizing similar NP pairs and VP pairs separately, avoiding penalties between an NP and a VP.Source: §2.2.2 · p. 4
2.2.3 Sentence Generation Constraints
- Linear ILP constraints require valid NP–VP sentence assignments and exclude VPs from very short sentences and pronouns as generated sentence subjects.Source: §2.2.3 · pp. 4–5
- Additional constraints prevent selecting nested phrases together and limit summary word length and sentence count.Source: §2.2.3 · p. 5
2.3 Postprocessing
- Postprocessing orders VPs by source-document order or timestamps, adds conjunctions when length permits, and orders sentences by their earliest VP timestamp.Source: §2.3 · p. 5
2.4 Relation to Existing MDS Approaches
- Extraction-based summaries are simulated by selecting original sentences, while compression-based summaries restrict each generated sentence's NP and VPs to one source sentence.Source: §2.4 · pp. 5–6
3 Experiments
On TAC 2011, the framework outperforms the leading systems under automated pyramid evaluation, while ROUGE and manual linguistic evaluation show a more qualified picture. The experiments also assess score reliability and use separate TAC 2010 data for parameter tuning.
3.1 Experimental Setup
- Evaluation uses 44 TAC 2011 topics, each with 10 news documents and four writer summaries; TAC 2010 supplies 46 tuning topics from the same categories.The tuned constants B and ρ are 6 and 0.5, and the alternative-VP similarity threshold is 0.75.Source: §3.1 · p. 6
3.2 Results with Pyramid Evaluation
- The framework achieves the best automated pyramid results at both thresholds and significantly outperforms TAC 2011 System 22 (paired t-test, p < 0.01).The authors report this result without domain adaptation, whereas the leading comparison systems used external linguistic resources.Source: Table 2 · p. 6, §3.2 · p. 7
3.3 Results with ROUGE Evaluation
- Under ROUGE, the framework scores slightly above System 22 but below Systems 43 and 17, which use category-specific features.The framework uses no category-specific features and tunes its general-purpose parameters on TAC 2010 data; the authors note that category-specific methods depend on predefined categories and matching training data.Source: Table 3 · p. 7, §3.3 · p. 7
3.2 Results with Pyramid Evaluation
- Automated pyramid scores correlate with manual scores at 0.91 and 0.93 for thresholds 0.6 and 0.65, respectively.The correlation analysis covers 50 TAC 2011 system submissions.Source: §3.2 · p. 7
3.4 Linguistic Quality Evaluation
- Manual linguistic ratings are close overall; excluding grammaticality, the framework averages 3.37 across the last four questions versus 3.33 for System 22.System 22 scores higher on grammaticality, while the framework scores higher on focus; the authors also describe referential relations in its generated summaries as basically clear.Source: §3.4 · p. 7
4 Qualitative Results
The framework produces new, compressed, and original sentences, with new constructions making up about a third of its output. The Amish Shooting case illustrates how phrases are combined and redundant or unimportant information omitted.
4.1 Analysis of Summary Sentence Type
- About 33% of summary sentences are newly constructed and 44% are compressed, showing that the framework performs phrase merging and compression in one model.The remaining sentence type is original, directly extracted from input documents.Source: §4.1 · p. 8
4.2 Case Study
- For the Amish Shooting topic, the example summary contains three compressed sentences, one new sentence, and two original sentences.The table identifies each selected phrase with its source sentence ID.Source: §4.2 · p. 8, Table 5 · p. 8
- In the case study, coreferent noun phrases allow VPs from different source sentences to be joined, while trivial or redundant details are omitted.The authors cite “authorities said” as trivial information and exclude a VP about killing five girls because it duplicates content in another summary sentence.Source: §4.2 · p. 8
5 Related Work
Related MDS work spans extraction, compression, and abstraction. The paper positions phrase selection and merging against the granularity limits of extraction and the cross-sentence limits of compression.
5 Related Work
- Extraction selects whole sentences and may retain unimportant or redundant phrases; unified models jointly select sentences and control redundancy but remain sentence-grained.Earlier greedy selection can become trapped in local optima.Source: §5 · p. 9
- Compression can jointly select and shorten sentences, but compressive methods cannot merge related facts from different sentences.Earlier approaches used a two-step selection-then-deletion process.Source: §5 · p. 9
- Abstraction generates sentences from facts across source sentences, with prior directions including information extraction, summary revision, word graphs, timeline and certainty modeling, and deep learning.Other explored settings include query-chain focused and hierarchical summarization, as well as mining sentence structures from news archives.Source: §5 · p. 9
6 Conclusions and Future Work
The framework builds summaries by globally selecting and merging noun and verb phrases while satisfying constraints such as NP/VP compatibility. On TAC 2011, it outperforms the top systems under the pyramid metric.
6 Conclusions and Future Work
- On TAC 2011, the framework outperforms the top systems under the pyramid metric.Source: §6 · p. 9
- Phrase selection and merging jointly optimize the summary globally, subject to summarization constraints such as NP/VP compatibility.Source: §6 · p. 9
- Future work targets grammar quality for generated and compressed sentences, plus efficiency, with ILP optimization the major time bottleneck.Source: §6 · p. 9