Source-linked AI summary
Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints
Greg Durrett, Taylor Berg-Kirkpatrick, Dan Klein
TL;DR
Single-document summarization requires selecting content without the redundancy available across multiple documents, while preserving grammaticality and coherent references. The paper trains an end-to-end structured model with compression and anaphora constraints, and reports substantial ROUGE improvements over baselines while maintaining good linguistic quality.
Problem
Single-document summarization is difficult because content selection lacks cross-document redundancy and simple positional information remains difficult to beat.
Method
The model uses end-to-end structured learning with syntactic and discourse compression constraints plus coreference-based anaphora constraints in an ILP framework.
Results
The system improves substantially over baseline systems on ROUGE while maintaining good linguistic quality.
Takeaways & Limitations
Combining expressive compression with grammaticality and anaphoricity constraints supports important content extraction without sacrificing fluency and coherence.
Takeaways & Limitations
The system omits cross-sentential RST constraints because experiments found no improvement from using them.
Abstract
from arXiv · showhide
We present a discriminative model for single-document summarization that integrally combines compression and anaphoricity constraints. Our model selects textual units to include in the summary based on a rich set of sparse features whose weights are learned on a large corpus. We allow for the deletion of content within a sentence when that deletion is licensed by compression rules; in our framework, these are implemented as dependencies between subsentential units of text. Anaphoricity constraints then improve cross-sentence coherence by guaranteeing that, for each pronoun included in the summary, the pronoun's antecedent is included as well or the pronoun is rewritten as a full mention. When trained end-to-end, our final system outperforms prior work on both ROUGE as well as on human judgments of linguistic quality.
1 Introduction
The paper addresses the difficult, underexplored problem of single-document summarization with an end-to-end structured model that combines compression, grammaticality, and anaphora constraints. On the New York Times corpus, the system improves ROUGE and human-rated linguistic quality over prior baselines.
- Motivation: Single-document summarization receives less recent attention and lacks the redundancy and positional guidance available in multi-document settings.
- Approach: The model is trained end-to-end on roughly 100,000 New York Times articles with abstractive summaries to learn important content selection.
- Approach: Syntactic, discourse, and coreference constraints support aggressive sentence compression while enforcing grammaticality and coherent pronoun references.
- Results: On the New York Times corpus, the system outperforms document-prefix, bigram-coverage, and discourse-informed baselines according to ROUGE.
- Results: Discursive and referential constraints improve human judgments of linguistic clarity and referential structure while the system approaches the clarity of a sentence-extractive baseline.
2 Model
The model selects scored textual units under a length constraint, using dependencies to enforce grammatical compression and anaphora constraints to preserve pronoun coherence. It combines discourse and syntactic compression options in an ILP and can rewrite pronouns when antecedents would otherwise be deleted.
- Model: The ILP selects scored textual units under a k-word length constraint, with binary inclusion variables and learned feature weights.Textual units range from whole sentences to finer-grained contiguous sentence parts.
- Grammaticality Constraints: Requirement dependencies prevent a textual unit from being included unless its prerequisite is also included, enforcing grammaticality.The dependencies represent both syntactic and discourse-based compression schemes.
- Grammaticality Constraints: RST compression deletes eligible discourse material while preserving required parent and SAME-UNIT relations.The example permits deleting an ELABORATION clause but requires the first and last EDUs to remain.
- Grammaticality Constraints: Syntactic compression deletes coordinated material or PP modifiers, and combined constraints allow units to be wholly included, partially included, or omitted.The combined scheme augments RST units with syntactic compression subtrees and requirement relations.
- Anaphora Constraints: Anaphora constraints either rewrite a pronoun with its likely antecedent when coreference confidence exceeds α or require probable antecedents to be included.Pronoun replacement is scored and incorporated into the length constraint; otherwise antecedent-inclusion constraints preserve interpretability.
3 Learning
The model learns feature weights with a structured SVM using a ROUGE-based loss for abstractive reference summaries, trained through loss-augmented ILP decoding. Its ROUGE-1 optimization also improves ROUGE-2, while constraints remain part of the achievable oracle.
- The model learns weights on document–reference pairs with a structured SVM and a ROUGE-based loss tailored to abstractive summaries.The loss accounts for the fact that abstractive references generally cannot be produced exactly by the model.
- The loss measures the gap between a hypothesis's ROUGE score and the oracle ROUGE score achievable under the model's constraints.Reference-summary n-gram indicators provide sufficient statistics for computing ROUGE.
- Loss-augmented decoding finds the most violated constraint by maximizing w⊤f(x)+ℓ(x, y) under the model's ILP constraints.Training uses an extended ILP to perform this decode.
- Optimizing for ROUGE-1 improved performance on both ROUGE-1 and ROUGE-2.The authors hypothesize that ROUGE-2 would yield a flatter preference structure because bigram matches are relatively rare in short abstractive summaries.
- The training objective is optimized with AdaGrad, ℓ1 regularization, minibatches of one, and held-out stopping criteria.The reported settings include λ = 10−8, step size 0.1, and 10 training iterations.
4 Experiments
The evaluation uses primarily the NYT50 corpus and compares learned, compressive summarization against extractive and heuristic baselines using ROUGE and human linguistic-quality judgments. Results show that end-to-end learning and combined compression improve content selection, while anaphoricity constraints improve referential quality.
- Evaluation setup: NYT50 contains 110,540 articles originally split into 100,834 training and 9,706 test examples; filtering summaries shorter than 50 words leaves 3,452 test examples.The filtering targets extremely short and formulaic summaries and aligns summary lengths more closely with standard evaluations.
- New York Times corpus: Oracle summaries contain useful sentences well beyond the document prefix, indicating room for content selection to outperform prefix extraction.The corpus includes longer-form articles whose early ledes can be relatively content-free.
- Evaluation setup: The evaluation measures content selection with ROUGE and linguistic quality through grammaticality ratings and counts of unclear pronoun references.Human evaluators rate grammaticality on a 10-point Likert scale without seeing the original document or reference summary.
- New York Times results: End-to-end learned systems substantially outperform heuristic baselines on content selection, and full syntactic-plus-discourse compression exceeds ablated compression variants on ROUGE.More compression options give the model greater flexibility to include key content words.
- New York Times results: Removing anaphora constraints slightly increases ROUGE but worsens linguistic-quality metrics, whereas adding them further improves clarity and reduces unclear pronouns.This separates content-selection flexibility from referential and linguistic quality.
5 Conclusion
The paper concludes with an end-to-end single-document summarization system that combines grammatical compression with pronoun-anaphoricity constraints. It improves substantially over baselines on ROUGE while maintaining good linguistic quality.
- The system is trained end-to-end on a large corpus for single-document summarization.
- Its compression model enforces grammaticality, while pronoun-anaphoricity constraints enforce coherence.
- The system improves substantially over baseline systems on ROUGE while maintaining good linguistic quality.
- The models and system are publicly available from the Berkeley NLP website.
- Comparison with Yoshida et al. uses a reimplementation because their system is unavailable, and differing discourse trees can substantially change metrics.