Source-linked AI summary
Variations of the Similarity Function of TextRank for Automated Summarization
Federico Barrios, Federico López, Luis Argerich, Rosa Wachenchauzer
TL;DR
Automated extractive summarization requires selecting important sentences, while TextRank provides a language- and domain-independent graph-based solution. This paper changes the similarity measures used to weight TextRank graph edges and evaluates the alternatives under the original configuration. BM25 and BM25+ achieve the strongest reported improvements, including a 2.92% gain over TextRank, while processing time also decreases.
Problem
Extractive summarization must identify important sentences, and TextRank's graph construction depends on a sentence-similarity function for weighting relations.
Method
The paper evaluates alternative similarity measures, including cosine similarity, BM25, BM25+, and LCS, as edge weights in the TextRank graph.
Results
BM25 with ε = 0.25 improves the original TextRank result by 2.92%; Cosine Similarity improves it by 2.54%, LCS by 1.40%, and processing 567 DUC2002 documents takes 84% of the original time.
Takeaways & Limitations
The authors suggest using BM25 with TextRank for unsupervised automatic summarization because it performs better than standard techniques without a performance penalty.
Abstract
from arXiv · showhide
This article presents new alternatives to the similarity function for the TextRank algorithm for automatic summarization of texts. We describe the generalities of the algorithm and the different functions we propose. Some of these variants achieve a significative improvement using the same metrics and dataset as the original publication.
1 Introduction
Extractive summarization selects important sentences at adjustable compression levels. TextRank is a widely used, language- and domain-independent graph-based approach, and this article proposes alternative graph constructions.
- 1 Introduction: Extractive summarization selects the most important sentences to create documents of arbitrary summary length.Different compression levels determine the resulting summary length.
- 1 Introduction: TextRank is a graph-based extractive summarization algorithm that does not require deep linguistic knowledge or annotated corpora.Its domain and language independence supports use across different settings.
- 1 Introduction: TextRank has performed well on structured news articles and has also been applied to meeting transcription summarization and web-content credibility assessment.
- 1 Introduction: The article proposes different graph constructions and reports their evaluation results.The article describes prior work, TextRank, proposed variations, evaluation metrics, and the dataset before presenting results.
2 Previous work
Previous summarization research includes statistical extractive methods and graph-based ranking approaches. Graph systems represent textual units as connected structures and use ranking techniques to select important sentences.
- 2 Previous work: Traditional extractive methods use statistical cues such as keywords, sentence position, length, word frequency, and titles.Several models combine these heuristics using training corpora.
- 2 Previous work: Graph-based approaches represent text structure explicitly rather than treating documents as simple aggregations of terms.This representation can capture richer information for identifying important concepts.
- 2 Previous work: Graph construction can use phrases, sentences, or paragraphs, with sentences often chosen to balance content richness and grammatical correctness.
- 2 Previous work: Graph edges may be based on overlapping words, cosine distance, query-sensitive similarity, or combinations of measures.
- 2 Previous work: TextRank and LexRank rank lexical units using PageRank variations, while HITS and Positional Function are other applicable graph-ranking algorithms.
3 TextRank
TextRank applies PageRank to a sentence graph whose edge weights represent sentence similarity. It ranks graph vertices and selects the highest-scoring sentences in document order for the summary.
- 3.1 Description: TextRank is an unsupervised summarization algorithm that can also identify important document keywords.
- 3.1 Description: The algorithm applies a PageRank variation to a task-specific graph, avoiding training corpora or labeling and supporting different languages.
- 3.2 Text as a Graph: TextRank models documents as graphs with sentences as nodes and uses sentence similarity to weight the edges between them.Higher similarity produces a more important edge, increasing the modeled likelihood of moving between similar sentences.
- 3.2 Text as a Graph: The original similarity function counts common lexical tokens and divides by sentence lengths to avoid promoting long sentences.
- 3.2 Text as a Graph: PageRank computes vertex importance on the resulting dense graph, after which the most significant sentences are selected in their original document order.
4 Experiments
The experiments compare alternative sentence-similarity functions for weighting TextRank graph edges, using the DUC2002 corpus and ROUGE evaluation. BM25-based variants achieved the strongest reported improvement over original TextRank, while processing time also decreased.
- Evaluation: 567 DUC2002 documents were summarized to 20% of their size using the same corpus as the original TextRank publication.The evaluation used the DUC2002 database.
- Evaluation: ROUGE-1, ROUGE-2, and ROUGE-SU4 were averaged for evaluation under DUC settings, with stemming and a 95% confidence level.The implementation used ROUGE version 1.5.5.
- Variations: The study tests LCS, cosine similarity, BM25, and BM25+ as alternative edge-weighting functions for TextRank.BM25 and BM25+ were tested with the corrective formula in equation 3.
- Results: The BM25 variation with ε = 0.25 improved results by 2.92% over original TextRank.The original TextRank result was independently reproduced as a 2.3% improvement over baseline.
- Results: Cosine similarity improved results by 2.54%, while LCS produced a 1.40% total improvement over original TextRank.Both alternatives performed better than the original method.
- Results: Processing the 567-document DUC2002 corpus required 84% of the time needed by the original version.The reported time performance was improved alongside summarization results.
5 Reference Implementation and Gensim Contribution
The authors provide a Python reference implementation of their proposals for testing and reproducing the reported results, and contribute BM25-TextRank to Gensim.
- A Python module implements the proposed variations.
- The implementation can be obtained for testing and reproducing results.
- BM25-TextRank was contributed to the Gensim project.
6 Conclusions
The article evaluates three TextRank variations and reports significant improvements over the original algorithm under the same test configuration. It concludes that combining TextRank with BM25 supports robust summarization performance without a performance penalty.
- Three TextRank variations significantly improved results under the original publication’s test configuration.
- Combining TextRank with BM25 and BM25+ produced a robust automatic summarization method that outperformed previously used standard techniques.
- The authors suggest using BM25 with TextRank for unsupervised automatic text summarization without a performance penalty relative to original TextRank.