Source-linked AI summary
Leveraging BERT for Extractive Text Summarization on Lectures
Derek Miller
TL;DR
Lecture summarization needs more effective approaches because existing methods often use dated natural language processing techniques and produce sub-par outputs. This paper presents a service that uses BERT embeddings and K-Means to select representative sentences, with examples indicating improved contextual quality over TextRank. The authors also identify weaknesses involving large lectures and context-dependent wording.
Problem
Existing lecture summarization approaches often use dated natural language processing algorithms, producing sub-par outputs and requiring frequent maintenance.
Method
The Lecture Summarization Service tokenizes lecture text, creates BERT sentence embeddings, clusters them with K-Means, and selects sentences closest to centroids.
Results
The authors report steady improvement over TextRank in summary quality, with examples showing BERT selecting sentences that provide broader context and more coherent explanations.
Takeaways & Limitations
The service provides students a configurable utility for summarizing lecture content and managing stored lectures and summaries.
Takeaways & Limitations
The current approach struggles with lectures of 100 or more sentences and with sentences containing context-dependent words whose antecedents may occur several sentences earlier.
Abstract
from arXiv · showhide
In the last two decades, automatic extractive text summarization on lectures has demonstrated to be a useful tool for collecting key phrases and sentences that best represent the content. However, many current approaches utilize dated approaches, producing sub-par outputs or requiring several hours of manual tuning to produce meaningful results. Recently, new machine learning architectures have provided mechanisms for extractive summarization through the clustering of output embeddings from deep learning models. This paper reports on the project called Lecture Summarization Service, a python based RESTful service that utilizes the BERT model for text embeddings and KMeans clustering to identify sentences closes to the centroid for summary selection. The purpose of the service was to provide students a utility that could summarize lecture content, based on their desired number of sentences. On top of the summary work, the service also includes lecture and summary management, storing content on the cloud which can be used for collaboration. While the results of utilizing BERT for extractive summarization were promising, there were still areas where the model struggled, providing feature research opportunities for further improvement.
METHOD
The Lecture Summarization Service combines lecture and summary management with a BERT-and-K-Means pipeline that selects representative sentences. Examples show improved contextual coherence over TextRank, while large lectures and context-dependent wording remain weaknesses.
- METHOD: The service manages lecture transcripts and summaries while using BERT embeddings and K-Means clustering to generate summaries.Users can create, edit, delete, and retrieve stored items; selected sentences are those closest to cluster centroids.
- METHOD: The pipeline tokenizes transcript text, generates BERT embeddings, clusters them with K-Means, and selects sentences nearest the centroids.Preprocessing includes converting transcript formats and removing some sentence types before inference.
- METHOD: Lectures with 100 or more sentences were difficult to represent with a small requested sentence ratio, and the proposed multi-sentence cluster approach was excluded because it exceeded that ratio.Using more sentences per cluster could preserve context but would add sentences beyond the user’s requested number.
- METHOD: The approach occasionally selected sentences containing context-dependent words such as “this,” “those,” “these,” and “also,” with antecedents sometimes occurring two or three sentences earlier.Replacing pronouns and keywords with proper values was identified as a possible future improvement.
- METHOD: BERT summaries better captured broader context than TextRank in a health-information-exchange example, although TextRank introduced IHIE as useful background.TextRank’s selected sentences flowed together less effectively, while BERT captured context about data governance and agreed on the final interview sentence.
- METHOD: For a TD(0) lecture, BERT selected the rule’s buildup and definition, whereas TextRank’s sentences were described as random and difficult to understand.The example reduced a 40-sentence lecture to 5 sentences.