PaperlayerView exact revision on arXiv

Source-linked AI summary

Unsupervised Abstractive Meeting Summarization with Multi-Sentence Compression and Budgeted Submodular Maximization

Guokan Shang, Wensi Ding, Zekun Zhang, Antoine Jean-Pierre Tixier, Polykarpos Meladianos, Michalis Vazirgiannis, Jean-Pierre Lorré

arXiv:1805.05271v2cs.CL

TL;DR

Meeting speech transcripts are often ill-formed fragments and can contain ASR noise, creating a need for effective summarization. The paper combines approaches from several tasks in a fully unsupervised framework, and reports state-of-the-art performance on AMI and ICSI. It also states that the method can be applied to other languages when the required resources are available.

  • Problem

    Spontaneous meeting transcripts consist of ill-formed utterance fragments, and ASR transcription and segmentation errors add noise.

    Source: §1 · p. 1

  • Method

    The paper combines approaches from keyword extraction, multi-sentence compression, and summarization into a fully unsupervised meeting-summarization framework.

    Source: §1 · p. 1

  • Results

    The authors report state-of-the-art performance on AMI and ICSI and reasonably grammatical abstractive summaries from noisy utterances without annotations or training data.

    Source: §7 · p. 9

  • Takeaways & Limitations

    The authors state that the method can be applied to other languages in an almost out-of-the-box manner when the required resources are available.

    Source: §2 · p. 1, §7 · p. 9

    The authors identify community detection and abstractive-sentence generation for improvement, and say limited meeting-domain training sets obstruct neural approaches.

    Source: §7 · p. 10

Abstract

from arXiv · show

We introduce a novel graph-based framework for abstractive meeting speech summarization that is fully unsupervised and does not rely on any annotations. Our work combines the strengths of multiple recent approaches while addressing their weaknesses. Moreover, we leverage recent advances in word embeddings and graph degeneracy applied to NLP to take exterior semantic knowledge into account, and to design custom diversity and informativeness measures. Experiments on the AMI and ICSI corpus show that our system improves on the state-of-the-art. Code and data are publicly available, and our system can be interactively tested.

1 Introduction

Meeting speech summarization is needed as meetings and ASR use grow, but spontaneous meeting transcripts contain ill-formed fragments and ASR noise. The paper combines six approaches from three tasks into a unified, fully unsupervised framework for this setting.

1 Introduction

  • The paper presents the framework as a response to the growing need for effective meeting-speech summarization amid widespread meetings and improving ASR adoption.The authors cite the ubiquity of meetings and the rapid improvement and adoption of ASR as motivating factors.Source: §1 · p. 1
  • Spontaneous multi-party meeting transcripts contain ill-formed, ungrammatical utterance fragments, with ASR transcription and segmentation errors adding noise.These characteristics distinguish the input from traditional documents.Source: §1 · p. 1
  • The framework combines six previously applied approaches from keyword extraction, multi-sentence compression, and summarization into a unified, fully unsupervised meeting-summarization system.It is designed to generate readable summaries despite noise in ASR transcriptions.Source: §1 · p. 1

2 Framework Overview

The framework preprocesses utterances, groups them into communities, generates an abstractive sentence for each, and selects sentences under a budget using a custom submodular quality function. It requires no annotations and uses a list of utterances plus specified language resources.

2 Framework Overview

  • The four modules preprocess text, detect utterance communities, generate one abstractive sentence per community, and select sentences under a budget.The selection is framed as maximizing a custom submodular quality function.Source: §2 · p. 1
  • The system needs no annotations or metadata; its input is a list of utterances, alongside language resources such as a POS tagger, language model, and pretrained word vectors.Stopwords and fillerwords are also needed, with a lexical database such as WordNet optional.Source: §2 · p. 1
  • The authors state that the system can work out of the box in most languages for which the required resources are available.Source: §2 · p. 1

3 Related Work and Contributions

This section presents the framework as combining six recent works and adding novel components. It traces the progression from compression and word-importance methods to fluency-aware compression and budgeted sentence selection.

3.1 Multi-Sentence Compression Graph (MSCG) (Filippova, 2010)

  • Filippova (2010) compresses overlapping sentences through weighted word-graph paths, but reports 48% missing important information and 36% imperfect grammaticality.Source: §3.1 · p. 2

3.2 More informative MSCG (Boudin and Morin, 2013)

  • Boudin and Morin (2013) use PageRank to favor informative paths; this paper notes its limits for word importance and adds grammaticality handling.Source: §3.2 · p. 2

3.3 Graph-based word importance scoring (Tixier et al., 2016a)

  • Core numbers better identify influential nodes than PageRank or degree; CoreRank further separates equal-core nodes with scores 7 vs 5 by summing neighbors' core numbers.Source: §3.3 · p. 3, Figure 3 · p. 3, Figure 4 · p. 3

3.4 Fluency-aware, more abstractive MSCG (Mehdad et al., 2013)

  • Unlike Mehdad et al. (2013)'s supervised setup, this framework uses unsupervised community detection and removes entailment filtering without deteriorating result quality.Source: §3.4 · pp. 3–4
  • The framework adds embedding-distance edge weights, path-level embedding-cluster diversity, and budgeted submodular selection of abstractive sentences.Source: §3.4 · p. 4

3.5 Submodularity for summarization (Lin and Bilmes, 2010; Lin, 2012)

  • Budgeted submodular maximization selects a near-optimal sentence subset under a word-count budget; its greedy guarantee requires monotone non-decreasing submodular quality.Source: §3.5 · p. 4

4 Our Framework

The framework groups noisy meeting utterances, compresses each community into an abstractive sentence, then selects sentences under a budget. Its modules use preprocessing, graph-based compression, and a submodular objective to produce the final summary.

4.1 Text preprocessing

  • Preprocessing removes repeated words, ASR tags, fillers, and boundary stopwords, then prunes utterances with fewer than three non-stopwords.The surviving utterances feed the subsequent steps.Source: §4.1 · p. 4

4.2 Utterance community detection

  • Utterances are clustered by applying k-means to TF-IDF vectors after LSA reduces the utterance-term matrix; word-embedding approaches were less effective for this clustering.The authors attribute this to participants’ tendency to reuse the same term, making synonymy counterproductive for distinguishing utterances.Source: §4.2 · pp. 4–5

Word importance scoring

  • For each community, the method builds a word co-occurrence network and scores words with CoreRank reweighted by an IDF-like measure across communities.The co-occurrence network uses a sliding window of six that does not span utterances; terms are weighted more when they are important within a community and appear in relatively few communities.Source: Word importance scoring · p. 5

Word graph building

  • The MSCG starts with the first utterance as a directed word sequence, then maps matching words from other utterances to shared nodes using form, part of speech, and context.Stopwords are mapped only when immediate context overlaps on a non-stopword; words from the same utterance cannot share a node, making each utterance a loopless path.Source: Word graph building · pp. 5–6

Edge Weight Assignment

  • Edge weights combine local co-occurrence statistics with a global word-embedding term, with lower weights preferred.The local term favors infrequent words that occur close together, while the embedding component favors paths through salient, semantically similar words to support readability.Source: Edge Weight Assignment · p. 6

Path re-ranking

  • The method finds K lowest-weight paths between START and END, filters paths that are too short or lack a verb, then selects the lowest-scoring reranked path.Reranking accounts for path length, fluency, coverage of important nouns, verbs, and adjectives, and diversity measured by unique embedding clusters per path length.Source: Path re-ranking · p. 6

4.4 Budgeted submodular maximization

  • The framework produces one abstractive sentence per utterance community, then selects a budget-compliant subset to address potential redundancy and off-topic content.The selection objective balances weighted word occurrences against coverage of word clusters.Source: §4.4 · p. 7
  • A greedy algorithm selects a near-optimal subset of abstractive sentences by maximizing a submodular objective, using CoreRank scores and word clusters from the full meeting transcription.The selection addresses summaries that exceed the size limit or contain redundancy and off-topic sentences.Source: §4.4 · p. 7

5 Experimental setup

Experiments use AMI and ICSI meeting corpora, compare the system with established baselines and MSCG variants, and tune parameters on development sets. Tuning indicates that diversity regularization is usually useful, while utterance costs are not always needed in the greedy decision.

5.1 Datasets

  • AMI and ICSI test sets contain 20 and 6 meetings, respectively; their ASR transcriptions have word error rates of 36% and 37%.AMI references average 290 words; ICSI has three references averaging 220, 220, and 670 words.Source: §5.1 · p. 7

5.2 Baselines

  • The comparison includes seven baselines and three system variants using the original MSCG, KeyRank, and FluCovRank.Preprocessing was identical for the system and baselines.Source: §5.2 · pp. 7–8

5.3 Parameter tuning

  • λ is mostly non-zero, while r is zero in some best configurations, indicating that cost need not affect every greedy decision.The searched parameters included community count n, minimum path length z, diversity trade-off λ, and cost scaling factor r.Source: §5.3 · p. 8, Table 1 · p. 8
  • The LSA settings retain 30 dimensions for AMI and 60 for ICSI, reflecting their smaller meeting-specific vocabularies.For final-summary generation, k-means uses k = 60; K = 200 shortest weighted paths was chosen as a complexity-performance trade-off.Source: §5.3 · p. 8

6 Results and Interpretation

On ASR-generated summaries, the proposed systems outperform most baselines across ROUGE comparisons, with the leading variant differing by corpus. Their results remain competitive with Oracle, and the authors attribute gains over the original MSCG variant on ROUGE-2 to improved fluency.

Results

  • On ROUGE-1, Our System is best on ICSI and Our System (KeyRank) is superior on AMI; the proposed systems outperform all baselines except Oracle on ICSI.Figure 7 shows the proposed systems consistently outperforming baselines across summary budgets from 150 to 500 words; Table 2 reports the fixed-budget comparisons.Source: §6 · p. 8, Table 2 · p. 9, Figure 7 · p. 9
  • On ROUGE-2, all proposed variants except Our System (Baseline) outperform the baselines other than Oracle.Our System and Our System (FluCovRank) consistently improve on Our System (Baseline), which the authors say improves summary fluency.Source: §6 · p. 9
  • On ROUGE-SU4, the proposed systems are competitive with all baselines, including Oracle; the better of Our System and KeyRank varies by corpus.ROUGE-SU4 measures in-order word-pair overlap.Source: §6 · pp. 8–9
  • The framework is very competitive with Oracle, which has direct access to human extractive summaries.The authors note that overlap between human extractive and abstractive summaries is only 19% on AMI and 29% on ICSI.Source: §6 · p. 9

7 Conclusion and Next Steps

The paper presents a fully unsupervised meeting-summarization framework that combines prior approaches and reports state-of-the-art performance on AMI and ICSI. It is designed for noisy meeting speech, while future work targets community detection and more abstractive generation despite limited meeting-domain training data.

7 Conclusion and Next Steps

  • The framework combines six approaches from keyword extraction, multi-sentence compression, and summarization into an unsupervised end-to-end system, with state-of-the-art performance on AMI and ICSI.It generates reasonably grammatical abstractive summaries from noisy utterances without annotations or training data.Source: §7 · p. 9
  • The meeting-domain design relies on redundancy for MSCG compression and uses graph-degeneracy-based path reranking to increase robustness to noise.The authors also use a language model to favor fluent paths, which they describe as crucial for meeting speech.Source: §7 · p. 10
  • Future work should improve community detection and generate more abstractive sentences, potentially using deep learning, but meeting-domain training data are scarce.The authors identify the lack of large training sets as an obstacle to neural approaches.Source: §7 · p. 10

A Use of WordNet

WordNet supports mapping words in a compression graph to semantically related nodes when an exact lowercased form and part-of-speech match is unavailable. The procedure checks several lexical relations and falls back to a new graph node if mapping fails.

A Use of WordNet

  • For a non-stopword without an exact lowercased-form and part-of-speech node, the procedure first attempts WordNet-based mapping.The listed relations are synonymy, hypernymy, a shared hypernym, and entailment.Source: A Use of WordNet · p. 12
  • For synonym, hypernym, and entailment matches, the word maps to the related node, which is relabeled when the incoming word has a higher TW-IDF score.When multiple candidate nodes exist in these attempts, the highest-TW-IDF candidate is selected.Source: A Use of WordNet · p. 12
  • A shared-hypernym match requires the product of WordNet path-distance similarities to exceed a threshold; the node is relabeled with the hypernym.Its TW-IDF score is set to the higher score of the two replaced words; among eligible nodes, the greatest similarity product wins.Source: A Use of WordNet · p. 12
  • If all mapping attempts fail, the system creates a new node for the word.Source: A Use of WordNet · p. 12

B Baseline Details

The section describes extractive baselines that select utterances directly, using random or length-based selection, graph ranking, or budgeted submodular maximization.

B Baseline Details

  • Random selects utterances without replacement until the summary budget is exceeded, while Longest Greedy repeatedly selects the longest remaining utterance.Random scores are averaged over 30 runs.Source: B Baseline Details · p. 12
  • TextRank builds a complete graph of utterances weighted by lexical similarity and selects top nodes by weighted PageRank.The implementation was publicly available.Source: B Baseline Details · p. 12
  • ClusterRank clusters utterances, scores them using cluster PageRank and centroid similarity, then greedily selects high-scoring utterances.The authors implemented it because the original code was unavailable.Source: B Baseline Details · p. 12
  • CoreRank submodular and PageRank submodular apply budgeted submodular maximization directly to all utterances, differing only in their ranking scores (Tixier et al., 2017).They implement the final step of the paper's pipeline.Source: B Baseline Details · p. 12
  • The Oracle samples utterances from human extractive summaries, which annotators assembled by selecting the best utterances from the manual transcription.Scores are averaged over 30 runs.Source: B Baseline Details · p. 13

C Results for Manual Transcriptions

Results for manual transcriptions are presented for ROUGE-1 F-1 across budgets and as macro-averaged scores at two summary lengths.

C Results for Manual Transcriptions

  • Figure 8 reports ROUGE-1 F-1 scores across various budgets for manual transcriptions.The caption does not state the scores or identify a winning system.Source: Figure 8 · p. 13
  • Table 3 presents macro-averaged results for 350- and 450-word summaries from manual transcriptions.The supplied caption does not provide the table's values or comparisons.Source: Table 3 · p. 13

D Example Summaries

The examples include a reference summary and system outputs for the AMI TS3003c meeting, illustrating discussion of remote-control design decisions and preferences.

D Example Summaries

  • The examples come from manual transcriptions of AMI meeting TS3003c, and the system can also be tested interactively.The passage gives a URL for the interactive system.Source: D Example Summaries · p. 13
  • The reference summary recounts design preferences, options discussed, decisions made about the remote, and the team's follow-up task.Topics include energy, speech recognition, screens, chips, case materials and shape, and button orientation.Source: D Example Summaries · pp. 13–14
  • The baseline example mentions changeable cases, button placement, speech recognition, a docking station, and competing casing shapes and materials.It also raises production price and product-line considerations.Source: D Example Summaries · pp. 14–15
  • The FluCovRank example includes remote-control features and preferences such as changeable covers, wood-like materials, dark colors, speech recognition, and accessible subtitles.It also mentions LCD screens, button placement, and design trade-offs.Source: D Example Summaries · pp. 14–15
Loading 1805.05271v2…