From Arguments to Key Points: Towards Automatic Argument Summarization

1 Introduction

• Prior clustering work grouped arguments by similarity, frames, or facets, but did not create a concise textual summary from those clusters.The paper frames the task against the difficulty of digesting hundreds or thousands of arguments per topic.Source: §1 · p. 1
• Key points are high-level talking points that balance generality sufficient to match many arguments with enough information to remain useful; salience is represented by matching-argument counts.The proposed summary maps arguments to a short list of key points for each side of a debate.Source: §1 · p. 1
• The study assumes each topic’s key points are given and focuses on automatically mapping arguments to them.The introduction also notes that mapping can measure key-point distributions, support interactive retrieval, and identify arguments that match none of the prespecified points.Source: §1 · pp. 1–2
• The ArgKP dataset provides about 24,000 argument–key point pairs labeled as matching or nonmatching, and the paper evaluates supervised and unsupervised configurations.The authors describe it as the first dataset for this task and report it is larger and more comprehensive than datasets for related tasks.Source: §1 · p. 2
• Extensive annotation and analysis support the feasibility and effectiveness of mapping large collections of arguments to a small set of key points.Across 28 controversial topics and 7,000 crowd-contributed pro and con arguments, the authors report that experts can compose a short, comprehensive list without seeing the arguments.Source: §1 · pp. 1–2

2.1 Argument Mining

• Topic-related argument mining identifies claims, evidence, or arguments from collections such as Wikipedia, the Web, and other large corpora.The current work starts from pro and con arguments collected for a topic, by survey or automatic mining.Source: §2.1 · p. 2

2.2 Argument Clustering and Summarization

• Prior clustering approaches group arguments by frames, similarity, or shared facets, whereas this work directly maps arguments to key points.Misra et al. (2016) detect shared facets between argument pairs but do not explicitly identify or generate the facets.Source: §2.2 · p. 3
• Egan et al. (2016) extract salient points defined as a verb and its syntactic arguments, while this work maps arguments to given points and aims to capture varied expressions of their meaning.The prior method matches only arguments with the same verb, subject, and object dependency nodes.Source: §2.2 · p. 3
• Earlier debate-reason mapping studies used smaller datasets or within-topic classification; this work covers 28 topics and over 6,500 crowd-contributed arguments and addresses cross-topic classification.The cited comparisons include datasets spanning 2–4 topics and a cross-topic experiment using two topics with fewer than 200 comments per topic.Source: §2.2 · p. 3

3.1 Arguments and Key Points

• For 28 Debatabase-linked topics, the team filtered IBM-Rank-30k arguments and commissioned a professional debater to write up to seven key points per side without seeing them.Debatabase headlines were unsuitable because their specificity varied and preliminary analysis found low coverage of crowd arguments.Source: §3.1 · pp. 3–4

3.2.1 Annotation Process

• Each argument was labeled by eight annotators, with quality controls; pairwise agreement was moderate (Fleiss’ Kappa 0.44; Cohen’s Kappa 0.5).Stance judgments were 98% correct.Source: §3.2.1 · p. 4

3.2.2 Annotation Results

• Independently composed key points covered 72.5% of arguments, and 5% mapped to multiple points.Adding points by matched-argument frequency increased coverage, with diminishing marginal contributions from the sixth and seventh points.Source: §3.2.2 · p. 5, Figure 1 · p. 5
• Arguments mapped to multiple key points were longest and highest quality; arguments with no match tended to be shorter and lower quality.22.8% of arguments were ambiguous because no key point received the 60% annotation threshold.Source: §3.2.2 · p. 5

3.3 Final Dataset Generation

• The final ArgKP dataset contains 24,093 labeled pairs, including 4,998 positives (20.7%), and 6,515 arguments with 243 key points.Pairs were positive at a label score of at least 0.6 and negative at 0.15 or below; intermediate pairs were removed. An expert check yielded Cohen’s Kappa of 0.82.Source: §3.3 · pp. 5–6

4.1 Experimental Setup

• The experiments use four-fold cross-validation, with each fold containing seven test topics, 17 training topics, and four development topics.Match scoring is followed by classification of pairs as matching or non-matching.Source: §4.1 · p. 6

Unsupervised Methods

• Match scores come from lexical TF-IDF cosine similarity, averaged GloVe or BERT embeddings, fine-tuned BERT classifiers, and BERT transferred from SNLI or MNLI.The supervised BERT models predict matches using a sigmoid output over the [CLS] representation.Source: Unsupervised Methods · pp. 6–7

4.1.2 Match Classification

• Best Match selects only the highest-scoring same-topic, same-stance point, whereas BM+Threshold rejects that match unless it clears a learned threshold.Best Match is the only fully unsupervised selection policy; it can force a match even though 27.5% of arguments have none.Source: §4.1.2 · p. 7

4.2.1 Match Scoring Methods

• Unsupervised methods struggle to capture matches: TF-IDF and GloVe perform worst, while BERT embeddings reach only F1 0.4.Simple lexical similarity is insufficient for this task.Source: §4.2.1 · p. 7
• Supervised-classifier errors include high-scoring non-matches with shared phrases or substantial sentence overlap that misses the main point.Some related arguments are labeled non-matching when another key point fits better.Source: §4.2.1 · pp. 7–8
• Fine-tuned BERT-large reaches F1 0.684 and BERT-base reaches F1 0.657, outperforming unsupervised methods and NLI-trained models.BERT embeddings reach F1 0.4; the best NLI model reaches F1 0.526, and cascading NLI training with ArgKP fine-tuning does not improve supervised results.Source: §4.2.1 · p. 7, Table 4 · p. 8

4.2.2 Key Point Selection Policies

• Dual Threshold achieves the best overall F1 score of 0.73 and improves recall for multiple matches while retaining good performance on single or no matches.The policy comparison uses BERT-large fine-tuned on ArgKP for match scoring.Source: Table 5 · p. 8, §4.2.2 · p. 8
• Dual Threshold offers the best mid-range precision–recall trade-off; BM+Threshold favors precision but has low recall because it selects at most one point.Threshold alone can raise recall to one at the cost of low precision.Source: Figure 2 · p. 9, §4.2.2 · p. 9

• The approach represents argument summaries as key points scored by relative salience and demonstrates that experts can quickly identify points capturing the gist of crowd-contributed arguments.Source: §5 · p. 9
• The paper studies automatic argument-to-key-point matching and introduces the first large-scale dataset for the task, made publicly available.Source: §5 · p. 9
• Supervised match scoring with an appropriate key-point selection policy achieved promising results, while unsupervised word- or sentence-level embedding methods were ineffective.Source: §5 · p. 9
• Future work targets automatic key-point generation, application to automatically mined arguments, and more implicit argument-to-key-point relations.Source: §5 · p. 9