Keyphrase Generation with Correlation Constraints

1 Introduction

• Conventional approaches extract and rank candidate spans, showing promise for present keyphrases but failing to handle absent keyphrases.Keyphrases can summarize documents and highlight important concepts.Source: §1 · p. 1
• Generative methods can predict absent keyphrases, but ignoring correlations among a document’s multiple target phrases leads to duplication and coverage issues.Duplication means at least two phrases express the same meaning; coverage gaps leave some document topics unrepresented.Source: §1 · pp. 1–2
• The paper motivates reviewing both the document and preceding phrases, generating new phrases until the document’s topics are covered.The example contrasts repeated phrases such as “agent systems” with a new phrase covering “norm conflict.”Source: §1 · p. 2
• CorrRNN models keyphrase correlation with a coverage mechanism that tracks source content summarized by earlier phrases and a review mechanism that uses previous phrases to avoid repetition.The authors describe the architecture as modeling the document-to-multiple-keyphrases relationship.Source: §1 · p. 2
• Across three benchmark datasets, CorrRNN outperforms state-of-the-art methods by a large margin and improves diversity over heuristic rules.The authors attribute this to modeling correlation end-to-end.Source: §1 · p. 2
• The paper’s contributions are modeling one-to-many correlation, proposing CorrRNN, and empirically verifying its effectiveness on public datasets.The introduction describes the remainder of the paper as covering related work, the model, experiments, results, and conclusion.Source: §1 · p. 2

2 Related Work

• Extraction-based approaches construct phrase candidates with heuristics and then rank them using supervised or unsupervised methods.Candidate construction includes important n-grams and text chunks selected by part-of-speech patterns.Source: §2 · p. 2
• Generation-based methods can generate absent keyphrases; Meng et al. (2017) applied an encoder-decoder framework with a copy mechanism.The cited approach is described as achieving state-of-the-art performance.Source: §2 · pp. 2–3
• Unlike prior re-ranking approaches to correlation, CorrRNN models keyphrase correlation end-to-end with review and coverage mechanisms.The paper links review to diversity and coverage to keyphrase coverage.Source: §2 · p. 3

3.1 Problem Formalization

• CorrRNN models keyphrase correlation to address coverage and duplication, unlike prior work that treats keyphrases independently given the source document.Its objective accounts for relationships among a document’s keyphrases.Source: §3.1 · p. 3

3.2 Seq2Seq Model with Copy Mechanism

• The Seq2Seq backbone encodes the input into hidden representations, uses attention to form source context, and decodes words; a copy mechanism can produce out-of-vocabulary source words.The copy mechanism selects appropriate words from the source text.Source: §3.2 · p. 3

3.3 Model Correlation

• CorrRNN captures one-to-many keyphrase relationships with source-side coverage to improve topic coverage and target-side review to reduce duplication.The framework updates and transmits both the coverage vector and review context while decoding multiple keyphrases.Source: §3.3 · p. 3, Figure 1 · p. 4

3.3.1 Coverage Mechanism

• The coverage vector sums earlier attention distributions and informs later attention, helping the model avoid repeatedly focusing on source locations and cover more topics.The vector represents how much attention the source words have received so far.Source: §3.3.1 · p. 4

3.3.2 Review Mechanism

• The review mechanism attends to hidden states from previously generated phrases, providing context for the next word and supporting less duplication and greater topic coherence.The target-side context is updated as decoding proceeds.Source: §3.3.2 · pp. 4–5

4.1 Implementation Details

• Training uses tokenized, lowercased text with digit replacement, a 10-phrase target cap, a bidirectional GRU encoder, a forward GRU decoder, and beam search for generation.Articles with more than 10 target phrases are split; the implementation reports 558830 training text-keyphrase pairs.Source: §4.1 · p. 5

4.2 Datasets

• KP20k contains 527,830 training articles and 20,000 validation articles; Krapivin contains 2,304 papers, with the first 400 in alphabetical order used for evaluation and the rest added to training.Source: §4.2 · pp. 5–6

4.3 Baseline Models

• Baselines include extractive methods and generative RNN and CopyRNN; CopyRNNF keeps only the first prediction among duplicates defined by substring overlap.The comparison uses the post-processed CopyRNN baseline to assess CorrRNN’s end-to-end learning against a heuristic diversity rule.Source: §4.3 · p. 6

4.4 Evaluation Metrics

• Present keyphrases are evaluated with F1@K and absent keyphrases with R@K; generative methods also use α-NDCG, denoted N@K, to measure diversity.The α-NDCG trade-off parameter is set to 0.5, and Porter Stemmer is used to determine phrase identity.Source: §4.4 · p. 6

5.1 Present Phrase Prediction

• CorrRNN and CopyRNN outperform other baselines on present-phrase F1, and CorrRNN significantly outperforms CopyRNN (p < 0.01).The results cover the top 5 and top 10 predictions.Source: §5.1 · p. 6, Table 2 · p. 7
• CorrRNN and its variants surpass other methods on α-NDCG for present-phrase prediction.Source: §5.1 · p. 6, Table 3 · p. 7
• CopyRNNF performs a little better than CopyRNN on α-NDCG, while heuristic rules may hurt generative approaches’ F1 performance.Source: §5.1 · pp. 6–7

5.2 Absent Phrase Prediction

• CopyRNN and CorrRNN outperform RNN on absent-phrase recall, though the improvement is smaller than for present-phrase prediction.Source: §5.2 · p. 7, Table 4 · p. 7
• CopyRNN and CorrRNN have comparable recall on absent phrases, but CorrRNN performs better on diversity.Source: §5.2 · p. 7
• The authors attribute the generative models’ advantage over RNN to the copy mechanism’s usefulness for predicting absent phrases.Source: §5.2 · p. 7

5.3 Generalization Ability

• On DUC-2001, CorrRNN beats TextRank, KeyCluster, TopicRank, and CopyRNN, but falls behind three other baselines.CorrRNN and CopyRNN are trained on scientific publications and evaluated on news.Source: §5.3 · p. 7, Table 6 · p. 7
• CorrRNN is applied directly to news articles without adaptive adjustment after training on scientific publications.DUC-2001 contains 308 news articles with 2,488 manually assigned keyphrases and an average article length of 740 words.Source: §5.3 · p. 7
• Despite vocabulary changes, more unknown words, and potentially inapplicable correlations, CorrRNN can use positional and syntactic features to predict phrases.Source: §5.3 · p. 7

5.4.1 Model Ablation

• CorrRNN achieves the best overall ablation performance because present phrases are the majority, while CorrRNNC performs better on absent-phrase prediction.The models perform better on NUS and SemEval than Kravipin, possibly because higher-coverage, higher-diversity assignments benefit more from them.Source: §5.4.1 · p. 8

5.4.2 Visualization

• Coverage redirects attention toward other source positions as earlier phrases cover words, while review attention uses all previous phrases to generate the last phrase.In the example, the coverage vector spans topics including “machine learning,” “mean average precision,” “svm,” and “ranked retrieval systems.”Source: §5.4.2 · p. 8

5.4.3 Comparison with Heuristic Rules

• Heuristic post-processing does not fundamentally resolve coverage and duplication: it cannot add uncovered topics, and forced duplicate removal can hurt accuracy.The authors report negative results for CopyRNNF with post-processing.Source: §5.4.3 · p. 8

5.4.4 Complexity Analysis

• CorrRNN has 94,886,750 parameters versus CopyRNN’s 78,835,750, yet the authors report better performance and faster convergence.The review mechanism accounts for most of the added parameters, while coverage accounts for few.Source: §5.4.4 · p. 8

5.4.5 Case Study

• On the Table 7 example, CorrRNN adds correct present “voip” and absent “real time audio”; CopyRNN produces four “conferencing” phrases, versus CorrRNN’s two.The two added correct phrases cover topics that CopyRNN misses.Source: §5.4.5 · pp. 8–9, Table 7 · p. 9

• CorrRNN models correlations among multiple keyphrases end-to-end using coverage and review mechanisms.Source: §6 · p. 9
• Empirical studies show that CorrRNN alleviates duplication and coverage issues and improves diversity and coverage for keyphrase generation.Source: §6 · p. 9
• The authors identify CorrRNN as the first encoder-decoder model used for one-to-many keyphrase generation.Source: §6 · p. 9
• Future work will investigate multi-document keyphrase generation and incorporate structure or syntax information.Source: §6 · p. 9