Source-linked AI summary
Neural Related Work Summarization with a Joint Context-driven Attention Mechanism
Yongzhen Wang, Xiaozhong Liu, Zheng Gao
TL;DR
Automatic related-work summarization relies on human-engineered features and makes authors read reference papers to find relevant material. This paper uses seq2seq with joint attention over textual and graph context; in evaluation, its approach ranked first more often than the compared systems and significantly outperformed all six contrast methods.
Problem
Automatic related-work summarization relies heavily on human-engineered features, and authors still must read reference papers carefully to locate relevant content.
Source: §1 · p. 1Method
The neural seq2seq summarizer uses joint context-driven attention to measure relevance in full texts and a heterogeneous bibliography graph simultaneously.
Source: §1 · p. 2, §3.3 · p. 5Results
40% of ratings ranked our approach first (NltkSum: 21%); ANOVA found it better than all six contrast methods (p < 0.01).
Source: §4.2 · p. 9Takeaways & Limitations
The experiments support using textual and graphic context to characterize publication relationships in automatic related-work summarization.
Source: §4.2 · p. 7, §5 · p. 9The extractive approach is not presented as a final solution because of plagiarism concerns; the authors plan a fully abstractive version.
Source: §5 · p. 9
Abstract
from arXiv · showhide
Conventional solutions to automatic related work summarization rely heavily on human-engineered features. In this paper, we develop a neural data-driven summarizer by leveraging the seq2seq paradigm, in which a joint context-driven attention mechanism is proposed to measure the contextual relevance within full texts and a heterogeneous bibliography graph simultaneously. Our motivation is to maintain the topic coherency between a related work section and its target document, where both the textual and graphic contexts play a big role in characterizing the relationship among scientific publications accurately. Experimental results on a large dataset show that our approach achieves a considerable improvement over a typical seq2seq summarizer and five classical summarization baselines.
1 Introduction
Automatic related-work summarization could reduce the labor of reading references, but it must select content that reflects the target paper’s topic rather than merely compressing sources. The paper proposes a neural approach combining textual and graph context.
1 Introduction
- Automatic related-work summarization relies heavily on human-engineered features, while authors must read reference papers and locate relevant content.Source: §1 · p. 1
- Lexical overlap can mislead selection: the DSSM literature review devotes about 58% of its text to Deep Learning despite other topics’ abstract similarity.Emerging concepts also pose terminology-variation and paraphrasing challenges.Source: §1 · pp. 1–2
- A heterogeneous bibliography graph captures citation, collaboration, keyword, and venue relations, but edge types vary in usefulness and existing graph-mining approaches often require human supervision.Source: §1 · p. 2
- On 8,080 papers, the approach significantly outperforms a typical seq2seq summarizer and five classical summarization baselines.Source: §1 · p. 2
- The proposed joint context-driven attention measures contextual relevance in full texts and a heterogeneous bibliography graph to characterize publication relationships.Source: §1 · pp. 1–2
2 Related Work
Prior related-work summarizers used rule-based, statistical, or graph-guided approaches that required manual intervention, while neural seq2seq summarizers addressed single-document tasks. This paper proposes a neural data-driven approach for related-work summarization and evaluates it on a larger dataset.
2 Related Work
- Prior related-work methods used rules, topic models, sentence classification, or keyword graphs, and their dependence on manual intervention limited application scale.Table 1 reports data scales of previous studies.Source: §2 · p. 2, Table 1 · p. 2
- Existing seq2seq summarizers addressed single-document scenarios, unlike related-work summarization’s reference-collection setting.Source: §2 · p. 2
- The paper presents a neural data-driven related-work summarizer with joint attention over full texts and a heterogeneous bibliography graph.The authors describe this as their first neural data-driven solution for this task.Source: §2 · p. 3
- The study uses an extractive sentential approach, assumes the reference collection is supplied as input, and excludes reference citation sentences.Source: §2 · p. 3
3 Methodology
The method formulates related-work generation as sequential sentence selection, then combines graph-based reference discovery with a hierarchical seq2seq extractor. The extractor’s attention uses textual and graph context alongside saliency and novelty to guide sentence decisions.
3.1 Problem Formulation
- Each reference is traversed in descending significance, candidate sentences receive binary inclusion labels, and the model maximizes their likelihood.The input is the target paper and its reference collection.Source: §3.1 · p. 3
3.2 Random Walk on Heterogeneous Bibliography Graph
- The graph represents four object types and ten edge types, while an unsupervised edge-type usefulness distribution guides random walks toward references matching the target collection.Node2vec embeds graph nodes, and an evolutionary algorithm tunes the distribution using reference-ranking fitness.Source: §3.2 · pp. 3–4
3.3 Neural Extractive Summarization
- A hierarchical CNN–LSTM encoder feeds an attention-based decoder that labels sentences using saliency, novelty, and textual and graph relevance to the target.The CNN derives sentence-level meanings from words; the LSTM models longer-range dependencies, and attention supplies context for each binary decision.Source: §3.3 · pp. 4–6
4 Experiment
On an 8,080-paper ACM dataset, the proposed summarizer outperformed six representative methods, with human ratings also favoring it over all six. Ablations and a reference-cluster example support the use of saliency, novelty, textual and graphic relevance, and edge-type usefulness.
4.1 Experimental Setup
- The 8,080-paper ACM sample required more than 15 in-dataset references and a related-work section of at least 500 words; evaluation used ten-fold cross-validation and ROUGE-1/2/L.Training labels were derived by selecting sentences with high ROUGE-2 scores against native related-work sections.Source: §4.1 · pp. 6–7
4.2 Results and Discussion
- Adding saliency and novelty as attention factors improved the model’s ability to locate required summary content.The comparison is between P.void, P.S, and P.S+N.Source: §4.2 · p. 7
- Textual relevance improved alignment, while comparisons indicate that graphic context and a heterogeneous graph provide relationship information beyond text or a homogeneous citation graph.The progression compares P.S+N with P.S+N+Rt, then textual-only with graphic relevance, and homogeneous with heterogeneous graph context.Source: §4.2 · p. 7
- Adding EUD to heterogeneous-graph relevance was reported as indispensable for organizing accurate contextual relevance.This compares P.S+N+Rteg with P.S+N+Rteg+EUD.Source: §4.2 · p. 7
- In the DSSM reference-cluster example, adding relevance—especially graphic context—improved sampling of “Deep Learning” and sharply reduced content from “Other Sources.”The authors attribute greater robustness to graph connectivity patterns rather than n-gram statistics.Source: Figure 3 · p. 7, §4.2 · pp. 7–8
- The proposed approach outperformed six representative methods on ROUGE evaluation; NltkSum led the generic baselines on ROUGE-1/2 and had comparable ROUGE-L fluency.The evaluation covered 8,080 ACM papers.Source: Table 2 · p. 8, §4.2 · p. 8
- Raters ranked the proposed approach first 40% of the time, and ANOVA found it significantly better than all six comparison methods (p < 0.01).Human evaluation covered 35 papers with more than 30 references.Source: §4.2 · pp. 8–9, Table 3 · p. 9
5 Conclusion
The proposed seq2seq summarizer uses joint context-driven attention over full texts and a heterogeneous bibliography graph, and experiments show superiority over six representative summarization baselines. The conclusion identifies logical sentence organization, citation sentences, and a fully abstractive version as future directions, citing plagiarism concerns about extractive methods.
5 Conclusion
- The approach outperforms six representative summarization baselines.Experiments also demonstrate the validity of the proposed attention mechanism.Source: §5 · p. 9
- The authors propose organizing selected sentences logically, potentially using a topic hierarchy tree.They also identify reference citation sentences as a concise, universal data source for scientific summarization.Source: §5 · p. 9
- The authors plan a fully abstractive version because they consider extractive methods insufficient for literature review generation due to plagiarism concerns.The abstractive version is reserved for further studies.Source: §5 · p. 9