Source-linked AI summary
Inter-sentence Relation Extraction with Document-level Graph Convolutional Neural Network
Sunil Kumar Sahu, Fenia Christopoulou, Makoto Miwa, Sophia Ananiadou
TL;DR
Inter-sentence relation extraction must capture local, non-local, syntactic, and semantic dependencies that existing methods do not fully exploit. The paper builds a labelled-edge GCNN on a document-level dependency graph and uses MIL with bi-affine scoring; it achieves comparable performance to state-of-the-art neural models on two biochemistry datasets, with all edge types effective for inter-sentence extraction.
Problem
Inter-sentence relation extraction requires local and non-local dependencies, but existing approaches do not adequately capture these dependencies across sentences.
Method
A document-level graph connects words through syntactic, sequential, coreference, and semantic dependencies, while labelled-edge GCNN and MIL-based bi-affine scoring model entity relations.
Results
Comparable performance to state-of-the-art neural models was achieved on two biochemistry datasets, and all graph edge types were effective for inter-sentence relation extraction.
Takeaways & Limitations
Combining multiple intra- and inter-sentence dependency types in a document-level graph supports inter-sentence relation extraction across the evaluated biochemistry datasets.
Takeaways & Limitations
The model is applied to biochemistry corpora, while broader applicability is stated as a future-use possibility rather than evaluated here.
Abstract
from arXiv · showhide
Inter-sentence relation extraction deals with a number of complex semantic relationships in documents, which require local, non-local, syntactic and semantic dependencies. Existing methods do not fully exploit such dependencies. We present a novel inter-sentence relation extraction model that builds a labelled edge graph convolutional neural network model on a document-level graph. The graph is constructed using various inter- and intra-sentence dependencies to capture local and non-local dependency information. In order to predict the relation of an entity pair, we utilise multi-instance learning with bi-affine pairwise scoring. Experimental results show that our model achieves comparable performance to the state-of-the-art neural models on two biochemistry datasets. Our analysis shows that all the types in the graph are effective for inter-sentence relation extraction.
1 Introduction
Inter-sentence relation extraction requires combining local and non-local dependencies across documents. The proposed model uses a labelled-edge GCNN over a document-level graph and MIL-based scoring to capture these dependencies.
- Motivation: Inter-sentence relations depend on local and non-local dependencies that sentence-level dependency trees and sequence models do not adequately capture.Co-reference can connect mentions such as Oxytocin and Oxt across sentences, while longer sequences challenge RNNs and CNNs.
- Proposed approach: The model represents words as graph nodes and uses labelled edges for syntactic, sequential, coreference, and semantic dependencies.The graph combines intra-sentence and inter-sentence information in a document-level structure.
- Proposed approach: Multi-instance learning with bi-affine pairwise scoring predicts relations from entity-node representations and multiple entity mentions.The approach aggregates mention-level evidence for the target entity pair.
- Evaluation and contributions: The model is evaluated on two biochemistry corpora and additionally introduces a distantly supervised chemical reactant-product dataset from PubMed abstracts.The contribution covers both model evaluation and dataset development.
2 Proposed Model
The proposed model builds a document-level dependency graph, encodes it with labelled-edge GCNN layers, and aggregates mention-pair predictions with MIL and bi-affine scoring.
- Task formulation: The task classifies a target entity pair using all of its document-level mentions, including a no-relation category.The model assumes the entity relationship can be inferred from the combined mention evidence.
- Graph Construction: The document graph uses words as nodes and labelled edges for syntactic, coreference, adjacent-sentence, adjacent-word, and self-node dependencies.These edge categories encode both local structure and non-local document dependencies.
- GCNN Layer: A labelled-edge GCNN updates each word representation from neighboring nodes while preserving graph structure and using edge-type-specific parameters.Stacked blocks gather information from increasingly distant neighbors, while edge-wise gating controls neighbor information.
- GCNN Layer: The model reduces over-parameterization by assigning separate parameters to top-N edge types and shared parameters to remaining rare types.This design is intended to avoid possible overfitting from separate parameters for every edge type.
- MIL-based Relation Classification: Two FFNNs project each encoded word into head and tail spaces before a bi-affine layer scores mention pairs and aggregates them into an entity-level score.MIL combines predictions across the multiple mention pairs for the target entities.
3 Experimental Settings
The model was evaluated on the CDR and newly created CHR biochemistry datasets against multiple baseline and state-of-the-art systems, using standardized preprocessing and training procedures.
- 3.1 Data Sets: The evaluation used the CDR dataset and a newly created CHR dataset with chemical relations derived through distant supervision.CHR was constructed from PubMed abstracts by aligning chemical entities with Biochem4j.
- 3.1 Data Sets: Entities sharing at least one knowledge-base identifier were grouped as the same entity, while ungrounded entities and self-relations were removed.The CDR data also underwent hypernym filtering.
- 3.2 Pre-processing: Syntactic dependencies came from Enju, coreference edges from Stanford CoreNLP, and tokenization from GENIA tools.These preprocessing tools supplied the linguistic structures used by the graph.
- 3.3 Baselines: The CDR comparison included five state-of-the-art systems plus reimplemented CNN-RE and RNN-RE baselines.All models used bi-affine pairwise scoring for relation detection.
- 3.4 Model Training: Models used 100-dimensional PubMed-trained GloVe embeddings, merged CDR training and development sets, and reported averages over five random seeds.Performance was measured with precision, recall, and F1-score.
4 Results
The GCNN outperformed the CNN-RE and RNN-RE baselines on both datasets and was comparable to a state-of-the-art neural model, while edge-type ablations supported the value of the graph dependencies.
- 4 Results: GCNN outperformed the CNN-RE and RNN-RE baselines on both the CDR and CHR datasets.The comparisons excluded additional enhancements such as joint NER training, ensembling, and heuristic rules.
- 4 Results: In CDR, GCNN was 1.6 percentage points below Gu et al. (2017) but comparable to Li et al. (2018).Gu et al. used separate models for intra- and inter-sentence pairs, whereas GCNN used one unified model.
- 4 Results: Top-4 edge types produced the best CDR development-set performance, with performance slightly deteriorating when more types were included.The top-4 setting was used in the other experiments.
- 4 Results: Removing any dependency category generally reduced inter-sentence and overall performance, although self-node and adjacent-sentence edges slightly harmed intra-sentence performance.Coreference did not affect intra-sentence pairs.
5 Related Work
Prior work addressed document-level relations with graph-based LSTMs, multi-instance learning, and Transformer encoders, while this study applies GCNNs to a document-level dependency graph.
- 5 Related Work: Graph-based LSTM networks were used for n-ary relation extraction across multiple sentences, with candidates restricted to spans of up to two sentences.Peng et al. and Song et al. studied protein-drug-disease associations.
- 5 Related Work: Verga et al. introduced multi-instance learning for document-level relation extraction.The present work differs by replacing Transformer encoding with GCNN encoding and entity-coreference dependencies.
- 5 Related Work: Earlier GCNN applications included citation networks, knowledge graphs, semantic role labeling, summarization, temporal relation extraction, and dependency-tree-based intra-sentence RE.This paper extends GCNNs to document-level graphs containing intra- and inter-sentence dependencies.
6 Conclusion
The paper concludes that labelled-edge GCNNs with document-level intra- and inter-sentence dependencies provide comparable performance to state-of-the-art neural models on two biochemistry datasets.
- 6 Conclusion: The proposed method uses a labelled-edge GCNN over a document-level graph whose word nodes are connected by multiple dependency types.MIL aggregates the multiple mention-level entity pairs.
- 6 Conclusion: The method achieved comparable performance to state-of-the-art neural models on two biochemistry datasets.The conclusion summarizes the reported cross-dataset result.
- 6 Conclusion: Ablation analysis found that all edge types were effective for inter-sentence relation extraction.The number of labelled edges was tuned to maintain the GCNN's parameter count.
- 6 Conclusion: The experiments were limited to biochemistry corpora, although the authors state that the method is applicable to other relation extraction tasks.Future work includes joint named entity recognition training and sub-word embeddings.
A Training and Hyper-parameter Settings
The proposed model used shared training settings across the CDR and CHR datasets, with hyperparameters tuned on the development set and summarized in Table 4.
- Hyperparameters were tuned on the development set, using the same settings for both CDR and CHR datasets.
- Training used Adam optimization with a 0.0005 learning rate, 0.75 decay, gradient clipping at 10, and early stopping after five patience epochs.
- Table 4 presents the best-performing hyperparameters selected for the proposed model.