Source-linked AI summary
Double Graph Based Reasoning for Document-level Relation Extraction
Shuang Zeng, Runxin Xu, Baobao Chang, Lei Li
TL;DR
Document-level relation extraction must capture relations among entities across multiple sentences, beyond sentence-level models. GAIN addresses this with double graphs and path reasoning, and experiments on DocRED show improved performance over previous methods.
Problem
Document-level relation extraction is needed because sentence-level models fail to recognize relations between entities across sentences.
Method
GAIN uses a heterogeneous mention-level graph for document-aware mention representations and an entity-level graph with path reasoning for relational inference.
Results
GAIN outperforms previous methods on DocRED, especially for inter-sentence and inferential relations.
Takeaways & Limitations
The double-graph design supports mention interactions across long-distance dependencies and relational inference among entities.
Takeaways & Limitations
The reported path reasoning considers only two-hop paths, although it can be extended to multi-hop paths.
Abstract
from arXiv · showhide
Document-level relation extraction aims to extract relations among entities within a document. Different from sentence-level relation extraction, it requires reasoning over multiple sentences across a document. In this paper, we propose Graph Aggregation-and-Inference Network (GAIN) featuring double graphs. GAIN first constructs a heterogeneous mention-level graph (hMG) to model complex interaction among different mentions across the document. It also constructs an entity-level graph (EG), based on which we propose a novel path reasoning mechanism to infer relations between entities. Experiments on the public dataset, DocRED, show GAIN achieves a significant performance improvement (2.85 on F1) over the previous state-of-the-art. Our code is available at https://github.com/DreamInvoker/GAIN .
1 Introduction
Document-level relation extraction addresses cross-sentence entities, repeated mentions, and relations requiring logical reasoning. GAIN tackles these challenges with double graphs and improves performance on DocRED.
- Sentence-level models miss relations between entities in different sentences, motivating document-level extraction for holistic knowledge understanding.
- Document-level extraction must aggregate repeated entity mentions and reason over relations distributed across sentences.The paper identifies cross-sentence entities, repeated mentions, and logical inference as major challenges.
- GAIN combines a heterogeneous mention-level graph with an entity-level graph and explicit path reasoning for multi-hop entity relations.The mention-level graph models mention interactions and document context; the entity-level graph supports relational inference.
- GAIN achieves a 2.85 F1-score improvement over the previous state-of-the-art on DocRED.The evaluation uses the public DocRED dataset.
2 Task Formulation
The task formalizes document-level relation extraction over entities, mentions, sentences, and a predefined set of relation types. It distinguishes intra-sentential from inter-sentential relations and defines K-hop relational reasoning through relation chains.
- A document consists of N sentences, and each entity is represented through its word-span mentions.
- The task extracts relation triples (e_i, r_ij, e_j) for entity pairs and relation types in a predefined set R.
- A relation is inter-sentential when the entities have no sentence in common, and intra-sentential when they share at least one sentence.
- K-hop relational reasoning predicts a relation from a K-length chain of existing relations connecting the head and tail entities.
3 Graph Aggregation and Inference Network (GAIN)
GAIN combines document-aware mention aggregation with entity-level path inference to model cross-sentence interactions and reason over relations before classification.
- GAIN comprises encoding, mention-level graph aggregation, entity-level graph inference, and classification modules.The architecture is presented as four main modules.
- Mention-level Graph Aggregation: The hMG represents mentions and the document with three edge types to model within-entity, co-occurrence, and document-wide interactions.Its document node acts as a pivot, making the distance between two mention nodes at most two.
- Mention-level Graph Aggregation: GCN aggregates neighboring features across the heterogeneous mention-level graph to produce document-aware node representations.The model concatenates hidden states from different GCN layers to retain features at multiple abstraction levels.
- Entity-level Graph Inference: The EG merges mentions of each entity into averaged entity representations and merges inter-entity edges connecting the same entity pair.The document node is excluded from the EG.
- Entity-level Graph Inference and Classification: GAIN models two-hop paths between entities, uses attention to fuse path information, and combines it with entity and document representations for multi-label relation prediction.The classifier input includes head and tail entities, comparison features, the document-node representation, and inferential path information.
4 Experiments
GAIN is evaluated on DocRED against sequential and graph-based baselines using F1, AUC, and their Ign variants, with analyses targeting module effectiveness, inter-sentence relations, inference, and a case study.
- 4.1 Dataset: DocRED contains 96 relation types, 132,275 entities, and 56,354 relational facts across 3,053 training, 1,000 development, and 1,000 test documents.Documents average about eight sentences; 40.7% of relation facts require multiple sentences and 61.1% require inference skills.
- 4.4 Results: 2.85/2.63 F1/Ign F1 is the test-set improvement of GAIN-BERTlarge over CorefRoBERTa-RElarge.GAIN-BERTbase also improves F1/Ign F1 over LSR-BERTbase by 2.19/2.03 on the test set.
- 4.6 Analysis & Discussion: Removing hMG decreases dev Ign F1 by 2.08/2.02 for GAIN-GloVe/GAIN-BERTbase, supporting its role in mention interaction and document-aware features.The ablation removes the heterogeneous Mention-level Graph and applies GCN directly to the Entity-level Graph.
- 4.6 Analysis & Discussion: Removing path inference decreases dev Ign F1 by 2.21/2.17 for GAIN-GloVe/GAIN-BERTbase, while removing the document node decreases it by 2.19/1.88.The results associate path inference with K-hop relational reasoning and the document node with information exchange among distant mentions.
- 4.6 Analysis & Discussion: GAIN improves both Intra-F1 and Inter-F1, and GAIN-BERTbase improves Infer-F1 by 5.11 over RoBERTa-REbase.Removing hMG affects Inter-F1 more than Intra-F1, while removing inference lowers GAIN-BERTbase Infer-F1 by 1.78.
5 Related Work
Related work moves from sentence-level extraction toward document-level models that capture cross-sentence information and relational reasoning, while GAIN separates aggregation and inference across two graphs.
- Sentence-level Relation Extraction: Sentence-level RE is restricted because many real-world relation facts can only be extracted across sentences.This limitation motivates the shift toward document-level relation extraction.
- Document-level Relation Extraction: Dependency-graph approaches capture document-specific features but may ignore relational inference, while HIN performs inference implicitly through a hierarchical network.GAIN instead adopts an explicit path reasoning mechanism.
- Graph-based Methods: GAIN selectively fuses all possible path information for an entity pair with attention, avoiding the extra overhead of walk-based path generation and its hyperparameter tuning.The comparison is made with prior graph-based relation extraction approaches.
- Graph-based Methods: GAIN uses separate mention-level and entity-level graphs to aggregate inter-sentence information and infer relations explicitly.This differs from approaches that place mention and entity nodes in one graph and conduct inference implicitly with GCN.
- Domain-specific Datasets: Some biomedical document-level RE datasets contain limited relation and entity types, making them less suitable testbeds for broad relational reasoning.CDR is cited as having one relation type and two entity types.
6 Conclusion
GAIN addresses the inter-sentence and relational-reasoning challenges of document-level relation extraction with double graphs at different granularities.
- Conclusion: GAIN combines a heterogeneous Mention-level Graph and an Entity-level Graph with path reasoning.The two graphs model mention interactions and support explicit relation inference, respectively.
- Conclusion: The Mention-level Graph captures document-aware features through interactions among mentions across the document.Its heterogeneous structure is part of GAIN’s double-graph design.
- Conclusion: On DocRED, GAIN outperforms previous methods, especially for inter-sentence and inferential relations.Ablation results also confirm the effectiveness of different model modules.
A Hyperparameter settings
GAIN’s hyperparameters are tuned on the development set using Ign F1, with separate settings reported for its GloVe, BERTbase, and BERTlarge variants.
- A Hyperparameter settings: Hyperparameters are manually tuned on the development set based on Ign F1, but not all hyperparameters are tuned.The final settings are listed separately for GAIN-GloVe, GAIN-BERTbase, and GAIN-BERTlarge.
- A Hyperparameter settings: The appendix reports separate configuration tables for GAIN-GloVe, GAIN-BERTbase, and GAIN-BERTlarge.These are identified as Tables 6, 7, and 8, respectively.