Source-linked AI summary
HIN: Hierarchical Inference Network for Document-Level Relation Extraction
Hengzhu Tang, Yanan Cao, Zhenyu Zhang, Jiangxia Cao, Fang Fang, Shi Wang, Pengfei Yin
TL;DR
Document-level relation extraction must infer facts spread across multiple sentences, and prior approaches often lack information beyond the entity level. HIN combines entity-, sentence-, and document-level inference, achieving state-of-the-art performance on DocRED.
Problem
Many relational facts span multiple sentences, while previous document-level methods often use only entity-level information and face challenges obtaining and aggregating inference at multiple granularities.
Source: §1 · p. 2Method
HIN uses translation and bilinear transformations for entity-pair inference and aggregates information across entity, sentence, and document levels.
Source: §1 · p. 2, §3.2 · p. 5, §4.2 · p. 8Results
HIN achieves state-of-the-art performance on the largest human-annotated DocRED dataset, and BERT representations further boost performance.
Source: §1 · p. 3, §6 · pp. 11–12Takeaways & Limitations
Experimental analysis indicates that both HIN's inference mechanism and hierarchical aggregation approach play an important role.
Source: §6 · p. 12BERT-Two-Step results are not counted because no official code was available.
Source: Analysis by the number of supporting… · p. 9
Abstract
from arXiv · showhide
Document-level RE requires reading, inferring and aggregating over multiple sentences. From our point of view, it is necessary for document-level RE to take advantage of multi-granularity inference information: entity level, sentence level and document level. Thus, how to obtain and aggregate the inference information with different granularity is challenging for document-level RE, which has not been considered by previous work. In this paper, we propose a Hierarchical Inference Network (HIN) to make full use of the abundant information from entity level, sentence level and document level. Translation constraint and bilinear transformation are applied to target entity pair in multiple subspaces to get entity-level inference information. Next, we model the inference between entity-level information and sentence representation to achieve sentence-level inference information. Finally, a hierarchical aggregation approach is adopted to obtain the document-level inference information. In this way, our model can effectively aggregate inference information from these three different granularities. Experimental results show that our method achieves state-of-the-art performance on the large-scale DocRED dataset. We also demonstrate that using BERT representations can further substantially boost the performance.
1 Introduction
Document-level relation extraction must combine evidence across sentences, but prior methods often rely only on entity-level information. HIN aggregates inference at entity, sentence, and document levels and achieves state-of-the-art performance on DocRED.
1 Introduction
- Sentence-level extraction misses facts distributed across sentences, while prior document-level methods using only entity-level information are inadequate.Source: §1 · p. 2
- HIN derives entity-level inference with translation and bilinear transformations, then compares it with sentence representations and hierarchically aggregates sentence-level evidence.Source: §1 · pp. 2–3
- The model combines entity-level and document-level representations for classification, aggregating inference from entity to sentence to document level.Source: §1 · p. 3
- HIN achieves state-of-the-art performance on DocRED, and BERT representations substantially boost its performance.Source: §1 · p. 3
- HIN performs much better than previous work when a relation requires many supporting sentences.Source: §1 · p. 3
2 Task Description
Document-level relation extraction takes a document with annotated entities and their mentions as input and identifies related entity pairs.
2 Task Description
- Given a document with annotated entities and their mentions, document-level relation extraction identifies all related entity pairs in that document.Source: §2 · p. 3
3 Proposed Approach
HIN builds relation evidence from entity representations, compares that evidence with sentence representations, and aggregates sentence-level inference into a document representation. It then combines entity- and document-level inference to predict relations.
3.1 Input Layer
- The input layer combines word, entity-type, and coreference embeddings into a vector for each word and arranges these vectors as document matrix X.Entity mentions of the same entity share an entity ID, which is embedded as a coreference feature.Source: §3.1 · p. 4
3.2 Entity-Level Inference Module
- HIN averages word embeddings into mention vectors and mention vectors into entity representations, then projects entities into K subspaces for parallel entity-level inference.The subspace representations and relative-distance embeddings are fed to an FFNN to form final entity-level inference information.Source: §3.2 · pp. 4–5
- Entity-level inference representations from different latent spaces and relative-distance embeddings feed an FFNN to form the final entity-level inference information.The model uses the distance between the entities’ first mentions as their relative distance.Source: §3.2 · p. 5
3.3 Hierarchical Document-Level Inference Module
- Sentence attention selects informative words, semantic matching compares each sentence vector with entity-level inference, and a BiLSTM-attention network aggregates sentence evidence into document-level inference.The final inference representation concatenates entity-level and document-level information before relation prediction.Source: Document-Level Inference · p. 6, Sentence-Level Inference · p. 6, §3.4 · p. 7
3.4 Prediction Layer
- The prediction layer concatenates entity- and document-level inference, uses a sigmoid FFNN for relation probabilities, and trains with binary cross entropy.A development-set F1 threshold controls how many predicted facts are extracted on the test set.Source: §3.4 · p. 7
4 Experiments
Experiments evaluate HIN on DocRED against document-level RE baselines using F1 and Ign F1. HIN-GloVe improves F1 over BiLSTM-RE, HIN-BERT outperforms previous methods, and HIN performs especially well as the number of supporting sentences increases.
4.1 Dataset
- DocRED contains over 5,053 documents, 40,276 sentences, 132,375 entities, and 96 frequent relation types; 40.7% of relational facts require multiple sentences.The dataset is human-annotated and constructed from Wikidata and Wikipedia.Source: §4.1 · p. 7
4.2 Comparison Models & Evaluation Metrics
- HIN is compared with CNN/LSTM/BiLSTM-RE, Context-Aware, BERT-RE, and BERT-Two-Step baselines.The paper describes each baseline as a document-level RE model.Source: §4.2 · pp. 7–8
- The experiments report F1 and Ign F1, with Ign F1 excluding relational facts present in both training and dev/test sets.The study uses GloVe embeddings or BERT representations; BERT base uncased representations are projected to 100 dimensions and fixed during training.Source: §4.2 · p. 8, §4.3 · p. 8
4.4 Experimental Results and Analyses
- Removing translation, bilinear transformation, multi-space projection, sentence-level inference, or hierarchical aggregation lowers HIN-BERT's DocRED dev F1 by 1.21%, 2.02%, 1.72%, 1.25%, or 2.81%, respectively.Removing all these factors lowers F1 by 4.21%; Table 2 presents the ablation results.Source: Table 2 · p. 9, Ablation Study · p. 9
- HIN-GloVe achieves a significant 2.24% F1 improvement; HIN-BERT consistently improves over BERT-based models, exceeds Context-Aware by 2.60% in F1, and outperforms all previous methods.Source: Overall Performance · p. 9
- HIN models perform better than other baselines as supporting-sentence counts increase; HIN-GloVe surpasses BERT-RE above four supporting sentences.Above seven supporting sentences, HIN-GloVe and other baselines behave alike, which the authors attribute to very few dev examples in that range.Source: Figure 3 · p. 10, Analysis by the number of supporting… · p. 10
- The case study covers logical inter-sentence reasoning, coreference reasoning, and a case combining context with common-sense knowledge.The authors leave adding external knowledge to address the third case as future work; Table 3 shows predictions by BERT-RE and HIN-BERT.Source: Case Study · p. 10, Table 3 · p. 11
5 Related Work
Earlier RE work largely modeled relations within single sentences, while document-level methods use information across documents. The section situates HIN among neural RE models and notes that prior document-level methods often relied mainly on entity-level information.
5 Related Work
- Most existing RE work modeled relations within a single sentence, although documents provide more information and motivate document-level RE.The section frames moving from sentence-level to document-level research as necessary.Source: §5 · p. 11
- Neural RE approaches included CNNs with lexical features, tree-structured LSTMs [9], and CNN/RNN models with attention.The section also notes that optimization algorithms [6,7,8] made these neural models more effective.Source: §5 · p. 11
- DocRED introduced a large-scale human-annotated document-level RE dataset; subsequent methods used entity representations with bilinear prediction or a two-step BERT process [13].The section states that most previous methods used only entity-level information, which it considers inadequate.Source: §5 · p. 11
6 Conclusion
The conclusion presents HIN as a hierarchical approach for aggregating entity-, sentence-, and document-level inference in document-level RE. It reports state-of-the-art performance on DocRED and says both the inference mechanism and hierarchical aggregation play an important role.
6 Conclusion
- HIN achieves state-of-the-art performance on the human-annotated DocRED dataset.The conclusion identifies DocRED as the largest human-annotated dataset.Source: §6 · pp. 11–12
- HIN hierarchically aggregates inference information across entity, sentence, and document levels for document-level RE.The approach is presented as making use of information at these different granularities.Source: §6 · p. 11
- Experimental analysis finds that both HIN's inference mechanism and hierarchical aggregation play an important role.Source: §6 · p. 12
- The authors plan to incorporate external knowledge to further improve HIN.Source: §6 · p. 12