Source-linked AI summary
Joint Extraction of Entities and Relations Based on a Novel Tagging Scheme
Suncong Zheng, Feng Wang, Hongyun Bao, Yuexing Hao, Peng Zhou, Bo Xu
TL;DR
The paper addresses joint extraction of entities and relations without separately identifying entities and relations. It introduces a tagging-based end-to-end approach and reports that its methods outperform most existing alternatives, with the best model achieving a 3% F1 improvement over CoType.
Problem
Separate entity recognition and relation classification can cause erroneous information transfer, motivating direct joint extraction of entity-relation triplets.
Method
A novel tagging scheme encodes entity positions, relation types, and relation roles, enabling neural end-to-end models to extract entity-relation triplets directly.
Results
3% improvement in F1 over CoType was achieved by LSTM-LSTM-Bias, which outperformed all other methods on the public dataset.
Takeaways & Limitations
Tagging-based methods were better than most existing pipelined and joint learning methods for jointly extracting entities and relations.
Takeaways & Limitations
The approach remains limited in distinguishing relationships between entity pairs and identifying overlapping relations.
Abstract
from arXiv · showhide
Joint extraction of entities and relations is an important task in information extraction. To tackle this problem, we firstly propose a novel tagging scheme that can convert the joint extraction task to a tagging problem. Then, based on our tagging scheme, we study different end-to-end models to extract entities and their relations directly, without identifying entities and relations separately. We conduct experiments on a public dataset produced by distant supervision method and the experimental results show that the tagging based methods are better than most of the existing pipelined and joint learning methods. What's more, the end-to-end model proposed in this paper, achieves the best results on the public dataset.
1 Introduction
Joint entity-relation extraction is important for knowledge extraction, but pipelined methods separate the subtasks and risk error propagation. The paper proposes a tagging scheme and end-to-end models that directly model entity-relation triplets, with experiments showing strong performance on a distant-supervision dataset.
- Motivation: Joint extraction detects entity mentions and their semantic relations simultaneously from unstructured text.Unlike Open IE, relations come from a predefined relation set that may not appear in the sentence.
- Existing approaches: Pipelined methods extract entities before relations, so independent subtasks can neglect their relevance and transmit recognition errors.The separated framework offers flexibility but may affect relation classification through erroneous entity-recognition results.
- Existing approaches: Joint learning integrates entity and relation information in one model but often relies on complicated feature engineering and external NLP toolkits.These dependencies may also lead to error propagation.
- Proposed approach: The proposed tagging scheme encodes entities and their relationships in novel tags, transforming joint extraction into a tagging problem.This enables neural-network modeling without complicated feature engineering and directly represents triplets containing two entities and one relation.
- Proposed approach: Different LSTM-based end-to-end models are investigated, including a biased loss designed to enhance the association between related entities.The study modifies decoding with a biased loss to suit the novel tags.
- Evaluation: On a public distant-supervision dataset, the tagging scheme is effective and the proposed end-to-end model achieves the best reported results.The method is supervised, while the evaluation dataset is produced using distant supervision.
2 Related Works
Prior work primarily uses either pipelined entity recognition and relation classification or joint models. The paper situates its approach as a special tagging-based end-to-end method that avoids separate NER and RC stages.
- Pipelined methods: Pipelined methods treat the task as separate named entity recognition and relation classification problems.Earlier NER models include HMMs and CRFs, while neural architectures have also been applied to sequential token tagging.
- Joint methods: Joint models extract entities and relations with a single model rather than treating them as two separated tasks.Most existing joint methods are feature-based structured systems.
- Joint methods: An LSTM-based joint model can reduce the manual work required by feature-based approaches.The cited approach uses LSTM to extract entities and relations jointly.
- End-to-end tagging: The proposed method uses a special tagging manner to extract results end to end without separate NER and RC components.End-to-end models map input sentences into meaningful vectors and produce output sequences, commonly for sequence tagging.
3 Method
The method converts joint entity–relation extraction into sequence tagging, then decodes tagged entities into relation triplets. Its end-to-end model uses Bi-LSTM encoding, LSTM decoding, and a biased objective that emphasizes relational tags.
- 3.1 The Tagging Scheme: The tagging scheme encodes entity position, predefined relation type, and relation role in each label, transforming joint extraction into tagging.BIES marks entity position, relation roles 1 and 2 identify triplet entities, and O marks unrelated words.
- 3.2 From Tag Sequence To Extracted Results: Tags with the same relation type are combined according to their roles to recover triplets such as (Entity1, RelationType, Entity2).The scheme assigns role 1 to the first entity and role 2 to the second entity in a triplet.
- 3.2 From Tag Sequence To Extracted Results: When multiple triplets share a relation type, the method pairs entities using the nearest principle.Each entity is assigned to the closest compatible entity for constructing the corresponding triplet.
- 3.2 From Tag Sequence To Extracted Results: The method considers cases where each entity belongs to a triplet and leaves overlapping-relation identification for future work.This defines the stated scope boundary of the extraction method.
- 3.3 The End-to-end Model: The end-to-end model uses a Bi-LSTM encoding layer and an LSTM-based decoding layer to produce the tag sequence.The Bi-LSTM combines forward and backward contextual representations, while the decoder models tag interactions.
- 3.3 The End-to-end Model: The biased objective distinguishes O tags from relational tags and increases relational-tag influence as the bias weight α grows.The model maximizes data log-likelihood, with the switching function separating O-tag and relational-tag losses.
4 Experiments
The experiments evaluate the proposed tagging methods on the NYT dataset and compare them with pipelined, jointly extracting, and end-to-end baselines. LSTM-LSTM-Bias achieves the strongest reported F1, while end-to-end and tagging approaches generally outperform traditional alternatives.
- Experimental setting: The NYT dataset contains 353k training triplets, 3,880 manually labeled test triplets, and 24 relation types.Training data is produced by distant supervision, while the test set is manually labeled for quality.
- Experimental setting: Table 1 compares pipelined methods, jointly extracting methods, and the proposed tagging methods using precision, recall, and F1.Standard deviations are also reported for the tagging methods.
- Experimental results: LSTM-LSTM-Bias outperforms all other methods in F1 and improves F1 by 3% over CoType, the best competing method.The result is reported as evidence for the effectiveness of the proposed method and tagging scheme.
- Experimental results: Jointly extracting methods outperform pipelined methods, while tagging methods outperform most jointly extracting methods.This comparison supports the reported validity of the tagging scheme for jointly extracting entities and relations.
- Experimental results: End-to-end models significantly improve precision over traditional methods, but only LSTM-LSTM-Bias balances precision and recall effectively.The authors attribute this pattern to neural models fitting the training data and potentially having lower expansibility.
- Experimental results: LSTM-LSTM performs better than LSTM-CRF because related tags may be distant, making LSTM decoding better suited than CRF decoding here.LSTM-LSTM-Bias further adds a bias weight that strengthens entity tags and weakens invalid tags.
5 Analysis and Discussion
The analysis examines entity-pair prediction, biased-loss behavior, and representative successes and failures of the end-to-end models. The biased model better associates related entities, while relation-role errors and misleading patterns remain challenges.
- 5.1 Error Analysis: Entity pairs have higher precision but lower recall than individual entities because some predicted entities lack their corresponding partners.This produces more single entities and fewer complete (E1, E2) pairs.
- 5.1 Error Analysis: About 3% of test data is predicted incorrectly because the relation type is wrong.
- 5.2 Analysis of Biased Loss: The biased approach yields relatively fewer single entities than LSTM-CRF and LSTM-LSTM, indicating stronger association between entity pairs.Single entities are entities that cannot find their corresponding entities.
- 5.2 Analysis of Biased Loss: When α = 10, LSTM-LSTM-Bias balances precision and recall and achieves the best F1 scores; overly large α harms accuracy, while overly small α reduces recall.
- 5.3 Case Study: In a long-distance example, LSTM-LSTM-Bias extracts both related entities, whereas LSTM-LSTM extracts only “Florida” and misses “Panama City Beach”.The example concerns entities whose relationship is difficult to detect because they are far apart.
- 5.3 Case Study: The biased model can identify an entity pair while still reversing the relational roles, showing that relationship distinction remains an improvement area.In the example, the roles of “Stephen A. Schwarzman” and “Blackstone Group” are reversed.
6 Conclusion
The paper proposes a novel tagging scheme and end-to-end models for jointly extracting entities and relations, but overlapping relations remain challenging.
- The paper proposes a novel tagging scheme and investigates end-to-end models for jointly extracting entities and relations.
- The proposed method enhances the effect of entity tags, while the association between corresponding entities still requires refinement.
- Overlapping relations remain a shortcoming because the current model cannot assign multiple tags to a word.
- The planned replacement of softmax with multiple classifiers would allow words to receive multiple tags and address overlapping relations.