Source-linked AI summary
More Data, More Relations, More Context and More Openness: A Review and Outlook for Relation Extraction
Xu Han, Tianyu Gao, Yankai Lin, Hao Peng, Yaoliang Yang, Chaojun Xiao, Zhiyuan Liu, Peng Li, Maosong Sun, Jie Zhou
TL;DR
Relation extraction must address sparse long-tail relations, noisy distant supervision, and increasingly complex extraction settings. This paper reviews existing methods, synthesizes four directions for more powerful RE, and investigates key challenges through statistics and experiments.
Problem
RE lacks convincing, quantitative analysis for important settings such as cross-sentence extraction, while real-world relation distributions leave most relations with few training examples.
Method
The paper comprehensively reviews RE methods and recent complex-scenario explorations, then organizes future progress around four directions and examines two key challenges with statistics and experiments.
Results
The review tracks RE's development from pattern and statistical methods to neural models, while experiments show that entity names and text both provide crucial information for RE.
Takeaways & Limitations
More powerful RE requires using more data, learning relations more efficiently, handling more complicated context, and extending to more open domains.
Takeaways & Limitations
Distant supervision inevitably introduces wrong labels because sentences mentioning an entity pair may not express their knowledge-graph relation.
Abstract
from arXiv · showhide
Relational facts are an important component of human knowledge, which are hidden in vast amounts of text. In order to extract these facts from text, people have been working on relation extraction (RE) for years. From early pattern matching to current neural networks, existing RE methods have achieved significant progress. Yet with explosion of Web text and emergence of new relations, human knowledge is increasing drastically, and we thus require "more" from RE: a more powerful RE system that can robustly utilize more data, efficiently learn more relations, easily handle more complicated context, and flexibly generalize to more open domains. In this paper, we look back at existing RE methods, analyze key challenges we are facing nowadays, and show promising directions towards more powerful RE. We hope our view can advance this field and inspire more efforts in the community.
1 Introduction
Relation extraction connects textual mentions to structured relational facts that support downstream applications, but current systems remain limited by simplified assumptions about data, relations, context, and domains. The paper reviews existing methods and organizes future work around four directions while identifying two additional challenges.
- Motivation: Relation extraction classifies relations between identified entity mentions from textual context, producing structured facts useful for knowledge graph completion, search, and question answering.Relational facts organize knowledge in triplet form and may be explicit or inferred from text.
- Current setting: Existing methods mainly train on large human-annotated datasets to classify entity pairs within one sentence and a pre-defined relation set.The paper notes that this setting simplifies real-world extraction.
- Challenges: Real-world RE requires addressing expensive annotation, sparse long-tail relations, multi-sentence context, and open-ended relation growth.These constraints motivate investigation of more complex RE scenarios.
- Promising directions: The paper reviews existing methods and proposes four directions: utilizing more data, more efficient learning, more complicated context, and more open domains.The directions target higher-quality data, limited-example learning, complex contexts, and undefined relations.
- Additional challenges: It also highlights challenges in learning from text or names and constructing datasets for special interests.The paper presents these as additional challenges for existing methods.
2 Background and Existing Work
Relation extraction developed from pattern-based and statistical approaches to neural models that automatically represent textual semantics. The review describes their methods and reports substantial performance gains associated with neural RE on established benchmarks.
- RE pipeline: A typical RE system identifies entities, links them to knowledge graphs when needed, and classifies their relation from context.The classification component is treated as the most crucial and difficult step because it requires semantic understanding.
- Pattern Extraction Models: Pattern-based methods automatically construct rules from analyzed sentences but generally require human examination because the patterns may contain mistakes.Later work sought better coverage, accuracy, pattern formats, and extraction efficiency.
- Statistical Relation Extraction Models: Statistical methods use engineered features, kernels, graphical models, or embeddings to represent entity pairs and context for relation classification.Feature and kernel methods require substantial design effort, while graphical and embedding methods have limited model capacity.
- Neural Relation Extraction Models: Neural RE replaces handcrafted features with learned representations and uses architectures including CNNs, RNNs, GNNs, attention networks, Transformers, and pretrained language models.These models target semantic, local-pattern, sequential, graph-based, or globally attended representations.
- Neural Relation Extraction Models: Neural models brought a vast increase in state-of-the-art performance on SemEval-2010 Task 8 compared with earlier years.The figure caption attributes great improvement since 2013 to adopting neural models.
3 “More” Directions for RE
The paper organizes promising paths toward more powerful relation extraction around using more data, learning efficiently from scarce examples, handling complex context, and opening relation inventories beyond predefined sets.
- Overview: Existing RE models achieve strong benchmark results but mainly rely on abundant human annotations and classify predefined relations within single sentences.This simplified setting limits their fit to complex real-world cases.
- 3.1 Utilizing More Data: Distant supervision scales training data by aligning knowledge-graph facts with sentences, but its automatic labels can be noisy when sentences do not express the matched relation.The paper reviews denoising methods and calls for schemes with better coverage and less noise.
- 3.1 Utilizing More Data: Wiki-Distant expands beyond NYT-10 with more relations and instances and a more reasonable N/A proportion, yet remains more challenging for current models.It is constructed from Wikipedia and Wikidata using the NYT-10 procedure.
- 3.2 Performing More Efficient Learning: Long-tail relation distributions leave many relations with fewer than 10 training instances, motivating few-shot methods that must generalize as relation counts grow and semantics become similar.The paper reports performance drops in both settings, indicating unresolved challenges for efficient learning.
- 3.3 Handling More Complicated Context: Most relational facts require document-level or cross-sentence context, but existing methods largely focus on intra-sentence extraction and lack targeted datasets for isolating cross-sentence performance.The paper identifies specialized evaluation resources as an open need.
- 3.4 Orienting More Open Domains: Because relations continually emerge and predefined inventories cannot cover them all, open-domain RE must detect undefined relations while also addressing redundant canonicalization and N/A cases.The review highlights these issues as unresolved aspects of relation discovery and Open IE.
4 Other Challenges
The paper examines how entity names and context contribute to relation extraction, finding that both are important and that names can sometimes outperform masked text. It also identifies a shortage of datasets designed for specific RE problems.
- 4.1 Learning from Text or Names: Both entity names and surrounding text provide crucial information for relation extraction.Experiments compare normal, masked-entity, and only-entity settings; removing either source causes a substantial performance drop.
- 4.1 Learning from Text or Names: In some cases, using only entity names outperforms using only text with entity names masked.The comparison is reported across state-of-the-art models and benchmarks evaluated under the three experimental settings.
- 4.1 Learning from Text or Names: The observed reliance on entity names differs from the human tendency to classify relations mainly from textual descriptions.The paper presents this mismatch as a problem requiring further investigation into how models understand relational language.
- 4.2 RE Datasets towards Special Interests: Existing datasets rarely target specific RE problems, limiting convincing quantitative analysis of capabilities such as cross-sentence extraction.Researchers often use handcrafted subsets or case studies, motivating well-recognized, well-designed, fine-grained datasets for special interests.
5 Conclusion
The paper reviews relation extraction research, organizes promising directions toward more powerful systems, and investigates two challenges with supporting statistics and experiments. It aims to clarify progress and problems while encouraging further work.
- 5 Conclusion: The paper reviews RE methods, organizes four directions toward more powerful systems, and investigates two challenges using literature, statistics, and experiments.The directions are utilizing more data, performing more efficient learning, handling more complicated context, and orienting toward more open domains.