PaperlayerView exact revision on arXiv

Source-linked AI summary

A Comprehensive Survey on Relation Extraction: Recent Advances and New Frontiers

Xiaoyan Zhao, Yang Deng, Min Yang, Lingzhi Wang, Rui Zhang, Hong Cheng, Wai Lam, Ying Shen, Ruifeng Xu

arXiv:2306.02051v3cs.CLcs.AI

TL;DR

Relation extraction is important for extracting structured relations from text, but real-world settings remain challenging because data, domains, languages, and relation structures vary. This survey synthesizes deep-learning and pre-trained-language-model approaches through a new taxonomy, reviews datasets, metrics, challenges, and solutions, and identifies future directions. It concludes that continued work is needed on domain resources, specialized models, cross-lingual extraction, and other emerging RE problems.

  • Problem

    RE supports knowledge-intensive NLP applications, while existing surveys insufficiently cover recent deep-learning, pre-trained-language-model, challenging-setting, and domain-specific advances.

  • Method

    The survey reviews RE resources and methods, proposes a taxonomy based on text representation, context encoding, and triplet prediction, and analyzes challenges and future directions.

  • Results

    The survey provides a comprehensive synthesis of recent RE techniques, performance across challenging settings and domains, and solutions for emerging RE problems.

  • Takeaways & Limitations

    The review identifies domain-specific resources and models, cross-lingual methods, and other frontier directions as continuing priorities for RE research.

  • Takeaways & Limitations

    Current domain-specific datasets can be either too narrow in semantic coverage or too broad in generic relations, limiting practical domain RE development.

Abstract

from arXiv · show

Relation extraction (RE) involves identifying the relations between entities from underlying content. RE serves as the foundation for many natural language processing (NLP) and information retrieval applications, such as knowledge graph completion and question answering. In recent years, deep neural networks have dominated the field of RE and made noticeable progress. Subsequently, the large pre-trained language models have taken the state-of-the-art RE to a new level. This survey provides a comprehensive review of existing deep learning techniques for RE. First, we introduce RE resources, including datasets and evaluation metrics. Second, we propose a new taxonomy to categorize existing works from three perspectives, i.e., text representation, context encoding, and triplet prediction. Third, we discuss several important challenges faced by RE and summarize potential techniques to tackle these challenges. Finally, we outline some promising future directions and prospects in this field. This survey is expected to facilitate researchers' collaborative efforts to address the challenges of real-world RE systems.

1 INTRODUCTION

Relation extraction identifies entities and their relations, supporting structured information access and downstream NLP applications. This survey reviews deep-learning advances, challenges, and future directions, including low-resource, cross-sentence, domain-specific, and emerging settings.

  • RE represents relations as entity triplets and supports applications including knowledge graph completion, question answering, and information retrieval.
  • Deep neural networks and pre-trained language models have substantially improved RE, while pipeline and joint approaches organize entity and relation prediction differently.
  • Low-resource RE remains difficult because long-tail relations lack labeled examples and deep learning generally requires large training corpora.
  • The survey addresses gaps in earlier reviews by organizing recent DL and PLM-based methods and analyzing challenging settings and domains.
  • Its contributions include datasets, a taxonomy, analyses of challenges and solutions, and future directions intended to support collaborative RE research.

2 PRELIMINARY

The preliminary section defines RE, introduces datasets and evaluation resources, and frames model comparison through standard and distantly supervised metrics. It also organizes representative datasets by level and domain.

  • The section establishes datasets and metrics as foundations for comparing deep RE models and their architectures.
  • 2.1 Problem Definition: RE predicts triplets containing a head entity, relation type, and tail entity from natural-language text.
  • 2.2 Datasets: Table 1 classifies representative datasets by sentence- versus document-level scope and by general, specific, or multilingual domains.
  • 2.2 Datasets: Transformer-based models dominate top performance across the listed datasets, reflecting their capability in complex language tasks.
  • 2.3 Evaluation Metrics: Supervised RE is commonly evaluated with precision, recall, and F1, while distantly supervised RE additionally uses Precision@K, PRC, and AUC.

3 DEEP LEARNING TECHNIQUES FOR RELATION EXTRACTION

The survey organizes deep-learning RE methods through a taxonomy spanning text representation, context encoding, and triplet prediction. Representative models illustrate the categories and their relationships.

  • The proposed taxonomy categorizes deep-learning RE approaches by text representation, context encoding, and triplet prediction.
  • The taxonomy supports a comprehensive review of representative methods and the distinctions among their modeling choices.
  • The survey frames deep-learning techniques as the main driver of recent progress in relation extraction.

3.1 Text Representation

Text representation methods encode linguistic information for RE using word, character, position, syntactic, or hybrid embeddings. These choices trade representational coverage against model complexity and generality.

  • Word embeddings provide semantic representations, while character, position, and syntactic embeddings encode complementary fine-grained information.
  • Character-level embeddings capture sub-word features such as n-grams, prefixes, and suffixes for relation extraction.
  • Position-enhanced embeddings exploit the relative distances between target entity pairs in sentences.
  • 3.1.4 Syntactic-level Embeddings.: Syntactic features such as dependency paths, part-of-speech tags, WordNet hypernyms, and grammatical relations add structural evidence beyond word embeddings.
  • Hybrid embeddings capture broader linguistic features but can increase complexity and affect model generality, so feature selection depends on the application.

3.2 Context Encoding

Context encoding learns sentence-level features by capturing contextual information and filtering irrelevant content from text representations. The survey reviews CNNs, RNNs, attention, GNNs, PLMs, and combinations of these encoders for RE.

  • 3.2 Context Encoding: Context encoding captures sentence-level information and filters irrelevant content after word-level embeddings provide lexical features.The survey presents context encoding as a stage for retaining information needed for relation extraction.
  • 3.2 Context Encoding: CNNs learn local, position-invariant contextual representations, while RNNs sequentially model context and are better suited to long-range dependencies.RNNs can struggle with very long sequences because of vanishing or exploding gradients.
  • 3.2 Context Encoding: Attention-based networks emphasize important correlations between relation and text representations and can combine multiple-grained entity and relation features.Multi-focused attention can incorporate dependency distance when identifying triplets.
  • 3.2 Context Encoding: GNNs construct semantic graphs to capture nonlinear input structure while jointly learning node and edge representations for relational reasoning.Their stated advantages include modeling global graph structure and relational information.
  • 3.2 Context Encoding: PLMs provide rich semantic knowledge through large-scale pretraining and are fine-tuned on annotated task-specific data for RE.Prompt tuning is described as a way to bridge pretraining and downstream RE formats, especially in few-shot settings.
  • 3.2 Context Encoding: Advanced RE systems often combine encoders, such as CNNs or RNNs with attention and GNNs with attention, to capture complementary information.The survey notes substantial overlap in how these architectures are applied.

3.3 Triplet Prediction

Triplet prediction decodes entity boundaries and relation types through pipeline and joint paradigms. The survey organizes joint methods into span-based, sequence-to-sequence, MRC-based, and sequence-labeling approaches, each handling triplet structure differently.

  • 3.3 Triplet Prediction: Triplet prediction detects entity boundaries and classifies relation types, with existing decoders grouped into five formulation-based categories.The categories correspond to different ways of representing and extracting relation triplets.
  • Pipeline-based approaches: Pipeline methods identify entities first and then classify relations between candidate entity pairs, but independent stages can accumulate errors and miss entity–relation dependencies.The survey contrasts this staged design with joint extraction.
  • Joint-extraction approaches: Joint extraction models identify entities and relations together, addressing no-overlap, single-entity-overlap, and entity-pair-overlap triplets.These overlap categories describe increasingly shared entities or entity pairs across triplets.
  • Span-based approaches: Span-based methods classify spans as entities and use detected entity pairs as candidate triplets for relation classification.The survey reports that span-based approaches outperform earlier pipeline methods and often use pretrained Transformer blocks.
  • Seq2Seq-based approaches: Seq2Seq methods directly generate ⟨head_entity, relationship, tail_entity⟩ triplets as sequential output from unstructured text.The target triplets are represented as a word sequence generated from the source sentence.
  • MRC-based approaches: MRC-based methods reformulate relation extraction as multi-turn question answering, extracting answer spans from context for queried relations.The approach uses questions about entity mentions and relation-specific partners.
  • Sequence labeling approaches: Sequence-labeling methods jointly extract relations end to end by assigning designed entity–relation tags to individual tokens.Functional decomposition can divide the original task into smaller components.

4 CHALLENGING RE PROBLEMS AND SOLUTIONS

The survey organizes challenging RE problems and their solutions, using examples to summarize each task’s input and output.

  • 4 CHALLENGING RE PROBLEMS AND SOLUTIONS: Table 4 provides examples of the inputs and outputs for each challenging relation extraction problem discussed in this section.The section introduces challenging RE problems before reviewing their corresponding solutions.

4.1 Low-resource Relation Extraction

Low-resource RE addresses settings where annotated data are scarce, including distant supervision and few-shot learning. The survey reviews methods that manage noisy supervision or generalize from few examples using metric learning and external knowledge.

  • 4.1 Low-resource Relation Extraction: Supervised RE requires large annotated corpora, which are difficult to obtain in real-world low-resource settings.This data constraint motivates dedicated low-resource RE methods.
  • 4.1.1 Distant Supervision Relation Extraction (DSRE): Distant supervision automatically generates weakly labeled triplets from knowledge bases, but individual sentences can receive incorrect relation labels.The approach aligns entity pairs and relations in resources such as Wikidata, DBpedia, and Freebase.
  • 4.1.1 Distant Supervision Relation Extraction (DSRE): DSRE methods operate at sentence or bag level, with some combining both granularities through cross-level interactions to address noisy supervision.Sentence-level methods use stricter labeling assumptions, whereas bag-level methods relax them.
  • 4.1.1 Distant Supervision Relation Extraction (DSRE): DSRE research develops models intended to improve robustness against noisy and incomplete data while extracting relations from large knowledge bases.The survey summarizes this development as a continuing effort to improve accuracy and robustness.
  • 4.1.2 Few-shot Relation Extraction (FSRE): Few-shot RE targets relations with only a small number of labeled instances, including long-tail relations that lack abundant relational facts.The survey groups FSRE methods into metric learning and knowledge-enhanced learning.
  • Metric Learning: Metric-learning FSRE assigns new instances to relation prototypes by minimizing distances between queries and corresponding class prototypes.Relation prototypes can be learned from contextual information to capture relation semantics.
  • Knowledge-enhanced Learning: Knowledge-enhanced FSRE enriches semantic information with unstructured descriptions or structured knowledge graphs.These resources provide auxiliary clues for relation classification when labeled examples are limited.

4.2 Cross-sentence Relation Extraction

Cross-sentence RE extends relation extraction beyond individual sentences to documents and dialogues, using graph-based, path-based, and contextual methods while facing sparse and implicit relational cues.

  • Document-level Relation Extraction: Document-level RE identifies entity-relation triplets across long documents, requiring models to focus on relevant context within richer and more complex structures than sentences.
  • Document-level Relation Extraction: Graph-based approaches construct document graphs for multi-hop reasoning, capturing local and global entity interactions but incurring computational and graph-quality costs.
  • Document-level Relation Extraction: Path-based approaches extract entity-connected paths to enrich local context and semantic patterns, offering efficiency but weaker coverage of global relations across distant text.
  • Dialogue Relation Extraction: Dialogue RE models contextualize turns and arguments, yet low information density and frequent personal pronouns make sparse semantics among multiple speakers difficult to capture.

4.3 Domain-specific Relation Extraction

Domain-specific RE addresses distribution shifts and specialized relational vocabularies across biomedical, financial, legal, and scientific settings. The survey highlights specialized models and resources while identifying continued needs for broader benchmarks and domain knowledge integration.

  • Domain-specific RE is needed because general-purpose models can perform unsatisfactorily when word distributions shift from general-domain to specialized corpora.
  • RE in the Biomedical Field: Biomedical RE covers interactions among entities such as drugs, diseases, genes, and proteins, increasingly using Transformer models, knowledge graphs, and n-ary datasets.
  • Financial, legal, and scientific RE target specialized relations in documents, including KPIs, contracts, and scientific entities, but their resources and methods remain unevenly developed.
  • Prospects on Domain-specific RE: The survey identifies a need for publicly available resources and effective approaches beyond biomedical RE, alongside domain-specific PLMs such as BioBERT, SciBERT, FinBERT, and Legal-BERT.
  • Prospects on Domain-specific RE: Future progress requires benchmark datasets covering practical domain relations and PLMs that inject domain knowledge to handle specialized vocabulary and complex relations.

5 RE WITH PRE-TRAINED LANGUAGE MODELS

The survey examines how pre-trained language models support relation extraction, especially under few-shot conditions, while noting annotation and deployment constraints. It also situates prompt tuning and large language models as responses to limited supervised data.

  • Performance comparison: Larger PLMs usually outperform smaller ones in few-shot biomedical RE, although some BERT-based models remain competitive with larger PLMs.The comparison covers PubMedBERT-base, BioBERT-large, RoBERTa-large, BART, and other BERT-based systems.
  • PLM-based RE: PLMs improve relation extraction, but supervised fine-tuning remains constrained by insufficient labeled RE data.This limitation is especially relevant to few-shot relation extraction.
  • Prompt tuning: Prompt tuning reformulates RE as text generation to bridge the mismatch between PLM pre-training and downstream fine-tuning objectives.Templates specify the ordering and relationship between entity spans and labels.
  • Performance comparison: Table 7 compares PLM-based RE performance on general-domain NYT and WebNLG datasets and biomedical ChemProt and DDI datasets.The table includes both standard and few-shot biomedical settings.
  • Large language models: LLMs can help obtain auto-labeled documents and operate with limited annotations, but API inference has higher latency and financial cost than fine-tuned PLMs.Their memorization and reasoning capabilities are highlighted as useful for relation extraction with limited annotations.

6 FUTURE DIRECTIONS

The survey identifies future RE directions spanning multimodal, cross-lingual, temporal, evolutionary, and explainable extraction. These directions address broader input modalities, language coverage, changing relation inventories, temporal reasoning, and model opacity.

  • Future directions: Future RE research must address diverse data, uneven relation distributions, and additional relation types in practical scenarios.The survey frames these as continuing challenges despite progress in neural RE model and subtask design.
  • 6.1 Multi-modal Relation Extraction: Multimodal RE combines visual and textual information to extract relations from media such as social-media posts.The MNRE dataset provides visual evidence collected from social media posts.
  • 6.2 Cross-lingual Relation Extraction: Cross-lingual RE seeks language-independent extraction because English-centered systems depend on annotated corpora, PLMs, and noisy translation-based projection.Multilingual training and language-agnostic representations are identified as possible directions.
  • 6.3 Temporal Relation Extraction: Temporal RE incorporates temporal constraints to identify entity relations and support more complex reasoning.The survey describes fine-grained temporal relations and event durations mapped to real-valued scales.
  • 6.4 Evolutionary Relation Extraction: Evolutionary RE targets unseen and emerging relations through open relation extraction and lifelong learning without forgetting established relations.Open extraction uses related phrases or clustering, while lifelong methods continuously learn new relations.
  • 6.5 Explainable Relation Extraction: Explainable RE remains difficult because deep learning models are opaque and their extracted features are not directly interpretable.This opacity complicates understanding the rationale behind model decisions.

7 CONCLUSION

The survey synthesizes recent deep neural relation extraction research through a new taxonomy, reviews challenging settings and domains, and identifies issues involving PLMs and LLMs. It concludes by outlining future directions for real-world RE systems.

  • Conclusion: The survey provides an up-to-date review of DNN-based RE architectures, subtasks, challenging settings, domains, and emerging PLM and LLM issues.It covers low-resource and cross-sentence settings as well as biomedical, finance, legal, and scientific domains.
  • Conclusion: Its taxonomy organizes existing RE approaches by model architectures and clarifies differences and connections between RE subtasks.The survey also analyzes challenges and corresponding solutions.
  • Conclusion: The survey identifies promising future directions intended to support wider implementation of real-life RE systems.The conclusion presents these directions as prospects for continued research.
Loading 2306.02051v3…