PaperlayerView exact revision on arXiv

Source-linked AI summary

Relation Extraction : A Survey

Sachin Pawar, Girish K. Palshikar, Pushpak Bhattacharyya

arXiv:1712.05191v1cs.CLcs.AIcs.IR

TL;DR

Relation extraction identifies relations between entity mentions, but approaches span different supervision settings and extraction paradigms. This survey reviews those methods and their development. It reports syntactic tree kernels as the strongest supervised techniques in its review, with the best ACE 2004 result around 77%, and notes growing use of distant supervision.

  • Problem

    Relation extraction seeks to identify relations linking entity mentions in text, supporting access to information hidden in large document collections.

    Source: §1 · pp. 1, 3

  • Method

    The survey reviews supervised and joint entity-relation methods alongside semi-supervised, unsupervised, Open IE, and distant-supervision approaches.

    Source: §9 · pp. 35–36

  • Results

    Syntactic tree kernel techniques were the most effective supervised methods, and the best reported ACE 2004 result after almost a decade was around 77%.

    Source: §9 · p. 35

  • Takeaways & Limitations

    The survey reports a continuing increase in RE research using distant-supervision techniques, which require negligible human supervision.

    Source: §9 · p. 36

    Without gold-standard entities, the best reported joint-model F-measure on ACE 2004 was around 48%, versus around 77% when gold-standard entity information was assumed.

    Source: §9 · p. 36

Abstract

from arXiv · show

With the advent of the Internet, large amount of digital text is generated everyday in the form of news articles, research publications, blogs, question answering forums and social media. It is important to develop techniques for extracting information automatically from these documents, as lot of important information is hidden within them. This extracted information can be used to improve access and management of knowledge hidden in large text corpora. Several applications such as Question Answering, Information Retrieval would benefit from this information. Entities like persons and organizations, form the most basic unit of the information. Occurrences of entities in a sentence are often linked through well-defined relations; e.g., occurrences of person and organization in a sentence may be linked through relations such as employed at. The task of Relation Extraction (RE) is to identify such relations automatically. In this paper, we survey several important supervised, semi-supervised and unsupervised RE techniques. We also cover the paradigms of Open Information Extraction (OIE) and Distant Supervision. Finally, we describe some of the recent trends in the RE techniques and possible future research directions. This survey would be useful for three kinds of readers - i) Newcomers in the field who want to quickly learn about RE; ii) Researchers who want to know how the various RE techniques evolved over time and what are possible future research directions and iii) Practitioners who just need to know which RE technique works best in various settings.

1. Introduction

The introduction frames relation extraction (RE) as identifying relations between entity mentions in text, then surveys approaches across settings and datasets. It distinguishes global from mention-level RE and situates the survey among earlier work.

1. Introduction

  • RE identifies relations of interest between entity mentions in sentences, following named-entity recognition; the survey assumes binary relations with both arguments in one sentence.Not every entity pair in a sentence is related.Source: §1 · pp. 2–3

1.1. Datasets for Relation Extraction

  • ACE evaluates entity detection and tracking and relation detection and characterization; ACE 2004 is the most widely used dataset for reporting RE performance.ACE 2003 and ACE 2004 specify relation types and subtypes, with occurrence counts shown in Figure 1.Source: §1.1 · p. 3, Figure 1 · p. 4

1.2. Relation Extraction : Global level Vs Mention level

  • Global RE outputs entity pairs for which a semantic relation exists, whereas mention-level RE judges whether a relation holds for an entity pair in a particular sentence.ACE calls mention-level RE Relation Detection and Characterization (RDC).Source: §1.2 · pp. 3–4

1.3. Previous Surveys

  • The survey organizes RE work into supervised, joint entity-relation, semi-supervised, unsupervised, Open IE, and distant-supervision approaches, followed by advanced techniques and future directions.Earlier surveys include broader IE coverage, an RE survey with limited recent advances, and surveys focused on Portuguese or biomedical RE.Source: §1.3 · pp. 4–5

2. Supervised Approaches

Supervised RE is framed as mention-level classification using labeled entity-mention pairs. The survey distinguishes feature-based and kernel-based methods.

2. Supervised Approaches

  • Supervised RE focuses on mention-level instances and requires labeled entity-mention pairs assigned predefined relation types.Pairs without a predefined relation receive the NONE label.Source: §2 · p. 5
  • The methods formulate RE as multiclass classification, with each class representing a relation type, including NONE.Source: §2 · p. 5
  • The survey groups supervised methods into feature-based and kernel-based approaches.Source: §2 · p. 5

2.1. Feature-based Methods

Feature-based RE represents labeled relation instances with designed features and trains classifiers to classify new instances. The surveyed work studies feature choices, structured cues, and class imbalance.

2.1. Feature-based Methods

  • Feature-based systems train a classifier on designed features for each labeled entity-mention pair, and their overall performance depends largely on feature effectiveness.Feature design requires analysis of each feature’s contribution and the underlying linguistic phenomena.Source: §2.1 · pp. 5, 7
  • Kambhatla used 49 classes, while Zhou et al. added word, phrase-chunking, and semantic-resource features, including cues for country-related and personal-social subtypes.Zhou et al. used an SVM with one-versus-others classification; their system outperformed Kambhatla’s.Source: §2.1 · pp. 5–6
  • Phrase-based chunking features contributed most to Zhou et al.’s increased accuracy, while dependency- and parse-tree syntactic features contributed only slightly.The authors attribute this pattern to ACE relations often being short-distance, where word and chunking features can suffice.Source: §2.1 · p. 6
  • Syntactic parse-tree features were the most effective of three studied feature subspaces, but combining sequence, syntactic-parse, and dependency-parse subspaces added little.Within each subspace, basic-unit features already performed reasonably well, and more complex features might not improve performance much.Source: §2.1 · pp. 6–7
  • Other feature-based approaches mined dependency-tree core-tree substructures and used constrained syntactico-semantic patterns to select specialized relation models.The core tree adds paths from the shortest dependency path to relation-cue keywords; the constrained patterns include premodifiers, possessives, prepositions, and formulaic forms.Source: §2.1 · p. 7
  • Class imbalance can make supervised classifiers overproduce NONE, raising precision while lowering recall; Kambhatla’s committee-voting solution significantly boosted recall.Source: §2.1 · p. 7

2.2. Kernel Methods

Kernel methods classify relation instances by computing similarities between representations such as sequences and syntactic trees, avoiding explicit feature engineering. The survey describes sequence, tree, dependency-path, and composite kernels, including reported performance comparisons.

2.2.1. Sequence Kernel

  • Sequence kernels compare shared sparse subsequences of feature vectors, while four subkernels capture fore-between, between, between-after, and modifier patterns.The feature vectors generalize words using lexical and linguistic feature domains.Source: §2.2.1 · pp. 8–9
  • A two-stage SVM—first detecting whether a relation exists, then classifying its type—performed better than a single multiclass SVM.Both scenarios use the relation kernel based on shared generalized sequence subsequences.Source: §2.2.1 · pp. 9–10

2.2.2. Syntactic Tree Kernel

  • Syntactic tree kernels count shared subtrees, and the Path-enclosed Tree representation performed best among the tested relation-instance representations.The Path-enclosed Tree is the smallest subtree containing both entity mentions.Source: §2.2.2 · pp. 10–11

2.2.3. Dependency Tree Kernel

  • Dependency-tree kernels compare augmented subtrees containing both mentions; matching child subsequences contribute similarity, with sparse subsequences penalized by a decay factor.Node features can include part-of-speech tags, entity type, mention level, and WordNet hypernyms.Source: §2.2.3 · pp. 13–14
  • On ACE, K4, combining contiguous tree and bag-of-words kernels, achieved the best performance among the dependency-tree kernels tested.The alternatives included sparse (K0), contiguous (K1), and bag-of-words (K2) kernels.Source: §2.2.3 · p. 14

2.2.4. Dependency Graph Path Kernel

  • Dependency-path kernels compare shortest paths between entity mentions; generalized word classes reduce sparsity, while convolution kernels allow paths to share subsequences without equal lengths.The basic path kernel requires equal path lengths.Source: §2.2.4 · pp. 14–15

2.2.5. Composite Kernels

  • Composite kernels outperformed individual kernels, and the polynomial-expansion kernel performed best; normalizing individual kernels prevents one from overwhelming another.The polynomial expansion combines normalized entity and syntactic-tree kernels.Source: §2.2.5 · p. 16

2.3. Evaluation

The survey compares supervised relation-extraction approaches on ACE 2004 using five-fold cross-validation, while cautioning that differing folds make comparisons approximate. It reports that kernel-based methods generally outperform feature-based methods, with syntactic-tree kernels strongest among the kernel methods.

2.3. Evaluation

  • Because approaches may use different five-fold splits, the reported figures provide only a rough comparative guide.Source: §2.3 · p. 16
  • The ACE 2004 comparison covers seven major relation types and reports precision, recall, and F-measure for non-NONE classes.Results use five-fold cross-validation.Source: §2.3 · p. 16, Table 5 · p. 17
  • Kernel-based methods generally outperformed feature-based methods, and syntactic-tree kernels performed best among the kernel methods.The comparison uses ACE 2004 results, but the actual cross-validation folds may differ across approaches.Source: §2.3 · pp. 16–17

3. Joint Extraction of Entities and Relations

Joint extraction models entity identification and relation extraction together, addressing error propagation from sequential pipelines. The survey covers global inference and joint models, including approaches that also handle mention boundaries or coreference.

3.1. Integer Linear Programming based Approach

  • Roth and Yih combine local entity and relation classifiers with an ILP that enforces domain constraints, improving relation extraction over a pipeline.The survey also reports improved entity classification, which it says the pipeline approach cannot provide.Source: §3.1 · pp. 17–18

3.2. Graphical Models based Approach

  • Roth and Yih use a Bayesian belief network to jointly assign entity and relation labels, with conditional probabilities encoding their dependencies.The survey reports better relation classification than an independent classifier for Born in and Kill, but no similar entity-classification improvement.Source: §3.2 · pp. 18–19
  • Yu and Lam jointly identify entities and relations without assuming known mention boundaries, but restrict relations to a principal entity and secondary entities.Source: §3.2 · p. 19
  • Singh et al. jointly model entity mentions, relations, and coreference across a document, using sparse-domain belief propagation to manage inference.Source: §3.2 · p. 19

3.3. Card-Pyramid Parsing

  • Kate and Mooney use a card-pyramid graph and CYK-like parsing to jointly label entity and relation nodes using local classifiers.The graph’s higher-level nodes represent possible relations between entities beneath them.Source: §3.3 · pp. 19–20, Figure 5 · p. 20

3.4. Structured Prediction

  • Li and Ji cast entities as nodes and relations as labeled arcs in a structured prediction problem, using beam search to find a high-scoring structure.Their framework incorporates mention-boundary detection and global features for entity and relation decisions.Source: §3.4 · pp. 20–21
  • Miwa and Sasaki map joint extraction to filling an n × n lower-triangular table whose diagonal stores entity types and other cells store relations.Source: §3.4 · p. 21
  • Most joint approaches report significant improvement over pipeline methods, but comparisons are difficult because no single standard dataset is used.Some methods jointly infer from independently trained local classifiers, while others learn one model for both tasks.Source: §3.4 · pp. 21–22

4. Semi-supervised Approaches

Semi-supervised RE seeks to reduce the effort of labeling data by exploiting unlabeled examples. The section surveys bootstrapping, active learning, label propagation, transfer learning, and their evaluation.

4.1. Bootstrapping Approaches

  • Given good patterns, bootstrapping can find relation tuples, and good tuples can in turn support learning good patterns.DIPRE is based on this pattern–tuple duality.Source: §4.1 · p. 22
  • Starting with 3 seed examples and about 24 million web pages, DIPRE generated 15,257 unique author–book pairs.Source: §4.1 · p. 23
  • SnowBall adds named-entity-aware, variation-tolerant patterns and evaluates patterns and tuples, discarding low-confidence extractions; performance still depends on seed choice.Its pattern confidence uses positive and negative matches.Source: §4.1 · pp. 23–24

4.2. Active Learning

  • Active learning reduces annotation effort by asking for labels on selected unlabeled instances; LGCo-Testing combines a local sentence-feature view with a global phrase-similarity view.The local view uses a maximum-entropy classifier, while the global view uses nearest neighbors.Source: §4.2 · pp. 24–25

4.3. Label Propagation Method

  • Label propagation represents relation instances as graph nodes and iteratively spreads labels over similarity-weighted edges, with unlabeled neighbors also informing predictions.The surveyed approach performed better than SVM and bootstrapping approaches.Source: §4.3 · p. 25

4.4. Other Methods

  • For a target relation with few seeds, multi-task transfer learning can use shared structures and a shared weight vector to transfer knowledge from other relation types.The framework also uses human guidance through entity-type constraints.Source: §4.4 · pp. 25–26

4.5. Evaluation

  • Mention-level methods can be evaluated on labeled data, but bootstrapping pair lists make recall difficult to estimate without manually annotating a subset of unlabeled data.Source: §4.5 · p. 26

5. Unsupervised Relation Extraction

Unsupervised RE approaches do not require labeled data. The survey covers clustering-based methods and other approaches that induce or extract relations from text.

5.1. Clustering based approaches

  • Hasegawa et al. cluster co-occurring named-entity pairs by context similarity and automatically label clusters; their contexts include only intermediate words between entities.They use ordered entity pairs, TF–IDF-weighted word vectors, cosine similarity, and hierarchical clustering.Source: §5.1 · pp. 26–27
  • Later clustering approaches remove noisy words or use Wikipedia concepts and dependency patterns before broadening coverage with surface patterns.The Wikipedia approach first forms precise clusters, then clusters remaining pairs using learned centroids and larger-web-corpus patterns.Source: §5.1 · p. 27
  • Other methods cluster semantically similar dependency paths or relation instances, including models with entity-type constraints and knowledge-based must-link or cannot-link rules.The survey also describes bilingual expression clustering using aligned named entities.Source: §5.1 · pp. 27–28

5.2. Other approaches

  • URES extracts relations from type definitions consisting of keywords and argument entity types, while paraphrase-based extraction expands an initial relation expression into alternatives.For example, an expression such as “X interact with Y” can yield paraphrases such as “X bind to Y.”Source: §5.2 · p. 28

6. Open Information Extraction

Open Information Extraction addresses the human effort involved in defining relations and preparing labeled data by discovering relations from text without domain-specific setup. The section describes TextRunner, ReVerb, and subsequent system comparisons, while noting that relation phrases may need consolidation.

6. Open Information Extraction

  • Open IE systems discover relations from text without human involvement, avoiding additional effort when switching domains.Traditional RE uses pre-specified relations and can require substantial human effort to design extraction rules or create labeled data.Source: §6 · pp. 28–29
  • TextRunner automatically labels tuples, trains a Naive Bayes classifier, extracts candidate noun-phrase pairs in one corpus pass, and scores merged tuples using sentence counts.Source: §6 · p. 29
  • ReVerb uses syntactic and lexical constraints and identifies relation phrases holistically, addressing incoherent, uninformative, and overly specific TextRunner extractions.Its syntactic constraint admits verb-centered patterns, while its lexical constraint requires a phrase to occur with at least k distinct argument pairs.Source: §6 · pp. 29–30, Table 7 · p. 30
  • ReVerb outperformed earlier open IE systems, R2A2 achieved better precision and recall than ReVerb, and DepOE achieved more precise extraction than ReVerb on the web scale.A remaining limitation is that one semantic relation can appear as multiple phrases, requiring post-processing to consolidate them.Source: §6 · pp. 30–31

7. Distant Supervision

Distant supervision uses knowledge-base entity pairs to label text automatically, trading human annotation for noisy labels and potential false negatives. Later work addresses overlapping relations, incomplete knowledge bases, and ways to combine or refine supervision.

Basic approach

  • Distant supervision uses semantic-database entity pairs to label sentences containing both entities, producing large but potentially noisy training data without labeled examples.The initial method used Freebase and combined a probabilistic classifier with thousands of features.Source: §7 · p. 31
  • Entity pairs absent from Freebase are sampled as negative instances, but database incompleteness can incorrectly label some as negative.A multi-class logistic classifier with Gaussian regularization is trained using lexical, syntactic, and named-entity-type features.Source: §7 · p. 31

Overlapping relations

  • MIML-RE models multiple mentions and multiple valid relation labels for an entity pair, addressing traditional systems’ failure to represent overlapping relations.Its entity-pair classifiers model label dependencies, including labels that cannot or tend to co-occur.Source: §7 · p. 32

MIML-RE results

  • MIML-RE outperforms the traditional distant-supervision approach by Mintz et al. and the multiple-instance approach by Hoffmann et al..Evaluations used Riedel’s dataset and the KBP dataset.Source: §7 · p. 32

Further developments

  • Later work targets false negatives from incomplete knowledge bases and non-convex MIML-RE likelihoods with positive-unlabelled learning, pseudorelevance feedback, and discriminative clustering.Guided distant supervision also reported significant improvement when small amounts of human-labeled data were combined with distant labels.Source: §7 · p. 32

Combining supervision

  • Joint direct and distant supervision improved F-measure by around 3% over ACE-only training, while active selection of uncertain, representative instances improved MIML-RE.The selected instances were labeled through Amazon Mechanical Turk crowdsourcing.Source: §7 · p. 33

8. Recent Advances in Relation Extraction

Recent advances broaden relation extraction beyond binary, within-sentence settings and explore shared relation representations, cross-lingual transfer, and domain adaptation. They also include neural methods that reduce reliance on complex preprocessing.

Universal Schemas

  • Universal Schemas unite structured-database relation types with Open IE surface forms and learn asymmetric implicatures for inferring additional relations.For example, CapitalOf can imply LocatedIn, but not the reverse.Source: §8 · p. 33

n-ary Relation Extraction

  • For n-ary relations, one approach turns binary relation predictions into a graph and finds maximal cliques corresponding to valid higher-order relations.The approach was demonstrated on biomedical data; Kraken is another Open IE system for n-ary extraction.Source: §8 · pp. 33–34

Semantic Roles and Cross-sentence Relations

  • Semantic role labeling offers another perspective on n-ary extraction by identifying a sentence’s predicate and its arguments.The survey also notes approaches to both intra- and inter-sentential relations.Source: §8 · p. 34
  • Table 8 presents relations spanning consecutive sentences; most cases like Example 1, but not Example 2, can be handled through coreference resolution.In Example 1, “son” refers to Hutomo “Tommy” Mandala Putra in the next sentence.Source: Table 8 · p. 34

Cross-lingual Annotation Projection

  • Cross-lingual annotation projection transfers relation labels from resource-rich English to Korean using parallel corpora, either in one pass or iteratively with a graph.The graph-based approach uses both entities and context information.Source: §8 · p. 35

Domain Adaptation

  • Domain adaptation addresses supervised RE degradation when training and test distributions mismatch, including out-of-domain use.Reported approaches incorporate semantic similarity into tree kernels, generalize lexical features with clusters and embeddings, or select relevant source-domain information.Source: §8 · p. 35

9. Conclusion and Future Research Directions

The survey organizes relation-extraction methods and reports that syntactic tree kernels were the strongest supervised techniques, while identifying several areas where evidence and methods remain limited. It also outlines the survey’s value for newcomers, researchers, and practitioners.

Conclusion

  • Syntactic tree kernels were the most effective supervised techniques, with best reported F-measure around 77% on ACE 2004 after nearly a decade of work.The strongest results used composite kernels or dynamically determined tree spans.Source: §9 · p. 35
  • The survey covers feature- and kernel-based supervised methods, joint entity-relation extraction, semi-supervised and unsupervised methods, Open IE, distant supervision, and recent advances.It reviews how techniques evolved and how they are evaluated.Source: §9 · pp. 35–36
  • Joint extraction supports two-way information flow between entity and relation tasks, while the survey reports an increasing research trend toward distant supervision.The paper notes that good entity extraction performance is important for good relation extraction performance.Source: §9 · pp. 35–36

Future Research Directions

  • Without gold-standard entities, the best reported ACE 2004 F-measure is around 48%, almost 30% below performance when gold-standard entity information is assumed.Other open directions include n-ary relations, under-resourced and free-word-order languages, and deeper semantic or discourse processing.Source: §9 · p. 36

Audience and Use

  • The survey is intended to help newcomers learn RE, researchers understand methods and open problems, and practitioners choose techniques suited to their applications.It summarizes methods’ relative performance and pros and cons.Source: §9 · p. 36
Loading 1712.05191v1…