Source-linked AI summary
Connecting Language and Knowledge Bases with Embedding Models for Relation Extraction
Jason Weston, Antoine Bordes, Oksana Yakhnenko, Nicolas Usunier
TL;DR
Relation extraction from free text must assign KB relationships under weak supervision, where labels can include relations not expressed by individual mentions. The paper jointly scores text mentions and KB triples using low-dimensional embeddings, and reports improved extraction over text-only systems while using large-scale Freebase information.
Problem
Weakly supervised relation extraction must assign KB relationships to relation mentions when entity pairs are known, although assigned labels may include relations not expressed by the mention.
Method
The model jointly learns a text-based relationship scorer and a KB-based entity–relationship scorer using low-dimensional embeddings and ranking-based training.
Results
The system improves over existing text-only relation extraction methods while using a Freebase subset containing 4M entities and 23k relationships.
Takeaways & Limitations
The framework leverages other KB triples to improve relation extraction and is described as applicable to settings such as entity linking.
Takeaways & Limitations
The experiments correct an error in a plot from Hoffmann et al. (2011), which the authors acknowledged.
Abstract
from arXiv · showhide
This paper proposes a novel approach for relation extraction from free text which is trained to jointly use information from the text and from existing knowledge. Our model is based on two scoring functions that operate by learning low-dimensional embeddings of words and of entities and relationships from a knowledge base. We empirically show on New York Times articles aligned with Freebase relations that our approach is able to efficiently use the extra information provided by a large subset of Freebase data (4M entities, 23k relationships) to improve over existing methods that rely on text features alone.
1 Introduction
The paper studies weakly supervised relation extraction, where text mentions linked to entity pairs receive KB relations that may not be expressed. It proposes jointly using weakly labeled text and existing KB triples through embedding-based scoring to improve extraction.
- Problem: Relation extraction assigns a KB relationship to each mention of a relation between an already detected entity pair.Weak supervision labels all mentions associated with an entity pair with every KB relationship connecting the pair, including relations not expressed in the mention.
- Approach: The model combines weakly labeled text mentions with known KB triples rather than relying on text features alone.It learns the plausibility of missing (h, r, t) triples by generalizing from the KB.
- Approach: A ranking-based embedding framework places relation mentions, entities, and relationships in a common low-dimensional vector space for scoring.The approach jointly learns representations and scores for the text and KB components.
- Prior work: Prior weakly supervised RE systems use Freebase-derived labels and textual information, including multi-instance learning frameworks.These approaches enable large-scale training without labeling numerous texts, but the cited systems use textual information for extraction.
- Prior work: A contemporaneous approach jointly models KB data and text through collaborative filtering but does not directly connect text mentions with KB relationships.The paper distinguishes its direct connection from that method's indirect connection through shared entity embeddings.
2 Embedding-based Framework
The framework jointly models text mentions and knowledge-base triples with embedding-based scoring functions. It combines mention–relationship compatibility with knowledge-base plausibility to make relation-extraction predictions.
- Framework overview: Two embedding models learn relation-mention scores and entity–relationship interactions from knowledge-base triples.Their joint prediction uses the connection between text and the knowledge base.
- Connecting text and relationships: The mention-to-relationship model projects a word window into R^k and scores its dot product with a relationship embedding.The projection is f(m) = W^TΦ(m), where Φ(m) is a sparse binary window representation.
- Connecting text and relationships: Ranking constraints train observed mention–relationship pairs to score above alternative relationships, with margin-based soft constraints used in practice.The model is trained with ranking loss and stochastic gradient descent, while constraining embedding norms.
- Encoding structured data of KBs: The knowledge-base model uses translation-style embeddings in which valid triples satisfy h + r ≈ t and invalid alternatives receive lower scores.Its ranking objective contrasts existing relations with alternative relationships and entities.
- Relation-extraction implementation: At test time, mentions for each entity pair are aggregated, and the predicted relationship receives a composite score from mention compatibility and knowledge-base plausibility.The composite score favors predictions supported by both sources; NA denotes no relation and leaves the score unchanged.
3 Experiments
The experiments evaluate the embedding-based approach on NYT articles aligned with Freebase, using a large external KB while preventing direct memorization of test entity pairs. Adding Freebase information substantially improves aggregate extraction performance, especially at low recall.
- Dataset and setup: The evaluation uses the NYT+FB dataset, with 52 relationships and 121,034 training mentions, alongside sentence-level linguistic features.The dataset aligns Freebase relations with New York Times text and retains the 100,000 most frequent features.
- Dataset and setup: The Freebase subset contains 4M entities and all 23k relationships, with test entity pairs removed to require generalization beyond memorization.The original KB has around 80M entities and 1.2B relations; the retained subset overlaps 80% with NYT+FB test entities.
- Evaluation and results: Figure 1 compares aggregate precision/recall curves for WSABIEM2R+FB, WSABIEM2R, and established relation-extraction methods.WSABIEM2R uses the mention model alone, whereas WSABIEM2R+FB combines it with the KB model.
- Evaluation and results: WSABIEM2R is comparable to, but slightly worse than, MIMLRE and HOFFMANN, possibly because it predicts a single relationship per entity pair.The comparison uses the training, test, evaluation framework, and baselines from prior work.
- Evaluation and results: The plot corrects an acknowledged error in the Hoffmann et al. comparison.The correction is noted directly alongside the experimental result.
- Evaluation and results: WSABIEM2R+FB provides superior performance to all other methods by a wide margin between 0 and 0.1 recall.The bottom plot zooms into this low-recall region.
4 Conclusion
The paper presents a general framework for improving relation extraction by training on both mention–relationship pairs and other knowledge-base triples. It also identifies entity linking as another possible application.
- Conclusion: The framework leverages large-scale knowledge bases by training on mention–relationship pairs and all other KB triples.This is the paper’s stated approach to improving relation extraction.
- Conclusion: The modeling approach is general and should apply to other settings, including entity linking.Entity linking is given as an example of a possible additional task.