Source-linked AI summary
STransE: a novel embedding model of entities and relationships in knowledge bases
Dat Quoc Nguyen, Kairit Sirts, Lizhen Qu, Mark Johnson
TL;DR
Incomplete knowledge bases motivate methods for predicting likely missing relationships. This paper introduces STransE, which combines SE-style relation-specific projections with a TransE-style translation, and reports better performance than related embedding models on two benchmark datasets. STransE is presented as a baseline for more complex link-prediction models, with future work extending it to relation-path information.
Problem
Knowledge bases are typically incomplete, so link prediction is needed to determine whether missing triples are likely to be true.
Method
STransE combines two relation-specific matrices from SE with a translation vector from TransE to model entity relationships.
Results
STransE performs better than SE, TransE, and other related models on WN18 and FB15k, including an 81.6% filtered Hits@10 result for M-M relations on FB15k.
Takeaways & Limitations
STransE is a suitable candidate as a baseline for more complex link-prediction models.
Takeaways & Limitations
The paper leaves incorporating relation-path information into STransE for future work.
Abstract
from arXiv · showhide
Knowledge bases of real-world facts about entities and their relationships are useful resources for a variety of natural language processing tasks. However, because knowledge bases are typically incomplete, it is useful to be able to perform link prediction or knowledge base completion, i.e., predict whether a relationship not in the knowledge base is likely to be true. This paper combines insights from several previous link prediction models into a new embedding model STransE that represents each entity as a low-dimensional vector, and each relation by two matrices and a translation vector. STransE is a simple combination of the SE and TransE models, but it obtains better link prediction performance on two benchmark datasets than previous embedding models. Thus, STransE can serve as a new baseline for the more complex models in the link prediction task.
1 Introduction
Knowledge bases are incomplete, motivating link prediction, while simpler embedding models remain valuable when external information is unavailable or costly. STransE combines SE and TransE and improves link prediction performance on WN18 and FB15k.
- Knowledge bases represent entity relationships as triples, but even very large knowledge bases remain incomplete.
- Link prediction predicts whether triples absent from a knowledge base are likely to be true.
- Simpler models are useful because external information may be unavailable and simpler systems are typically faster to train.
- STransE combines SE relation-specific matrices with a TransE translation vector in a single model.The matrices identify relation-dependent aspects of the head and tail, while the vector describes their relationship in that subspace.
- STransE performs better than SE, TransE, and other state-of-the-art models on WN18 and FB15k, providing a new KB-completion baseline.
2 Our approach
STransE scores triples by measuring how closely relation-specific projections of the entities match after translation. Its parameters are learned with a margin-based objective using corrupted triples and constrained stochastic gradient descent.
- STransE assigns lower scores to plausible triples than to implausible triples.
- The model learns relation-specific matrices and a translation vector so projected head and tail representations satisfy a translation-based relation.
- The margin-based objective compares correct training triples with incorrect triples generated by replacing the head or tail entity.
- Incorrect triples are generated by corrupting a correct triple with alternative entities not already present as correct triples.
- Training uses stochastic gradient descent with norm constraints on entity vectors, relation vectors, and projected representations.
3 Related work
Related embedding models differ in their score functions, optimization algorithms, and ways of representing relation-specific transformations. Subsequent models extend projection, bilinear, path-based, or external-information approaches for link prediction.
- Embedding models differ in their score functions and optimization methods, including SGD, AdaGrad, AdaDelta, and L-BFGS.
- Bilinear models represent relations with matrices or tensors, while DISTMULT simplifies this representation using diagonal matrices.
- TransH, TransD, and TransR project entity vectors into relation-specific spaces using hyperplanes, projection vectors, or matrices.
- STransE and TranSparse can be viewed as extensions of TransR, with STransE using full relation-specific matrices.
- Relation-path information and textual mentions from external corpora provide additional information used by other link-prediction models.
4 Experiments
The experiments evaluate STransE for link prediction on WN18 and FB15k using standard ranking protocols and compare it with prior models. STransE performs strongly against models without path or external-corpus information, especially across several relation categories.
- Task and evaluation protocol: The experiments compare STransE with published results on the WN18 and FB15k benchmark datasets.The comparison uses the same experimental setup as prior work.
- Task and evaluation protocol: Link prediction ranks candidate head or tail entities using the score function, with Raw and Filtered evaluation protocols.Filtered evaluation removes corrupted triples already present in the knowledge base.
- Main results: Without path or external information, STransE achieves the highest filtered mean rank on WN18 and the highest filtered Hits@10 and mean reciprocal rank on FB15k.These comparisons are reported against models in the corresponding evaluation setting.
- Main results: STransE outperforms SE, TransE, TransR, CTransR, TransD, and TranSparse on both WN18 and FB15k.The comparison concerns closely related models that do not necessarily use the same additional information sources.
- Main results: On FB15k M-M relations, STransE reaches 81.6% Hits@10, 0.5% below TranSparse, while exceeding TranSparse by 2.5% for M-1 relations.STransE also performs better than TransD for 1-M and M-1 relation categories.
- Main results: The authors attribute gains on diverse relation categories to STransE's full matrices, while noting lower performance than TransD on relatively rare 1-1 relations.They suggest the extra parameters may hurt performance for 1-1 relations.
5 Conclusion and future work
The paper concludes that STransE combines SE and TransE into a simple embedding model that performs competitively on standard link prediction evaluations. It presents STransE as a candidate baseline and plans to extend it with relation-path information.
- Conclusion: STransE combines SE and TransE by representing each relation with a low-dimensional vector and two projection matrices.Entities are represented through the embedding model for link prediction and knowledge base completion.
- Conclusion: STransE produces highly competitive results and scores better than the embedding-based models it builds on.The conclusion identifies it as a suitable future baseline for more complex link prediction models.
- Future work: Future work will extend STransE to exploit relation path information in knowledge bases.The planned extension follows approaches that incorporate relation paths.