Source-linked AI summary
A Novel Embedding Model for Knowledge Base Completion Based on Convolutional Neural Network
Dai Quoc Nguyen, Tu Dinh Nguyen, Dat Quoc Nguyen, Dinh Phung
TL;DR
Knowledge bases are incomplete, creating a need for methods that predict whether missing triples are valid. ConvKB applies convolution to triple embeddings to capture global and transitional relationships, and it outperforms previous state-of-the-art embedding models on WN18RR and FB15k-237.
Problem
Knowledge bases contain many missing valid triples, motivating knowledge base completion and link prediction.
Method
ConvKB applies CNN filters to a k × 3 matrix of head, relation, and tail embeddings, concatenates feature maps, and computes a triple score.
Results
ConvKB outperforms previous state-of-the-art embedding models on WN18RR and FB15k-237, with strong MR, MRR, and Hits@10 results.
Takeaways & Limitations
ConvKB combines convolutional modeling of same-dimensional embedding relationships with transitional characteristics for knowledge base completion.
Takeaways & Limitations
Extending ConvKB to new applications involving triple-form data remains future work.
Abstract
from arXiv · showhide
In this paper, we propose a novel embedding model, named ConvKB, for knowledge base completion. Our model ConvKB advances state-of-the-art models by employing a convolutional neural network, so that it can capture global relationships and transitional characteristics between entities and relations in knowledge bases. In ConvKB, each triple (head entity, relation, tail entity) is represented as a 3-column matrix where each column vector represents a triple element. This 3-column matrix is then fed to a convolution layer where multiple filters are operated on the matrix to generate different feature maps. These feature maps are then concatenated into a single feature vector representing the input triple. The feature vector is multiplied with a weight vector via a dot product to return a score. This score is then used to predict whether the triple is valid or not. Experiments show that ConvKB achieves better link prediction performance than previous state-of-the-art embedding models on two benchmark datasets WN18RR and FB15k-237.
1 Introduction
Knowledge bases contain incomplete relational triples, motivating link prediction methods that score triple validity. ConvKB introduces a CNN-based embedding model that captures relationships across corresponding embedding dimensions and reports strong benchmark performance.
- Knowledge bases are collections of entity-relation triples used in applications including semantic search, question answering, and machine reading.
- Missing valid triples motivate knowledge base completion, which predicts whether candidate triples are valid.
- Embedding models represent entities and relations as vectors or matrices and assign lower implausibility scores to valid triples than invalid ones.
- Prior approaches include transition-based models, tri-linear scoring models, and relation-path methods for incorporating contextual information.
- ConvKB represents each triple as a k × 3 embedding matrix and applies 1 × 3 filters across rows to capture global relationships among same-dimensional entries.
- ConvKB concatenates convolutional feature maps, scores the resulting vector with a weight vector, and uses that score to infer triple validity.
- ConvKB reports the best MR and Hits@10 on WN18RR and the highest MRR and Hits@10 on FB15k-237 among the compared models.
2 Proposed ConvKB model
ConvKB represents each triple with entity and relation embeddings, applies convolutional filters to model relationships among same-dimensional entries, and produces a triple score. The model generalizes transitional characteristics and can reduce to TransE under specific settings.
- A knowledge base contains valid triples (h, r, t), and embedding models assign lower implausibility scores to valid than invalid triples.
- Each triple’s embeddings form a matrix A = [v_h, v_r, v_t] ∈ R^k×3, with each row containing same-dimensional entries across the triple.
- A 1×3 filter scans every row of A to extract global relationships among same-dimensional embedding entries and generalize transitional characteristics.
- Multiple filters generate feature maps that are concatenated and multiplied by a weight vector to produce the triple score.The filter set Ω has τ filters, yielding τ feature maps and a concatenated vector in R^τk×1.
- With one fixed filter, zero bias, specified activation, ω = [1, 1, −1], and w = 1, ConvKB reduces to the plain TransE model.
- ConvKB is trained with Adam by minimizing a loss function with L2 regularization on the weight vector w.
3 Experiments
ConvKB is evaluated on WN18RR and FB15k-237 using standard link-prediction ranking protocols and metrics. It outperforms several baselines, with especially strong gains over TransE on FB15k-237 and mixed comparisons with ConvE and KBLRN.
- Datasets: WN18RR and FB15k-237 are evaluated because their source datasets contain reversible relations that make WN18 and FB15k comparatively easy.The evaluation uses these reduced benchmarks to avoid performance dominated by simple reversal rules.
- Evaluation protocol: Link prediction ranks valid test triples against filtered corrupted triples using mean rank, mean reciprocal rank, and Hits@10.For each test triple, either the head or tail is replaced by other entities, excluding corrupted triples already present in the knowledge base.
- Main results: ConvKB achieves the best MR and highest Hits@10 on WN18RR, plus the highest MRR and Hits@10 on FB15k-237.Table 3 compares ConvKB with previously published results under the same experimental setup.
- Comparison with TransE: 90 MR points, 0.102 MRR, and 5.2% absolute Hits@10 separate ConvKB from TransE on FB15k-237.These correspond to about 26% relative MR improvement, more than 34% relative MRR improvement, and 5.2% absolute Hits@10 improvement.
- Comparison with other models: ConvKB beats ConvE on both datasets except WN18RR MRR and FB15k-237 MR, while KBLRN retains better FB15k-237 MR.On FB15k-237, ConvKB improves over ConvE by 0.078 MRR and 2.6% Hits@10, and has 25% higher MRR than KBLRN.
4 Conclusion
The paper concludes that ConvKB applies CNNs to model global relationships among same-dimensional embedding entries and generalize transitional characteristics. It reports stronger performance than other state-of-the-art models and proposes extending the approach to triple-formatted applications.
- Conclusion: ConvKB applies a convolutional neural network to global relationships among same-dimensional entity and relation embedding entries.The model is presented as an embedding approach for knowledge base completion.
- Conclusion: ConvKB generalizes the transitional characteristics used by transition-based embedding models.This characteristic is part of the model’s stated design motivation.
- Conclusion: ConvKB outperforms other state-of-the-art models on WN18RR and FB15k-237.The conclusion summarizes the reported experimental results on both benchmark datasets.
- Future application: The authors plan to apply ConvKB to new applications whose data can be formulated as triples, including query-user-document relationships.Search personalization is given as an example of this proposed extension.