Source-linked AI summary
TuckER: Tensor Factorization for Knowledge Graph Completion
Ivana Balažević, Carl Allen, Timothy M. Hospedales
TL;DR
Knowledge graphs contain only a subset of possible facts, so link prediction must infer missing triples from known ones. TuckER applies Tucker decomposition to this problem as a straightforward linear model. It achieves state-of-the-art results on standard datasets while remaining fully expressive and encompassing several earlier linear models.
Problem
Knowledge graphs omit many possible facts, creating the need to infer missing triples automatically.
Method
TuckER factorizes the binary tensor of known triples with Tucker decomposition, using entity and relation embeddings plus a core tensor.
Results
TuckER achieves state-of-the-art results on standard link-prediction datasets and outperforms many linear and more complex neural architectures.
Takeaways & Limitations
TuckER’s low-rank core supports multi-task learning across relations, while its parameter count grows linearly with the number of entities or relations.
Takeaways & Limitations
Full expressiveness may require dimensionalities larger than the stated bound in practice, although structured data is expected to need much smaller dimensions.
Abstract
from arXiv · showhide
Knowledge graphs are structured representations of real world facts. However, they typically contain only a small subset of all possible facts. Link prediction is a task of inferring missing facts based on existing ones. We propose TuckER, a relatively straightforward but powerful linear model based on Tucker decomposition of the binary tensor representation of knowledge graph triples. TuckER outperforms previous state-of-the-art models across standard link prediction datasets, acting as a strong baseline for more elaborate models. We show that TuckER is a fully expressive model, derive sufficient bounds on its embedding dimensionalities and demonstrate that several previously introduced linear models can be viewed as special cases of TuckER.
1 Introduction
Knowledge graphs omit many real-world facts, motivating link-prediction methods that infer missing triples. TuckER addresses this gap with a Tucker-decomposition linear model that is expressive, subsumes prior linear models, and achieves strong benchmark performance.
- Knowledge graphs store facts as entity-relation-entity triples, but manually adding omitted information is costly.
- Link prediction infers which unknown tensor entries represent missing true triples rather than false facts.
- TuckER is a straightforward linear link-prediction model based on Tucker decomposition of the binary triple tensor.Entity and relation embeddings occupy factor matrices, while a low-rank core tensor determines their interactions and shares information across relations.
- TuckER is fully expressive and has a dimensionality bound guaranteeing that expressiveness.It can represent any ground truth over triples by separating true from false facts with suitable embeddings.
- RESCAL, DistMult, ComplEx, and SimplE are special cases of TuckER, which also achieves state-of-the-art results across standard datasets.
2 Related Work
Prior link-prediction work includes linear tensor-factorization models and newer nonlinear architectures. These methods trade expressiveness, parameterization, interpretability, and structural constraints in different ways.
- RESCAL uses full-rank relation matrices and is expressive, but its parameter count can cause overfitting as relations and embedding dimensions grow.
- DistMult reduces RESCAL’s parameters with diagonal relation matrices but cannot model asymmetric relations because its tensor is symmetric in subject and object modes.
- ComplEx introduces asymmetry by using complex-conjugate subject and object embeddings, while SimplE modifies CP decomposition with dependent head-tail terms.
- ConvE and HypER use nonlinear convolutional mechanisms, with HypER generating relation-specific filters through a hypernetwork.The paper notes that ConvE’s reshaping and convolution operations are unintuitive, while HypER imposes sparsity through many zero core-tensor elements.
3 Background
Link prediction learns a scoring function for triples from a subset of known facts. Tensor methods represent the task through structured decompositions of the binary knowledge-graph tensor.
- A triple (es, r, eo) consists of subject and object entities from E and a relation from R.
- Link prediction learns a scoring function φ(es, r, eo) that assigns real-valued scores to triples and aims to score missing triples correctly.Positive scores typically indicate predicted true facts, while negative scores indicate predicted false ones.
- Table 1 compares state-of-the-art models by scoring function, relation-parameter dimensionality, and significant space-complexity terms.Its notation includes entity and relation dimensions, entity and relation counts, convolution, dot products, and Tucker tensor products.
- Tucker decomposition factorizes a tensor into three factor matrices and a smaller core tensor whose entries encode component interactions.The core can provide a compressed representation when its dimensions are smaller than those of the original tensor.
4 Tucker Decomposition for Link Prediction
TuckER applies Tucker decomposition to the binary tensor of knowledge-graph triples, using shared entity embeddings, relation embeddings, and a learned core tensor. Its scores are converted into triple probabilities, and shared core parameters avoid growth with the numbers of entities or relations.
- Model formulation: TuckER factorizes the binary knowledge-graph tensor with shared subject and object entity embeddings and separate relation embeddings.The entity matrix satisfies E = A = C, while the relation matrix satisfies R = B.
- Model formulation: The scoring function combines subject and object embeddings with a relation embedding through the core tensor W.W has dimensions de×dr×de, while es, eo ∈ R^de and wr ∈ R^dr.
- Prediction: Applying the logistic sigmoid to each score produces the predicted probability that a triple is true.The architecture is illustrated in Figure 1.
- Parameter sharing: TuckER’s core tensor stores knowledge shared across entities and relations rather than encoding all learned information in their embeddings.This distinguishes it from simpler models such as DistMult, ComplEx, and SimplE.
- Training: TuckER is trained numerically with reciprocal-relation augmentation, 1-N scoring, and Bernoulli negative log-likelihood.The loss operates on predicted probabilities and binary labels for all entities paired with a subject-relation combination.
5 Theoretical Analysis
TuckER is theoretically fully expressive with a sufficient embedding-size bound and subsumes several established tensor-factorization models. Its shared core enables asymmetric relation modeling while keeping relation-dependent parameter growth linear, although full reconstruction may require large dimensions.
- 5.1 Full Expressiveness and Embedding Dimensionality: TuckER is fully expressive with entity dimension de = ne and relation dimension dr = nr.The theorem guarantees representation of any ground truth over a set of entities and relations.
- 5.1 Full Expressiveness and Embedding Dimensionality: The proof assigns one-hot entity and relation embeddings and sets each core-tensor entry to 1 for true facts and −1 otherwise.After the logistic sigmoid, the resulting scores represent the original binary tensor.
- 5.1 Full Expressiveness and Embedding Dimensionality: The full-expressiveness theorem establishes potential reconstruction of all information and noise, while practical reconstruction is expected to need smaller dimensions because data follows structure.Low decomposition rank is desirable because it encourages learning structure and generalizing rather than memorizing inputs.
- 5.2 Relation to Previous Linear Models: RESCAL, DistMult, ComplEx, and SimplE can each be represented as special cases of TuckER through constrained core tensors or decompositions.RESCAL corresponds to Tucker2, DistMult uses a superdiagonal core, and ComplEx and SimplE use structured zero-and-nonzero patterns.
- 5.2 Relation to Previous Linear Models: DistMult cannot represent asymmetric relations because identical subject and object embeddings enforce symmetry.ComplEx and SimplE instead introduce asymmetry through distinct entity embeddings and diagonal relation structures.
- 5.3 Representing Asymmetric Relations: TuckER represents relations as vectors while its asymmetric shared core produces full-rank relation matrices, making parameter growth linear in the number of relations.The resulting matrices can perform rotations, reflections, stretches, and other linear transformations.
6 Experiments and Results
TuckER is evaluated on four standard link-prediction datasets using filtered ranking metrics and tuned hyperparameters. It outperforms prior models broadly, while parameter-sharing experiments show strong performance with fewer parameters, especially at smaller embedding sizes.
- Implementation and Experiments: TuckER is evaluated on FB15k, FB15k-237, WN18, and WN18RR using filtered MRR and hits@1, hits@3, and hits@10.At evaluation, candidate entities are ranked for each test entity-relation pair after removing other known true triples.
- Link Prediction Results: TuckER outperforms previous state-of-the-art models on all metrics across all datasets except hits@10 on WN18, where R-GCN performs better.Its results exceed those of linear models and many deeper neural-network or reinforcement-learning architectures.
- Link Prediction Results: TuckER consistently outperforms ComplEx and SimplE on WN18RR despite using approximately 9.4 million rather than 16.4 million parameters.The authors attribute this advantage to knowledge sharing between relations through the core tensor.
- Link Prediction Results: On FB15k, TuckER improves over ComplEx by 14% and SimplE by 8% on hits@1.The margin over other linear models increases on datasets with many relations.
- Influence of Parameter Sharing: At embedding size 20 on FB15k-237, TuckER uses approximately 300,000 parameters while nearly matching ComplEx and SimplE at size 200 with approximately 6 million parameters.The MRR gap grows at smaller sizes: 4.2% for ComplEx and 9.9% for SimplE at size 20.
7 Conclusion
The paper concludes that TuckER is a straightforward, fully expressive linear model whose parameter count grows linearly with entities or relations. It achieves strong benchmark results, supports multi-task learning across relations, and subsumes several earlier linear models.
- Conclusion: TuckER is a Tucker-decomposition model that achieves state-of-the-art results on standard link-prediction datasets.Its performance is partly attributed to multi-task learning across relations.
- Conclusion: TuckER is fully expressive while its parameter count grows linearly with the number of entities or relations.The paper also derives a dimensionality bound for full expressiveness.
- Conclusion: RESCAL, DistMult, ComplEx, and SimplE can be interpreted as special cases of TuckER.The conclusion identifies this subsumption as a unifying relationship among linear tensor-factorization models.
- Conclusion: Future work may incorporate background knowledge about individual relation properties into TuckER.
A Hyper-parameters
The appendix reports the best-performing hyperparameter values for TuckER across datasets and for ComplEx and SimplE on FB15k-237.
- Hyper-parameters: Table 5 lists TuckER’s best-performing hyperparameter values across all datasets.The table includes learning rate, decay rate, label smoothing, and dropout settings.
- Hyper-parameters: Table 6 lists the best-performing hyperparameter values for ComplEx and SimplE on FB15k-237.These settings were used to produce the results shown in Figure 4.