Source-linked AI summary
Quaternion Knowledge Graph Embeddings
Shuai Zhang, Yi Tay, Lina Yao, Qi Liu
TL;DR
Incomplete knowledge graphs require methods for predicting missing links from learned entity and relation representations. QuatE addresses this with quaternion embeddings and relational rotations, generalizes ComplEx, and achieves state-of-the-art performance across four established benchmarks.
Problem
Real-world knowledge graphs are usually incomplete, motivating methods that predict missing entity links from learned representations.
Method
QuatE represents entities and relations with quaternion embeddings and models relations as rotations using Hamilton product followed by quaternion inner products.
Results
QuatE achieves state-of-the-art performance on four well-established knowledge graph completion benchmarks.
Takeaways & Limitations
QuatE provides a quaternion-space framework that generalizes ComplEx while modeling relational patterns with richer rotational degrees of freedom.
Abstract
from arXiv · showhide
In this work, we move beyond the traditional complex-valued representations, introducing more expressive hypercomplex representations to model entities and relations for knowledge graph embeddings. More specifically, quaternion embeddings, hypercomplex-valued embeddings with three imaginary components, are utilized to represent entities. Relations are modelled as rotations in the quaternion space. The advantages of the proposed approach are: (1) Latent inter-dependencies (between all components) are aptly captured with Hamilton product, encouraging a more compact interaction between entities and relations; (2) Quaternions enable expressive rotation in four-dimensional space and have more degree of freedom than rotation in complex plane; (3) The proposed framework is a generalization of ComplEx on hypercomplex space while offering better geometrical interpretations, concurrently satisfying the key desiderata of relational representation learning (i.e., modeling symmetry, anti-symmetry and inversion). Experimental results demonstrate that our method achieves state-of-the-art performance on four well-established knowledge graph completion benchmarks.
1 Introduction
Knowledge graphs are often incomplete, motivating low-dimensional embeddings for predicting missing links. QuatE extends complex-valued embeddings with quaternion representations and rotations, offering expressive interactions, multiple rotational degrees of freedom, and state-of-the-art benchmark performance.
- Incomplete real-world knowledge graphs motivate link prediction using low-dimensional entity and relation representations.Knowledge graphs support relational reasoning and structural representation learning across applications including question answering, search, and natural language processing.
- QuatE represents entities and relations as quaternions and scores triples by rotating the head with Hamilton product before taking a quaternion inner product with the tail.Quaternion embeddings use one real and three imaginary components.
- Hamilton product captures inter-latent interactions among all four quaternion components, producing a highly expressive model.The formulation is presented as more expressive than complex Hermitian operators and Euclidean inner products.
- Quaternion rotations provide two planes of rotation and more degrees of freedom than complex rotations, while QuatE inherits symmetry, anti-symmetry, and inversion modeling.The authors also state that QuatE can use equal or fewer parameters while outperforming previous work.
- QuatE achieves state-of-the-art performance on WN18, FB15K, WN18RR, and FB15K-237.
2 Related Work
Prior knowledge graph embedding work includes translational, bilinear semantic-matching, and neural-network-based models. QuatE instead uses quaternion geometry for expressive semantic matching and generalizes complex-valued approaches while differing from RotatE’s translational formulation.
- Translational models: Translational models interpret relations as vector additions, with later variants introducing relation-specific hyperplanes, spaces, or projections.
- Semantic matching models: Semantic-matching models score plausibility by matching latent entity and relation semantics, using bilinear matrices, diagonal matrices, or related constructions.
- QuatE: QuatE uses relational rotation quaternions for expressive semantic matching and can generalize DistMult and ComplEx to more advanced hypercomplex spaces.
- QuatE: QuatE has two rotation planes and is a semantic-matching model, whereas RotatE has one rotation plane and is a translational model.
3 Hamilton’s Quaternions
Hamilton’s quaternions extend complex numbers into a four-dimensional hypercomplex system with non-commutative multiplication. The paper defines quaternion conjugation, inner products, and Hamilton products, which can represent four-dimensional rotations.
- A quaternion has one real and three imaginary components, Q = a+bi+cj+dk, extending complex numbers to four-dimensional space.The imaginary units satisfy i^2 = j^2 = k^2 = ijk = −1 and follow non-commutative multiplication rules.
- Quaternion conjugation negates the three imaginary components while retaining the real component.For Q = a+bi+cj+dk, the conjugate is a−bi−cj−dk.
- The quaternion inner product sums the component-wise inner products of the real, i, j, and k parts.This produces a scalar similarity across corresponding quaternion components.
- The Hamilton product combines all component multiplications to produce another quaternion and is not commutative.Its algebraic form includes the real, i, j, and k components of both operands.
- Multiplying quaternions can scale one quaternion by another’s magnitude and apply a special four-dimensional rotation.
4 Method
QuatE embeds entities and relations as quaternions, rotates heads with normalized relation quaternions, and scores triplets by quaternion inner products. The formulation supports major relation patterns, variable composition, and reductions to established complex- and real-valued models.
- Quaternion embeddings: QuatE embeds entities and relations as k-dimensional quaternion vectors with one real and three imaginary components.Entity embeddings are stored in Q ∈ H^N×k and relation embeddings in W ∈ H^M×k.
- Relational rotation: The model normalizes each relation quaternion, then rotates the head embedding through a Hamilton product before comparing it with the tail.Unit-quaternion normalization removes scaling effects; the rotated head and tail are scored with a quaternion inner product.
- Relational rotation: Hamilton products couple the real, i, j, and k components, while quaternion rotations provide two rotation planes and more degrees of freedom than complex rotations.Right multiplication by a unit quaternion gives a right-isoclinic rotation; left multiplication provides an alternative with the same basic geometrical meaning.
- Scoring and optimization: QuatE uses quaternion inner products as scores and learns parameters with regularized logistic loss, negative sampling, and Adagrad.The framework compares rotated heads with tails for triplet scoring and applies ℓ2 regularization to entity and relation parameters.
- Relation patterns: Quaternion representations can model symmetry, antisymmetry, and inversion through suitable relation components and quaternion conjugation.Symmetry follows when relation imaginary parts are zero, antisymmetry when they are nonzero, and inversion uses conjugation.
- Connections and composition: QuatE permits composition patterns influenced jointly by relations and entity embeddings, and reduces to ComplEx or DistMult under specified restrictions.Setting j and k coefficients to zero recovers complex multiplication, while removing imaginary parts and normalization yields DistMult.
5 Experiments and Results
QuatE is evaluated on four knowledge graph completion benchmarks using filtered ranking metrics and comparisons with translational and semantic-matching baselines. Results show strong performance, relation-pattern modeling, parameter savings, and sensitivity to scoring-function design and normalization.
- Experimental Setup: Experiments use WN18, FB15K, WN18RR, and FB15K-237, with MR, MRR, and Hit@n under filtered evaluation.Baselines include translational models such as TransE, TorusE, and RotatE, plus semantic-matching and neural models.
- Results: QuatE performs competitively across metrics and outperforms representative complex-valued models ComplEx and RotatE on the four datasets.The reported gains over RotatE are attributed to quaternion rotation relative to rotation in the complex plane.
- Results: QuatE outperforms all baselines on WN18 except Hit@10, achieves the best results on FB15K, and captures symmetry, antisymmetry, and inversion patterns.These patterns account for a large portion of the relations in WN18 and FB15K.
- Results: QuatE achieves a large performance gain on WN18RR and FB15K-237, including datasets where trivial inverse relations are removed.Relation-level MRR analysis reports superior representation capability across different relation types, while fixed-composition methods are sometimes weaker.
- Model Variants: N3 regularization and reciprocal learning boost performance, especially on FB15K and FB15K-237, but require a large embedding dimension.QuatE2 also avoids relation normalization because N3 regularization reduces embedding norms.
- Model Analysis: QuatE reduces parameter size relative to RotatE, saving up to 80% of parameters on datasets without trivial inverse relations while maintaining superior performance.QuatE3 uses almost the same number of free parameters as QuatE1, whereas TorusE uses embedding dimension 10000 on WN18 and FB15K.
- Ablations: Removing relation normalization harms embedding performance, while replacing relational rotation with head–tail Hamilton products loses the geometric rotation property and performs poorly.An additional rotational quaternion for the tail achieves competitive results but may reduce efficiency.
6 Conclusion
QuatE is a quaternion-space knowledge graph embedding model with mathematical and physical interpretations. It is designed to model key relation patterns, provide expressive representations, generalize effectively, and achieve state-of-the-art results with fewer free parameters.
- Conclusion: QuatE operates in quaternion space and is designed to model key relation patterns with expressive representations and good generalization.The conclusion reports overall state-of-the-art performance across four datasets, with fewer free parameters.
7 Appendix
The appendix verifies relational patterns, lists QuatE configurations, and extends the framework from quaternions to octonions. It reports that OctonionE performs equally to QuatE on WN18 and WN18RR, while noting possible side effects from lost associativity.
- Proof of antisymmetry and inversion: The appendix proves antisymmetry by showing that corresponding terms differ in sign when imaginary components are nonzero.
- Proof of antisymmetry and inversion: It proves inversion by expanding both sides of the relevant equality and checking that the terms match.
- Hyperparameters and training: QuatE hyperparameters vary across WN18, FB15K, WN18RR, and FB15K-237, including embedding dimensions, regularization weights, and negative-sample counts.
- Octonion extension: Octonions extend quaternions to one real component and seven imaginary components, with multiplication rules encoded by the Fano Plane.
- Octonion extension: OctonionE treats octonions as two quaternions and retains QuatE's scoring functions, but performs equally to QuatE on WN18 and WN18RR.The authors suggest that lost associativity in octonions might introduce side effects.